Capturing traces to DuckDB
How route-switch logs every call to a per-prompt SQLite dataset and a DuckDB analytics store, and how to read per-prompt cost, latency, and success rate.
route-switch treats analytics as a first-class part of the gateway, not a bolt-on. Every served call is written to two places at once.
Two stores, two jobs
- Per-prompt SQLite dataset — one file per template under
dataset.base_path. Each row records the rendered input, the output, the variables, the model, the provider, whether the call succeeded, and its cost. This is the raw material the optimizer replays. Datasets are trimmed todataset.max_records. - DuckDB analytics store — request/response summaries, latency, error rates, and cost rollups, aggregated per prompt and globally.
Reading the numbers
The gateway surfaces aggregates through operational endpoints:
# per-prompt stats
curl http://localhost:8080/v1/prompts/support-reply/stats
# global analytics rollup
curl http://localhost:8080/v1/system/analyticsBecause the store is DuckDB, you can also point any DuckDB client at the file and run SQL directly — useful for ad-hoc cost-per-prompt or success-rate-over-time queries.
Why this matters
Without per-prompt economics, routing is guesswork. With it, performance_based routing has real signal to act on, and you can see prompt drift before it becomes support tickets — which is exactly what prompt optimization from traces needs.
route-switch publishes no synthetic benchmark numbers. The figures you see are the ones your own traffic produced.
Frequently asked questions
Is the SQLite dataset a response cache?
No. It captures every call for optimization and analytics, but identical requests are not deduplicated or served from cache. route-switch has no response cache.
What columns does the per-prompt dataset record?
Rendered input, output, variables, model, provider, success, and cost, with a created_at timestamp. Datasets are trimmed to dataset.max_records.
Related
Choosing a routing strategy
When to use round_robin, weighted, or performance_based routing across registered prompt+model+provider combinations — and how fallback fits in.
GuideRunning MIPROv2 on production traces
How route-switch closes the loop: replaying captured traces to score instruction and few-shot candidates with goptuna, then shipping the winner.
GuideDrop-in OpenAI base_url swap
Migrate an existing OpenAI client to route-switch by changing one field — the base URL — without rewriting request code or prompts.