Prompt optimization from production traces
route-switch captures production traffic into per-prompt datasets and reruns MIPROv2 (goptuna Bayesian search) against it, scoring instruction and few-shot candidates and writing the winner back to the registry on a configurable interval.
The prompt that shipped at launch is rarely the prompt production needs three months later. route-switch treats that drift as something to measure and correct, not discover through support tickets.
The closed loop
Traffic captured → dataset accumulated → optimizer rerun → prompt updated. Concretely, MIPROv2 bootstraps a calibration sample, generates instruction candidates, drives a goptuna Bayesian search over instruction and few-shot combinations, scores each by replaying captured rows, and writes the winner back to the registry.
gateway:
optimization:
enabled: true
interval: 6hGrounded, not magical
Optimization is per-prompt and scored with the evaluation strategy you choose. It reliably improves format adherence, demonstration selection, and instruction specificity. It cannot make a model do a task it fundamentally can’t, fix a mismatched scorer, or rescue a non-representative dataset — the honesty about inputs and loss function is the point.
Prerequisites
You need captured traffic first: set up trace capture to DuckDB, then enable the optimizer. This is the capability that makes route-switch a prompt platform, not just a router — the thing a plain aggregator like OpenRouter does not do.
Frequently asked questions
Why optimize against captured traffic instead of a static eval set?
Because production traffic is what your prompt actually faces. Optimizing against it targets the distribution you serve, not a curated benchmark that may not represent real inputs.
How often does optimization run?
On a configurable interval when the background optimizer is enabled, or on demand via the CLI. It is per-prompt, so different templates optimize independently.
Related
Latency-aware routing
Use performance-based routing and DuckDB latency data to favor combinations that respond faster, with fallback when a provider slows down.
Use caseMulti-provider failover
Front six providers behind one endpoint, and fall back automatically when a provider combination's success rate drops below its threshold.
Use caseCost-aware routing
Bias traffic toward cheaper model combinations with weighted routing, and see per-prompt cost in DuckDB so routing decisions are grounded in real spend.