Choosing a routing strategy

route-switch routes each request across the combinations you register — tuples of (prompt template, model, provider, weight, fallbacks). The strategy decides which eligible combination serves a given request. There are three.

round_robin

Distributes requests evenly across all eligible combinations. Predictable and unbiased.

Use it when you are A/B-comparing combinations and want equal exposure, or when all combinations are roughly equivalent and you just want even spread.

weighted

Biases selection toward combinations by a configured integer weight. A weight-3 combination is picked three times as often as a weight-1 combination.

Use it when you want a primary/secondary split — say, most traffic to a cheaper model with a slice going to a stronger one — without hard-coding percentages in application logic.

combinations:
  - prompt: support-reply
    provider: openai
    model: gpt-4
    weight: 3
  - prompt: support-reply
    provider: anthropic
    model: claude-3-5-sonnet
    weight: 1

performance_based

Favors combinations with better observed success from captured traffic. The balancer leans on what the DuckDB analytics actually show rather than a static weight.

Use it when you have accumulated enough traffic for success rates to be meaningful and you want routing to adapt to real outcomes.

Fallback is orthogonal

Any strategy supports fallback: when a combination’s recent success rate drops below a configured threshold, the balancer redirects to the next eligible combination. Fallback is a policy you set per combination, not a fourth strategy.

Putting it together

Start with round_robin while you gather data, mark a resilient combination as fallback: true, then graduate to performance_based once success rates are trustworthy. The full request lifecycle is on the how it works page; see also multi-provider failover and cost-aware routing.