What is route-switch?
route-switch is an OpenAI-compatible LLM gateway written in Go. It fronts multiple providers behind a single /v1/chat/completions endpoint, routes across registered prompt+model+provider combinations, records analytics in DuckDB, and reruns MIPROv2 prompt optimization against captured production traces.
What language is route-switch written in?
Go (1.24+). It ships as a single binary that runs as either a CLI or an HTTP gateway. There is no daemon and no external runtime dependency beyond the SQLite and DuckDB files it manages.
What license is route-switch under?
MIT. route-switch is fully open source and developed in the open at github.com/Skelf-Research/route-switch. There are no paid tiers or feature gates.
Is route-switch a managed service?
No. route-switch is self-hosted. You build the binary, you run it, and you keep the per-prompt SQLite datasets and the DuckDB analytics store on your own infrastructure. Your traffic does not pass through a third party.
Who builds route-switch?
route-switch is built by Skelf Research, a research lab. It is one of several developer tools published under the skelfresearch.com family.
Which providers does route-switch support?
Through the gollm adapter: OpenAI, Anthropic, Google, Ollama, Cohere, and Mistral, plus others gollm supports. Credentials are supplied per provider in config.yaml — route-switch is bring-your-own-key.
What routing strategies are available?
Three: round_robin distributes evenly across combinations, weighted biases toward combinations by a configured weight, and performance_based favors combinations with better observed success. Each supports automatic fallback when a combination's success rate drops below a threshold.
Is route-switch a semantic router?
No. It does not classify a free-form user query and pick "the best" of hundreds of models in real time. Routing happens across the prompt+model+provider combinations you register, selected by strategy — not by an LLM judging the request.
Does route-switch support streaming?
Yes. The /v1/chat/completions endpoint supports both streaming (OpenAI chunk format, with usage totals in the final chunk) and non-streaming responses.
Do I have to rewrite my code to use route-switch?
No. Because the endpoint is OpenAI-compatible, you point your existing OpenAI client at the gateway by changing its base URL. Provider choice, routing strategy, and fallbacks move into config the gateway owns.
What does "automatic prompt optimization" mean here?
route-switch implements MIPROv2: it bootstraps a calibration sample from a prompt's dataset, generates instruction candidates, drives a Bayesian search (goptuna) over instruction and few-shot demo combinations, scores each candidate by replaying captured rows, and writes the winner back to the prompt registry. It is per-prompt and per-(prompt, model).
Is the optimizer RLHF or fine-tuning?
No. It does not train or fine-tune a model. It rewrites the prompt templates you registered, scored against the data your traffic produced, using the evaluation strategy you configured. It is a prompt search, not model training.
How are optimization candidates scored?
By replaying dataset rows under one of three built-in evaluation strategies: Similarity (token-overlap heuristics), ExactMatch (trimmed equality), or KeywordMatch (presence of declared keywords). If those do not fit, implement the EvaluationStrategy Go interface with your own scorer.
Does route-switch include an evaluation harness with held-out ground truth?
No. The optimizer scores against your captured traffic, not a held-out labeled eval set. Bring your own ground truth via a custom EvaluationStrategy if Similarity, ExactMatch, or KeywordMatch is not sufficient.
What does route-switch store, and where?
Per-prompt SQLite datasets (one file per template, under dataset.base_path) recording rendered input, output, variables, model, provider, success, and cost; a DuckDB analytics store for cross-prompt aggregates; and YAML template manifests. All local files you control.
Does route-switch cache responses?
No. The per-prompt SQLite dataset captures every call for optimization, but it is not a response cache — identical requests are not deduplicated or served from cache.
Does route-switch provide guardrails or PII redaction?
No. Guardrails, content moderation, and PII redaction are out of scope. If you need them, place a dedicated guardrails layer in front of route-switch.
Can I move a prompt between environments?
Yes. route-switch bundles a template plus its dataset snapshot and recent logs into a portable "package" (a tarball) that you can move between environments — the prompt, its data, and its history travel together.
Frequently asked questions
Everything about route-switch — the OpenAI-compatible LLM gateway with closed-loop prompt optimization.
General
What is route-switch?
route-switch is an OpenAI-compatible LLM gateway written in Go. It fronts multiple providers behind a single /v1/chat/completions endpoint, routes across registered prompt+model+provider combinations, records analytics in DuckDB, and reruns MIPROv2 prompt optimization against captured production traces.
What language is route-switch written in?
Go (1.24+). It ships as a single binary that runs as either a CLI or an HTTP gateway. There is no daemon and no external runtime dependency beyond the SQLite and DuckDB files it manages.
What license is route-switch under?
MIT. route-switch is fully open source and developed in the open at github.com/Skelf-Research/route-switch. There are no paid tiers or feature gates.
Is route-switch a managed service?
No. route-switch is self-hosted. You build the binary, you run it, and you keep the per-prompt SQLite datasets and the DuckDB analytics store on your own infrastructure. Your traffic does not pass through a third party.
Who builds route-switch?
route-switch is built by Skelf Research, a research lab. It is one of several developer tools published under the skelfresearch.com family.
Routing & providers
Which providers does route-switch support?
Through the gollm adapter: OpenAI, Anthropic, Google, Ollama, Cohere, and Mistral, plus others gollm supports. Credentials are supplied per provider in config.yaml — route-switch is bring-your-own-key.
What routing strategies are available?
Three: round_robin distributes evenly across combinations, weighted biases toward combinations by a configured weight, and performance_based favors combinations with better observed success. Each supports automatic fallback when a combination's success rate drops below a threshold.
Is route-switch a semantic router?
No. It does not classify a free-form user query and pick "the best" of hundreds of models in real time. Routing happens across the prompt+model+provider combinations you register, selected by strategy — not by an LLM judging the request.
Does route-switch support streaming?
Yes. The /v1/chat/completions endpoint supports both streaming (OpenAI chunk format, with usage totals in the final chunk) and non-streaming responses.
Do I have to rewrite my code to use route-switch?
No. Because the endpoint is OpenAI-compatible, you point your existing OpenAI client at the gateway by changing its base URL. Provider choice, routing strategy, and fallbacks move into config the gateway owns.
Prompt optimization
What does "automatic prompt optimization" mean here?
route-switch implements MIPROv2: it bootstraps a calibration sample from a prompt's dataset, generates instruction candidates, drives a Bayesian search (goptuna) over instruction and few-shot demo combinations, scores each candidate by replaying captured rows, and writes the winner back to the prompt registry. It is per-prompt and per-(prompt, model).
Is the optimizer RLHF or fine-tuning?
No. It does not train or fine-tune a model. It rewrites the prompt templates you registered, scored against the data your traffic produced, using the evaluation strategy you configured. It is a prompt search, not model training.
How are optimization candidates scored?
By replaying dataset rows under one of three built-in evaluation strategies: Similarity (token-overlap heuristics), ExactMatch (trimmed equality), or KeywordMatch (presence of declared keywords). If those do not fit, implement the EvaluationStrategy Go interface with your own scorer.
Does route-switch include an evaluation harness with held-out ground truth?
No. The optimizer scores against your captured traffic, not a held-out labeled eval set. Bring your own ground truth via a custom EvaluationStrategy if Similarity, ExactMatch, or KeywordMatch is not sufficient.
Operations
What does route-switch store, and where?
Per-prompt SQLite datasets (one file per template, under dataset.base_path) recording rendered input, output, variables, model, provider, success, and cost; a DuckDB analytics store for cross-prompt aggregates; and YAML template manifests. All local files you control.
Does route-switch cache responses?
No. The per-prompt SQLite dataset captures every call for optimization, but it is not a response cache — identical requests are not deduplicated or served from cache.
Does route-switch provide guardrails or PII redaction?
No. Guardrails, content moderation, and PII redaction are out of scope. If you need them, place a dedicated guardrails layer in front of route-switch.
Can I move a prompt between environments?
Yes. route-switch bundles a template plus its dataset snapshot and recent logs into a portable "package" (a tarball) that you can move between environments — the prompt, its data, and its history travel together.