# route-switch — full text dump for LLM consumption

> Intelligent LLM routing with automatic prompt optimization. Route-Switch is
> an OpenAI-compatible Go gateway from Skelf Research that fronts multiple
> LLM providers, load-balances across registered prompt+model combinations,
> and reruns MIPROv2 (instruction + few-shot search via goptuna's Bayesian
> optimizer) against captured production traces to keep prompts honest.

## Identity

- Name: route-switch
- Organization: Skelf Research
- Repository: https://github.com/Skelf-Research/route-switch
- Documentation: https://docs.skelfresearch.com/route-switch/
- Marketing site: https://route-switch.skelfresearch.com
- License: MIT
- Language: Go 1.24+

## What it is

Route-Switch is a single Go binary that runs as either a CLI or an HTTP
gateway. The gateway exposes an OpenAI-compatible /v1/chat/completions
endpoint (streaming + non-streaming) plus operational endpoints (/health,
/status, /v1/prompts/{id}/stats, /v1/system/analytics).

The gateway holds a registry of "combinations" — tuples of
(prompt template, model, provider, weight, fallbacks). A request lands on
the gateway, the load balancer picks a combination according to the
configured strategy (round_robin, weighted, performance_based), the prompt
template is rendered with variables from the request, the call is made
through the gollm provider adapter, the response is returned, and the call
is logged to a per-prompt SQLite dataset and to the DuckDB analytics store.

## What "automatic prompt optimization" means concretely

Route-Switch implements MIPROv2:

1. Bootstrap a calibration sample from the prompt's SQLite dataset.
2. Generate instruction candidates by calling the configured provider with
   an instruction-proposal meta-prompt.
3. Drive a Bayesian search (goptuna) over (instruction, few-shot demo set)
   combinations.
4. Score each candidate by replaying dataset rows under one of three
   evaluation strategies (Similarity, ExactMatch, KeywordMatch — or a
   custom Go implementation of EvaluationStrategy).
5. Write the winning prompt back to the registry.

This is per-prompt and per-(prompt, model) — not a generic "rewrite the
user query" layer, not RLHF, not training. Run it on demand via
`--optimize-prompt`, or enable the background optimizer
(`gateway.optimization.enabled: true`, configurable interval).

## What it is not

- Not a semantic router that classifies arbitrary user queries and picks
  one of 200 models in real time. Routing is across combinations you
  registered, by strategy.
- Not an evaluation harness with held-out ground truth. Scoring runs
  against captured traffic with the strategy you configured.
- Not a guardrails / PII / moderation layer.
- Not a vector database or feature store.
- Not a response cache.
- Not a managed cloud service.

## Provider coverage

Via the gollm adapter: OpenAI, Anthropic, Google, Ollama, Cohere, Mistral,
and others. Credentials are supplied per-provider in `config.yaml`.

## State

- Per-prompt SQLite databases under `dataset.base_path` (one file per
  template). Records: rendered_input, output, variables, model, provider,
  success, cost, created_at. Trimmed to `dataset.max_records`.
- DuckDB analytics store: request/response summaries, latency, error
  rates, cost rollups.
- YAML template manifests per registered prompt.

## Pages on the marketing site

### Home (/)
Hero, three-up features, "what automatic prompt optimization actually
means", honest "yes / not" list, links to compare pages and the three
recent notes.

### About (/about)
Why route-switch exists, the three problems it tries to address
(provider lock-in, prompt drift, missing cost/quality/latency view), what
ships in the binary, what it deliberately isn't.

### Notes (/blog)

- Multi-model routing as a feature, not a hack
  https://route-switch.skelfresearch.com/blog/multi-model-routing-as-a-feature/

  Argues that an in-application if-tree around provider SDKs has four
  hidden taxes: prompts welded to providers, cost data per-client instead
  of per-prompt, reactive fallback, and provider switches that require
  deploys. Shows how a gateway lifts each by making the prompt a
  first-class registered object, fallback a policy, and per-combination
  economics legible via the /v1/prompts/{id}/stats endpoint. Honest about
  when this is the wrong move: one provider, no tracking needs, very
  tight latency budgets.

- Per-model prompt optimization: what actually moves the score
  https://route-switch.skelfresearch.com/blog/per-model-prompt-optimization-what-moves-the-score/

  Explains MIPROv2 as two searches glued together (instruction search +
  few-shot demo search) driven by goptuna's Bayesian optimizer. Walks
  through what the three built-in evaluation strategies actually compute
  (Similarity = token-overlap heuristics; ExactMatch = trimmed equality;
  KeywordMatch = presence of pre-declared keywords). Lists what the
  optimizer reliably moves (format adherence, demo selection,
  instruction specificity) and what it does not (a task the model can't
  do, a mismatched score function, a non-representative dataset).
  Practical checklist for setting up the first optimization run.

- Quality / cost / latency: the routing triangle
  https://route-switch.skelfresearch.com/blog/quality-cost-latency-the-routing-triangle/

  Frames LLM serving as a trilemma you can't optimize all three of at
  once. Walks through three postures (quality-constrained,
  cost-constrained, latency-constrained) with config snippets for each,
  showing how the strategy, weights, and fallback threshold encode the
  posture. Honest about a current limitation: the gateway can't enforce
  explicit p95 latency budgets — you exclude slow models or run an
  upstream timeout. Closes with how analytics let you check that the
  trade-off you thought you were making is the one you're actually
  making.

### Compare

- vs. OpenRouter — https://route-switch.skelfresearch.com/compare/openrouter
  Hosted aggregator (200+ models behind one API key, unified billing,
  per-request routing preferences) vs. self-hosted gateway with prompt
  optimization and local state. Adjacent problems, not substitutes.

- vs. Portkey — https://route-switch.skelfresearch.com/compare/portkey
  Observability-first AI gateway (rich dashboards, guardrails, semantic
  cache, prompt library UI) vs. optimization-first gateway (closed-loop
  MIPROv2 on captured traffic, prompt-as-registered-object). Different
  bets about which engineering problem deserves the most attention.

## Full site taxonomy

In addition to Home, About, Notes, and the two Compare pages, the site now includes:

- Features (/features): the full capability surface grouped into gateway, routing &
  prompts, prompt optimization, and operations.
- How it works (/how-it-works): the seven-stage request lifecycle (request → template
  render → route → provider call → fallback → log → optimize) with an architecture diagram
  and a description of the state the gateway keeps (per-prompt SQLite, DuckDB, YAML manifests).
- Quickstart (/quickstart): build → config.yaml → register prompt → run gateway → point
  OpenAI client → enable optimizer (marked up as HowTo).
- FAQ (/faq): general, routing & providers, prompt optimization, and operations questions,
  emitted as FAQPage structured data.
- Glossary (/glossary): LLM gateway, OpenAI-compatible endpoint, combination, routing
  strategy, fallback, prompt registry, MIPROv2, goptuna, gollm, DuckDB, evaluation strategy,
  trace/dataset, portable package.
- Guides (/guides): drop-in OpenAI base_url swap; choosing a routing strategy; capturing
  traces to DuckDB; running MIPROv2 on production traces.
- Use cases (/use-cases): multi-provider failover; cost-aware routing; latency-aware routing;
  prompt optimization from production traces; OpenAI-compatible migration.

All content pages link up to the Skelf Research primary site at https://www.skelfresearch.com.

## Trust / honesty stance

The site avoids fabricated benchmarks. It does not list pricing or
tokenomics. The "automatic prompt optimization" feature is explained in
its actual algorithmic terms (MIPROv2, goptuna, evaluation strategies),
not as marketing magic. Where the project does not cover something
(guardrails, response caching, latency-budget enforcement), the site says
so explicitly rather than implying coverage.