Routing

The AICP gateway selects a model and provider for each request. You can name a specific model, or let AICP pick the cheapest model that can handle each prompt. Every routing decision is logged so you can audit and replay any request.

This page covers how to configure routing. For how the router decides (the two stages, the quality dial, escalation and failover), see Intelligent routing. For how it was evaluated and what it delivers, see Routing performance.

Automatic model selection

Send model: "auto" instead of a specific model and AICP routes each prompt to the cheapest model that can answer it well. Easy prompts go to a small, cheap model; hard prompts are escalated to a stronger one. You keep quality while cutting cost on the majority of requests that don't need a frontier model.

const res = await client.chat.completions.create({
  model:    'auto',
  messages: [{ role: 'user', content: 'Summarize this in one line: ...' }],
});

How it works, from your side:

• AICP classifies each prompt into a tier (small / medium / large). A tier is a quality floor: every model in it is at least as capable as the tier's reference model.
• Within that tier, your routing strategy (below) picks the model: cheapest, fastest, best-quality, or balanced.
• If your enabled providers can't serve the chosen tier, AICP escalates upward (never to a weaker model), and returns a clear error only if nothing suitable is configured.

Each response includes headers describing the decision:

Header	Meaning
X-Router-Tier	The tier chosen for the prompt (0 small · 1 medium · 2 large)
X-Router-Model	The model actually served
X-Router-Estimated-Saving	Estimated fraction saved vs always using the large-tier model
X-Router-Tier-Escalated	Present only when the request was escalated to a higher tier

You control how aggressively auto saves cost by choosing your routing strategy and enabling the providers/models you want it to use.

Strategies

Your routing strategy decides which model wins, both when you name a model (across providers) and when auto picks within a tier.

01

balancedDefault

Weighs cost, latency, and quality together.

02

cheapest

Picks the lowest-cost option.

03

fastest

Picks the lowest-latency option.

04

highest_quality

Picks the most capable option available.

05

manual

Pins a specific provider you choose.

The latency that fastest and balanced rank on starts as a benchmark estimate but is replaced by latency observed from your own traffic once a model has enough recent requests, so routing reflects how each model actually performs for you, not a generic benchmark. Models with little traffic keep using the estimate until real data accumulates.

Configure the strategy via the dashboard (Settings → Routing) or the SDK:

// JavaScript
await client.routing.updateConfig({ strategy: 'balanced' });

# Python
client.routing.update_config({"strategy": "balanced"})

Fallback

fallbackBehavior controls what happens when no provider passes your policy constraints (blocked providers, region restrictions, cost/latency limits, etc.). It is not a retry-on-5xx setting:

Value	Behaviour
fail	Return an error if no eligible provider is found
allow_commercial	Fall back to any enabled commercial provider (default)
allow_global	Fall back to any enabled provider globally
allow_cheapest	Fall back to the cheapest available provider

await client.routing.updateConfig({
  strategy: 'balanced',
  policy:   { fallbackBehavior: 'allow_commercial' },
});

Model aliases

Model aliases abstract away provider-specific model names. Define them once and your application code never has to change when you switch providers:

Your code          →  Gateway resolves  →  Provider
model: "smart"        gpt-4o              OpenAI
model: "fast"         gpt-4o-mini         OpenAI
model: "coding"       claude-sonnet-4-6   Anthropic

Aliases can also be set per-project, overriding the org-level aliases for that project's requests.

Routing history

Every routed request creates a routing decision record. You can list decisions and inspect why a particular provider was chosen:

const { decisions } = await client.routing.listHistory({ limit: 10 });

const detail = await client.routing.getDecision(decisions[0].id);
// detail.selectedProvider, detail.candidates, detail.strategy, detail.latencyMs

See also

• Intelligent routing: how the routing decision is made
• Routing performance: how the router was evaluated, and what it delivers