Free & open-source · Self-host or Cloud

Stop hard-coding AI models.

AICP classifies every request, routes it to the right model, and enforces your org's policies in under 10ms.

without AICP

// Before: fragile, expensive, ungoverned
import OpenAI from 'openai'
import Anthropic from '@anthropic-ai/sdk'

// Hard-coded. No failover. No cost visibility.
// Every team picks differently. No shared policy.
const res = await openai.chat.completions.create({
  model: 'gpt-4o',   // always frontier, always $$$
  messages: [...]
})

with AICP

// After: one endpoint, right model, full control
import { AICPClient } from '@aicp/sdk'

const ai = new AICPClient({ apiKey: process.env.AICP_API_KEY })

const res = await ai.chat.complete({
  model: 'auto',  // ML classifier picks: Mistral Medium, −87%
  messages: [...] // policies enforced · decision logged · <10ms
})

// ✓ ML complexity tier       ✓ EU data residency applied
// ✓ p50/p95 latency tracked  ✓ full trace in Request Explorer

ML routing that earns its overhead.

The routing pipeline runs in under 10ms and typically saves 40-87% on inference cost by routing simple requests to smaller, cheaper models, without sacrificing the quality your application needs.

Routing strategies

POST org_routing_configs · payloadcheapest

{
  "strategy": "cheapest",
  "params": {
    "default_prompt_tokens":  "2000",
    "default_output_tokens":  "512"
  }
}

Under the hood

ML complexity classifier

Every request is embedded and scored by a purpose-built ML router to predict small, medium, or large complexity tier. Falls back to a token-length heuristic when embeddings are unavailable.

Observed-latency feedback loop

Real p50 and p95 latency per provider/model is tracked and refreshed hourly, overriding static catalog estimates. Routes naturally shift away from degraded providers without any manual intervention.

Explainable decisions in Request Explorer

Every routing decision is logged with its full reasoning: tier predicted, models scored, policy applied, winner selected. Fully queryable and replayable from the dashboard.

Context compaction

When a prompt exceeds the largest available model's context window, AICP automatically compacts it before routing. Available on Business plans with the VS Code Copilot client.

Capability filtering runs before every strategy

Before any strategy executes, AICP filters the candidate pool to models that support the request's detected requirements: streaming, vision, function-calling, JSON mode, long-context, audio, or reasoning. No config knob.

Eight native adapters. Every OpenAI-compatible endpoint.

Provider adapters are maintained in the AICP core, not wrappers around the provider's SDK. That means consistent retry semantics, streaming behaviour, and capability detection across every provider you connect.

OpenAI

GPT-4o, o1, o3-mini, …

Anthropic

Claude 3.5 Sonnet, Haiku, …

Google Gemini

Gemini 1.5 Pro/Flash, …

Mistral

Medium, Small, Codestral, …

Azure OpenAI

GPT-4o via Azure endpoint

Azure AI Foundry

Phi-3, Llama, Cohere via Hub

AWS Bedrock

Titan, Llama, Claude on AWS

AWS SageMaker

Custom & fine-tuned endpoints

Any OpenAI-compatible endpoint: Nebius, Together, Groq, Fireworks, Ollama, and more

If it speaks the OpenAI chat completions API, AICP can route to it. No custom adapter required. Point it at your base URL and it's available in the routing pool immediately.

base_url: https://…

BYOK, AES-256-GCM encrypted. Keys never leave the control plane in plaintext.

No code? Use the n8n connector instead.

Set policy once. Enforced on every request, across every team.

Engineering leadership shouldn't have to audit each team's model choices manually. Define data residency rules, spend limits, and model allowlists at the org level, and AICP enforces them automatically.

PII detection and redaction (Presidio)

Cloud / Enterprise

Outbound prompts are scanned for personally identifiable information using Microsoft Presidio. Configurable fail-open or fail-closed. Cloud and Enterprise plans only.

Data residency rules

Bind projects or orgs to specific geographic regions. Requests that cannot be satisfied by a compliant model are rejected, never silently routed to a non-compliant provider.

Model allow/blocklists and org-level policies

Define which models each team or project can access. Set org-wide defaults. Override at the project level. Policies are evaluated on every request, not at deploy time.

Budget guards per project

Set monthly spend caps per project. Requests that would breach the cap are rejected with a structured error before a provider is called. No surprise bills.

GDPR-minded by design

PII redaction, data residency, no training on your data

EU AI Act-aware

Architecture supports per-jurisdiction model filtering

ISO 27001

Certification in progress

Every request logged. Every decision replayable.

A ClickHouse-backed logging pipeline captures every prompt, completion, routing decision, latency, and cost across four dashboard views.

Live request volume, cost by model, provider health, and active alerts at a glance.

Requests (30d)

284,425

Total cost

$1,134.20

Avg p50 latency

1.4s

Avg quality score

0.91

eval-service

Response quality scoring and cross-model benchmarking. Run regression comparisons when you add or rotate a model, and catch quality drops before they reach production.

logging-service

Durable ClickHouse request/response trail. Every prompt, completion, latency, cost, and routing decision is stored and queryable. Retention is configurable per plan.

notification-service

Email alerts and outbound webhooks for budget breaches, API key expiry, and provider health incidents. Integrate into your existing on-call tooling.

Docker to EKS. Your infra, your rules.

Run AICP wherever your data lives. Docker Compose for local and small-scale deploys, Helm for production Kubernetes, and a full Terraform module for AWS EKS with VPC, RDS, and ElastiCache pre-wired.

Community

Docker Compose

Up in a few minutes. Free forever.

Dashboard at :3001 · Gateway at :3000

# Clone and start
git clone https://github.com/aicp/community.git
cd community
cp .env.community.example .env
docker compose -f docker-compose.community.yml up -d
./scripts/migrate-community.sh

Production

Helm chart

Production-hardened Kubernetes deploy.

HA config · persistent ClickHouse · configurable replicas

helm repo add aicp https://charts.aicp.ai
helm install aicp aicp/aicp \
  --set license.key=$LICENSE_KEY \
  --set db.clickhouse.url=$CH_URL \
  -n aicp --create-namespace

Enterprise

Terraform + AWS EKS

Full VPC + EKS cluster, provisioned in one apply.

EKS · VPC · RDS · ElastiCache · all wired

module "aicp" {
  source  = "aicp/aicp/aws"
  version = "~> 1.0"

  license_key = var.aicp_license
  region      = "eu-west-1"
  vpc_cidr    = "10.0.0.0/16"
}

Self-host free. Pay only for what you route.

The Community tier is fully open-source with no feature time-bombs. Cloud adds managed infrastructure and compliance features. Enterprise adds audit, licensing, and dedicated SLAs.

Community

Free

Open source · Self-hosted

Self-host now

Full ML routing engine
All 8 native provider adapters + OpenAI-compat
BYOK with AES-256-GCM key storage
Owner / member role management
Docker Compose deploy

Hard-coding AI models is the new technical debt.

Stop writing routing logic. Stop overpaying for inference. Stop guessing which model fits the task. AICP handles all of it, automatically, on every request.

Get started free Book a Demo Read the Docs