Skip to main content

Providers (BYOK)

Bring your own key: connect your own provider credentials and they're used for real inference. Auto-discovered models are merged into your catalog immediately, and your own key always takes priority over any provider your admin has enabled platform-wide, for that same provider. This is available on every plan.

Connecting a provider​

client.providers.add(provider="openai", api_key="sk-...")

Models are discovered automatically by calling the provider's own /models endpoint with your key, so you don't need to list them yourself. If discovery fails (e.g. the key is invalid or the endpoint is unreachable), the provider is still saved but starts with no models; call client.providers.test() again once the key is fixed to retry discovery.

Self-hosted / OpenAI-compatible providers

The underlying API supports a base_url field for self-hosted or custom-endpoint providers, but the Python SDK's client.providers.add() doesn't expose it yet. Until it does, add these via a direct request: POST /v1/providers with {provider, base_url, api_key?}.

Managing your providers​

result = client.providers.list()
# result["configs"]: each entry has id, provider, enabled, enabled_models, created_at, updated_at

client.providers.enable("openai")
client.providers.disable("openai")
client.providers.delete("openai")

test = client.providers.test("openai")
# {"healthy": bool, "latency_ms": int, "error": str | None}

Disabling a provider (rather than deleting it) keeps the stored key but removes it from routing consideration, useful for temporarily pulling a provider out of rotation without losing the discovered model list.

Precedence: your key vs. your admin's​

If your organization's admin has also enabled a given provider platform-wide, and you connect your own key for that same provider, your key wins. Every request that would route to that provider uses your credentials, not the platform's. Every other provider you haven't connected yourself keeps using whatever your admin has enabled. There's no separate setting to toggle this; it's automatic based on which providers you've personally connected.

Seeing what's actually available​

client.models.list() is not this list yet

client.models.list() currently returns a static reference list of commonly-supported model IDs. It does not reflect your org's actual connected providers or BYOK keys. The dashboard's Model Catalog tab is the accurate, real-time view (it merges your own connected providers with whatever your admin has enabled platform-wide); the SDKs don't have a method for it yet. If you need this programmatically today, call GET /v1/models/catalog directly.

Until the SDKs catch up, use the dashboard's Model Catalog tab, right next to the Providers tab where you connect your own keys, to see what's actually routable for your org.