One API for all your AI providers
Integrate once with the platform contract. Switch providers centrally. No code changes in your products.
Why AI Gateway
Provider Abstraction
Your products talk to the platform, not to OpenAI or Anthropic directly. Switch providers in config, not in code.
Tier-Based Routing
Request small/medium/large — the platform maps tiers to concrete models. Upgrade models without touching product code.
Fallback & Resilience
Automatic provider failover. If one provider goes down, traffic reroutes transparently to the next.
Cost & Usage Control
Per-tenant rate limits, budget enforcement, and token tracking. See exactly who spends what.
Self-Hosted & OSS
Runs on your infrastructure. No data leaves your network. Full source code access and auditability.
OpenAI-Compatible API
Standard chat completions endpoint. Drop-in replacement for any OpenAI SDK integration.
How it works
Products call the Gateway
POST /llm/v1/chat/completions with tier-based model field (small, medium, large)
→Platform routes to provider
LLMRouter resolves tier to configured provider and model. Applies rate limits and policies.
→Response flows back
Standard OpenAI format response. Analytics tracked automatically. Products never see the provider.
curl -X POST http://gateway:4000/llm/v1/chat/completions \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "medium",
"messages": [
{"role": "user", "content": "Explain this code"}
]
}'Stop integrating AI providers in every product
One platform contract. Centralized switching. Full control.