LLM Gateway

LLM Gateway is an open-source API gateway for large language models, built by theopenco and licensed under AGPLv3 for its core layer. It lets developers route requests to 40+ providers—including OpenAI, Anthropic, Google, AWS Bedrock, Azure, DeepSeek, and more—through a single unified API endpoint that is drop-in compatible with the OpenAI SDK. The project has accumulated over 1,400 GitHub stars since its April 2025 launch and is SOC 2 Type II certified.

What It Is

LLM Gateway sits as a middleware layer between your application and any LLM provider. Instead of managing separate API keys, dashboards, and SDKs for each provider, you point your existing OpenAI SDK at LLM Gateway's base URL and it handles routing, failover, cost tracking, and analytics automatically. The gateway supports 200+ models across text generation, embeddings, vision, reasoning, image generation, video generation, and more.

How the Routing Layer Works

The core value proposition is intelligent, automatic routing with built-in reliability:

• Automatic failover: When a provider goes down, traffic is rerouted to a healthy alternative in real time. The site claims that while individual providers average ~94% uptime (~22 days of downtime per year), combining multiple providers through automatic failover yields effective uptime of 99.9999%.
• Bring Your Own Keys (BYOK): Users can supply their own provider API keys and pay providers directly, with LLM Gateway providing routing, tracking, and analytics at no platform cost.
• Per-project routing overrides: Enterprise users can pin production workloads to specific regions or providers, set fallback chains, and apply cost ceilings per project.
• Prompt caching: Supported to reduce latency and cost on repeated prompts.

Open-Source Deployment Model

The core gateway is AGPLv3-licensed and self-hostable via Docker or Terraform modules. The repository's folder structure includes a Next.js dashboard (apps/ui), a playground (apps/playground), a Hono backend (apps/api), and the gateway routing layer (apps/gateway). Enterprise features—advanced billing, extended data retention, SSO, white-labeling, and team management—live in the ee/ directory under a separate commercial license. Self-hosting is free forever; the cloud-hosted version charges a 5% flat fee on credit top-ups, with no markup on token prices.

Observability and Cost Analytics

Every request through LLM Gateway is tracked with cost, latency, and token usage. The dashboard provides:

• Requests, tokens, total spend, and average cost per 1K tokens over 7 or 30 days
• Per-model and per-provider breakdowns to identify expensive outliers
• Error rate, cache hit rate, and reliability trend monitoring
• Project-level usage explorer with dedicated charts and tables
• Organization-wide analytics for enterprise deployments, rolling up spend across teams

Enterprise Capabilities

The enterprise tier adds a set of compliance and multi-team controls built for production deployments:

• SSO/SAML: SAML 2.0 and OIDC with SCIM provisioning and group-based role mapping
• Audit logs: Tamper-evident, SIEM-ready trails for every config change, key rotation, and admin action
• Guardrails: Server-side detection for prompt injection, PII, secrets, and policy violations, enforced at the gateway layer
• Provider compliance policies: Route only to providers meeting specified certifications (SOC 2, ISO 27001, GDPR, no prompt training)
• White-label chat playground: Deployable under a custom domain with custom branding
• Terraform modules: One-command provisioning on AWS, GCP, or Azure

Update: v1.8.0

The latest release is v1.8.0, published July 13, 2026, with the repository last updated July 15, 2026. The project has been actively developed since its April 2025 creation, with 25+ contributors and 157 forks on GitHub. Recent additions highlighted on the site include DevPass (fixed-price monthly plans for coding tools like Claude Code and Cursor), an MCP server, agent skills registry, and an AI SDK provider integration. The SOC 2 Type II certification was announced as a recent milestone.