Otari

Otari is an open-source LLM gateway built by Mozilla.ai that routes requests from your applications to 40+ AI providers through a single OpenAI-compatible endpoint. It handles authentication, budget enforcement, provider credential management, and usage logging — either self-hosted on your own infrastructure or connected to the hosted otari.ai platform. The project is licensed under Apache 2.0 and its latest release is v0.2.0, published in July 2026.

What It Is

Otari is a proxy server that sits between your applications and LLM providers like OpenAI, Anthropic, Mistral, Gemini, and local models. Instead of managing separate SDKs, API keys, and dashboards for each provider, your apps call one OpenAI-compatible endpoint at port 8000, and Otari handles routing, authentication, budget checks, and usage logging before forwarding the request. The core routing layer is powered by any-llm, a Mozilla.ai Python SDK that abstracts 40+ providers. Otari can run in two modes: standalone (fully local, manages its own database and credentials) or hybrid/connected (linked to otari.ai, which handles provider routing, auth, and usage tracking in the cloud).

Deployment Model: Self-Hosted or Hosted

Otari is designed to be run in your own infrastructure, keeping provider keys and usage data in your environment. The quickstart gets a gateway running with a single Docker command — no config file or database setup required for standalone mode. For production, a Docker Compose stack adds a Postgres backend, built-in tool services, and guardrail services. One-click Railway deployment is also supported via a published template. The hosted otari.ai platform is available for teams that want provider routing, auth, and usage reporting managed for them without running the gateway themselves.

Key Capabilities

• Single OpenAI-compatible endpoint in front of 40+ providers via the any-llm routing library
• Virtual keys: clients receive scoped, revocable gw- keys; real provider credentials never leave Otari
• Pre-request budget enforcement: per-user and per-key spending limits are checked before a request runs, not reconciled after billing
• Usage tracking: all requests are logged and queryable via /v1/usage
• Built-in tools: opt-in otari_code_execution (sandboxed Python REPL) and otari_web_search tools that give any model, including open-weight ones, parity with frontier API managed tools
• Guardrails: request-level checks (e.g., prompt-injection detection) that run before the provider is called, with monitor or block modes
• Automatic failover: keeps requests flowing through provider outages in connected mode
• Multi-tenant workspaces: workspace-level routing rules, budgets, and access control

SDK and Integration Surface

Otari ships client SDKs for Python (pip install otari), TypeScript, Rust, and Go, plus a CLI (pip install otari-cli). Because the gateway is OpenAI-compatible, any existing OpenAI client works by pointing base_url at the Otari endpoint — no code rewrites required. The docs include integration guides for OpenCode and Claude Code. A Postman collection and a built-in Swagger UI at /docs are available for exploring the API without a separate client.

Update: v0.2.0

The latest release, v0.2.0, was published on July 3, 2026. The repository was created in April 2026 and has been actively developed, with the last push in July 2026. GitHub topics tag it as a litellm-alternative, signaling its positioning in the LLM gateway/proxy space. The project is part of the broader Mozilla.ai open-source ecosystem, which also includes any-llm and the Otari SDK repositories.