Metrum AI Router
Open-source, self-hosted LLM gateway and smart router that selects an upstream model per request from a deployment-owned policy and records why.
At a Glance
Self-hosted core router under Apache-2.0; no license key required.
Engagement
Available On
Alternatives
Listed Nov 2026
About Metrum AI Router
Metrum AI Router is an open-source, self-hosted LLM gateway from Metrum AI. It selects a different upstream model for each request based on a deployment-owned policy and records the reason for each decision, while keeping provider keys, budgets, telemetry and request-time cost accounting under platform control. The source is released under Apache-2.0, and the documentation lists v4.7.1 as the latest docs version.
What It Is
The router sits between GenAI clients and upstream models. Clients call one governed endpoint using OpenAI Chat, OpenAI Responses or Anthropic Messages shapes, including SDKs, Codex CLI and Claude Code. Platform teams define model groups, which are quality and cost contracts, instead of exposing raw provider IDs. Routing picks the cheapest eligible candidate that has evidence of meeting the contract.
Upstreams can be hosted providers or enterprise-owned vLLM, SGLang and other OpenAI-compatible services, mixed behind one API.
How Routing Works
Before a strategy runs, request-shape eligibility filters targets by tool support, images, dialect, structured outputs and max-token requirements. If no target remains, the router returns a no-eligible-target error without calling an upstream. Group strategies include static, weighted, failover, dynamic_score and external policy services.
Optional contracts can require tags, minimum evaluation quality and pass rate, and a maximum evaluation age, so stale validation removes a target from eligibility. Token budgets and quotas are admitted before any cache-miss upstream call.
Learned Routing Policy
An optional Learned Routing Policy trains offline on operator outcome data, using LightGBM quality and output-token models. It chooses only among router-eligible targets and abstains when uncertainty is high. It is promoted through explicit shadow and enforce modes, and rollback is a single config change. The README's published results come from a synthetic holdout and are documented as not authorizing live promotion.
Operations and Evidence
Usage rows record provider and model, tokens, cache, latency, attempts, fallbacks and request-time cost. An admin reports dashboard and a request-evidence endpoint join this data by request ID. Prompts and responses are not retained by default, and content capture is opt-in, redacted and encrypted. Documented runtimes are a Linux binary, Docker Compose and Kubernetes.
Tradeoffs to Know
The project is aimed at teams that run the gateway themselves rather than a hosted service. Its documentation states that it does not guarantee provider uptime, model quality or compliance outcomes. SQLite state is not meant for horizontally scaled replicas, and the in-process response cache is cleared on restart.
Community Discussions
Be the first to start a conversation about Metrum AI Router
Share your experience with Metrum AI Router, ask questions, or help others learn from your insights.
Pricing
Open Source (Apache-2.0)
Self-hosted core router under Apache-2.0; no license key required.
- Apache-2.0 licensed core
- No license key required
- Self-hosted: Linux binary, Compose, Kubernetes
- On-premises or air-gapped operation
- Bring your own provider keys (external provider costs apply)
Capabilities
Key Features
- Per-request upstream model selection from deployment-owned policy
- Policy-owned model groups with quality and cost contracts
- Request-shape eligibility filtering (tools, images, dialect, max tokens)
- Pre-call token budget admission and quotas
- Learned Routing Policy with abstention and shadow/enforce modes
- Dynamic score, weighted, failover, static and external routing strategies
- Request-time cost capture and usage reports
- Admin reports dashboard and request evidence endpoint
- PII filtering with redaction
- Response cache
- Runtime model-group swap via Admin API and routerctl
- Mixed private and hosted upstreams with fallback
- Claude Code and Codex CLI support
