ngrok AI Gateway

ngrok AI Gateway is a hosted routing layer that unifies every AI model you use—public providers like OpenAI and Anthropic, custom endpoints, and locally self-hosted LLMs—behind a single URL. Built by the team behind the widely used ngrok tunneling platform, it is designed to let developers swap a single baseURL and API key to start routing without deploying additional infrastructure.

What It Is

ngrok AI Gateway sits between your application and any number of AI model providers, acting as a centralized control plane for LLM traffic. It is accessed via https://gateway.ngrok.ai and is compatible with the OpenAI SDK, Anthropic SDK, and Vercel AI SDK out of the box. The gateway handles routing, retries, failover, observability, and access control so application code does not have to.

How Routing Works

The gateway accepts a standard API call and forwards it to whichever provider or model you have configured. Key routing behaviors include:

• Automatic failover: When a model or key slows or fails, requests reroute to a healthy alternative without user-visible errors.
• Automatic retries: Failed requests are retried instantly without requiring error-handling logic in the application.
• Local LLM support: Self-hosted models reachable from the gateway (e.g., via Ollama) can be routed to over private connectivity, with no public IP or inbound port required.
• Fully programmable: The gateway is configurable entirely through APIs, making it compatible with coding agents, Terraform, CLI tooling, or custom automation.

Key Management and BYOK

ngrok AI Gateway supports a bring-your-own-keys (BYOK) model. Developers drop their existing OpenAI, Anthropic, or custom provider keys into the gateway, and the gateway routes through them. Billing for inference goes directly to the provider at the developer's existing rates. Alternatively, ngrok keys can be used and inference is passed through at cost. All keys are managed in one place regardless of provider.

Access Control and Observability

Each application or developer can be issued a separate scoped access key that restricts which providers and models it is permitted to call. This eliminates the need to share a single key that grants broad access. On the observability side, the gateway aggregates token usage, latency, and error data across every routed call—broken down by app, developer, and model—filling a gap that provider dashboards alone cannot address.

Pricing Model

The gateway charges a flat per-token routing fee on top of inference costs. Credits are purchased up front and drawn down as requests are routed, with no subscription or commitment required. When BYOK is used, inference is billed directly by the provider and only the routing fee applies.

Current Status

ngrok AI Gateway is live and accepting signups at app.ngrok.ai. It was featured on Product Hunt (post ID 1204933) and is positioned as a standalone product alongside ngrok's existing universal gateway and localhost-sharing offerings.