Tokenless

Tokenless is an AI inference router built by researchers from Google DeepMind, Princeton, and UC Berkeley, backed by Y Combinator. It acts as a drop-in replacement for existing OpenAI and Anthropic API calls, automatically routing each request to the most cost-efficient model that can handle it well.

What It Is

Tokenless sits between your application and the LLM providers, functioning as an intelligent proxy that reduces inference spend. Rather than sending every request to an expensive frontier model, it fans out each request to a group of candidate models simultaneously and monitors their outputs in real time. Once a model is clearly on track to produce a high-quality response, Tokenless selects it, cancels the other in-flight requests, and charges only for the tokens actually used. The result, according to the vendor, is roughly half the inference bill with equivalent output quality.

How the Routing Works

The core mechanism is speculative multi-model execution with early cancellation:

• A request is sent in parallel to multiple models
• Tokenless watches each model's output as it streams, tracking per-model confidence
• The first model that demonstrates it is "clearly on track" is selected
• All other model calls are cancelled immediately
• The user is billed only for the selected model's tokens

This approach differs from static routing rules or simple model tiering — it makes a dynamic, per-request decision based on observed generation quality.

Benchmark Evidence

The homepage presents benchmark data comparing Tokenless routing modes against individual frontier models on public agentic benchmarks (τ³-Banking, Terminal-Bench 2.1, DeepSWE 1.1). According to vendor-published results, Tokenless Pro achieves a 40.2% solve rate at $0.57 average cost per task, outperforming GPT-5.6 Sol (33.0% at $1.50/task) and Claude Opus 5 (32.8% at $1.64/task) on those benchmarks. The vendor presents these as measured against the best published run of each frontier model.

Integration and Setup

Tokenless exposes OpenAI- and Anthropic-compatible endpoints, meaning integration requires only a two-line change to point existing API calls at the Tokenless endpoint. No SDK installation or model-specific code changes are needed. The vendor offers a savings calculator on the homepage that estimates monthly bill reduction based on current LLM spend and model mix.

Audience and Use Case

The primary audience is teams and companies with meaningful monthly LLM API spend — the homepage savings calculator defaults to $40K/month as an example input. The tool is particularly relevant for agentic workloads where many requests are routine and do not require the most capable (and expensive) frontier model. The vendor positions it as suitable for any team already using OpenAI or Anthropic APIs who wants to reduce costs without re-engineering their stack.