TypeLLM

TypeLLM is an open-source Python library, licensed under Apache 2.0, that brings type-safe structured output to existing autoregressive LLMs without changing their architecture or weights. It is built on top of SGLang and is currently in early access, with the latest release being v0.1.5 published on September 24, 2026.

What It Is

TypeLLM solves a core reliability problem in LLM-powered software: models that generate free text can hallucinate values outside the expected schema. TypeLLM constrains the decoding process so that outputs are guaranteed to conform to a JSON Schema definition — returning native Python types (str, int, float, bool) that software can use directly. It is inspired by TypeSafe AI's Jev system and targets developers who self-host open autoregressive models via SGLang.

How It Works

Developers define a questions dict using JSON Schema field types and optional instructions. The TypeLLMClient points at a running SGLang HTTP server and calls client.generate(). TypeLLM handles the constrained decoding internally:

• Enum and boolean fields consume a single output token each, keeping cost negligible.
• Numeric fields decode in lockstep, one batched request per digit, bounded to 32 digits by default.
• String fields without an enum generate free text.
• Thinking mode can be enabled per-client with an optional per-field token budget, allowing reasoning before the constrained answer.
• Image input is supported for vision-language models, accepting local paths, URLs, data URIs, raw bytes, or PIL images.

Dependency-Aware Execution Model

TypeLLM supports three execution modes — batch, sequential, and DAG — selectable per request. The depends_on key on any field activates DAG mode, letting developers declare which earlier results a field needs. TypeLLM then:

• Runs independent fields in parallel within each layer.
• Extends parent prompts along dependency paths for KV cache reuse.
• Passes resolved dependency values as context to downstream fields.

A benchmark on Qwen3.8-27B with 16 Boolean fields showed batch execution at 1.61 s end-to-end versus 9.35 s sequential — a 5.8× throughput improvement — with 1,088 cached tokens reused per branch.

Benchmark Results

TypeLLM was evaluated on 231 public JevBench tasks. The README reports TypeLLM + Qwen3.8-27B scored 84.42% accuracy without thinking and 98.70% with thinking enabled, compared to non-type-safe models ranging from 85% to 100% on the same benchmark. Full per-task results are published in the repository.

Update: TypeLLM v0.1.5

The project was created on September 17, 2026, and has moved quickly through several updates:

• v0.1.5 (September 24, 2026) — latest release; added image input for vision-language models tested with Qwen3.8-27B.
• September 23 — added JevBench evaluation results and permutation averaging to reduce option-order bias on enum questions.
• September 22 — added depends_on dependency graphs with incremental prefix reuse.
• September 19 — added optional thinking mode with per-field budget.
• September 18 — added constrained integer and number output types.

The project is available on GitHub under the Apache License 2.0 and is accepting early access requests via the TypeLLM website.