# TypeLLM

> An open-source Python library that extends autoregressive LLMs with type-safe generation, producing guaranteed typed outputs (string, integer, number, boolean, enum) via JSON Schema without modifying model weights.

TypeLLM is an open-source Python library, licensed under Apache 2.0, that brings type-safe structured output to existing autoregressive LLMs without changing their architecture or weights. It is built on top of SGLang and is currently in early access, with the latest release being v0.1.5 published on September 24, 2026.

## What It Is

TypeLLM solves a core reliability problem in LLM-powered software: models that generate free text can hallucinate values outside the expected schema. TypeLLM constrains the decoding process so that outputs are guaranteed to conform to a JSON Schema definition — returning native Python types (`str`, `int`, `float`, `bool`) that software can use directly. It is inspired by TypeSafe AI's Jev system and targets developers who self-host open autoregressive models via SGLang.

## How It Works

Developers define a `questions` dict using JSON Schema field types and optional `instructions`. The `TypeLLMClient` points at a running SGLang HTTP server and calls `client.generate()`. TypeLLM handles the constrained decoding internally:

- **Enum and boolean fields** consume a single output token each, keeping cost negligible.
- **Numeric fields** decode in lockstep, one batched request per digit, bounded to 32 digits by default.
- **String fields** without an enum generate free text.
- **Thinking mode** can be enabled per-client with an optional per-field token budget, allowing reasoning before the constrained answer.
- **Image input** is supported for vision-language models, accepting local paths, URLs, data URIs, raw bytes, or PIL images.

## Dependency-Aware Execution Model

TypeLLM supports three execution modes — batch, sequential, and DAG — selectable per request. The `depends_on` key on any field activates DAG mode, letting developers declare which earlier results a field needs. TypeLLM then:

- Runs independent fields in parallel within each layer.
- Extends parent prompts along dependency paths for KV cache reuse.
- Passes resolved dependency values as context to downstream fields.

A benchmark on Qwen3.8-27B with 16 Boolean fields showed batch execution at 1.61 s end-to-end versus 9.35 s sequential — a 5.8× throughput improvement — with 1,088 cached tokens reused per branch.

## Benchmark Results

TypeLLM was evaluated on 231 public JevBench tasks. The README reports TypeLLM + Qwen3.8-27B scored 84.42% accuracy without thinking and 98.70% with thinking enabled, compared to non-type-safe models ranging from 85% to 100% on the same benchmark. Full per-task results are published in the repository.

## Update: TypeLLM v0.1.5

The project was created on September 17, 2026, and has moved quickly through several updates:

- **v0.1.5** (September 24, 2026) — latest release; added image input for vision-language models tested with Qwen3.8-27B.
- **September 23** — added JevBench evaluation results and permutation averaging to reduce option-order bias on enum questions.
- **September 22** — added `depends_on` dependency graphs with incremental prefix reuse.
- **September 19** — added optional thinking mode with per-field budget.
- **September 18** — added constrained `integer` and `number` output types.

The project is available on GitHub under the Apache License 2.0 and is accepting early access requests via the TypeLLM website.

## Features
- Type-safe generation with no out-of-schema hallucinations
- Supports string, integer, number, boolean, and enum output types
- JSON Schema-based output definition
- Batch, sequential, and DAG execution modes
- Dependency-aware execution with depends_on field graphs
- KV prefix caching and shared-prefix reuse via SGLang
- Optional thinking mode with per-field token budget
- Image input for vision-language models
- Permutation averaging to reduce option-order bias on enum fields
- Return probabilities for enum and boolean fields
- Negligible output-token cost for enum and boolean fields
- Compatible with open autoregressive models served via SGLang

## Integrations
SGLang, Qwen3.8-27B, Qwen3.5 family, MiniCPM5, Ling-mini-2.0, Ring-mini-2.0, Hugging Face model hub, PIL (image input)

## Platforms
API, CLI, DEVELOPER_SDK

## Pricing
Open Source

## Version
v0.1.5

## Links
- Website: https://typellm.ai
- Documentation: https://typellm.ai/docs
- Repository: https://github.com/TypeLLM/TypeLLM
- EveryDev.ai: https://www.everydev.ai/tools/typellm