Guidance

Guidance is an open-source Python library from the guidance-ai organization for controlling how language models generate output. It lets developers constrain generation with regular expressions, selection lists, JSON schemas, and context-free grammars, while mixing in conditionals, loops, and tool use. The latest release listed is 0.3.2, published 2026-03-18.

What It Is

Guidance is a programming interface for steering language models. Users build prompts with Python idioms such as system(), user(), and assistant() context blocks and append generation calls like gen() and select() to an immutable model object. It supports backends including Transformers, llama.cpp, and OpenAI. The project states that it can reduce latency and cost compared with conventional prompting or fine-tuning.

Constrained Generation Workflow

A gen() call can be limited by a regex, and select() picks from a known list of options. Developers can also write their own functions with the @guidance decorator and compose stateless functions into full context-free grammars, for example to produce a simple HTML page. JSON generation is supported directly, including from Pydantic models, and results can be captured by name and parsed with Python's JSON library or Pydantic.

Efficiency and Debugging

When a grammar makes some tokens predictable, Guidance can insert them without a model forward pass. This is called token fast-forwarding, and it reduces GPU usage. Grammars can be tested offline with the Mock model and by matching strings against a grammar, without calling a model API. In Jupyter notebooks, a widget shows generation output with highlighting.

Setup Path

Install from PyPI with pip install guidance, along with the backend needed for your model.