LM Format Enforcer

LM Format Enforcer is an open-source Python library by Noam Gat that constrains a language model's output to a required format such as a JSON Schema or a regular expression. It filters which tokens the model may generate at every timestep, so the final text always conforms to the specified format. The latest listed release is v0.11.2, published in August 2025.

What It Is

LM Format Enforcer is a structured-output tool for developers running language models locally or through Python inference stacks. Instead of relying only on prompt engineering, it combines a character-level parser with a prefix tree of the model's tokenizer vocabulary to determine which tokens can legally come next. When the model picks a token, the parser advances and the next set of allowed tokens is computed.

Supported formats include JSON Schema (with required and optional fields, nested objects, arrays, dictionaries, and recursive classes), schemaless JSON mode, and regular expressions. Regex support uses the interegular library, so it does not cover the full regex standard.

How the Enforcement Works

The README describes parser implementations such as JsonSchemaParser, RegexParser, and StringParser, all built on a CharacterLevelParser interface. Rather than forcing an exact output layout, the library allows any token sequence that parses into the target format. The model therefore keeps control over whitespace and field ordering in JSON, which the author argues avoids pushing the model into unnatural output styles. A strict field-order option is available through configuration.

Batched generation and beam search are supported, with different tokens filtered for each input or beam.

Integrations and Usage

The library works with any Python language model and tokenizer, and the README lists integrations with transformers, LangChain, LlamaIndex, llama.cpp, vLLM, Haystack, NVIDIA TensorRT-LLM, and ExLlamaV2. In vLLM's OpenAI-compatible server it can be selected as the guided decoding backend, either server-wide or per request with guided_regex, guided_json, or guided_choice parameters. Installation is via pip.

Diagnostics and Limitations

The format is guaranteed, but semantic correctness is not, and heavy enforcement may increase hallucinations. When using transformers, passing output_scores and return_dict_in_generate yields a token-by-token table comparing the chosen token with the model's preferred one, helping tune prompts. Behavior can be adjusted via environment variables (whitespace limits, array length, alphabet, field order) or a CharacterLevelParserConfig object. Because it needs access to output logits through a Python API, it cannot be used with API-only services such as OpenAI ChatGPT.