Ax LLM

Ax is an open-source LLM programming framework built around typed signatures — a one-line contract that declares what data goes in and what typed data comes out. Instead of writing raw prompts, developers write signatures like "review:string -> sentiment:class \"positive, negative, neutral\"" and Ax handles prompt rendering, provider calls, output parsing, validation, retries, streaming, and tracing. The TypeScript package (@ax-llm/ax) is the reference implementation, and the same programming model is compiled via the AxIR intermediate representation into verified native packages for Python, Java, C++, Go, and Rust.

What It Is

Ax is described by its authors as "DSPy for TypeScript" — a framework that brings the declarative, self-improving pipeline ideas from the DSPy research lineage into a multi-language, production-ready library. The core abstraction is the signature: a typed input/output contract that Ax lowers into a full pipeline including prompt rendering, streaming parsers, validators, retry feedback loops, trace metadata, and optimization hooks. The same contract scales from a single typed generation call to long-running RLM-grade agents operating on large datasets without stuffing bulk data into the prompt.

The AxIR Compiler Architecture

Rather than porting TypeScript code six times, Ax uses a portable intermediate representation (AxIR). The TypeScript runtime is the source of truth; the AxIR compiler emits native package surfaces for each target language — native names, native error types, native builders, and native runtime profiles — while sharing the same Ax semantics. A conformance gate (axir verify) checks generated package examples, API metadata, capability manifests, and conformance fixtures to ensure every backend earns its place in the language matrix. Packages are published on npm, PyPI, Maven Central, crates.io, and Go modules, and are installable via CMake FetchContent for C++.

Agent Harness and Runtime Model

Ax ships a three-stage agent pipeline — distiller → executor → responder — built around the principle that the model computes on data rather than reading it. Bulky inputs live in a runtime session; the agent writes small code steps against them; only compact evidence enters the prompt. Key agent capabilities include:

• Context maps: a persistent orientation cache for recurring long-context work, with configurable evolve steps and snapshot persistence
• Memories: vector/BM25/KV lookup via recall(...), with results injected into the next turn
• Skills: on-demand guidance bodies loaded via consult(...), persisted across .forward() calls
• Sandboxed JS runtime (AxJSRuntime): hardened by default, portable across Node, Bun, Deno, and browser, with opt-in permissions for network, filesystem, storage, and child process access
• Function discovery: large tool catalogs, MCP servers, child agents, and runtime state are discovered and loaded as needed rather than stuffed into the base prompt

Optimization and Research Lineage

Ax incorporates ideas from several research papers, including DSPy (Stanford), DSPy Assertions, GEPA (Genetic-Pareto prompt evolution from Stanford/Berkeley), ACE (Agentic Context Engineering from Stanford), RLM (Recursive Language Models from MIT/Stanford), and PEEK (context maps from MIT/Stanford). The AxGEPA optimizer tunes prompts, demos, flows, and agents against evals, returning a Pareto front that makes quality, latency, cost, and brevity tradeoffs explicit. AxFlow provides typed, chainable workflow graphs with branching, loops, parallel execution, and mermaid diagram round-trip support.

Provider Coverage and Operational Features

Ax supports 15+ LLM providers through a unified ai() abstraction: OpenAI (including Responses API and realtime), Anthropic/Claude, Google Gemini (including Live API), Mistral, Cohere, Reka, DeepSeek, Grok/xAI, Azure OpenAI, and any OpenAI-compatible endpoint. Audio is a first-class citizen — batch speech-to-text (ai.transcribe), batch text-to-speech (ai.speak), typed speech:audio signature outputs, and conversational/realtime audio turns are all supported. Operational features built in include OpenTelemetry distributed tracing (40+ metrics), cost tracking per request, streaming with field-boundary parsers, rate limiting, provider routing, and enterprise-grade error handling. The repo ships 1,000+ tests and 220+ runnable examples across all six languages.

Update: Release 23.0.9

The latest published release is 23.0.9, published on 2026-07-31, with the repository last updated on 2026-08-03. The project is under active development with recent additions including the GEPA optimizer, mermaid-based flow definitions, the AxIR multi-language compiler pipeline, and the PEEK-inspired context map system. The GitHub repository reports 2,855 stars and 90,400+ npm downloads per week as stated on the project homepage.