Kapitsa

Kapitsa is an open-source inference engine for Apple silicon, published by Nace AI under the Apache 2.0 license. It runs Qwen, Nemotron and Recursive's Flint models locally and is built for coding agents. It can launch opencode, Claude Code, Codex or pi on a local model, and it serves OpenAI and Anthropic APIs. The latest listed release is v0.2.17.

What It Is

Kapitsa is a local LLM runtime for Macs with Apple silicon (M1 or newer) on macOS 14 Sonoma or later. Users install it through Homebrew or an install script, then start a model with a command such as kapitsa serve or kapitsa claude. The server exposes a chat interface plus OpenAI- and Anthropic-compatible APIs at a local address, so existing agent tooling can point at a local model. Its decoding path uses hand-written Metal code.

How KFlash Decoding Works

Kapitsa decodes with KFlash, a tree-based form of speculative decoding. A draft model guesses upcoming tokens with several candidates per position, and the large model verifies a best-first tree of those candidates in a single pass, so more tokens survive each pass. The README states that the tree size is measured per Mac and that output is unchanged, because the big model checks every token. It works with published drafters including z-lab's DFlash and DFlash2 and NVIDIA's DSpark and LiLiCorr.

CLI and Workflow

The command line covers serving, launching agents (claude, codex, opencode, pi), downloading and listing models (pull, list, rm), and importing Qwen or Flint GGUF files from LM Studio, Ollama or llama.cpp. kapitsa doctor reports the Mac's GPU and memory and which models fit. kapitsa cluster finds other Macs on the network running Kapitsa, and a Mac that cannot hold a model can use another one on the network. kapitsa bench benchmarks the machine, and bench compare runs the same test against other installed engines.

Models and Benchmarks

The catalog includes Qwen3.5-4B and 9B, Qwen3.8-27B, Qwen3.6-35B-A3B, Qwen3.8-Flash-Next, several Nemotron models, and Recursive Flint 40B-A2.5B, with 2-bit GGUF variants for 16-24 GB Macs. The project's own benchmarks across M1 Pro to M5 Max MacBook Pros claim the fastest decode in 32 of 36 model and Mac pairs against nine engines; these are vendor-published measurements. Other documented features include an SSD prefix cache, prefix reuse for subagents, and image input on most models.