# Reame

> A lean, fully-tested LLM inference server built on llama.cpp for cheap CPU hardware, with persistent disk caching of prompts and past generations so each repeated request costs less than the last.

Reame is an MIT-licensed LLM inference server built by Marco Caciotti (swellweb) on top of llama.cpp, designed specifically for CPU-first deployment on free tiers, shared VPS instances, and 2-core ARM boxes. It exposes an OpenAI-compatible REST API and ships a zero-config CLI that downloads models, auto-configures threads and KV cache, and starts serving in a single command. The project reached v0.1.5 in July 2026 and is actively developed.

## What It Is

Reame is a self-hosted local inference server whose central thesis is: **on a CPU, never compute the same thing twice.** It targets narrow, repetitive AI workloads over private data — document extraction, batch tagging, RAG pipelines, privacy-bound legal or medical text — where the answer lives in the provided context rather than in broad model knowledge. The README reports 100% accuracy on long-context extraction with a 7B model on a free 2-core ARM box, and positions Reame as a cost-effective alternative to metered API calls for thin-margin SaaS products.

## Core Caching Architecture

Reame's performance model rests on several layered caching mechanisms:

- **Persistent shared-prefix KV cache** — prompts are split into fixed token blocks; a chain hash keys a KV snapshot at every block boundary. Different prompts sharing a prefix restore the longest cached boundary and decode only their own tail. Snapshots are stored on NVMe with zstd compression, LRU budgeting, and checksums, and survive restarts.
- **Palimpsest generation archive** — every completed generation feeds an on-disk n-gram archive; future requests draft from it at zero inference cost. The README reports a measured 2.3× throughput improvement (22→51 tok/s) on an M3 Pro.
- **ARCA shared-memory daemon** — a Redis-compatible service (`reame arca`) that provides an exact-response cache (~0.02 s vs ~1 s of inference) and a fleet-wide generation corpus so one node's output can draft another's.

## Speculative Decoding and Quality Features

Beyond caching, Reame includes several inference acceleration and quality mechanisms:

- **Self-regulating speculative decoding** — a small draft model or zero-cost n-gram lookup proposes tokens; the target verifies them in one batched pass. A feedback controller measures whether speculation pays on the current hardware and disables it automatically when it doesn't.
- **Il Suggeritore** — inverts grammar-constrained decoding to *propose* structure tokens (list numbering, bullets, format tokens) for free on novel content.
- **The Conclave** — `--best-of N` generates N candidate answers in one interleaved batch using KV cloning (one prefill, cloned into N attempts), then elects a winner by majority vote. The README reports it squeezes roughly one extra correct answer per quiz out of the same model, correcting variance rather than bias.
- **Interleaved multi-user serving** — N concurrent generations advance together inside single multi-sequence batches, sharing every model-weight read.

## Deployment and Setup Path

Installation is available via Homebrew (`brew tap swellweb/reame && brew install reame`), prebuilt binaries for Linux x64/arm64 and macOS arm64 on the GitHub releases page, and build-from-source with CMake ≥ 3.16 and a C++17 compiler. The CLI resolves catalog names like `qwen2.5-1.5b`, downloads to `~/.reame/models` on first use, and auto-configures threads, KV quantization, and cache directory. A config file is only needed for fine-grained control. The server exposes `/v1/completions`, `/v1/chat/completions`, SSE streaming, session snapshots, bearer auth, and a `/metrics` endpoint, making it a drop-in target for any OpenAI client.

## Update: v0.1.5

The latest release is v0.1.5, published on 2026-07-13. The repository was created in early July 2026 and has seen active pushes through mid-July 2026. The project is explicitly described as "young and deliberately opinionated and focused" — CPU-only serving, one model per process, correctness pinned by 220+ isolated test cases. Planned roadmap items mentioned in the README include warm-ahead prefill, a semantic (L2) cache layer for the ARCA daemon, and first-class MoE serving. GPU offload, training, and model management UX are explicitly out of scope.

## Features
- Persistent shared-prefix KV cache to disk (zstd, checksummed, LRU-budgeted)
- Palimpsest generation archive for zero-cost drafting from past outputs
- Self-regulating speculative decoding with automatic disabling when not beneficial
- Il Suggeritore: grammar-based structure token speculation
- The Conclave: best-of-N consensus via interleaved batching and KV cloning
- ARCA Redis-compatible shared-memory daemon for fleet-wide caching
- OpenAI-compatible REST API (/v1/completions, /v1/chat/completions, SSE streaming)
- Zero-config CLI with automatic model download and host auto-configuration
- Interleaved multi-user serving with shared weight reads
- Bearer auth, session snapshots, /metrics endpoint
- 220+ isolated test cases covering multi-sequence, speculative, and KV-clone paths
- Homebrew, prebuilt binary, and build-from-source installation options

## Integrations
llama.cpp, OpenAI API clients, Continue.dev, Redis clients, GGUF model format, Homebrew

## Platforms
CLI, API, LINUX, MACOS

## Pricing
Open Source

## Version
v0.1.5

## Links
- Website: https://github.com/swellweb/reame
- Documentation: https://github.com/swellweb/reame
- Repository: https://github.com/swellweb/reame
- EveryDev.ai: https://www.everydev.ai/tools/reame