# MTPLX

> Native Mac app and CLI that runs local language models on Apple Silicon using multi-token prediction speculative decoding, with an OpenAI- and Anthropic-compatible local server.

MTPLX is a native Mac app and command line tool, built by Youssof Altoukhi, that runs local language models on Apple Silicon using each model's own multi-token prediction (MTP) heads. It supports Qwen 3.8 Flash Next, Qwen 3.8 27B, Prism ML's Ternary Bonsai 2 27B, Qwen 3.6, Qwen 3.5 and Gemma 4. The project is open source under Apache-2.0, and the latest release listed is v2.12.2.

## What It Is

MTPLX is a local inference engine for Macs. The model drafts several tokens ahead, a single batched forward pass verifies the draft, and tokens are committed through exact rejection sampling with residual correction. This works at the model's native temperature and needs no separate draft model. The README reports roughly twice the speed of plain decoding, for example 1.6x on a 16 GB M4 Mac mini and 2.24x on an M5 Max.

## Setup path

The Mac app checks the hardware, recommends a model that fits the machine's memory, downloads it, sets up its own Python engine, installs fan control, puts `mtplx` on the PATH, and measures the machine to pick a decoding depth. The CLI is available through Homebrew or pip, and `mtplx start` walks through model, mode and surface selection. Requirements are Apple Silicon (M1 or newer) and macOS 14+.

## Local server and integrations

`mtplx start` or `mtplx serve` exposes an OpenAI-compatible API on 127.0.0.1:8000 (chat completions, responses, completions, models, optional embeddings and rerank) plus an Anthropic-compatible messages endpoint with streaming and tool calls. Clients such as OpenCode, Claude Code, Cline, Continue, Cursor and Open WebUI can point at it. A session cache keeps multi-turn chats fast, and the same daemon can serve MLX embedding and reranker models for RAG setups.

## Tuning, Forge and modes

Auto-tune measures autoregressive decoding against each MTP depth on the user's own Mac and saves a depth only if it wins. Forge converts a Hugging Face repo to MLX, trains an MTP adapter, verifies the result is faster and exact, and can publish it to the Hub. Turbo, Sustained, Sustained Max and Burst modes trade speed, context length and fan noise, with a watchdog restoring automatic fan control if the process dies.

## Tradeoffs to know

MTPLX is MLX-native and Apple Silicon first; it is not a CUDA project. It does not attach a separately supplied MTP sidecar to an arbitrary MLX trunk. Larger models like Qwen 3.8 Flash Next need 96 GB or more of unified memory. Products built on MTPLX must display a visible 'Powered by MTPLX' attribution.

## Features
- Multi-token prediction speculative decoding with exact rejection sampling
- Native macOS app with live decode dashboard and chat
- CLI for serving, pulling, tuning, inspecting and benchmarking models
- OpenAI-compatible and Anthropic-compatible local server
- Auto-tune of draft depth per Mac
- Forge for building and verifying custom MTP models
- Embeddings and reranking endpoints
- Warm-prefix and SSD session caching
- Fan control modes
- Multiple model library folders
- Built-in AIME benchmark runner

## Integrations
OpenCode, Claude Code, Cline, Continue, Cursor, Open WebUI, Hermes, Pi, Codex, Hugging Face, MLX

## Platforms
MACOS, LINUX, WEB, API, CLI

## Pricing
Open Source

## Version
2.12.2

## Links
- Website: https://mtplx.com
- Repository: https://github.com/youssofal/MTPLX
- EveryDev.ai: https://www.everydev.ai/tools/mtplx