Rapid-MLX

Rapid-MLX is an Apache 2.0-licensed LLM inference server and macOS desktop app built exclusively for Apple Silicon, using Apple's MLX framework as its native inference engine. It began as vLLM-MLX by Wayner Barrios and was renamed to Rapid-MLX on 2026-03-13, with extensive modifications and rewrites since. The project is actively maintained on GitHub with 3,863 stars and a latest release of v0.15.2 as of September 2026.

What It Is

Rapid-MLX is a local LLM inference server that exposes OpenAI-compatible (/v1/chat/completions, /v1/responses) and Anthropic-compatible (/v1/messages) HTTP endpoints, allowing any client that supports those APIs to use it as a drop-in local backend. It runs entirely on-device on M1 through M4 Apple Silicon Macs, with no llama.cpp fallback and no Metal shim — pure MLX kernels throughout. Beyond text, it also supports image generation, video generation, audio (TTS, transcription, voice cloning), and embeddings via opt-in extras.

Architecture and Performance

Rapid-MLX is built around continuous batching, a radix prefix cache (with state snapshots for hybrid models, saved to disk on shutdown and restored at startup), and a quantized live KV cache (int4/int8 on the continuous-batching cache plus a TurboQuant K8V4 codec). The project's own benchmark claims 3.0× Ollama's aggregate decode throughput at 8 concurrent streams on Qwen3.6-35B-A3B on a 32 GB M2 Pro Mac mini (82.9 vs 27.2 tok/s aggregate decode), with the methodology, raw data, and caveats published on the project blog. Tool-call parsing is handled by 27 parser modules including an auto-detect fallback, which the README positions as a key differentiator for coding agent reliability.

Agent and Ecosystem Integration

Rapid-MLX wire-verifies 12 coding agents and 3 Python frameworks against real model weights before every release:

• Tier-1 (release-blocking): Claude Code, Codex CLI, Hermes Agent, Aider, DeepSeek Harness
• Tier-2 (wire-verified): OpenCode, Qwen Code, OpenHands, Kilo Code, GitHub Copilot, Factory Droid, Moonshot Kimi Code
• Frameworks: LangChain (+ LangGraph), PydanticAI, smolagents

The rapid-mlx launch <agent> command auto-configures supported agents (e.g., rapid-mlx launch claude-code patches ~/.claude/settings.json) so no manual environment variable editing is required.

Multimodal Capabilities

Beyond text, Rapid-MLX ships opt-in extras for:

• Image generation: OpenAI-compatible /v1/images/generations and /v1/images/edits endpoints, with 12 registered image model aliases including FLUX.2 Klein, SDXL, HiDream, and Qwen Image families
• Video generation: Text-to-video and image-to-video via /v1/videos, backed by Wan 2.1/2.2, CogVideoX-Fun, and LTX-2.3 across 8 checkpoints (requires Python 3.11+)
• Audio: 44 aliases across 13 families behind /v1/audio/* endpoints, covering TTS (Kokoro, Qwen3-TTS), transcription (Whisper, Parakeet, SenseVoice), zero-shot voice cloning, voice design, forced alignment, and text-to-music

Installation and Setup

Rapid-MLX installs via Homebrew (in homebrew-core since 0.10.12), a guided curl installer, uv tool install, or pip. A macOS desktop app (Rapid-MLX Desktop) bundles the same engine in a one-click signed app for users who prefer a GUI. The CLI supports rapid-mlx chat, rapid-mlx serve, rapid-mlx benchmark, rapid-mlx recipe (RAM-tier model recommendations), and rapid-mlx doctor for self-diagnostics. The project catalogs 262 total model aliases (196 text, 12 image, 10 video, 44 audio) browsable at models.rapidmlx.com.

Update: v0.15.2

The latest release is v0.15.2, published 2026-09-25. Version 0.13.4 introduced a verified MTP (multi-token prediction) path for Qwen3.8-27B, delivering measured decode speedups of 1.43× to 2.34× depending on context length on an M3 Ultra. Version 0.15.0 added anonymous, metadata-only telemetry (opt-out via rapid-mlx telemetry off or DO_NOT_TRACK=1). The project's community benchmark leaderboard allows Mac users to publish standardized results from their own hardware.