# BaseRT

> BaseRT is the fastest LLM inference runtime for Apple Silicon, offering up to 6.4x faster prefill than llama.cpp and 3.9x faster than MLX, with an OpenAI-compatible server and multi-language bindings.

BaseRT is an open-source LLM inference runtime built by Base Compute, an AI inference lab with teams in Melbourne and Berlin. It targets Apple Silicon (M1 or later, macOS 14+) and delivers benchmark-leading throughput through hand-written Metal GPU kernels. The project repository is licensed under Apache-2.0, while the prebuilt engine binary ships under its own separate proprietary license.

## What It Is

BaseRT is a local inference engine that lets developers pull models from HuggingFace, chat with them interactively, or serve an OpenAI-compatible HTTP API — all from a single CLI. It introduces its own `.base` model format with affine quantization (Q2–Q8), optional AWQ calibration, and signed bundles. The runtime supports text, vision, and audio on compatible models, and exposes a stable C API with bindings for Python, Node, Rust, and Swift.

## Performance on Apple Silicon

The BaseRT homepage publishes benchmark results measured on an Apple M5 Pro. According to those figures:

- **Prefill**: up to 6.4x faster than llama.cpp and 3.9x faster than MLX (e.g., Gemma 4 E2B at pp2048 Q8: 16,264 tokens/sec vs. 2,547 for llama.cpp)
- **Decode**: up to 1.33x faster than MLX (e.g., Qwen3 0.6B Q4: 531 tokens/sec vs. 398 for MLX)

These gains come from hand-tuned Metal kernels optimized per hardware and per model architecture.

## CLI and Workflow

The `basert` CLI is the single entry point for all operations:

- `basert pull <id>` — download from HuggingFace and convert to `.base`
- `basert chat <model>` — interactive chat session
- `basert serve <model>` — OpenAI-compatible HTTP server with continuous batching, paged-KV, and prefix caching
- `basert convert <src>` — convert GGUF, HF, or MLX checkpoints to `.base`
- `basert bench <model>` — throughput benchmark
- `basert sign` / `verify` / `keygen` — ed25519 signing of `.base` bundles

The server supports chat completions, embeddings, transcription, and tool calls out of the box.

## Supported Models and Coding Agent Integration

BaseRT supports a broad set of popular open-weight model families including Qwen3, Qwen3.5, Qwen3.6, Llama 3.1, Llama 3.2, Gemma 3, Gemma 4, Mistral, Phi-3, and Nomic BERT. It also functions as a drop-in local backend for the `pi` coding agent via the `pi-basert` extension, which auto-discovers running models. This allows developers to run a full local coding agent workflow — model serving, agent execution, and tool calling — entirely on-device with no API keys and no data leaving the machine.

## Update: BaseRT Engine v0.1.7

The latest release is **BaseRT engine 0.1.7**, published on July 21, 2026. The GitHub repository was created in late June 2026 and has seen active development, with the last push on July 21, 2026. The project is in early versioning (0.x) and the repository currently has 88 stars and 6 forks. Base Compute describes its research direction as focused on GPU kernel optimization and automated research pipelines combining human and AI collaboration.

## Open-Source Licensing and Repository Structure

The repository is organized into distinct components: `base-convert/` (the Rust CLI, model hub, converter, and `.base` format specs), `include/baseRT/` (the public C API headers), `bindings/` (Python, Node, Rust, Swift), `benchmarks/`, and `docs/`. The CLI, `.base` format, public headers, and language bindings are all Apache-2.0 licensed. The prebuilt engine binary is distributed separately under a proprietary license and is not open source.

## Features
- Fastest LLM inference runtime for Apple Silicon
- Hand-written Metal GPU kernels for maximum throughput
- OpenAI-compatible HTTP server (chat, completions, embeddings, transcription, tool calls)
- Continuous batching, paged-KV, and prefix caching
- Own .base model format with affine quantization (Q2–Q8) and AWQ calibration
- Signed model bundles with ed25519
- Single CLI: pull, chat, serve, convert, bench, inspect
- HuggingFace model hub integration with convert-on-pull
- Multimodal support: text, vision, and audio
- Python, Node, Rust, and Swift bindings over a stable C API
- Drop-in local backend for pi coding agent
- Multi-model serving with --model flag
- GGUF, HF, and MLX checkpoint conversion to .base
- Fully on-device: no API keys, no data leaving the device

## Integrations
HuggingFace Hub, pi coding agent (pi-basert extension), OpenAI-compatible API clients, Python, Node.js, Rust, Swift, GGUF models, MLX models

## Platforms
MACOS, CLI, API, DEVELOPER_SDK

## Pricing
Open Source

## Version
v0.1.7

## Links
- Website: https://www.basecompute.co/getbasert
- Documentation: https://docs.basecompute.co
- Repository: https://github.com/basecompute/baseRT
- EveryDev.ai: https://www.everydev.ai/tools/basert