TurboFieldfare

TurboFieldfare is an open-source, model-specific inference runtime built by Andrey Mikhaylov, an iOS and Metal engineer. It runs the instruction-tuned Gemma 4 26B-A4B large language model on Apple Silicon Macs — including the base 8 GB M2 MacBook Air — by keeping only a ~1.35 GB shared core and FP16 KV cache in memory and streaming routed experts from SSD on demand. The entire stack — runtime, streaming installer, CLI, loopback server, and native Mac app — is written in Swift and Metal, with no dependency on MLX or llama.cpp.

What It Is

TurboFieldfare is a local inference engine purpose-built for a single model: Gemma 4 26B-A4B, a 26-billion-parameter mixture-of-experts model with roughly 3.88 billion parameters active per token. Rather than loading the full 14.3 GB model into RAM, TurboFieldfare streams only the expert weights needed for each token from SSD into Metal-visible buffers, making the model runnable on hardware that would otherwise be unable to fit it. The project is licensed under Apache 2.0 and is not affiliated with, sponsored by, or endorsed by Google.

How the Inference Engine Works

The core loop operates at the transformer layer level. Metal computes attention and the MoE router from resident weights; the CPU uses the router's top-8 expert IDs to plan against a 16-slot LFU cache per layer, then fills cache misses with bounded parallel pread calls. Metal computes the shared-expert branch while those reads are in flight, then combines shared and routed outputs. Key architectural details:

• Weights use MLX affine 4-bit quantization (group 64), 8-bit router, 4-bit shared and routed experts
• Prompt prefill uses chunks of up to 128 tokens so one fetched expert can serve multiple rows
• KV cache uses FP16 with circular storage for 25 sliding-window layers and linear storage for 5 full-attention layers
• The streaming installer repacks remote Hugging Face byte ranges directly into the .gturbo layout without staging a full checkpoint, keeping scratch memory bounded

Six Products in One Package

The Swift package ships six distinct products that share the same .gturbo model directory:

• TurboFieldfare — Swift library with the runtime and Metal kernels
• TurboFieldfareMac — Native SwiftUI/AppKit Mac app for installation and generation
• TurboFieldfareDecodeService — One-shot local model and Metal owner used by the Mac app
• TurboFieldfareCLI — Command-line instruction chat and raw completion
• TurboFieldfareServer — Loopback OpenAI-compatible Chat Completions server (supports streaming, function tools, and single-prefix prompt reuse)
• TurboFieldfareRepack — Streaming model installer and install verifier

Platform and System Requirements

TurboFieldfare targets a narrow, specific hardware and software configuration:

• Apple Silicon Mac (arm64-only; validated on 8 GB M2 MacBook Air)
• macOS 26 with Metal 4
• Xcode 26 and Swift 6.2 or newer
• ~14.3 GB free storage for the installed model
• Internet connection for first-time model download

Older macOS versions and non-Apple-Silicon hardware are not supported. The project's stated future work includes iPhone and iPad apps and benchmarks on additional Apple Silicon configurations.

Update: TurboFieldfare 0.3 — Apple10 Long-Context Prefill

Version 0.3, named "Apple10 Long-Context Prefill," was published on 2026-07-29. The repository was created on 2026-07-17 and had accumulated over 2,000 GitHub stars and 75 forks within roughly two weeks of creation, according to the repository metadata. The experiment record documents 103 audited measured results across kernels, caching, I/O, prefill, and decode. Measured decode throughput on an 8 GB M2 MacBook Air is reported at 5.1–6.3 tokens/second; on a 24 GB M5 Pro, 31–35 tokens/second.