# TurboFieldfare

> A custom Swift + Metal runtime that runs Gemma 4 26B-A4B inference in approximately 2 GB of RAM on any Apple Silicon Mac, including 8 GB models.

TurboFieldfare is an open-source, model-specific inference runtime built by Andrey Mikhaylov, an iOS and Metal engineer. It runs the instruction-tuned Gemma 4 26B-A4B large language model on Apple Silicon Macs — including the base 8 GB M2 MacBook Air — by keeping only a ~1.35 GB shared core and FP16 KV cache in memory and streaming routed experts from SSD on demand. The entire stack — runtime, streaming installer, CLI, loopback server, and native Mac app — is written in Swift and Metal, with no dependency on MLX or llama.cpp.

## What It Is

TurboFieldfare is a local inference engine purpose-built for a single model: Gemma 4 26B-A4B, a 26-billion-parameter mixture-of-experts model with roughly 3.88 billion parameters active per token. Rather than loading the full 14.3 GB model into RAM, TurboFieldfare streams only the expert weights needed for each token from SSD into Metal-visible buffers, making the model runnable on hardware that would otherwise be unable to fit it. The project is licensed under Apache 2.0 and is not affiliated with, sponsored by, or endorsed by Google.

## How the Inference Engine Works

The core loop operates at the transformer layer level. Metal computes attention and the MoE router from resident weights; the CPU uses the router's top-8 expert IDs to plan against a 16-slot LFU cache per layer, then fills cache misses with bounded parallel `pread` calls. Metal computes the shared-expert branch while those reads are in flight, then combines shared and routed outputs. Key architectural details:

- Weights use MLX affine 4-bit quantization (group 64), 8-bit router, 4-bit shared and routed experts
- Prompt prefill uses chunks of up to 128 tokens so one fetched expert can serve multiple rows
- KV cache uses FP16 with circular storage for 25 sliding-window layers and linear storage for 5 full-attention layers
- The streaming installer repacks remote Hugging Face byte ranges directly into the `.gturbo` layout without staging a full checkpoint, keeping scratch memory bounded

## Six Products in One Package

The Swift package ships six distinct products that share the same `.gturbo` model directory:

- **TurboFieldfare** — Swift library with the runtime and Metal kernels
- **TurboFieldfareMac** — Native SwiftUI/AppKit Mac app for installation and generation
- **TurboFieldfareDecodeService** — One-shot local model and Metal owner used by the Mac app
- **TurboFieldfareCLI** — Command-line instruction chat and raw completion
- **TurboFieldfareServer** — Loopback OpenAI-compatible Chat Completions server (supports streaming, function tools, and single-prefix prompt reuse)
- **TurboFieldfareRepack** — Streaming model installer and install verifier

## Platform and System Requirements

TurboFieldfare targets a narrow, specific hardware and software configuration:

- Apple Silicon Mac (arm64-only; validated on 8 GB M2 MacBook Air)
- macOS 26 with Metal 4
- Xcode 26 and Swift 6.2 or newer
- ~14.3 GB free storage for the installed model
- Internet connection for first-time model download

Older macOS versions and non-Apple-Silicon hardware are not supported. The project's stated future work includes iPhone and iPad apps and benchmarks on additional Apple Silicon configurations.

## Update: TurboFieldfare 0.3 — Apple10 Long-Context Prefill

Version 0.3, named "Apple10 Long-Context Prefill," was published on 2026-07-29. The repository was created on 2026-07-17 and had accumulated over 2,000 GitHub stars and 75 forks within roughly two weeks of creation, according to the repository metadata. The experiment record documents 103 audited measured results across kernels, caching, I/O, prefill, and decode. Measured decode throughput on an 8 GB M2 MacBook Air is reported at 5.1–6.3 tokens/second; on a 24 GB M5 Pro, 31–35 tokens/second.

## Features
- Gemma 4 26B-A4B inference in ~2 GB of RAM
- SSD-backed routed-expert streaming with bounded LFU expert cache
- Custom Metal kernels for quantized GEMV, attention, MoE, normalization, RoPE, sampling, and production fusions
- 4-bit MLX affine quantized weights with 8-bit router
- FP16 KV cache with circular storage for sliding-window layers
- Chunked single-prompt prefill (up to 128 tokens per chunk)
- Native SwiftUI/AppKit Mac app with one-shot local decode service
- Command-line interface for instruction chat and raw completion
- Loopback OpenAI-compatible Chat Completions server with streaming and function tools
- Streaming model installer that repacks Hugging Face byte ranges without staging full checkpoint
- Install verification via manifest and file hash validation
- Configurable sampling: temperature, Top-K, Top-P, repetition penalty, seed
- Text-only inference; no image, audio, or video support

## Integrations
Hugging Face (model download), OpenAI Chat Completions API (loopback-compatible server), Swift Package Manager, Metal 4, OpenCode

## Platforms
MACOS, IOS, API, CLI

## Pricing
Open Source

## Version
0.3

## Links
- Website: https://github.com/drumih/turbo-fieldfare
- Documentation: https://github.com/drumih/turbo-fieldfare/blob/main/docs/SYSTEM_DESIGN.md
- Repository: https://github.com/drumih/turbo-fieldfare
- EveryDev.ai: https://www.everydev.ai/tools/turbo-fieldfare