kimi-k3-in-c
A portable C99 inference engine that runs the 2.78-trillion-parameter Kimi K3 model on a single CPU in about 8 GB of RAM by streaming weights from disk.
At a Glance
Self-hosted Kimi K3 inference engine in portable C99, licensed under Apache License 2.0. Runs on the user's own hardware.
Engagement
Available On
Alternatives
Listed Nov 2026
About kimi-k3-in-c
kimi-k3-in-c is an open-source inference engine written by Fareed Khan (FareedKhan-dev) that runs Kimi K3, a 2.78-trillion-parameter mixture-of-experts model, on a single CPU with no GPU. It is written in portable C99 with no BLAS or ML framework, and the README reports a measured peak of 8.24 GB RAM against a 1.56 TB checkpoint. The latest release listed is v1.1.0.
What It Is
The project is a from-scratch CPU inference engine for the Kimi K3 checkpoint. It reads the 96 safetensors shards directly, tokenizes text with a C byte-level BPE tokenizer, and generates tokens through every layer of the model. It also includes a chat mode that uses the official Kimi K3 XTML format, with optional sampling and a JSONL history file.
The workflow is: build with make, verify with a weightless test suite, download the checkpoint, pack the dense trunk into one file, then run the k3 binary with a memory preset. Output is generated text, token ids, JSON run results, and a report with peak RSS and I/O share.
How the Memory Reduction Works
The README describes four reductions. The routed experts already ship in MXFP4 at about half a byte per weight, and the engine multiplies directly from the packed nibbles instead of dequantizing. 69 of the 93 layers use Kimi Delta Attention, whose recurrent state stays a fixed size regardless of context. The other 24 layers use latent attention. Finally, the dense trunk is streamed from disk, and only 16 of 896 experts per layer are loaded per token, with an LRU cache for experts.
The README states that output is byte-identical across memory budgets; more memory only improves speed.
Presets and Speed
Presets (ultra, laptop, desktop, workstation, server, max) set the trunk and expert-cache budgets. The README's measured figures range from about 26.5 s per token at 8 GB to about 5.6 s per token on a 128 GB+ machine with a fast NVMe drive. The ultra preset is described as a proof-of-life path for 8 GB-class machines rather than an interactive one.
Validation and Requirements
The README says kernels use a fixed floating-point reduction order so scalar, AVX2, and NEON paths produce identical bits. The weightless test suite checks the engine against a PyTorch reference on a 13-layer model with the same tensor graph. The checkpoint needs about 1.56 TB of storage plus a 109 GB packed trunk. Linux x86-64 is the reference platform, and macOS/arm64 and Windows/x86-64 are also documented as building.
Community Discussions
Be the first to start a conversation about kimi-k3-in-c
Share your experience with kimi-k3-in-c, ask questions, or help others learn from your insights.
Pricing
Open Source (Apache-2.0)
Self-hosted Kimi K3 inference engine in portable C99, licensed under Apache License 2.0. Runs on the user's own hardware.
- Apache License 2.0 (OSI-approved)
- Portable C99, no BLAS, no framework, no GPU
- Runs on 8 GB RAM and up with streaming from disk
- Linux, macOS and Windows builds
- Requires ~1.7 TB free storage for checkpoint and packed trunk (user-provided hardware)
Capabilities
Key Features
- Runs the 2.78T-parameter Kimi K3 on a single CPU with no GPU
- Peak RSS of 8.24 GB reported on the laptop preset
- Portable C99 with no BLAS or framework dependencies
- Direct MXFP4 matrix multiply without dequantization
- Streamed dense trunk with pinned layers and ring buffer
- LRU cache and prefetch for routed experts
- Memory presets: ultra, laptop, desktop, workstation, server, max
- Incremental decoding with KV cache and carried KDA state
- Official Kimi K3 XTML chat mode with JSONL history
- Greedy decoding and optional seeded sampling
- Byte-level BPE tokenizer loaded from tiktoken.model
- Bit-exact kernels across scalar, AVX2 and NEON paths
- Weightless test suite validated against PyTorch reference
- Cache trace dump and replay tools
