EveryDev.ai
Subscribe
Home
Tools

5,923+ AI tools

  • Featured
  • Gems
  • Trending
  • New
  • Rate tools
  • Compare
  • Arena
Categories
  • Agents2992
  • Coding2260
  • Infrastructure1068
  • Projects824
  • MCP795
  • Research744
  • Analytics567
  • Marketing527
  • Design489
  • Testing476
  • Data442
  • Security417
  • Integration288
  • Communication286
  • Prompts284
  • Extensions239
  • Voice214
  • DevOps191
  • Commerce184
  • Web128
  • Learning123
  • Finance44
AI Tools by Topic
  • AI Coding Assistants
  • Agent Frameworks
  • MCP Servers
  • AI Prompt Tools
  • Vibe Coding Tools
  • AI Design Tools
  • AI Database Tools
  • AI Website Builders
  • AI Testing Tools
  • LLM Evaluations
Follow Us
  • X / Twitter
  • LinkedIn
  • Reddit
  • Discord
  • Threads
  • Bluesky
  • Mastodon
  • YouTube
  • GitHub
  • Instagram
Get Started
  • Users
  • Rate Tools
  • About
  • Editorial Standards
  • Corrections & Disclosures
  • Community Guidelines
  • Advertise
  • Contact Us
  • Newsletter
  • Submit a Tool
  • Start a Discussion
  • Write A Blog
  • Share A Build
  • Terms of Service
  • Privacy Policy
Explore with AI
  • ChatGPT
  • Gemini
  • Claude
  • Grok
  • Perplexity
Agent Experience
  • llms.txt
Theme
With AI, Everyone is a Dev. EveryDev.ai ©
    1. Home
    2. Tools
    3. kimi-k3-in-c
    kimi-k3-in-c icon

    kimi-k3-in-c

    AI Infrastructure
    Featured

    A portable C99 inference engine that runs the 2.78-trillion-parameter Kimi K3 model on a single CPU in about 8 GB of RAM by streaming weights from disk.

    Visit Website

    At a Glance

    Pricing
    Open Source

    Self-hosted Kimi K3 inference engine in portable C99, licensed under Apache License 2.0. Runs on the user's own hardware.

    Engagement

    Available On

    Windows
    macOS
    Linux
    Web
    API

    Resources

    WebsiteDocsGitHubllms.txt

    Topics

    AI InfrastructureCompute OptimizationLocal Inference

    Alternatives

    DwarfStar 4 (ds4)samir-vllmBitNet
    Developer
    Fareed KhanI do research on AI

    Listed Nov 2026

    About kimi-k3-in-c

    kimi-k3-in-c is an open-source inference engine written by Fareed Khan (FareedKhan-dev) that runs Kimi K3, a 2.78-trillion-parameter mixture-of-experts model, on a single CPU with no GPU. It is written in portable C99 with no BLAS or ML framework, and the README reports a measured peak of 8.24 GB RAM against a 1.56 TB checkpoint. The latest release listed is v1.1.0.

    What It Is

    The project is a from-scratch CPU inference engine for the Kimi K3 checkpoint. It reads the 96 safetensors shards directly, tokenizes text with a C byte-level BPE tokenizer, and generates tokens through every layer of the model. It also includes a chat mode that uses the official Kimi K3 XTML format, with optional sampling and a JSONL history file.

    The workflow is: build with make, verify with a weightless test suite, download the checkpoint, pack the dense trunk into one file, then run the k3 binary with a memory preset. Output is generated text, token ids, JSON run results, and a report with peak RSS and I/O share.

    How the Memory Reduction Works

    The README describes four reductions. The routed experts already ship in MXFP4 at about half a byte per weight, and the engine multiplies directly from the packed nibbles instead of dequantizing. 69 of the 93 layers use Kimi Delta Attention, whose recurrent state stays a fixed size regardless of context. The other 24 layers use latent attention. Finally, the dense trunk is streamed from disk, and only 16 of 896 experts per layer are loaded per token, with an LRU cache for experts.

    The README states that output is byte-identical across memory budgets; more memory only improves speed.

    Presets and Speed

    Presets (ultra, laptop, desktop, workstation, server, max) set the trunk and expert-cache budgets. The README's measured figures range from about 26.5 s per token at 8 GB to about 5.6 s per token on a 128 GB+ machine with a fast NVMe drive. The ultra preset is described as a proof-of-life path for 8 GB-class machines rather than an interactive one.

    Validation and Requirements

    The README says kernels use a fixed floating-point reduction order so scalar, AVX2, and NEON paths produce identical bits. The weightless test suite checks the engine against a PyTorch reference on a 13-layer model with the same tensor graph. The checkpoint needs about 1.56 TB of storage plus a 109 GB packed trunk. Linux x86-64 is the reference platform, and macOS/arm64 and Windows/x86-64 are also documented as building.

    kimi-k3-in-c - 1

    Community Discussions

    Be the first to start a conversation about kimi-k3-in-c

    Share your experience with kimi-k3-in-c, ask questions, or help others learn from your insights.

    Pricing

    OPEN SOURCE

    Open Source (Apache-2.0)

    Self-hosted Kimi K3 inference engine in portable C99, licensed under Apache License 2.0. Runs on the user's own hardware.

    • Apache License 2.0 (OSI-approved)
    • Portable C99, no BLAS, no framework, no GPU
    • Runs on 8 GB RAM and up with streaming from disk
    • Linux, macOS and Windows builds
    • Requires ~1.7 TB free storage for checkpoint and packed trunk (user-provided hardware)

    Capabilities

    Key Features

    • Runs the 2.78T-parameter Kimi K3 on a single CPU with no GPU
    • Peak RSS of 8.24 GB reported on the laptop preset
    • Portable C99 with no BLAS or framework dependencies
    • Direct MXFP4 matrix multiply without dequantization
    • Streamed dense trunk with pinned layers and ring buffer
    • LRU cache and prefetch for routed experts
    • Memory presets: ultra, laptop, desktop, workstation, server, max
    • Incremental decoding with KV cache and carried KDA state
    • Official Kimi K3 XTML chat mode with JSONL history
    • Greedy decoding and optional seeded sampling
    • Byte-level BPE tokenizer loaded from tiktoken.model
    • Bit-exact kernels across scalar, AVX2 and NEON paths
    • Weightless test suite validated against PyTorch reference
    • Cache trace dump and replay tools

    Integrations

    Hugging Face
    PyTorch
    OpenMP
    CMake
    API Available
    View Docs

    Ratings & Reviews

    No ratings yet

    Be the first to rate kimi-k3-in-c and help others make informed decisions.

    Rate other tools you’ve used

    Developer

    Fareed Khan

    I do research on AI

    Read more about Fareed Khan
    WebsiteGitHubLinkedIn
    1 tool in directory

    Similar Tools

    DwarfStar 4 (ds4) icon

    DwarfStar 4 (ds4)

    A narrow C inference engine that runs frontier open-weight models locally on high-memory Mac, CUDA and ROCm machines.

    samir-vllm icon

    samir-vllm

    A small LLM inference engine in C++ and CUDA that teaches inference from first principles with video tutorials and study notes.

    BitNet icon

    BitNet

    Microsoft's official implementation of BitNet, enabling efficient 1-bit large language model inference on CPUs without requiring GPUs.

    Browse all tools

    Related Topics

    AI Infrastructure

    Infrastructure designed for deploying and running AI models.

    589 tools

    Compute Optimization

    Tools for optimizing computational resources and performance.

    90 tools

    Local Inference

    Tools and platforms for running AI inference locally without cloud dependence.

    353 tools
    Browse all topics
    Back to all toolsSuggest an edit
    ratings
    discussions
    8views