Chitu

Chitu (赤兔) is an open-source large model inference framework from the thu-pacman organization, with the license file crediting Qingcheng.AI. It is positioned as a production-grade inference engine for taking enterprise AI deployments from small experiments to large-scale serving. The latest listed release is v0.6.0, published 2026-07-02.

What It Is

Chitu is an LLM serving and inference engine written primarily in Python. It targets deployment scenarios ranging from CPU-only and single-GPU setups to large clusters. The project states it is built to run on both NVIDIA hardware, from the latest flagship to older cards, and on domestic Chinese chips.

Hardware and Model Coverage

The homepage lists official images for NVIDIA (including sm_80, sm_89 and sm_90 architectures), Ascend, MetaX, Hygon and Moore Threads, plus other platforms. Supported models listed include DeepSeek (V4-Pro, V4-Flash, V3.2), GLM (5.1 to 5.3) and Qwen (including Qwen3.6 and Qwen3.5) families, with a longer list in the docs. Earlier milestones mention FP4-to-FP8/BF16 operators and CPU+GPU heterogeneous hybrid inference for running DeepSeek-R1 671B on a single GPU.

Setup Path

The README recommends deploying with the chitu.run executable, downloaded from the GitHub Releases assets. According to the milestones, v0.6.0 added this single-file executable for launching multi-node, multi-instance and prefill/decode-disaggregated tasks. For quick single-machine verification, the homepage suggests the official Docker image, run with GPU access and a mounted model path. A development manual covers full installation and usage.

Community and Support

The project accepts contributions and issues on GitHub and runs a WeChat group. The maintainers note they cannot guarantee timely resolution of every user issue and offer professional technical services through solution@chitu.ai. It is licensed under Apache License 2.0.