# Chitu

> High-performance open-source large language model inference framework focused on efficiency, flexibility, and usability across diverse hardware.

Chitu (赤兔) is an open-source large model inference framework from the thu-pacman organization, with the license file crediting Qingcheng.AI. It is positioned as a production-grade inference engine for taking enterprise AI deployments from small experiments to large-scale serving. The latest listed release is v0.6.0, published 2026-07-02.

## What It Is

Chitu is an LLM serving and inference engine written primarily in Python. It targets deployment scenarios ranging from CPU-only and single-GPU setups to large clusters. The project states it is built to run on both NVIDIA hardware, from the latest flagship to older cards, and on domestic Chinese chips.

## Hardware and Model Coverage

The homepage lists official images for NVIDIA (including sm_80, sm_89 and sm_90 architectures), Ascend, MetaX, Hygon and Moore Threads, plus other platforms. Supported models listed include DeepSeek (V4-Pro, V4-Flash, V3.2), GLM (5.1 to 5.3) and Qwen (including Qwen3.6 and Qwen3.5) families, with a longer list in the docs. Earlier milestones mention FP4-to-FP8/BF16 operators and CPU+GPU heterogeneous hybrid inference for running DeepSeek-R1 671B on a single GPU.

## Setup Path

The README recommends deploying with the chitu.run executable, downloaded from the GitHub Releases assets. According to the milestones, v0.6.0 added this single-file executable for launching multi-node, multi-instance and prefill/decode-disaggregated tasks. For quick single-machine verification, the homepage suggests the official Docker image, run with GPU access and a mounted model path. A development manual covers full installation and usage.

## Community and Support

The project accepts contributions and issues on GitHub and runs a WeChat group. The maintainers note they cannot guarantee timely resolution of every user issue and offer professional technical services through solution@chitu.ai. It is licensed under Apache License 2.0.

## Features
- High-performance LLM inference engine
- Support for NVIDIA GPUs and domestic Chinese chips (Ascend, MetaX, Hygon, Moore Threads)
- Scalable from CPU-only and single GPU to large clusters
- CPU+GPU heterogeneous hybrid inference
- FP4 and FP8 online conversion operators
- chitu.run single executable for multi-node, multi-instance and PD-disaggregated deployments
- Official Docker images
- Supports DeepSeek, GLM, Qwen and other models
- Performance benchmark documentation

## Integrations
Docker, Hugging Face, PyTorch

## Platforms
LINUX, CLI

## Pricing
Open Source

## Version
v0.6.0

## Links
- Website: https://chitu.ai/
- Documentation: https://github.com/thu-pacman/chitu/tree/public-main/docs/en
- Repository: https://github.com/thu-pacman/chitu
- EveryDev.ai: https://www.everydev.ai/tools/chitu