MACaQ
Open-source model-agnostic calibrated quantizer for 4-bit weights and activations across text, image, video, audio and 3D models.
At a Glance
MACaQ is open source under the Apache License 2.0, free to use, modify and distribute. Bundled third-party components retain their own licenses.
Engagement
Available On
Alternatives
Listed Nov 2026
About MACaQ
MACaQ (Model Agnostic Calibrated Quantization) is an open-source quantization toolkit from Huawei Computing Systems Lab in Switzerland. It calibrates low-precision weights and activations one linear layer at a time, and it targets text, image, video, audio and 3D reconstruction models. It ships workflows for llama.cpp, vLLM, Diffusers and PyTorch under the Apache 2.0 license.
What It Is
MACaQ is a post-training quantizer. For each linear layer it applies channel balancing, small Hadamard transforms and weight refinement (Flip-Refinement), and every change stays inside that layer. Calibration and refinement happen offline, and the saved checkpoint is then served or run without recalibration. The 4-bit formats it uses include MXFP4 with W4A4 and W4A8 configurations, and it can also produce 2-, 3- and 4-bit GGUF files.
Workflow and Backends
Users install a separate Python 3.12 environment per backend, supply representative calibration text or data, quantize, and then run or serve the result.
- llama.cpp: quantizes text models to GGUF and runs them with the bundled runtime. CPU execution is also supported.
- vLLM: produces compact checkpoints for generation or HTTP chat serving.
- Diffusers: quantizes diffusion transformers for image, video and audio generation.
- PyTorch: quantizes speech generators such as Qwen3-TTS and Chatterbox Turbo.
The docs list example models including Qwen3 variants, FLUX.2 klein 4B, Wan2.2 and MapAnything.
MACaQ-R
MACaQ-R refines the weight codes of an existing quantized GGUF while preserving its native formats and block scales. The resulting files run on a standard llama.cpp installation. The README publishes a Qwen3.6-35B-A3B MACaQ-R GGUF on Hugging Face.
Reported Results and Ascend Kernels
The README states that MACaQ can quantize a 122B-parameter model in 64 minutes using 32.5 GB of device memory. It also reports lower KL divergence than other published Qwen3-4B GGUFs at comparable file sizes. These figures come from the authors. MACaQ W4A4 dense and grouped matmul kernels for Ascend are available through CANN pull requests, and full-model execution there needs integration with an Ascend inference runtime.
Community Discussions
Be the first to start a conversation about MACaQ
Share your experience with MACaQ, ask questions, or help others learn from your insights.
Pricing
Open Source (Apache 2.0)
MACaQ is open source under the Apache License 2.0, free to use, modify and distribute. Bundled third-party components retain their own licenses.
- Apache License 2.0
- Quantize text, image, video and audio models
- Backends: llama.cpp, vLLM, Diffusers, PyTorch
- MACaQ-R to refine existing GGUF models
- Bundled third-party components retain their respective licenses
Capabilities
Key Features
- Layer-local calibrated quantization with channel balancing, block-Hadamard transforms and Flip-Refinement
- 4-bit weight and activation (W4A4/W4A8 MXFP4) quantization
- 2-, 3- and 4-bit GGUF export for llama.cpp
- vLLM checkpoint quantization and HTTP serving
- Diffusers support for image, video and audio generation models
- PyTorch speech generator quantization
- MACaQ-R refinement of existing GGUF models for standard llama.cpp
- Ascend W4A4 dense and grouped matmul kernels via CANN pull requests
