h3.c

h3.c is a native C inference engine for the MiniMax-H3 video-audio generation model, built specifically for Apple Silicon Macs using Metal GPU acceleration. Created by Salvatore Sanfilippo (antirez), the project is open source under the MIT License and hosted on GitHub. It targets M3 Max and M5 Max hardware and supports end-to-end prompt-to-video/audio generation, first/last-frame conditioning, and ordered Ref2VA image/video/audio references.

What It Is

h3.c is a local inference runtime for MiniMax-H3, a 33B diffusion transformer (DiT) model capable of generating short video clips with synchronized audio from text prompts. Rather than relying on Python frameworks or cloud APIs, it is written in C and uses Apple's Metal and MPSGraph APIs directly, making it a self-contained binary that runs the full generation pipeline on Mac hardware. The project is structured as a sequence of working vertical slices, progressing from model metadata and Metal block parity through prompt encoding, media generation, and reference conditioning.

Core Capabilities

• Prompt-to-video/audio: Generates H.264 video with 32 kHz stereo AAC audio from text prompts, using a Context-IR-style description format covering subject, action, camera, lighting, and sound.
• First/last-frame conditioning (FL2VA): Anchors the generated clip to specific start and end images using the released visual VAE encoder and Qwen3-VL vision tower.
• Ordered Ref2VA references: Accepts image, silent video, video-with-audio, and standalone audio references in command-line order, exposing them to the model as <Picture N> or <Video N> tokens.
• Interactive session: Without a -p prompt flag, the binary starts an Iris-style REPL that keeps BF16 prompt conditioning, the prepared DiT, and the video decoder in memory across multiple generations.
• SSD streaming: --ssd-streaming reduces DiT unified-memory use from ~36.5 GiB to ~2.0 GiB by keeping only two transformer blocks resident and reading ahead from SSD while the GPU runs the current block.

Performance Architecture

The engine implements a layered set of speed/quality controls that can be combined independently:

• Denoising passes (--steps): Default 20; 4–7 passes for fast previews; 50 for reference quality.
• Whole-denoiser reuse (--reuse): Extrapolates skipped velocity evaluations; --reuse 2 at 20 steps runs 11 fresh DiT evaluations instead of 20.
• Layer thinning (--layers): Ranks AdaLN gates and drops tail blocks; --layers 45 reduces both compute and resident transformer weights.
• Token reduction (--token-reduction): Pairs adjacent horizontal video tokens inside middle DiT blocks; measured 28.3% denoise time reduction on M5 Max at 512×512.
• Internal canvas scaling (--render-width/--render-height): Runs DiT and VAE at a lower resolution, then upscales with vImage; 384→512 measured 33% DiT time reduction.
• int8 MLP and QKV quantization: On M5, the default path uses native Metal 4 TensorOps int8 for MLP, QKV, and attention-output projections; a fixed 50-layer, 19-transition 512×512 render measured 36.30 s (BF16 MPS) vs. 19.18 s (full int8 path) on M5 Max.

Metal 4 TensorOps are used automatically on M5 GPUs for QKV and attention-output projections at sequence lengths up to 2,048. The DiT core is split across two ordered Metal command buffers so GPU execution of the first half overlaps CPU encoding of the second.

Supported Resolutions and Durations

Width and height must each be multiples of 32 and their product must not exceed 768×1344 pixels. Validated canvases include 512×512, 768×768, 1344×768, 768×1344, and 1024×768. Frame counts align to the H3 temporal shape 5 + 17*n; --seconds N converts at 24 fps and rounds up to the next legal shape. The released workflow targets roughly 4–15 second clips (107–362 frames).

Current Status

The repository was created in August 2026 and had 1,803 stars and 116 forks as of mid-August 2026, reflecting rapid early adoption. The README describes the project as actively under incremental development, with current work focused on H3-specific Metal performance and memory optimization on M3 Max and M5 Max. All major generation paths — prompt-to-video/audio, first/last-frame conditioning, and ordered Ref2VA references — are described as working end to end. The project is MIT-licensed and requires the MiniMax-H3 Hugging Face snapshot, FFmpeg, and FFprobe to be available locally.