AI Performance Engineering
Code, labs, and resources accompanying the O'Reilly book AI Systems Performance Engineering, covering GPU optimization, distributed training, and inference scaling.
At a Glance
Code, tooling, and resources repository for the AI Systems Performance Engineering book, licensed under Apache 2.0. The O'Reilly book itself is sold separately on Amazon and is not covered by this plan.
Engagement
Available On
Alternatives
Listed Nov 2026
About AI Performance Engineering
AI Performance Engineering is a public GitHub repository by Chris Fregly that accompanies the O'Reilly book AI Systems Performance Engineering. It collects code, tooling, a performance checklist, and community resources on GPU optimization, distributed training, inference scaling, and full-stack tuning for AI workloads.
What It Is
The repository is a companion resource for a book aimed at AI/ML engineers, systems engineers, researchers, and platform teams who build or operate training and inference at scale. The README says the book contains thousands of lines of PyTorch and CUDA C++ examples for modern NVIDIA GPUs. It is released under the Apache 2.0 license.
What the Material Covers
The book follows a profile-first, goodput-oriented methodology. Topics include Nsight Systems and Nsight Compute profiling, the PyTorch profiler, memory and bandwidth optimization, and compiler stacks such as torch.compile and OpenAI Triton. Distributed training coverage includes DP, FSDP, TP, PP, CP, and MoE parallelism, with overlap of computation and communication. On the serving side it covers vLLM, SGLang, TensorRT-LLM, and NVIDIA Dynamo, including disaggregated prefill/decode and KV cache management. Twenty chapters run from hardware and OS/Kubernetes tuning through CUDA kernel optimization to AI-assisted optimization.
Checklist and Community Resources
The book ships with a 200+ item performance checklist covering the lifecycle from system architecture and drivers to GPU tuning, distributed training, inference serving, and power management. The repo also links monthly meetup recordings, slide decks, and a YouTube channel for the AI Performance Engineering community.
Community Discussions
Be the first to start a conversation about AI Performance Engineering
Share your experience with AI Performance Engineering, ask questions, or help others learn from your insights.
Pricing
Open Source (Apache 2.0)
Code, tooling, and resources repository for the AI Systems Performance Engineering book, licensed under Apache 2.0. The O'Reilly book itself is sold separately on Amazon and is not covered by this plan.
- Apache License 2.0 (OSI-approved)
- Code, labs, and tooling for GPU optimization, distributed training, and inference scaling
- 200+ item performance checklist
Capabilities
Key Features
- Code examples in PyTorch and CUDA C++ for NVIDIA GPUs
- 200+ item performance checklist
- Profiling guidance with Nsight Systems, Nsight Compute, and PyTorch profiler
- Triton and torch.compile kernel tuning material
- Distributed training parallelism strategies
- High-throughput inference with vLLM, SGLang, TensorRT-LLM, and NVIDIA Dynamo
- Disaggregated prefill/decode and KV cache tuning
- Meetup slides and recordings
