EveryDev.ai
Subscribe
Home
Tools

5,901+ AI tools

  • Featured
  • Gems
  • Trending
  • New
  • Rate tools
  • Compare
  • Arena
Categories
  • Agents2972
  • Coding2248
  • Infrastructure1067
  • Projects818
  • MCP794
  • Research744
  • Analytics565
  • Marketing527
  • Design488
  • Testing476
  • Data442
  • Security412
  • Integration288
  • Communication284
  • Prompts283
  • Extensions238
  • Voice213
  • DevOps190
  • Commerce183
  • Web128
  • Learning123
  • Finance44
AI Tools by Topic
  • AI Coding Assistants
  • Agent Frameworks
  • MCP Servers
  • AI Prompt Tools
  • Vibe Coding Tools
  • AI Design Tools
  • AI Database Tools
  • AI Website Builders
  • AI Testing Tools
  • LLM Evaluations
Follow Us
  • X / Twitter
  • LinkedIn
  • Reddit
  • Discord
  • Threads
  • Bluesky
  • Mastodon
  • YouTube
  • GitHub
  • Instagram
Get Started
  • Users
  • Rate Tools
  • About
  • Editorial Standards
  • Corrections & Disclosures
  • Community Guidelines
  • Advertise
  • Contact Us
  • Newsletter
  • Submit a Tool
  • Start a Discussion
  • Write A Blog
  • Share A Build
  • Terms of Service
  • Privacy Policy
Explore with AI
  • ChatGPT
  • Gemini
  • Claude
  • Grok
  • Perplexity
Agent Experience
  • llms.txt
Theme
With AI, Everyone is a Dev. EveryDev.ai ©
    1. Home
    2. Tools
    3. AI Performance Engineering
    AI Performance Engineering icon

    AI Performance Engineering

    AI Courses
    Featured

    Code, labs, and resources accompanying the O'Reilly book AI Systems Performance Engineering, covering GPU optimization, distributed training, and inference scaling.

    Visit Website

    At a Glance

    Pricing
    Open Source

    Code, tooling, and resources repository for the AI Systems Performance Engineering book, licensed under Apache 2.0. The O'Reilly book itself is sold separately on Amazon and is not covered by this plan.

    Engagement

    Available On

    API

    Resources

    WebsiteGitHubllms.txt

    Topics

    AI CoursesAI InfrastructureCompute Optimization

    Alternatives

    tiny-vllmAwesome AI ArchitectMLC LLM
    Developer
    Chris FreglyAI startup founder/advisor · ex-AWS/Databricks/Netflix · 3×…

    Listed Nov 2026

    About AI Performance Engineering

    AI Performance Engineering is a public GitHub repository by Chris Fregly that accompanies the O'Reilly book AI Systems Performance Engineering. It collects code, tooling, a performance checklist, and community resources on GPU optimization, distributed training, inference scaling, and full-stack tuning for AI workloads.

    What It Is

    The repository is a companion resource for a book aimed at AI/ML engineers, systems engineers, researchers, and platform teams who build or operate training and inference at scale. The README says the book contains thousands of lines of PyTorch and CUDA C++ examples for modern NVIDIA GPUs. It is released under the Apache 2.0 license.

    What the Material Covers

    The book follows a profile-first, goodput-oriented methodology. Topics include Nsight Systems and Nsight Compute profiling, the PyTorch profiler, memory and bandwidth optimization, and compiler stacks such as torch.compile and OpenAI Triton. Distributed training coverage includes DP, FSDP, TP, PP, CP, and MoE parallelism, with overlap of computation and communication. On the serving side it covers vLLM, SGLang, TensorRT-LLM, and NVIDIA Dynamo, including disaggregated prefill/decode and KV cache management. Twenty chapters run from hardware and OS/Kubernetes tuning through CUDA kernel optimization to AI-assisted optimization.

    Checklist and Community Resources

    The book ships with a 200+ item performance checklist covering the lifecycle from system architecture and drivers to GPU tuning, distributed training, inference serving, and power management. The repo also links monthly meetup recordings, slide decks, and a YouTube channel for the AI Performance Engineering community.

    AI Performance Engineering - 1

    Community Discussions

    Be the first to start a conversation about AI Performance Engineering

    Share your experience with AI Performance Engineering, ask questions, or help others learn from your insights.

    Pricing

    OPEN SOURCE

    Open Source (Apache 2.0)

    Code, tooling, and resources repository for the AI Systems Performance Engineering book, licensed under Apache 2.0. The O'Reilly book itself is sold separately on Amazon and is not covered by this plan.

    • Apache License 2.0 (OSI-approved)
    • Code, labs, and tooling for GPU optimization, distributed training, and inference scaling
    • 200+ item performance checklist

    Capabilities

    Key Features

    • Code examples in PyTorch and CUDA C++ for NVIDIA GPUs
    • 200+ item performance checklist
    • Profiling guidance with Nsight Systems, Nsight Compute, and PyTorch profiler
    • Triton and torch.compile kernel tuning material
    • Distributed training parallelism strategies
    • High-throughput inference with vLLM, SGLang, TensorRT-LLM, and NVIDIA Dynamo
    • Disaggregated prefill/decode and KV cache tuning
    • Meetup slides and recordings

    Integrations

    PyTorch
    CUDA
    OpenAI Triton
    NVIDIA Nsight
    vLLM
    SGLang
    TensorRT-LLM
    NVIDIA Dynamo
    NCCL
    Kubernetes
    Docker
    API Available

    Ratings & Reviews

    No ratings yet

    Be the first to rate AI Performance Engineering and help others make informed decisions.

    Rate other tools you’ve used

    Developer

    Chris Fregly

    AI startup founder/advisor · ex-AWS/Databricks/Netflix · 3× O'Reilly author · AI Systems Performance Engineer

    Read more about Chris Fregly
    GitHubLinkedInX / Twitter
    1 tool in directory

    Similar Tools

    tiny-vllm icon

    tiny-vllm

    A hands-on course and full source code for building a high-performance LLM inference engine in C++ and CUDA, implementing features like KV cache, PagedAttention, and continuous batching.

    Awesome AI Architect icon

    Awesome AI Architect

    An open-source curated knowledge base of plain-language guides blending AI engineering and solution architecture, covering RAG, agents, cloud, security, and career development.

    MLC LLM icon

    MLC LLM

    Machine learning compiler and high-performance deployment engine for running large language models natively across GPUs, browsers, and mobile devices.

    Browse all tools

    Related Topics

    AI Courses

    Structured courses, workshops, and comprehensive training programs for AI, machine learning, and development.

    82 tools

    AI Infrastructure

    Infrastructure designed for deploying and running AI models.

    593 tools

    Compute Optimization

    Tools for optimizing computational resources and performance.

    91 tools
    Browse all topics
    Back to all toolsSuggest an edit
    ratings
    discussions