AI-Infra-Auto-Driven-SKILLS

AI-Infra-Auto-Driven-SKILLS gives coding agents reusable workflows for LLM serving and inference engineering. Maintained by Xiaoyu Zhang (BBuf), it combines skill instructions, Python helpers, reference material, and model implementation histories. Developers use it with their own agent runtime, serving environment, logs, and traces.

What It Is

The project packages SKILL.md workflows for Claude Code and other skill-capable agents, including Codex and Kimi. It covers SGLang, vLLM, TensorRT-LLM, and TokenSpeed, with FlashInfer optimization references. Install it through the Claude Code plugin marketplace or copy or link the skill directories into your runtime.

Serving Benchmarks and Capacity Planning

The benchmark workflow compares deployment configurations for the same model, GPU budget, workload, and latency requirements. Configuration validators render candidate commands, and reporting helpers compare normalized results. It is a playbook and helper toolchain: the operator still launches servers, runs workloads, and records results.

Capacity planning uses SGLang and vLLM startup logs to explain GPU memory allocation, KV-cache budgets, request capacity, and out-of-memory pressure. Compute simulation estimates operator shapes, FLOPs, and model FLOPs utilization for a specified serving configuration.

Profiling and Debugging

Profiler workflows analyze existing Torch Profiler traces or capture supported running servers. They produce kernel attribution and overlap and fusion opportunity tables, keeping prefill and decode separate. Pipeline analysis and layer-track helpers add navigation guides for Perfetto. Existing trace analysis does not require a GPU or an installed serving framework.

SGLang workflows also organize new-model support into dependent pull requests and a validation matrix, assist with source-grounded PR review, and turn production symptoms into replay and debugging steps.

Model History and Technical Explanations

A queryable knowledge base contains bilingual model-family dossiers covering implementation files and upstream pull requests. Generated source inventories are marked as awaiting manual diff review; they are research leads rather than proof that an optimization is active or validated.

Additional skills find original public model architecture diagrams and turn SGLang pull requests or technical articles into Chinese explainer videos with bilingual captions, using HyperFrames and edge-tts.

Evidence and Environment Requirements

Performance conclusions depend on the model, hardware, precision, workload, and framework revision tested. Source audits and historical traces are distinguished from fresh GPU runs. Live benchmarks require a suitable serving environment and GPU access; documentation and existing-trace workflows have different requirements.