Transformers - The Definitive Guide

This is the official companion code repository for the O'Reilly book Transformers - The Definitive Guide by Nicole Koenigstein. It provides 12 chapters of Jupyter notebooks covering transformer architectures and their applications across a wide range of domains, all runnable on Google Colab with GPU support. The repository is licensed under Apache License 2.0 and freely available on GitHub.

What It Is

This repository is a structured, chapter-by-chapter code companion to a published technical book on transformer models. Each chapter folder (CH01 through CH12) contains Jupyter notebooks that demonstrate specific transformer concepts and applications — from foundational attention mechanisms to deploying production models. The notebooks are designed to run on cloud GPU environments (primarily Google Colab) rather than requiring local setup, making them accessible without specialized hardware.

Chapter Coverage and Scope

The repository spans a broad curriculum of transformer use cases:

• Chapter 1: Attention mechanism variations, embeddings, and perplexity
• Chapter 2: Time series transformers (Chronos, PatchTST, fine-tuning with PyTorch)
• Chapter 3: Vision tasks — image classification, ViT embeddings, SAM fine-tuning with W&B and Optuna, video segmentation with SAM 2
• Chapter 4: Image generation — DiT, KV compression, PixArt-Σ XL inference, quantizing text-to-image models
• Chapter 5: Video generation — LTX, Latte, Tora
• Chapter 6: Audio tasks — Kimi-Audio transcription, Qwen2-Audio, music generation, waveform and spectrogram analysis
• Chapter 7: Reinforcement learning transformers — STORM, Decision Transformer, replay buffers
• Chapter 8: Planning, reasoning, and coding — Qwen3, MCTS, TreeQuest, Kimi K2 Instruct
• Chapter 9: AI agents — LangGraph multi-turn conversation, multi-agent market research teams
• Chapter 10: Optimization — AdaptThink, ART·E with LangGraph, rLLM
• Chapter 11: Deployment — LLM performance evaluation, advanced LoRA fine-tuning, LangGraph code interpreter
• Chapter 12: Forward-looking topics — SAM 3 agent

How the Notebooks Are Organized

Each notebook follows a consistent naming convention (ch01_attention_mechanism_variations.ipynb) that ties it to its chapter and topic. Every notebook includes a self-cloning snippet that pulls the full repository into the Colab environment, giving access to shared utility functions and per-chapter requirements installation. A create_env.sh bash script is also provided for local virtual environment setup via conda or pipenv on Windows (WSL), macOS, or Linux, though local setup is described as optional since GPU cloud execution is the primary path.

Audience and Prerequisites

The repository targets practitioners and researchers who want hands-on code alongside the book's theoretical content. Readers are expected to have Python and ML familiarity; the notebooks handle dependency installation automatically per chapter. GPU access is required — the README explicitly notes that notebooks are designed for cloud services with GPU support.

Current Status

The repository was created in March 2024 and last pushed in August 2026, indicating active maintenance. As of the latest data, it has accumulated 412 stars and 99 forks on GitHub. The project is open source under the Apache License 2.0, authored by Nicole Koenigstein.