LitGPT

LitGPT is an Apache 2.0-licensed project from Lightning AI that provides from-scratch implementations of more than 20 large language models, along with command-line workflows for pretraining, finetuning, evaluating, chatting with, and serving them. The latest listed release is v0.5.13, published 2026-06-29.

What It Is

LitGPT is a Python library and CLI for working directly with LLM code. The README says each model is implemented without abstraction layers, in single-file implementations, so that debugging and modification are straightforward. It supports model families such as Llama, Gemma, Phi, Qwen, Mistral, Falcon, and others. It is built on Lightning Fabric and extends Lit-LLaMA and nanoGPT.

Workflow and Outputs

The CLI follows a litgpt [action] [model] pattern. Users can download weights, finetune on a custom dataset (for example JSON), chat with the result, evaluate on benchmarks such as MMLU and TruthfulQA, and serve a model as a web server with a /predict endpoint. Pretraining from scratch and continued pretraining on custom text files are also documented. A Python API lets users load a model with LLM.load(...) and call generate.

Training Features and Recipes

Documented capabilities include Flash Attention, fully-sharded data parallelism, optional CPU offloading, TPU/XLA support, low-precision modes (FP16, BF16, mixed), and quantization (4-bit, 8-bit, double quantization). Parameter-efficient finetuning covers LoRA, QLoRA, Adapter, and Adapter v2. YAML configuration recipes in a config hub can be overridden on the command line. Models can be exported to other weight formats.

Lightning AI Context

The project is maintained by Lightning AI, creators of PyTorch Lightning. Its Studios provide hosted environments for running LitGPT workflows, and the README points to Lightning's GPU cloud for compute.