# LitGPT

> Open-source Python library and CLI with from-scratch implementations of 20+ LLMs and recipes to pretrain, finetune, evaluate, and deploy them.

LitGPT is an Apache 2.0-licensed project from Lightning AI that provides from-scratch implementations of more than 20 large language models, along with command-line workflows for pretraining, finetuning, evaluating, chatting with, and serving them. The latest listed release is v0.5.13, published 2026-06-29.

## What It Is

LitGPT is a Python library and CLI for working directly with LLM code. The README says each model is implemented without abstraction layers, in single-file implementations, so that debugging and modification are straightforward. It supports model families such as Llama, Gemma, Phi, Qwen, Mistral, Falcon, and others. It is built on Lightning Fabric and extends Lit-LLaMA and nanoGPT.

## Workflow and Outputs

The CLI follows a `litgpt [action] [model]` pattern. Users can download weights, finetune on a custom dataset (for example JSON), chat with the result, evaluate on benchmarks such as MMLU and TruthfulQA, and serve a model as a web server with a `/predict` endpoint. Pretraining from scratch and continued pretraining on custom text files are also documented. A Python API lets users load a model with `LLM.load(...)` and call `generate`.

## Training Features and Recipes

Documented capabilities include Flash Attention, fully-sharded data parallelism, optional CPU offloading, TPU/XLA support, low-precision modes (FP16, BF16, mixed), and quantization (4-bit, 8-bit, double quantization). Parameter-efficient finetuning covers LoRA, QLoRA, Adapter, and Adapter v2. YAML configuration recipes in a config hub can be overridden on the command line. Models can be exported to other weight formats.

## Lightning AI Context

The project is maintained by Lightning AI, creators of PyTorch Lightning. Its Studios provide hosted environments for running LitGPT workflows, and the README points to Lightning's GPU cloud for compute.

## Features
- From-scratch implementations of 20+ LLMs
- Pretraining and continued pretraining
- Finetuning with LoRA, QLoRA, Adapter, and Adapter v2
- Quantization (4-bit, 8-bit, double quantization)
- Flash Attention and FSDP multi-GPU training
- TPU and XLA support
- Model evaluation on benchmarks such as MMLU and TruthfulQA
- Interactive chat command
- Model serving via web server
- YAML training recipes
- Export to other weight formats
- Python API

## Integrations
PyTorch Lightning Fabric, Hugging Face, Lightning AI Studios, EleutherAI Evaluation Harness, bitsandbytes, Weights & Biases, TensorBoard

## Platforms
LINUX, WEB, API, VSC_EXTENSION, JETBRAINS_PLUGIN, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Version
v0.5.13

## Links
- Website: https://lightning.ai
- Documentation: https://github.com/Lightning-AI/litgpt/blob/main/tutorials/0_to_litgpt.md
- Repository: https://github.com/Lightning-AI/litgpt
- EveryDev.ai: https://www.everydev.ai/tools/litgpt