torchtune

torchtune is an open-source, native-PyTorch library for fine-tuning and post-training large language models, published under the meta-pytorch GitHub organization. It provides training recipes, model implementations, and a command-line tool for running them from YAML configs. The README states that torchtune is no longer actively maintained and that development wound down in 2025.

What It Is

torchtune is a PyTorch library for authoring, post-training, and experimenting with LLMs. Users pick a recipe and a config, then launch it with the tune CLI (for example tune run lora_finetune_single_device --config llama3_1/8B_lora_single_device). Configs can be overridden from the command line or copied locally with tune cp and edited.

Recipes and Supported Models

The recipes cover supervised finetuning (full, LoRA and QLoRA), knowledge distillation, DPO, PPO, GRPO, and quantization-aware training. Support varies by recipe across single-device, multi-device, and multi-node setups. Documented model families include Llama (up to Llama 4 Scout), Gemma2, Mistral, Phi3 and Phi4, and Qwen2, Qwen2.5 and Qwen3. Model weights can be downloaded from the Hugging Face Hub or Kaggle Hub with tune download.

Memory Efficiency and Devices

The README documents memory and speed levers including packed datasets, torch.compile, chunked cross entropy, activation checkpointing and offloading, fused optimizer steps, and 8-bit AdamW. It also lists LoRA and QLoRA for lower memory use. Recipes can run on NVIDIA GPU, Intel XPU, AMD ROCm, Apple MPS, and Ascend NPU. Datasets can be instruct, chat, preference, multimodal, or text-completion styles.

Ecosystem Integrations

torchtune works with Hugging Face Hub and Datasets, EleutherAI's LM Eval Harness, PyTorch FSDP2, torchao, Weights & Biases, Comet, ExecuTorch, bitsandbytes, and PEFT.

Current Status

The README notes that torchtune is no longer actively maintained. The latest listed release is v0.6.1, published April 7, 2025.