# LlamaFactory

> Open-source framework for fine-tuning 100+ large language models and vision-language models through a zero-code CLI and web UI.

LlamaFactory is an open-source training and fine-tuning platform for large language models and vision-language models, maintained under the GitHub handle hiyouga. It lets users fine-tune more than a hundred pretrained models locally through a command-line interface or the LLaMA Board web UI, without writing training code. The latest listed release is v0.9.5, published 2026-05-30.

## What It Is

LlamaFactory is a unified fine-tuning framework. Users prepare a dataset, choose a model and training method in a YAML config or the web UI, and run training, inference, evaluation, and export. The documentation lists supported model families including LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen, Gemma, DeepSeek, GLM, and Phi.

## Training Methods and Efficiency

Supported approaches include continued pre-training, supervised fine-tuning, reward modeling, PPO, DPO, KTO, ORPO, and SimPO. Precision and parameter-efficiency options include 16-bit full tuning, freeze tuning, LoRA, and 2- to 8-bit QLoRA via AQLM, AWQ, GPTQ, LLM.int8, HQQ, and EETQ. It also documents optimizers and tricks such as GaLore, BAdam, APOLLO, Adam-mini, Muon, OFT, DoRA, LoRA+, PiSSA, FlashAttention-2, Unsloth, and Liger Kernel. Distributed training is documented with DDP, DeepSpeed, FSDP, FSDP2, and Ray, plus a Megatron-core backend.

## Workflow and Outputs

The quickstart uses `llamafactory-cli train`, `chat`, and `export` to run LoRA fine-tuning, inference, and adapter merging. Models can be served through an OpenAI-style API with vLLM or SGLang workers. Experiments can be tracked with LlamaBoard, TensorBoard, Weights & Biases, MLflow, or SwanLab. Models and datasets can be downloaded from Hugging Face, ModelScope, or Modelers Hub.

## Hardware and Deployment Options

Installation is available from source, via Docker images for CUDA, Ascend NPU, and AMD ROCm, and with Windows-specific guidance. The README gives estimated memory requirements by method and model size, and documents Ascend NPU support.

## Features
- Zero-code CLI and LLaMA Board web UI for fine-tuning
- Support for 100+ LLMs and vision-language models
- Pre-training, SFT, reward modeling, PPO, DPO, KTO, ORPO, and SimPO
- Full, freeze, LoRA, and QLoRA tuning with 2-8 bit quantization
- Advanced optimizers such as GaLore, BAdam, APOLLO, Muon, and OFT
- Acceleration via FlashAttention-2, Unsloth, and Liger Kernel
- Distributed training with DeepSpeed, FSDP, and Ray
- OpenAI-style API serving with vLLM or SGLang
- LoRA merging and model export
- Experiment monitoring with TensorBoard, W&B, MLflow, and SwanLab
- Multimodal tasks including image, video, and audio understanding
- Ascend NPU and AMD ROCm support

## Integrations
Hugging Face, ModelScope, Modelers Hub, vLLM, SGLang, DeepSpeed, Weights & Biases, SwanLab, TensorBoard, MLflow, Gradio, Docker, Ollama, Unsloth, FlashAttention-2

## Platforms
WINDOWS, MACOS, LINUX, WEB, API, CLI

## Pricing
Open Source

## Version
v0.9.5

## Links
- Website: https://llamafactory.readthedocs.io/en/latest/
- Documentation: https://llamafactory.readthedocs.io/en/latest/
- Repository: https://github.com/hiyouga/LlamaFactory
- EveryDev.ai: https://www.everydev.ai/tools/llamafactory
