# verl

> Open-source RL training library for large language models implementing the HybridFlow framework, with support for PPO, GRPO and multiple training and rollout backends.

verl is an open-source reinforcement learning training library for large language models, initiated by the ByteDance Seed team and maintained by the verl community. It is the open-source version of the HybridFlow RLHF framework paper and is released under the Apache License 2.0. The latest release listed on the repository is v0.9.1.

## What It Is

verl is a post-training library for running supervised fine-tuning and reinforcement learning on LLMs and vision-language models. Its hybrid-controller programming model lets users express RL dataflows such as GRPO and PPO in a few lines of code. It decouples computation and data dependencies so it can plug into existing LLM infrastructure, and it supports flexible placement of models onto different sets of GPUs.

## Training and Rollout Stack

FSDP, FSDP2 and Megatron-LM are supported for training, while vLLM, SGLang and HF Transformers handle rollout generation. The README lists algorithms including PPO, GRPO, GSPO, ReMax, REINFORCE++, RLOO, PRIME and DAPO, with model-based and function-based (verifiable) rewards. It also covers multi-turn rollouts with tool calling, LoRA RL, and multi-modal RL. According to the README, it scales to 671B models and hundreds of GPUs using expert parallelism, and a 3D-HybridEngine reduces memory redundancy when switching between training and generation. Experiment tracking works with wandb, swanlab, mlflow and tensorboard.

## Hardware and Ecosystem

The README documents support for NVIDIA, AMD ROCm GPUs (MI300X, MI325X, MI355X) and Ascend hardware. The verl-project organization also publishes related projects built on verl, including verl-vla for vision-language-action post-training, VeRL-Omni for diffusion and omni-modal RL, verl-SpeCo for speculative decoding co-training, uni-agent for training long-horizon agents, and RL-Insight for observability. A project blog covers releases and case studies.

## Getting Started

The documentation on Read the Docs provides installation and quickstart guides, a programming guide, PPO and GRPO algorithm pages, data and reward-function preparation steps, and a performance tuning guide. Algorithm recipes live in a separate verl-recipe repository.

## Features
- Hybrid-controller programming model for RL dataflows
- PPO, GRPO, GSPO, ReMax, REINFORCE++, RLOO, PRIME, DAPO algorithms
- FSDP, FSDP2 and Megatron-LM training backends
- vLLM, SGLang and HF Transformers rollout generation
- Supervised fine-tuning
- Model-based and verifiable function-based rewards
- Multi-turn rollouts with tool calling
- Vision-language model and multi-modal RL
- LoRA RL support
- Scales to 671B models via expert parallelism
- 3D-HybridEngine actor resharding
- Experiment tracking with wandb, swanlab, mlflow and tensorboard
- NVIDIA, AMD ROCm and Ascend hardware support

## Integrations
FSDP, FSDP2, Megatron-LM, vLLM, SGLang, Hugging Face Transformers, ModelScope Hub, Ray, wandb, swanlab, mlflow, tensorboard, DeepSpeed Ulysses, Liger-kernel

## Platforms
LINUX, WEB, API, JETBRAINS_PLUGIN, DEVELOPER_SDK

## Pricing
Open Source

## Version
v0.9.1

## Links
- Website: https://verl-project.github.io
- Documentation: https://verl.readthedocs.io/en/latest/
- Repository: https://github.com/verl-project/verl
- EveryDev.ai: https://www.everydev.ai/tools/verl