# rLLM

> Open-source framework for training language agents with reinforcement learning on any harness, sandbox, backend, and benchmark.

rLLM is an open-source framework for training language agents with reinforcement learning, developed as part of Berkeley Sky Computing Lab. It lets users bring an existing agent, score its outputs with a reward function, and update the underlying model, using the same agent code for evaluation and training. The latest listed release is v0.3.0-pre, published 2026-04-30.

## What It Is

rLLM is a post-training framework for agentic programs. Users can wrap agent code built with LangGraph, the OpenAI Agents SDK, or plain `openai.OpenAI` using the `@rllm.rollout` decorator, define an evaluator as a reward function, and pass both to an `AgentTrainer`. A command-line interface covers model provider setup, benchmark evaluation (`rllm eval`), RL training (`rllm train`), and agent scaffolding (`rllm init`).

## How the Training Pipeline Works

The documented pipeline is: run the agent, collect traces, compute rewards, update the model. A model gateway captures LLM calls, including token IDs and logprobs, and structures them into Episodes (one task) containing Trajectories (one agent run) made of Steps (one LLM call). A workflow engine runs parallel agent instances to collect rollouts, a transform pipeline groups trajectories for advantage computation, and a training backend performs the policy update.

## Backends, Harnesses, and Benchmarks

Three training backends share one API and are switched with a flag: `verl` for distributed multi-GPU training, `tinker` for single-machine setups, and `fireworks` for the Fireworks platform. The README lists 10+ CLI harnesses (including Claude Code, Codex, and opencode) plus Harbor-compatible task directories. Rollouts can run in Docker, Daytona, Modal, or local sandboxes. It integrates 60+ benchmarks spanning math, code, QA, search, VLM, and agentic tasks, and supports methods such as GRPO, REINFORCE, RLOO, SFT, and on-policy distillation.

## Setup Path

rLLM requires Python 3.11 or newer and installs via pip or uv, with optional extras for the verl and Fireworks backends; source and Docker installs are also documented. It is released under the Apache License 2.0. The project says models trained with it include rLLM-FinQA-4B, DeepSWE, DeepCoder-14B, and DeepScaleR-1.5B.

## Features
- Reinforcement learning training for AI agents
- @rllm.rollout decorator that traces LLM calls automatically
- CLI for model setup, eval, train, and agent scaffolding
- 60+ integrated benchmarks
- Support for 10+ CLI harnesses and Harbor-compatible tasks
- Sandboxes: Docker, Daytona, Modal, local
- Training backends: verl, tinker, fireworks
- Algorithms: GRPO, REINFORCE, RLOO, SFT, on-policy distillation
- LoRA training and VLM support
- Multi-agent workflows
- rLLM UI for observability

## Integrations
LangGraph, SmolAgents, Strands, OpenAI Agents SDK, Google ADK, verl, Tinker, Fireworks AI, Docker, Daytona, Modal, HuggingFace, LiteLLM, AWS Bedrock AgentCore, Claude Code, Codex

## Platforms
API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Version
v0.3.0-pre

## Links
- Website: https://docs.rllm-project.com
- Documentation: https://docs.rllm-project.com
- Repository: https://github.com/rllm-org/rllm
- EveryDev.ai: https://www.everydev.ai/tools/rllm