# Verifiers

> An open-source Python library by Prime Intellect for creating environments to train and evaluate LLMs using reinforcement learning.

Verifiers is an open-source Python library developed by Prime Intellect AI for building environments that train and evaluate large language models (LLMs) via reinforcement learning (RL). Originally created by Will Brown, the project is hosted on GitHub under the MIT License and integrates tightly with Prime Intellect's broader platform, including the Environments Hub, the `prime-rl` training framework, and Hosted Training infrastructure.

## What It Is

Verifiers provides a structured way to define self-contained RL environments for LLMs. Each environment bundles three core components: a **dataset** of task inputs, a **harness** for the model (covering tools, sandboxes, and context management), and a **rubric** (reward function) to score model performance. These environments can be used for RL training, capability evaluation, synthetic data generation, and agent harness experimentation.

## Architecture and Core Concepts

The library supports both a legacy single-turn API and a newer v1 Taskset/Harness API introduced in v0.1.14. Key abstractions include:

- **Taskset**: Defines the dataset rows, reward functions, and split configuration for a task.
- **Harness**: Manages how the model interacts with tools, sandboxes, and multi-turn context.
- **Rubric**: Scores model completions using async reward functions with configurable weights.
- **Environment types**: `SingleTurnEnv`, `MultiTurnEnv`, `RLMEnv`, `CliAgentEnv`, `BrowserEnv`, and `OpenEnv` integrations cover a wide range of interaction protocols.

Environments are self-contained Python modules installable via `uv`, and the same package is used for both evals and RL training runs.

## Workflow and Tooling

The `prime` CLI drives the development workflow end-to-end:

- `prime lab setup` scaffolds a workspace with recommended configs for training, eval, and prompt optimization (GEPA).
- `prime env init` generates a new environment template.
- `prime eval run` executes local evaluations against any OpenAI-compatible endpoint.
- `prime env push` publishes environments to the public Environments Hub.
- `vf-tui` provides a terminal UI for reviewing eval results.

Configuration is TOML-based, with separate config shapes for RL training (model, batch size, rollouts) and environment-specific options (taskset split, harness max turns, reward weights).

## Environments Hub Integration

Verifiers connects to the Prime Intellect [Environments Hub](https://app.primeintellect.ai/dashboard/environments), a shared registry where community-built environments can be published, discovered, and installed. Environments from the hub (e.g., `primeintellect/math-python`) can be installed directly into a local project and used for both evaluation and training without modification.

## Update: v0.1.14 and Active Development

The project has seen rapid release cadence since its creation in January 2025. As of the latest available data:

- **v0.1.15.dev7** is the most recent pre-release (published May 15, 2026).
- **v0.1.14** (released May 7, 2026) introduced the v1 Taskset/Harness API, shared eval/training config shapes, model-family starter configs, OpenAI Responses and renderer-backed clients, per-turn timing, GEPA prompt artifacts, Lean guard markers, and infrastructure hardening.
- Earlier notable releases include v0.1.12 (RLMEnv improvements, multi-worker env server), v0.1.11 (unified client stack, eval TUI), v0.1.10 (OpenEnv and BrowserEnv integrations), and v0.1.8 (trajectory-based rollout tracking for token-in/token-out training).

The repository reports 4,123 stars and 551 forks as of the last recorded update, according to GitHub metadata.

## Features
- RL environment creation for LLM training and evaluation
- Taskset/Harness v1 API for reusable environment components
- SingleTurnEnv, MultiTurnEnv, RLMEnv, CliAgentEnv, BrowserEnv, OpenEnv support
- Rubric-based reward functions with async scoring
- prime CLI for workspace setup, env init, eval run, and env push
- Environments Hub integration for publishing and installing environments
- TOML-based training and eval configuration
- OpenAI-compatible API endpoint support
- Per-turn timing and token tracking
- GEPA prompt optimization support
- vf-tui terminal UI for eval result review
- Sandbox lifecycle management
- Trajectory-based rollout tracking
- Pass@k and ablation sweep support
- Integration with prime-rl training framework

## Integrations
prime-rl, Prime Intellect Hosted Training, Prime Intellect Environments Hub, Prime Inference, OpenAI API (compatible endpoints), OpenCode, OpenEnv, BrowserEnv, Harbor task directories, uv (Python package manager)

## Platforms
WEB, API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Version
v0.1.15.dev7

## Links
- Website: https://github.com/PrimeIntellect-ai/verifiers
- Documentation: https://docs.primeintellect.ai/verifiers
- Repository: https://github.com/PrimeIntellect-ai/verifiers
- EveryDev.ai: https://www.everydev.ai/tools/verifiers