rl-env-use

rl-env-use is an open-source project from Browser Use that turns a one-line skill description into a verified reinforcement learning environment. Browser Use agents mine real cases from the web, and the tool pairs the resulting dataset with a hillclimbed system prompt for the model being trained. It is MIT licensed and installs from the GitHub repository.

What It Is

rl-env-use builds RL environments for any domain from real cases. It follows the workflow in Anthropic's "Automating eval design and hillclimbing", with one change: the tasks come from real cases that Browser Use agents find on the web. The README describes the goal as environments that frontier models do not saturate.

How the Workflow Runs

• Browser Use agents find source materials (such as a trial protocol or a 10-Q) and the outcome published later.
• Gold answers are computed by code, and tasks that a frontier-model panel solves are rewritten to be harder.
• Bad tasks are rejected, including leaked answers, memorized outcomes and ambiguous prompts.
• A model's system prompt is hillclimbed using a train/test split, a noise floor and paired confidence intervals.
• The dataset can be exported for an RL trainer or served in code with Env.

Example Results

The shipped example, envs/trial-forecast, contains 35 forecasts from 29 ClinicalTrials.gov trials whose results were posted after the models' training data. The README reports that a climbed Qwen 3.6 35B-A3B rose from 0.53 to 0.78 on held-out tasks, level with Opus 5.5. It notes that always answering 50% scores 0.75, so most of the gain is calibration.

Setup Path

It can be set up by asking Claude Code, Codex or OpenCode to read install.md and SKILL.md. Manual install needs Python 3.11+, optionally Node.js 18+ for HTML reports, and Browser Use and OpenRouter API keys. Commands such as rl-env-use create and rl-env-use climb run the build and prompt-climb steps.