SWE-Gym

SWE-Gym is a research environment for training language-model-based software engineering agents, released with the ICML 2025 paper "Training Software Engineering Agents and Verifiers with SWE-Gym" by authors from UC Berkeley, UIUC, CMU and Apple. The repository holds the code for the paper, while the dataset and models are published on Hugging Face.

What It Is

SWE-Gym is a training environment for agents that solve repository-level software engineering problems. According to the paper, it contains 2,438 real-world Python task instances, and the README describes them as drawn from 11 Python repositories. Each instance combines a codebase with an executable runtime environment, unit tests, and a natural-language task description. A Lite split of 234 instances is also provided.

How It Is Used

Researchers sample agent trajectories in the environment and fine-tune language models on them. The README documents experiments with the OpenHands and MoatlessTools agent scaffolds, including rejection-sampling fine-tuning for self-improvement. Verifiers can also be trained on agent trajectories. These verifiers pick the most promising solution through best-of-n selection at inference time.

Reported Results

The authors report up to 19% absolute gains in resolve rate on SWE-Bench Verified and Lite. Combining fine-tuned agents with verifiers, they report 32.0% on SWE-Bench Verified and 26.0% on Lite, which they describe as a new state of the art for open-weight SWE agents. The README lists downstream work that builds on SWE-Gym, such as SkyRL and OpenHands LM.

Reproducing the Work

The dataset and models are on the Hugging Face hub. Environment constants live in the SWE-Bench-Fork repository, and pre-built Docker images for each instance are on Docker Hub. The repository's docs/OpenHands.md and docs/MoatlessTools.md describe how to reproduce the training and inference-time experiments.