RoboLab

RoboLab is an NVIDIA Research simulation benchmark for evaluating task-generalist robot manipulation policies. It is built on NVIDIA Isaac Lab and ships 120 manipulation tasks with language instructions and automated success detection. The code is open source under Apache 2.0, with the latest release listed as v0.3.1.

What It Is

RoboLab is a robot- and policy-agnostic benchmarking platform. According to the project page, it is designed for policies trained on real-world data that are evaluated directly in simulation, without co-training on simulation data, to measure task generalization. Tasks are not tied to a specific robot embodiment, so any Isaac Lab-compatible robot can be used.

How the Workflow Operates

Users build scenes by physically arranging objects in simulation, then create tasks by attaching a language instruction to a scene. Environments are generated by specifying the robot, policy, action and observation configs, and scene variations. Policies run through a server-client architecture: the model runs as a standalone server and RoboLab connects through a lightweight inference client. Pi0.5 via OpenPI is the documented quick-start example. Episodes can run in parallel across multiple environments, and scene variations cover lighting, camera pose, backgrounds, textures, and shadows.

AI-Assisted Scene and Task Generation

The /robolab-scenegen and /robolab-taskgen Claude Code skills generate USD scenes and task files from natural-language descriptions. The scene generator picks from a library of 312 objects and uses a predicate solver for collision-free layouts. The task generator produces default, vague, and specific instruction variants.

Benchmark and Analysis

The RoboLab benchmark covers three competency axes: visual, relational, and procedural. A self-contained web dashboard lets users browse scenes and tasks, replay episode videos, compare results across experiments, and inspect failure events. The project page also describes a sensitivity analysis using Neural Posterior Estimation to quantify which environment parameters most affect policy success.

Setup Path

Installation uses uv and system ffmpeg, with a choice of IsaacSim 5.0 / IsaacLab 2.2.0 (default) or IsaacSim 5.1 / IsaacLab 2.3.2.post1. Python 3.11 and Ubuntu 22.04+ are required. An NVIDIA RTX GPU is required, with 48GB+ VRAM recommended, and the documentation lists about 8 GB of disk space.