Show-Harness

Show-Harness is an open-source embodied harness from Show Lab at the National University of Singapore. It lets a vision-language model (VLM) "play" a robot by emitting discrete semantic action units that embodiment-specific interpreters ground into motion. The repository includes the harness, GUMI demonstration collectors, a plugin suite, and a training pipeline, released under Apache-2.0.

What It Is

Show-Harness is a robot-control framework in which a VLM reasons over parameter-free action units such as MV_LEFT, GRASP, RELEASE and ROTATE_CW. The interpreter supplies the metric magnitude for each unit, so the same vocabulary can drive a Franka arm, an AgileX Piper (single and dual arm), Isaac Lab and LIBERO simulators. Switching embodiments changes only the interpreter and its primitives config; the model-facing prompts stay the same.

Two Modes, One Interface

In zero-shot mode a frontier VLM operates the full plugin harness with no robot-specific training. In fine-tuned mode a small open VLM, trained on GUMI demonstrations, emits one action token per step without a planner. The authors state that fine-tuning needs only a few GPU-hours. Released LoRA adapters cover Qwen3.5 (0.8B, 2B, 4B, 9B) and Gemma 4 E4B backbones, plus a simulation policy, alongside a demonstration dataset on Hugging Face.

GUMI and Data Collection

GUMI (GUI Manipulation Interface) maps each action unit to a key or button. A human or a GUI-driving agent demonstrates a task in the browser, and every step is recorded as an observation-action training pair. A synthetic tabletop mode lets users try it without hardware.

Plugins and Setup

Plugins mount on perception, reasoning, or action stages of the loop and are each toggled by one boolean. Examples include Multi-View Guidance, Proprioception, Subtask Planning, Action Chunking, Adaptive Step, Action History and Failure Recovery. Setup uses separate Python virtual environments for the harness, local VLM serving via vLLM, and simulators. A real-robot run needs site config, an API key or a served VLM, a calibrated safety floor, and a preflight check.

Reported Results

The project page reports that zero-shot Gemini-3.1 Pro and a fine-tuned Qwen3.5-2B agent outperform VLA, VLA-centric agent and code-as-policy baselines across tasks, environments and embodiments.