# Show-Harness

> An embodied harness that lets vision-language model agents control robots through a compact semantic action interface.

Show-Harness is an open-source embodied harness from Show Lab at the National University of Singapore. It lets a vision-language model (VLM) "play" a robot by emitting discrete semantic action units that embodiment-specific interpreters ground into motion. The repository includes the harness, GUMI demonstration collectors, a plugin suite, and a training pipeline, released under Apache-2.0.

## What It Is

Show-Harness is a robot-control framework in which a VLM reasons over parameter-free action units such as MV_LEFT, GRASP, RELEASE and ROTATE_CW. The interpreter supplies the metric magnitude for each unit, so the same vocabulary can drive a Franka arm, an AgileX Piper (single and dual arm), Isaac Lab and LIBERO simulators. Switching embodiments changes only the interpreter and its primitives config; the model-facing prompts stay the same.

## Two Modes, One Interface

In zero-shot mode a frontier VLM operates the full plugin harness with no robot-specific training. In fine-tuned mode a small open VLM, trained on GUMI demonstrations, emits one action token per step without a planner. The authors state that fine-tuning needs only a few GPU-hours. Released LoRA adapters cover Qwen3.5 (0.8B, 2B, 4B, 9B) and Gemma 4 E4B backbones, plus a simulation policy, alongside a demonstration dataset on Hugging Face.

## GUMI and Data Collection

GUMI (GUI Manipulation Interface) maps each action unit to a key or button. A human or a GUI-driving agent demonstrates a task in the browser, and every step is recorded as an observation-action training pair. A synthetic tabletop mode lets users try it without hardware.

## Plugins and Setup

Plugins mount on perception, reasoning, or action stages of the loop and are each toggled by one boolean. Examples include Multi-View Guidance, Proprioception, Subtask Planning, Action Chunking, Adaptive Step, Action History and Failure Recovery. Setup uses separate Python virtual environments for the harness, local VLM serving via vLLM, and simulators. A real-robot run needs site config, an API key or a served VLM, a calibrated safety floor, and a preflight check.

## Reported Results

The project page reports that zero-shot Gemini-3.1 Pro and a fine-tuned Qwen3.5-2B agent outperform VLA, VLA-centric agent and code-as-policy baselines across tasks, environments and embodiments.

## Features
- Semantic action space of parameter-free units interpreted per embodiment
- Zero-shot mode with frontier VLMs
- Fine-tuned mode with small open VLMs emitting one action token per step
- GUMI browser-based demonstration collection without teleoperation hardware
- Toggleable plugin suite for perception, reasoning and action
- Released LoRA adapters and demonstration dataset
- LoRA fine-tuning pipeline built on LLaMA-Factory
- Preflight check and safety floor calibration for real robots

## Integrations
Franka, AgileX Piper, Isaac Lab, LIBERO, ManiSkill, vLLM, LLaMA-Factory, Polymetis, Qwen3.5, Gemma 4, InternVL3.5, Gemini

## Platforms
LINUX, API, CLI

## Pricing
Open Source

## Links
- Website: https://showlab.github.io/Show-Harness/
- Documentation: https://github.com/showlab/Show-Harness/tree/main/docs
- Repository: https://github.com/showlab/Show-Harness
- EveryDev.ai: https://www.everydev.ai/tools/show-harness