S-Agent 2.0

S-Agent 2.0 is a research system from Ropedia that gives a general coding agent (OpenHands) a command-line perception toolkit for spatial reasoning over images and video. The agent acquires 3D scene evidence, keeps it in persistent scene memory, and reads measurements back in plain spatial language. The repository contains the task instruction, perception gateway, benchmark loaders, agent harness and GPU perception server; model weights are not included.

What It Is

S-Agent 2.0 is an agentic spatial reasoning framework rather than a trained model. It needs no training and no custom controller: the toolkit is a single executable (perception.py) the agent calls from its shell, and rendered images are files the agent opens with its ordinary file viewer. The paper and project page present it as closing an "evidence loop": acquire, remember, interpret.

How the Toolkit Works

Tool perception commands build the scene: recon creates a metric, gravity-aligned reconstruction of all frames, and segment lifts SAM 3 text-prompted masks into named 3D instances. Model perception commands such as add_instance let the agent register a point or an oriented observer implied by the question, and drop removes errors. Readout commands (list_instance, bev, distance, depth, side, elevation, motion, compass) return JSON with metric values, units, and a conclusion sentence that states the reference frame. Frozen models behind the toolkit include Depth Anything 3, Pi3MOS, GeoCalib and SAM 3.

Evaluation and Results

The project page reports results on MMSI-Bench, ReVSI, MindCube, SPAR-Bench and MMSI-Video with Qwen3.8-27B and Gemini-3.8-Flash backbones. It states 74.1 on MMSI-Bench and 90.5 on ReVSI with Gemini-3.8-Flash, and compares against SpatialClaw, pySpatial and SpaceTools. Ablations and runs with Codex and Claude Code as the coding agent are also reported, along with eleven browsable successful trajectories.

Setup and Limitations

Setup involves four services plus a task runner: a GPU perception server, a gateway, an LLM endpoint for the backbone, and the OpenHands runtime image, with a scripts/run.sh launcher for benchmarks. Documented limitations: grounding depends on the backbone, which may misplace an imagined observer or confuse object height with height above the floor, and reconstruction errors in scale or pose propagate to readouts. Code under agent/ and docs/ is Apache 2.0, while the perception component keeps NVIDIA's non-commercial license.