Meta-Harness

Meta-Harness is an open-source framework from the Stanford IRIS lab for automatically searching over model harnesses: the code around a fixed base model that decides what to store, retrieve, and show while the model works. The repository contains the reusable framework, an onboarding flow for new domains, and two reference experiments from the accompanying paper.

What It Is

Meta-Harness treats harness engineering as an optimization problem. A proposer agent reads a filesystem containing the source code, scores, and execution traces of all prior candidate harnesses, then proposes a new harness. Each candidate is evaluated on held-out tasks, its logs are written back to the filesystem, and the loop repeats. The examples use Claude Code as the proposer, and the project says it can be swapped by adapting the provided wrapper scripts, which must log proposer interactions.

How the Search Loop Works

The project page contrasts this with prior text-optimization methods that compress history into summaries, scalar scores, or sliding windows. Meta-Harness instead lets the proposer inspect raw history with standard tools such as grep and cat, so it can trace failures to specific harness decisions. The authors report up to 10M tokens of diagnostic context per step.

Reference Experiments

The repo includes a text classification memory-system search and a Terminal-Bench 2.0 scaffold evolution example. The authors report that on text classification, the best discovered harness reached 48.6% versus ACE's 40.9% with fewer context tokens. They also report a retrieval harness for math reasoning that improved average accuracy across five held-out models, and 76.4% on TerminalBench-2 with Claude Opus 4.6 and 37.6% with Claude Haiku 4.5. The optimized Terminal-Bench 2 harness is in a separate artifact repository.

Applying It to a New Domain

Users point a coding assistant at ONBOARDING.md and hold a conversation that produces a domain_spec.md containing an implementation and evaluation plan. Quick-start commands use uv to run the text classification and Terminal-Bench 2 examples. The authors describe the code as a cleaned-up version of the paper code that has only been verified to run.