# Terminal-Bench-Science

> An open-source benchmark for evaluating AI agents on expert-curated research workflows across life, physical, earth, mathematical, and engineering sciences.

Terminal-Bench-Science is an open academic benchmark designed to measure the frontier of AI agent capabilities on challenging, expert-curated scientific research workflows. It is hosted by Stanford University and the Laude Institute, and built by the Terminal-Bench and Harbor Framework team. The project is licensed under Apache 2.0 and released on GitHub, with version v0.1.0 published in August 2026.

## What It Is

Terminal-Bench-Science is a continuous, community-driven benchmark that evaluates AI agents on real scientific research tasks drawn from five broad domains: life sciences, physical sciences, earth sciences, mathematical sciences, and engineering sciences. Tasks are authored and reviewed by domain experts, and the benchmark evolves alongside frontier AI to create a feedback loop between scientific needs and AI development. It is part of the Terminal-Bench franchise and uses the Harbor Framework as its execution and evaluation infrastructure.

## Task Coverage and Structure

The benchmark currently contains 70 expert-curated tasks and is growing toward 100+. Each task is designed to be genuinely challenging for frontier AI agents and must produce outcomes that can be objectively verified in a terminal environment. The contribution flow follows a three-stage process:

- **Propose**: Submit a task idea via a structured proposal form for feedback and approval.
- **Build**: Implement the task and open a pull request following the contributing guide.
- **Review**: Tasks undergo automated checks (static validation, Docker build, oracle/nop validation, agent trials, cheat trials), parallel domain and technical review, and final bar-raiser approval before merge.

A public task dashboard tracks every proposal, pull request, and review status alongside domain coverage.

## Leaderboard and Evaluation

The benchmark publishes a live leaderboard ranking AI models and agents by resolution rate. As of the latest data, Anthropic's Opus 5 running via Claude Code leads with a 30.0% resolution rate, followed by GPT-5.6 Sol via Codex at 22.4%. The leaderboard also tracks token usage and cost per run, providing a Pareto view of performance versus efficiency. Evaluations are run using the Harbor CLI tool against sandboxed environments such as Modal or Daytona.

## Open-Source Architecture and Running the Benchmark

The benchmark is built on the Harbor Framework, an open-source agent harness. Users install Harbor via `uv tool install` or `pip install` and run tasks against the published dataset on Harbor Hub. The oracle solution can be run 5x to confirm task correctness in a given sandboxing environment. Agent and model selection is handled via CLI flags (`--agent`, `--model`), and concurrent runs are supported with `--n-concurrent`.

## Community and Research Partners

Terminal-Bench-Science is a large-scale scientific community effort. The project lists contributors from institutions including Stanford University, MIT, Caltech, Princeton, Oxford, ETH Zurich, and many others. Research partners include the Stanford AI Lab (SAIL), Stanford HAI, the Allen Institute for AI, and the NSF AI Institute for Foundations of Machine Learning (IFML). Industry sponsors listed by the project include Anthropic, Google, Modal, Moonshot AI, and others. The community coordinates via a dedicated `#tb-science` Discord channel and weekly project calendar meetings.

## Update: v0.1.0 Release

Version v0.1.0 was published on August 26, 2026, with a concept DOI (10.5281/zenodo.22110253) that always resolves to the latest release. The project is actively accepting pull requests for v0.2, with the README noting that early contribution is encouraged due to multi-round review requirements. The GitHub repository shows active development with the last push on August 29, 2026.

## Features
- Expert-curated scientific research tasks across 5 domains
- Live leaderboard with resolution rate, token usage, and cost tracking
- Continuous benchmark that evolves alongside frontier AI
- Automated task review pipeline (static checks, Docker build, oracle/nop, agent trials, cheat trials)
- Community contribution workflow (Propose → Build → Review)
- Public task dashboard for tracking proposals and reviews
- Harbor Framework integration for sandboxed agent execution
- Support for multiple AI agents and models via CLI flags
- DOI-versioned releases on Zenodo
- Apache 2.0 open-source license

## Integrations
Harbor Framework, Modal, Daytona, Claude Code, OpenAI Codex, Anthropic API, OpenAI API, Gemini API, Harbor Hub, Zenodo, Airtable (task proposal form)

## Platforms
WEB, CLI, API

## Pricing
Open Source

## Version
v0.1.0

## Links
- Website: https://www.terminal-bench-science.ai
- Documentation: https://harborframework.com/docs
- Repository: https://github.com/harbor-framework/terminal-bench-science
- EveryDev.ai: https://www.everydev.ai/tools/terminal-bench-science