# WebsiteBench

> An open-source benchmark that tests whether AI agents can rebuild production-level websites through browser exploration.

WebsiteBench is a benchmark from TIGER-AI-Lab that evaluates AI agents on a hard task: given a production-level website they can explore through a browser, they must reverse-engineer a full-stack version that matches its appearance and behavior as closely as possible. It comprises 70 websites and 46,916 hidden test cases, with a public leaderboard comparing eight models.

## What It Is

WebsiteBench is an agent-evaluation benchmark for website reconstruction. Agents explore a reference site via a browser, then build their own implementation. Tasks are distributed through Harbor Hub and Hugging Face, and can be fetched with the websitebench PyPI package. The code, task definitions, and verifier are released under Apache-2.0, while the reference websites reproduce third-party sites for non-commercial research only.

## How Scoring Works

Each task has 302 to 2,016 hidden test cases. A case passes when its functional checks pass and every visual checkpoint reaches a normalized SSIM of 0.99. A task's reward is the share of cases that pass. The leaderboard reports overall, request, browser, and system pass rates alongside cost, token use, and turns per task.

## Running the Benchmark

Users install Docker and the Harbor framework, then run an agent (for example Codex with a chosen model) across all 70 tasks or a single task using the harbor CLI. A reference oracle agent can check a task's reference solution. Each trial's reward is written to a verifier file in the job output directory. The project site also provides pages for tasks, evaluation, and analysis, plus leaderboard views as table, Pareto, and waffle charts with CSV download.

## Data Handling Note

Task definitions carry a harbor-canary GUID, and the authors ask that documents containing it be excluded from training data.

## Features
- 70 website reconstruction tasks with 46,916 hidden test cases
- Functional and visual scoring using normalized SSIM of 0.99
- Public leaderboard with overall, request, browser, and system pass rates
- Cost, token, and turn usage metrics per model
- Leaderboard CSV, PNG, and SVG export
- Task download via the websitebench PyPI package
- Runs through the Harbor framework with Docker
- Oracle reference solution check per task

## Integrations
Harbor, Docker, Hugging Face, PyPI

## Platforms
LINUX, WEB, CLI

## Pricing
Open Source

## Links
- Website: https://website-bench.com
- Documentation: https://github.com/TIGER-AI-Lab/WebsiteBench/blob/main/docs/README.md
- Repository: https://github.com/TIGER-AI-Lab/WebsiteBench
- EveryDev.ai: https://www.everydev.ai/tools/websitebench
