# Next.js Evals

> An open-source benchmark that measures AI coding agent performance on Next.js code generation and migration tasks, tracking success rate and execution time across models.

Next.js Evals is an open-source evaluation suite published by Vercel that benchmarks AI coding agents on real Next.js coding tasks. It is hosted at nextjs.org/evals and backed by the MIT-licensed `next-evals-oss` repository on GitHub. The project measures how well models like Claude, GPT, Gemini, Cursor, and others handle Next.js-specific code generation and migration challenges, reporting both success rate and average execution time.

## What It Is

Next.js Evals is a structured benchmark harness — not a general-purpose LLM leaderboard — focused specifically on Next.js competency. Each evaluation is a self-contained Next.js project containing a `PROMPT.md` task description and a `EVAL.ts` vitest assertion file that is withheld from the agent. Agents are scored on whether they can complete the task such that the hidden tests pass. The suite is powered by the `@vercel/agent-eval` npm package and is designed to run incrementally, skipping already-completed model/eval pairs.

## What the Evals Cover

The current eval set spans 20 distinct tasks, including:

- Pages Router → App Router migration (simple and hard variants)
- Avoiding anti-patterns like `fetch` in `useEffect`, `getServerSideProps`, redundant `useState`, and serial `await`
- Preferring Next.js primitives: `Link`, `Image`, Font, Server Actions, and the `use cache` directive
- Next.js 16-specific features: proxy/middleware, `connection()` for dynamic rendering, `after()` for post-response work, `updateTag()` for read-your-own-writes, and `revalidatePath`
- Auth patterns like `Forbidden` and async `cookies`/`headers`

## AGENTS.md and Documentation-Augmented Scoring

A notable feature of the leaderboard is a second scoring column showing results when agents are given access to an `AGENTS.md` file containing bundled Next.js documentation. The results page notes that this file provides "bundled Next.js documentation for AI coding agents." The gap between the base success rate and the AGENTS.md-augmented rate is often significant — for example, Kimi K2.5 jumps from 21% to 58% with documentation access — illustrating how much framework-specific context affects agent performance.

## How the Runner Works

The eval runner uses memoization so only new or missing (model, eval) pairs are executed on each run. Infrastructure or timeout failures are automatically deleted, ensuring only valid model results are exported. Results are exported to `agent-results.json` and then copied into the Next.js front-end site for publication at nextjs.org/evals. Adding a new eval or model is straightforward: create a directory or config file, and the runner automatically detects and fills in the missing pairs.

## Current Status and Activity

The repository was created in October 2025 and shows active development, with the last push recorded on July 18, 2026. The leaderboard page lists a last run date of July 17, 2026, and includes 25 model/agent combinations across agents including Claude Code, Codex, Cursor, Gemini CLI, and OpenCode. The project is MIT-licensed and has accumulated 291 stars and 39 forks on GitHub as of the last recorded update.

## Features
- AI coding agent benchmarking on Next.js tasks
- Success rate and execution time tracking per model
- AGENTS.md documentation-augmented scoring column
- 20+ self-contained Next.js eval tasks
- Incremental memoized eval runner (skips completed pairs)
- Automatic detection of new models and evals
- Vitest-based hidden assertion files
- Export to JSON for publishing
- Covers Pages Router to App Router migration
- Tests Next.js 16-specific APIs

## Integrations
Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Vercel, Next.js, Vitest, @vercel/agent-eval

## Platforms
WEB, API, CLI

## Pricing
Open Source

## Version
main

## Links
- Website: https://nextjs.org/evals
- Documentation: https://nextjs.org/docs/app/guides/ai-agents
- Repository: https://github.com/vercel/next-evals-oss
- EveryDev.ai: https://www.everydev.ai/tools/nextjs-evals