Next.js Evals

Next.js Evals is an open-source evaluation suite published by Vercel that benchmarks AI coding agents on real Next.js coding tasks. It is hosted at nextjs.org/evals and backed by the MIT-licensed next-evals-oss repository on GitHub. The project measures how well models like Claude, GPT, Gemini, Cursor, and others handle Next.js-specific code generation and migration challenges, reporting both success rate and average execution time.

What It Is

Next.js Evals is a structured benchmark harness — not a general-purpose LLM leaderboard — focused specifically on Next.js competency. Each evaluation is a self-contained Next.js project containing a PROMPT.md task description and a EVAL.ts vitest assertion file that is withheld from the agent. Agents are scored on whether they can complete the task such that the hidden tests pass. The suite is powered by the @vercel/agent-eval npm package and is designed to run incrementally, skipping already-completed model/eval pairs.

What the Evals Cover

The current eval set spans 20 distinct tasks, including:

• Pages Router → App Router migration (simple and hard variants)
• Avoiding anti-patterns like fetch in useEffect, getServerSideProps, redundant useState, and serial await
• Preferring Next.js primitives: Link, Image, Font, Server Actions, and the use cache directive
• Next.js 16-specific features: proxy/middleware, connection() for dynamic rendering, after() for post-response work, updateTag() for read-your-own-writes, and revalidatePath
• Auth patterns like Forbidden and async cookies/headers

AGENTS.md and Documentation-Augmented Scoring

A notable feature of the leaderboard is a second scoring column showing results when agents are given access to an AGENTS.md file containing bundled Next.js documentation. The results page notes that this file provides "bundled Next.js documentation for AI coding agents." The gap between the base success rate and the AGENTS.md-augmented rate is often significant — for example, Kimi K2.5 jumps from 21% to 58% with documentation access — illustrating how much framework-specific context affects agent performance.

How the Runner Works

The eval runner uses memoization so only new or missing (model, eval) pairs are executed on each run. Infrastructure or timeout failures are automatically deleted, ensuring only valid model results are exported. Results are exported to agent-results.json and then copied into the Next.js front-end site for publication at nextjs.org/evals. Adding a new eval or model is straightforward: create a directory or config file, and the runner automatically detects and fills in the missing pairs.

Current Status and Activity

The repository was created in October 2025 and shows active development, with the last push recorded on July 18, 2026. The leaderboard page lists a last run date of July 17, 2026, and includes 25 model/agent combinations across agents including Claude Code, Codex, Cursor, Gemini CLI, and OpenCode. The project is MIT-licensed and has accumulated 291 stars and 39 forks on GitHub as of the last recorded update.