ARC-AGI-2

ARC-AGI-2 is the task dataset for the second version of the Abstraction and Reasoning Corpus, published by the ARC Prize Foundation under the Apache License 2.0. It contains 1,000 public training tasks and 120 public evaluation tasks stored as JSON grid puzzles. It is intended for both humans and AI systems that aim to emulate human-like general fluid intelligence.

What It Is

ARC-AGI-2 is a benchmark dataset that can be treated as an artificial general intelligence benchmark, a program synthesis benchmark, or a psychometric intelligence test. Each task shows a few demonstration input/output grid pairs, and the test-taker must construct the correct output grid for the test input(s). Grids are rectangular matrices of integers 0 to 9 (visualized as colors), from 1x1 up to 30x30.

Dataset Structure

The data directory has two subsets: training (1,000 tasks, combining ARC-AGI-1 tasks with new ones) and evaluation (120 tasks). Training tasks demonstrate the task format and Core Knowledge priors and can be used to train models. Evaluation tasks are for testing models that have not seen them; the README reports average human performance of 66% on them in its test sample, and each was solved by at least two people in two attempts or fewer.

Two private test sets are not in the repository: a semi-private set for remotely hosted commercial models and a fully private set for the ARC Prize competition. Both are calibrated to the same difficulty as the public evaluation set.

Scoring and Testing Interface

A task counts as solved only when the output grid exactly matches for all test inputs, including output dimensions, with 2 trials per test input. Tasks can be tried in the browser on the ARC Prize play page or with the ARC-AGI-1 testing interface, which offers grid resize, copy-from-input, reset, color edit, select, copy/paste, and floodfill tools, plus a Submit button for answer checking.