SkillsBench

SkillsBench is the first evaluation framework designed to measure how AI agent skills perform across diverse, expert-curated tasks spanning high-GDP-value domains. It provides a structured approach to benchmarking AI agents by evaluating them across three abstraction layers that mirror traditional computing systems: Skills, Agent Harness, and Models.

The framework enables researchers and developers to understand how domain-specific capabilities and workflows extend agent functionality, similar to how applications work on an operating system. SkillsBench includes a comprehensive task registry with 84 tasks across multiple domains including engineering, research, security, data visualization, and more.

• Three-Layer Evaluation Architecture provides a systematic approach to benchmarking AI agents across Skills (domain-specific capabilities), Agent Harness (execution environment), and Models (foundational AI models) layers.

• Comprehensive Task Registry includes 84 expert-curated tasks spanning diverse domains such as 3D geometry, control systems, BGP routing, citation verification, game mechanics, legal document processing, materials science, and seismology.

• Agent Performance Leaderboard tracks pass rates across multiple agent-model configurations with detailed metrics including confidence intervals and normalized gain calculations.

• Skills Impact Measurement quantifies the improvement in agent performance when using domain-specific skills versus without, showing gains of up to +23.3% in pass rates.

• Open Source Framework released under MIT License, allowing the community to contribute tasks, evaluate agents, and extend the benchmark.

• Multiple Agent Support evaluates various agent-model combinations including Gemini CLI, Claude Code, and Codex with different underlying models.

To get started with SkillsBench, visit the documentation to learn how to run evaluations on your coding agent's ability to use domain-specific skills. The framework supports community contributions, allowing developers to add new tasks to expand the benchmark's coverage across additional domains and use cases.