Scope

Scope is an open-source agentic experience evaluation platform from Microsoft, labeled a research preview on GitHub. It lets teams give coding agents the same task, define success criteria, and inspect the evidence behind each result through a browser Portal, CLI, or REST API. The homepage lists GitHub Copilot and Claude Code as supported today, with OpenAI Codex and OpenCode planned.

What It Is

Scope evaluates how AI agents use a product and approach real tasks. A user writes a task prompt, defines criteria, and runs agents against it repeatedly to compare outcomes rather than relying on a generic leaderboard.

How evaluations work

Criteria are organized into ordered gates, such as requirements, build, and test. Inside each gate, criteria form a dependency graph, so if a prerequisite check fails, dependent checks are skipped. Agent configuration (model, skills, tools) is saved in versioned profiles so experiments can be rerun from a known setup. Runs can be followed through live logs, with iterations and criterion feedback available for inspection, and results cover gates passed, quality, tokens, and duration.

Deployment model

Scope runs on Kubernetes, using containerized agent workers, KEDA queue-driven scaling, and Flux GitOps deployments. Self-hosting guides cover Helm deployment. The source is released under the MIT License.