Gauge CLI
Open-source CLI that runs coding agents against your Skills, CLIs, SDKs, MCPs and docs to find where they get stuck and test fixes.
At a Glance
MIT-licensed Gauge CLI that runs Agent Experience and Agent Preference evaluations locally with no Gauge account, subscription, or credits required.
Engagement
Available On
Listed Nov 2026
About Gauge CLI
Gauge CLI is the open-source command-line tool from Gauge, built for improving Agent Experience: how easily coding agents discover and use a product's Skills, CLIs, SDKs, MCPs and docs. It runs real agents in Docker sessions, captures where they get stuck, and lets your own coding agent test fixes before publishing. The full loop runs locally without a Gauge account, with an optional hosted Gauge for managed compute.
What It Is
Gauge CLI is a testing and measurement tool for product teams who want coding agents to succeed with their developer-facing artifacts. It supports two kinds of measurement: Agent Experience evals, which check whether an agent can complete a task with your product, and Agent Preference prompts, which examine which tools agents choose for a task. Agents supported include Claude Code, Codex, Pi and OpenCode, using native logins or API keys.
How the improvement loop works
You define realistic tasks with observable criteria, then Gauge starts fresh Docker sessions that inject the working-tree version of your configured inputs, such as Skills, SDKs, CLIs and files. Events from the agent harnesses are normalized into consistent transcripts. In local mode your own coding agent reads the evidence, judges the outcome against the criteria and saves structured findings. After editing the product or its guidance, you rerun the same eval and compare batches on completion, turns, duration and criterion judgments. Findings it describes include blocking interactive commands, deprecated API usage, unused Skills and incorrect claims of limitations.
Local or hosted
Local mode uses Docker and your own agent credentials, storing measurements in a private SQLite workspace. Hosted Gauge adds managed compute, automated judging, scheduled measurements, GitHub App PR checks, and support for docs, authenticated MCPs and APIs. Web fixtures let you test changes to content an agent encounters, such as a docs page, before publishing.
Setup path
Requirements are Node.js 22.13+, npm, Docker running Linux containers, and credentials for the agent under test. Install with npm, pull the runtime image, run init in your project, and install the Gauge skill so your coding agent can find the CLI's bundled instructions. The CLI collects anonymous command-usage telemetry by default, which can be disabled.
Community Discussions
Be the first to start a conversation about Gauge CLI
Share your experience with Gauge CLI, ask questions, or help others learn from your insights.
Pricing
Open-source CLI (local)
MIT-licensed Gauge CLI that runs Agent Experience and Agent Preference evaluations locally with no Gauge account, subscription, or credits required.
- MIT License
- Runs locally with Docker
- No Gauge account required
- Uses your own Claude Code or Codex subscription or API keys (billed separately by the provider)
- Gauge does not charge for local execution
Hosted Gauge Agents Free
Hosted Gauge Agents free plan with 500 credits at signup and a weekly refill to 100 credits; no card required.
- 500 credits at signup, then weekly refill to 100 credits
- Open models on OpenCode; first onboarding run uses Claude Code and Codex at no cost
- 1 new Optimization per week and 2 active runs at a time
- No AI Traffic or bring-your-own-key
Hosted Gauge Agents Pro
Hosted Gauge Agents Pro plan with 5,000 credits per month.
- 5,000 credits per month
- About 20 credits per Agent Experience run and about 12 credits per Agent Preference run
Hosted Gauge Agents Enterprise
Custom credit volume and pricing for teams needing more than 5,000 credits per month.
- Custom credit volume
Capabilities
Key Features
- Runs real coding agents in isolated Docker sessions against your product
- Agent Experience evals with criteria and observable outcomes
- Agent Preference studies on which tools agents choose
- Normalized transcripts across Claude Code, Codex, Pi and OpenCode
- Findings and judgments saved to a local SQLite workspace
- Batch comparison of before/after runs
- Tests working-tree Skills, npm CLIs and SDKs, and files
- Web fixtures to test docs changes before publishing
- Shared Markdown cases and gauge.json manifest for teams and CI
- Bundled agent instructions via gauge instructions
- Optional hosted execution with PR checks and scheduled runs
- JSON output for scripts and agent analysis
