EveryDev.ai
Subscribe
Home
Tools

5,923+ AI tools

  • Featured
  • Gems
  • Trending
  • New
  • Rate tools
  • Compare
  • Arena
Categories
  • Agents2992
  • Coding2260
  • Infrastructure1068
  • Projects824
  • MCP795
  • Research744
  • Analytics567
  • Marketing527
  • Design489
  • Testing476
  • Data442
  • Security417
  • Integration288
  • Communication286
  • Prompts284
  • Extensions239
  • Voice214
  • DevOps191
  • Commerce184
  • Web128
  • Learning123
  • Finance44
AI Tools by Topic
  • AI Coding Assistants
  • Agent Frameworks
  • MCP Servers
  • AI Prompt Tools
  • Vibe Coding Tools
  • AI Design Tools
  • AI Database Tools
  • AI Website Builders
  • AI Testing Tools
  • LLM Evaluations
Follow Us
  • X / Twitter
  • LinkedIn
  • Reddit
  • Discord
  • Threads
  • Bluesky
  • Mastodon
  • YouTube
  • GitHub
  • Instagram
Get Started
  • Users
  • Rate Tools
  • About
  • Editorial Standards
  • Corrections & Disclosures
  • Community Guidelines
  • Advertise
  • Contact Us
  • Newsletter
  • Submit a Tool
  • Start a Discussion
  • Write A Blog
  • Share A Build
  • Terms of Service
  • Privacy Policy
Explore with AI
  • ChatGPT
  • Gemini
  • Claude
  • Grok
  • Perplexity
Agent Experience
  • llms.txt
Theme
With AI, Everyone is a Dev. EveryDev.ai ©
    1. Home
    2. Tools
    3. Gauge CLI
    Gauge CLI icon

    Gauge CLI

    Automated Testing
    Featured

    Open-source CLI that runs coding agents against your Skills, CLIs, SDKs, MCPs and docs to find where they get stuck and test fixes.

    Visit Website

    At a Glance

    Pricing
    Open Source
    Free tier available

    MIT-licensed Gauge CLI that runs Agent Experience and Agent Preference evaluations locally with no Gauge account, subscription, or credits required.

    Hosted Gauge Agents Pro: $499/mo
    Hosted Gauge Agents Enterprise: Custom/contact

    Engagement

    Available On

    Windows
    macOS
    Linux
    Web
    API

    Resources

    WebsiteDocsGitHubllms.txt

    Topics

    Automated TestingLLM EvaluationsCommand Line Assistants

    Alternatives

    CortadCrabboxOctrafic
    Developer
    Gauge Technologies, Inc.San Francisco, CAEst. 2024$2.4M raised

    Listed Nov 2026

    About Gauge CLI

    Gauge CLI is the open-source command-line tool from Gauge, built for improving Agent Experience: how easily coding agents discover and use a product's Skills, CLIs, SDKs, MCPs and docs. It runs real agents in Docker sessions, captures where they get stuck, and lets your own coding agent test fixes before publishing. The full loop runs locally without a Gauge account, with an optional hosted Gauge for managed compute.

    What It Is

    Gauge CLI is a testing and measurement tool for product teams who want coding agents to succeed with their developer-facing artifacts. It supports two kinds of measurement: Agent Experience evals, which check whether an agent can complete a task with your product, and Agent Preference prompts, which examine which tools agents choose for a task. Agents supported include Claude Code, Codex, Pi and OpenCode, using native logins or API keys.

    How the improvement loop works

    You define realistic tasks with observable criteria, then Gauge starts fresh Docker sessions that inject the working-tree version of your configured inputs, such as Skills, SDKs, CLIs and files. Events from the agent harnesses are normalized into consistent transcripts. In local mode your own coding agent reads the evidence, judges the outcome against the criteria and saves structured findings. After editing the product or its guidance, you rerun the same eval and compare batches on completion, turns, duration and criterion judgments. Findings it describes include blocking interactive commands, deprecated API usage, unused Skills and incorrect claims of limitations.

    Local or hosted

    Local mode uses Docker and your own agent credentials, storing measurements in a private SQLite workspace. Hosted Gauge adds managed compute, automated judging, scheduled measurements, GitHub App PR checks, and support for docs, authenticated MCPs and APIs. Web fixtures let you test changes to content an agent encounters, such as a docs page, before publishing.

    Setup path

    Requirements are Node.js 22.13+, npm, Docker running Linux containers, and credentials for the agent under test. Install with npm, pull the runtime image, run init in your project, and install the Gauge skill so your coding agent can find the CLI's bundled instructions. The CLI collects anonymous command-usage telemetry by default, which can be disabled.

    Gauge CLI - 1

    Community Discussions

    Be the first to start a conversation about Gauge CLI

    Share your experience with Gauge CLI, ask questions, or help others learn from your insights.

    Pricing

    OPEN SOURCE

    Open-source CLI (local)

    MIT-licensed Gauge CLI that runs Agent Experience and Agent Preference evaluations locally with no Gauge account, subscription, or credits required.

    • MIT License
    • Runs locally with Docker
    • No Gauge account required
    • Uses your own Claude Code or Codex subscription or API keys (billed separately by the provider)
    • Gauge does not charge for local execution
    FREE

    Hosted Gauge Agents Free

    Hosted Gauge Agents free plan with 500 credits at signup and a weekly refill to 100 credits; no card required.

    • 500 credits at signup, then weekly refill to 100 credits
    • Open models on OpenCode; first onboarding run uses Claude Code and Codex at no cost
    • 1 new Optimization per week and 2 active runs at a time
    • No AI Traffic or bring-your-own-key

    Hosted Gauge Agents Pro

    Hosted Gauge Agents Pro plan with 5,000 credits per month.

    $499
    per month
    • 5,000 credits per month
    • About 20 credits per Agent Experience run and about 12 credits per Agent Preference run

    Hosted Gauge Agents Enterprise

    Custom credit volume and pricing for teams needing more than 5,000 credits per month.

    Custom
    contact sales
    • Custom credit volume
    View official pricing

    Capabilities

    Key Features

    • Runs real coding agents in isolated Docker sessions against your product
    • Agent Experience evals with criteria and observable outcomes
    • Agent Preference studies on which tools agents choose
    • Normalized transcripts across Claude Code, Codex, Pi and OpenCode
    • Findings and judgments saved to a local SQLite workspace
    • Batch comparison of before/after runs
    • Tests working-tree Skills, npm CLIs and SDKs, and files
    • Web fixtures to test docs changes before publishing
    • Shared Markdown cases and gauge.json manifest for teams and CI
    • Bundled agent instructions via gauge instructions
    • Optional hosted execution with PR checks and scheduled runs
    • JSON output for scripts and agent analysis

    Integrations

    Claude Code
    Codex
    Pi
    OpenCode
    Docker
    npm
    GitHub
    MCP
    API Available
    View Docs

    Ratings & Reviews

    No ratings yet

    Be the first to rate Gauge CLI and help others make informed decisions.

    Rate other tools you’ve used

    Developer

    Gauge Technologies, Inc.

    Gauge Technologies, Inc. is based in San Francisco and builds the Gauge agent-led growth platform. Its products cover AI search visibility, agent preference and agent experience. The company also publishes the open-source Gauge CLI.

    Founded 2024
    San Francisco, CA
    $2.4M raised
    5 employees

    Used by

    PostHog
    Neo4j
    Braintrust
    Clerk
    +13 more
    Read more about Gauge Technologies, Inc.
    WebsiteGitHubLinkedInX / Twitter
    1 tool in directory

    Similar Tools

    Cortad icon

    Cortad

    Tests AI apps with simulated users and analyzes production conversations so coding agents can fix AI behavior and verify the results.

    Crabbox icon

    Crabbox

    A remote software testing and execution control plane that leases cloud capacity, syncs your local checkout, runs commands remotely, and releases — keeping the local edit-save-run loop intact.

    Octrafic icon

    Octrafic

    Open source CLI tool for AI-powered API testing in plain English, straight from the terminal. No code required.

    Browse all tools

    Related Topics

    Automated Testing

    AI-powered platforms that automate end-to-end testing processes with intelligent test case generation, execution, and reporting for faster, more reliable software delivery.

    208 tools

    LLM Evaluations

    Platforms and frameworks for evaluating, testing, and benchmarking LLM systems and AI applications. These tools provide evaluators and evaluation models to score AI outputs, measure hallucinations, assess RAG quality, detect failures, and optimize model performance. Features include automated testing with LLM-as-a-judge metrics, component-level evaluation with tracing, regression testing in CI/CD pipelines, custom evaluator creation, dataset curation, and real-time monitoring of production systems. Teams use these solutions to validate prompt effectiveness, compare models side-by-side, ensure answer correctness and relevance, identify bias and toxicity, prevent PII leakage, and continuously improve AI product quality through experiments, benchmarks, and performance analytics.

    242 tools

    Command Line Assistants

    AI-powered command-line assistants that help developers navigate, search, and execute terminal commands with intelligent suggestions and context awareness.

    409 tools
    Browse all topics
    Back to all toolsSuggest an edit
    ratings
    discussions
    8views