# VoiceGremlin

> E2E testing for AI phone agents using real phone calls and plain-English test goals via a REST API.

VoiceGremlin is a SaaS tool founded in 2026 by Nick Hall that automates end-to-end testing of intelligent voice agents that answer phones. A test run places a real call to the agent's phone number, and the call transcript is evaluated by a judge model that returns a pass/fail result. It is driven entirely by a REST API, with no SDK.

## What It Is

VoiceGremlin is a testing service for voice agents (IVAs). Each test is a plain-English string describing what to validate, and a test suite is an array of strings, so there is no test case repository or scripted conversation path to maintain. One customized agent calls the target and role-plays as a customer. A second agent reads the transcript and judges whether the target met the goal, using inference rather than keyword matching or regex.

## How the Workflow Runs

A CI job makes two calls. First, POST /api/runs with a phone number and a list of test goals (1 to 50, with optional concurrency and a custom run_id). Then poll GET /api/runs/:run_id every 5 to 10 seconds until the status moves from queued to running to complete. Results include a boolean passed value per test, a reason, latency, and the full call transcript, so failures can be diagnosed directly. Calls go out over a real carrier line, which exercises the SIP integration, STT/TTS and the agent's behavior. Supported numbers are US/Canada.

## CI Validation Targets

Special no-charge phone numbers let teams check their pipeline without placing real calls: instant pass, slow pass (180s), always-fail and flaky targets. Two simulated agents, an airline reservation agent and a hotel front desk agent, run LLM-to-LLM conversations without deducting minutes.

## Limits to Know

The about page says adversarial or red-team testing is possible but the tool is not optimized for it and should not be relied on. There are no webhooks or callbacks, so results must be polled. The service is intended only for calling voice agents, and cold calls or marketing use can lead to account suspension.

## Features
- Real outbound phone calls over a carrier line
- Plain-English test goals
- LLM caller role-plays as a customer
- LLM judge returns pass/fail with reason
- Full call transcript on results
- Batch runs of up to 50 tests with configurable concurrency
- No-charge CI validation targets
- Simulated airline and hotel voice agents
- REST API with Bearer token auth
- Account info endpoint for plan and balance

## Integrations
REST API, CI pipelines

## Platforms
API, WEB

## Pricing
Subscription-based

## Links
- Website: https://voicegremlin.com
- Documentation: https://voicegremlin.com/docs.html
- EveryDev.ai: https://www.everydev.ai/tools/voicegremlin