NovaSynth by Noveum

NovaSynth is the simulation product from Noveum, an AI agent evaluation platform. It sends synthetic users at voice and chat agents over SIP phone calls, LiveKit real-time audio and chat, so failures show up in rehearsal rather than with real customers.

What It Is

NovaSynth is a pre-production testing tool for voice and chat agents. Teams define synthetic callers with goals, moods, accents, languages, behaviors and interruptions, then run them against a test number, a staging SIP endpoint or an unshipped agent. Each call is captured as a full trace, including audio, and scored with voice AI metrics. Noveum says it provides 30+ voice metrics for these runs, and 100+ across the platform.

How a Run Works

The workflow is a loop: define a persona, set a scenario, score the run, find the cause and a fix, then validate the fix by running again. Example personas include a frustrated repeat caller, a rapid interrupter who talks over the agent, and a prompt-injection attempt that probes safety boundaries. Personas and scenarios can be generated from agent context such as the system prompt, PRD or workflow docs, and teams can also submit their own scenarios. Batch matrix runs execute personas against scenarios under a shared batch id.

Connections and Scale

Supported endpoints are phone SIP (Twilio, Plivo, Vobiz), LiveKit, Pipecat, LangChain/LangGraph, OpenAI and Anthropic SDKs, and chat over HTTP. Custom agents can connect via the SDK or light hooks around LLM calls. Runs are scheduled by an async queue built on BullMQ, PostgreSQL and Redis. Noveum says concurrency can go up to 2,000 lines for load testing. Tool virtualization replays reads and sandboxes writes, so changed decisions are exercised without touching production systems.

Current Status

NovaSynth is described as pre-production, not a live-production caller. It was announced as live on Product Hunt.