PandaProbe

PandaProbe is an open-source agent engineering platform built by Chirpz AI that gives developers the observability, evaluation, and monitoring tools needed to ship reliable AI agents with confidence. It supports the full agent development lifecycle — from first run to continuous improvement — with a single instrument() call. The platform is self-hostable under the Apache 2.0 license, with no vendor lock-in, and integrates seamlessly with all major agent frameworks and LLM providers.

• Tracing — Automatically captures every span (chains, agents, LLMs, tools) with one instrument() call, tracking model types, params, token usage, and key metadata.
• Evals & Metrics — Run trace-level and session-level evaluations to measure agent quality and catch regressions before they reach production.
• Live Monitoring — Monitor agents in real time to detect failures, latency spikes, and unexpected behaviors as they happen.
• Framework Integrations — Plug-and-play support for LangGraph, LangChain, CrewAI, Google ADK, Claude Agent SDK, and OpenAI Agents SDK via a Python SDK.
• LLM Provider Support — Works seamlessly with OpenAI, Anthropic, Google Gemini, and more through built-in wrappers.
• Human Annotation — Supports human-in-the-loop annotation for labeling and improving agent evaluation datasets.
• Self-Hosting — Deploy the full platform on your own infrastructure using the open-source repo; all core features and APIs are included at no cost.
• Scalable Cloud Option — Use PandaProbe Cloud for a managed experience with pay-as-you-go scaling beyond plan limits.
• Open by Default — Apache 2.0 licensed core platform built by a team of PhD researchers specializing in uncertainty and robustness in AI agents.