Judgment Labs
Judgment Labs is an applied-research and infrastructure company building a continuous-improvement stack for AI agents. Its platform uses production traces and agent behavior data to monitor agents, discover failure patterns, evaluate changes, and help teams make agents better with every interaction.
At a Glance
- AI-native and agent-native startups
- Teams building deep, long-horizon autonomous agents
- Enterprise software teams operating agents in production
- Customer support, sales, legal, finance, and software-engineering organizations deploying agents
AI Tools by Judgment Labs
(1)
Judgeval
Python SDK for LLM Agent Evaluation
Discussions
No discussions yet
Be the first to start a discussion about Judgment Labs
Latest News
A Missing Primitive for AI Agent Improvement: Judgment introduces Behaviors as semantic tags and indexes over production sessions.
Agent Judge: Solving Long-Horizon Evals for Production Agents, describing search, verification, and adaptive rubric capabilities.
Contrario published a case study describing its Judgment Labs partnership, a 20-person team, and use of Judgment to improve Contrario's Rio recruiting agent.
Judgment Labs announced its public launch and $32M combined seed and Series A funding led by Lightspeed Venture Partners.
Products & Services
Commercial continuous-improvement and Agent Behavior Monitoring platform for capturing production executions, investigating failures, defining judges and behaviors, building production-derived datasets, testing candidate changes, and monitoring regressions.
Open-source Python and TypeScript SDK/framework for tracing agent executions, defining and running agent judges, monitoring production traffic, managing datasets, querying project data with read-only SQL, and validating fixes.
Agentic evaluation harness for long-horizon production agents. It searches full trajectories, verifies actions against tool evidence and source-of-truth environments, and adapts evaluation rubrics using production feedback.
Semantic tags over production sessions that describe user, agent, and environment patterns; behaviors become online monitoring signals, semantic indexes, datasets, and regression-test inputs.
Market Position
Judgment positions itself as an agent behavior monitoring and continuous-improvement layer rather than a conventional exception/latency observability product or a generic single-prompt LLM judge. Its differentiation is production-derived behavior indexing, full-trajectory search, verification of stateful actions against the environment, and adaptive rubrics that connect monitoring to datasets, evals, and shipped fixes. Relevant adjacent competitors include LLM evaluation/observability platforms such as Arize/Phoenix, LangSmith, Braintrust, Humanloop, and Helicone, while Judgment emphasizes long-horizon agent behavior and post-deployment improvement.
Leadership
Founders
Alex Shan
Co-founder and CEO. Previously an AI researcher in Stanford's NLP group within the Stanford AI Lab, working under Professor Chris Manning.
Andrew Li
Co-founder and Chief Scientist. Previously an early research hire at Together AI, a training and inference startup; he also co-authored Judgment's research on production-data-driven agent evaluation.
Joseph Sripramong Camyre
Co-founder and CTO. Previously a systems engineer at Datadog, where he built large-scale infrastructure.
Executive Team
Alex Shan
Co-founder & CEO
Former Stanford NLP/Stanford AI Lab researcher under Professor Chris Manning.
Andrew Li
Co-founder & Chief Scientist
Former early research hire at Together AI; co-author of Judgment research on production-data-driven evaluations.
Founding Story
The three founders had been childhood friends, with Andrew teaching Alex NLP and Alex being Joseph's first customer for a middle-school Python course. They came together when the first wave of deep agents reached production and the failure modes they had been studying began appearing in real systems; they set out to build tools that could understand what agents were actually doing across long trajectories rather than judging only input and final output.
Business Model
Revenue Model
Business-to-business SaaS/platform revenue sold through demos and customer deployments; the commercial Judgment platform is paired with open-source Judgeval SDKs and tools.
Target Markets
- AI-native and agent-native startups
- Teams building deep, long-horizon autonomous agents
- Enterprise software teams operating agents in production
- Customer support, sales, legal, finance, and software-engineering organizations deploying agents
- Monitoring and improving customer-support agents
- Evaluating coding agents and long-running software-engineering workflows
- Sales and CRM agents that update records, send messages, or book meetings
- Legal, finance, and other high-consequence autonomous workflows
- Detecting hallucinations, instruction drift, context-retrieval loss, premature stopping, unsafe actions, PII leakage, and unauthorized edits
- Turning real production interactions and user feedback into evals, rewards, datasets, and improvement loops
- E3 Group
- Contrario