# Braintrust

> AI observability and evaluation platform that helps teams trace production AI, run experiments, score outputs, and ship quality AI at scale.

Braintrust is an AI observability and evaluation platform built for teams shipping AI products in production. Founded by Ankur Goyal, who previously led the AI team at Figma and founded Impira, Braintrust connects three core workflows — tracing, evaluation, and automation — into a single platform so teams can continuously measure and improve AI quality.

## What It Is

Braintrust sits at the intersection of AI observability and evaluation infrastructure. Where traditional observability tools answer "is the system operational?", Braintrust is designed to answer "is the system producing good outputs, and how do we make them better?" It captures every trace — inputs, outputs, prompt state, model versions, tool calls, retrieved context, and control flow — and connects that production data to a structured eval and improvement loop.

## Core Workflow

The platform is organized around five stages that form a closed feedback loop:

- **Instrument** — Integrate with AI providers and frameworks using native SDKs for Python, TypeScript, Go, Ruby, C#, and more to send traces to Braintrust with minimal code changes.
- **Observe** — Inspect every trace and tool call in real time, search across millions of logs, and track latency, cost, and quality via live dashboards powered by Brainstore, Braintrust's purpose-built AI database.
- **Annotate** — Add human feedback, build versioned datasets from real production failures, and create task-specific annotation interfaces without frontend work.
- **Evaluate** — Run experiments against real datasets, compare prompts and models side-by-side, and score outputs with LLMs, code, or human reviewers.
- **Deploy** — Use quality gates and online scoring to block bad releases before they reach production.

## Brainstore: Purpose-Built AI Database

Braintrust ships its own database layer called Brainstore, designed specifically for the nested, large-scale structure of AI traces. According to Braintrust's published benchmarks, Brainstore delivers significantly faster full-text search, write latency, and span load times compared to general-purpose databases — enabling exploratory queries across millions of traces at production scale.

## Automation and Pattern Discovery

A key differentiator is the Topics feature, which automatically surfaces patterns across production traces in real time — clustering by task type, sentiment, issues, and custom business dimensions called "facets." Teams can define their own facets (e.g., customer segment, compliance risk, tone) and Topics continuously classifies every trace against them. The Loop agent, Braintrust's built-in AI agent, can autonomously run evaluations, generate test cases, and iterate on prompts based on production signals. An MCP server integration lets coding agents in IDEs like Cursor, Claude Code, and Windsurf query logs, run evals, and update prompts directly.

## Security and Compliance

Braintrust is SOC 2 Type II certified, GDPR compliant, and supports HIPAA compliance, SAML SSO, MFA, and role-based access control. A hybrid deployment option lets teams run the Brainstore data plane on their own infrastructure for privacy-sensitive workloads.

## Who It's Built For

Braintrust targets cross-functional AI teams — software engineers, PMs, data scientists, and subject matter experts — who need a shared system for shipping and improving AI features. The platform is described by the vendor as being used by teams at companies including Notion, Vercel, Dropbox, Replit, Coursera, Graphite, and Navan, among others. Customer quotes on the site include Notion's AI Lead noting "there are some problems we wouldn't know were problems without Braintrust" and Vercel's CTO saying "we didn't realize we needed deep observability until Braintrust."

## Features
- Real-time trace inspection
- LLM-as-a-judge scoring
- Human annotation and review
- Versioned datasets
- Prompt experimentation and comparison
- Model comparison side-by-side
- Topics: automatic pattern discovery
- Custom facets for domain-specific clustering
- Loop agent for autonomous eval iteration
- Quality gates and online scoring
- MCP server for IDE integration
- Brainstore purpose-built AI database
- Native SDKs for Python, TypeScript, Go, Ruby, C#
- Custom trace views and annotation interfaces
- Trace-to-dataset with one click
- RBAC and SSO
- Hybrid deployment
- SOC 2 Type II, GDPR, HIPAA compliance
- CI/CD integration for eval pipelines
- Live performance dashboards

## Integrations
OpenAI, Anthropic Claude, Google Gemini, Mistral, Cursor, Claude Code, Windsurf, Cline, GitHub Copilot, LangChain, LlamaIndex, Vercel AI SDK, AWS, S3

## Platforms
WEB, API, CLI, DEVELOPER_SDK

## Pricing
Freemium — Free tier available with paid upgrades

## Links
- Website: https://braintrust.dev/
- Documentation: https://www.braintrust.dev/docs
- Repository: https://github.com/braintrustdata/eval-101-course
- EveryDev.ai: https://www.everydev.ai/tools/braintrust