# Braintrust

> AI observability platform for building, evaluating, monitoring, and shipping quality AI products.

Braintrust provides an AI observability platform that helps teams iterate on prompts, run systematic evaluations, and monitor production AI features. It combines evaluation tooling, production monitoring, and an AI-optimized log store to support end-to-end quality workflows for models and agents. The product includes a web UI, programmatic APIs and SDKs, an automation agent called Loop, and Brainstore, a purpose-built database for AI traces and logs.

- **Evals and Playgrounds**: build datasets, tasks, and scorers to run automated and human-in-the-loop evaluations; start by creating an experiment and running batch or playground tests to compare prompts and models.
- **Loop automation agent**: automates prompt optimization, synthetic dataset generation, and scorer building to accelerate evaluation cycles and reduce manual work.
- **Brainstore (AI log database)**: an optimized storage and query layer for traces and spans that enables fast full-text search and large-scale analysis of AI interactions.
- **Production monitoring & alerts**: capture live model responses, track latency and custom quality metrics, and configure alerts and automations to prevent regressions from reaching users.
- **API & SDKs**: REST API with OpenAPI spec plus language SDKs enable programmatic ingestion, querying (BTQL), and integration into CI/CD and observability pipelines.
- **Security, access control & deployment**: role-based access control, SOC 2 Type II compliance, and self-hosting/on-prem options for privacy-sensitive or high-volume deployments.

To get started, sign up for a free account, use the evaluation playground to author and run evals, instrument your application to send traces to Braintrust, and iterate using Loop and the SDKs to track improvements and set production quality gates.

## Features
- Evaluation framework for datasets, tasks, and scorers
- Playgrounds for fast prompt iteration and side-by-side diffs
- Loop agent for prompt optimization and synthetic data generation
- Brainstore: AI-optimized log and trace database
- Real-time production monitoring, metrics, and alerts
- Automated and human-in-the-loop scoring workflows
- REST API and language SDKs (examples in docs)
- Role-based access control and SOC 2 Type II compliance
- Self-hosting / on-prem deployment option
- Batch testing and scalable ingestion of traces

## Integrations
REST API (OpenAPI), Postman (OpenAPI import), Python SDK, TypeScript SDK, Go SDK, GitHub (organization: braintrustdata), Discord community

## Platforms
WEB, API, DEVELOPER_SDK

## Pricing
Freemium — Free tier available with paid upgrades

## Links
- Website: https://braintrust.dev/
- Documentation: https://www.braintrust.dev/docs/guides/api
- Repository: https://github.com/braintrustdata/
- EveryDev.ai: https://www.everydev.ai/tools/braintrust