# Evidently

> Open-source Python framework to evaluate, test, and monitor ML and LLM-powered systems and data pipelines.

Evidently is an open-source Python library from Evidently AI for evaluating, testing, and monitoring ML and LLM systems, from experiments through production. It works with tabular and text data and ships 100+ built-in metrics, from data drift detection to LLM judges. The latest listed release is v0.7.23.

## What It Is

Evidently is an ML and LLM observability framework distributed as a Python package. Users compute evaluations over datasets, summarize them in reports, and optionally track results over time in a monitoring UI. It supports both predictive and generative tasks, from classification to RAG, and lets users add custom metrics through a Python interface.

## Reports and Test Suites

Reports compute and summarize data, ML and LLM quality evaluations using presets and built-in metrics, or custom ones. They can be viewed interactively in Python or exported as JSON, a Python dictionary, or HTML, or viewed in the monitoring UI. Adding pass/fail conditions (such as greater-than or less-than) turns a Report into a Test Suite for regression testing, CI/CD checks, or data validation. Test conditions can also be auto-generated from a reference dataset.

## What You Can Evaluate

Documented evaluation areas include:

- Text descriptors such as length, sentiment, toxicity, and regex matches
- LLM outputs such as semantic similarity and retrieval relevance
- Data quality, including missing values and duplicates
- Data distribution drift, using 20+ statistical tests and distance metrics
- Classification and regression metrics
- Ranking and RAG metrics
- Recommendation metrics

## Monitoring Dashboard

A Monitoring UI service visualizes metrics and test results over time. It can be self-hosted from the open-source version (for example with the evidently ui command and demo projects at localhost:8000) or used through Evidently Cloud, which the README says adds dataset and user management, alerting, and no-code evals.

## Features
- 100+ built-in metrics
- Data drift detection
- LLM-as-a-judge evaluators
- Text descriptors
- Reports with JSON, dict and HTML export
- Test Suites with pass/fail conditions
- Auto-generated test conditions from reference data
- Custom metrics via Python
- Self-hostable monitoring UI
- Classification, regression, ranking and recommendation metrics

## Integrations
Pandas, Jupyter, scikit-learn, Conda, PyPI

## Platforms
LINUX, WEB, API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Version
v0.7.23

## Links
- Website: https://discord.gg/xZjKRaNp8b
- Documentation: https://docs.evidentlyai.com
- Repository: https://github.com/evidentlyai/evidently
- EveryDev.ai: https://www.everydev.ai/tools/evidently