OpenLIT

OpenLIT is an open-source AI engineering platform built to help teams build, evaluate, and observe AI applications across the entire lifecycle from development to production. It is available under the Apache License 2.0 and self-hosted, meaning data stays within your own infrastructure. The project follows and maintains the OpenTelemetry Semantic Conventions for AI observability, consistently updating to align with the latest community standards.

What It Is

OpenLIT is a full-stack observability and engineering platform for generative AI and LLM applications. It combines OpenTelemetry-native auto-instrumentation SDKs (Python, TypeScript, Go), a self-hosted UI with dashboards, a prompt management hub, LLM evaluation tooling, a secrets vault, and a model playground — all in one open-source package. The platform is designed so that adding observability requires as little as a single line of code, with zero-code options available for Kubernetes, Docker, and Linux environments via eBPF-based automatic instrumentation.

Architecture and Deployment Model

OpenLIT's architecture centers on three components:

• OpenLIT Platform — The self-hosted UI and backend for tracing, prompt management, evaluations, dashboards, metrics, logs, and remote collectors.
• OpenLIT SDKs — OpenTelemetry-native auto-instrumentation libraries that trace LLMs, agents, vector databases, and GPUs with zero-code changes.
• OpenLIT Controller — A zero-code observability layer for Kubernetes, Docker, and Linux using eBPF and automatic SDK injection.

Data flows from the SDK through an OpenTelemetry Collector into ClickHouse, where the OpenLIT UI pulls it for visualization. As of version 1.15.0 (which introduced Fleet Hub), the OpenTelemetry Collector is integrated directly into the OpenLIT container, simplifying Docker Compose deployments.

Supported Integrations

The project auto-instruments over 50 LLM providers, AI frameworks, and vector databases. Key integrations include:

• LLM Providers: OpenAI, Anthropic, Cohere, Mistral AI, Groq, Google AI Studio, Together AI, Ollama, AWS Bedrock, Azure AI Inference, Vertex AI, vLLM, Hugging Face, LiteLLM, and more.
• AI Frameworks: LangChain, LlamaIndex, CrewAI, Pydantic AI, LangGraph, OpenAI Agents, Google ADK, Smolagents, Strands Agents, Claude Agent SDK, AG2 (AutoGen), Haystack, Mem0, and others.
• Vector Databases: Pinecone, ChromaDB, Qdrant, Milvus, Astra DB, PostgreSQL.
• Specialized Tools: ElevenLabs, AssemblyAI, MCP.

Core Feature Set

OpenLIT ships a broad set of capabilities beyond basic tracing:

• Distributed Tracing — OpenTelemetry-native traces for LLMs, agents, vector databases, and GPUs; compatible with OpenInference and OpenLLMetry instrumentation frameworks.
• Evaluations — 11 built-in LLM-as-a-Judge evaluation types (hallucination, bias, toxicity, safety, relevance, coherence, faithfulness, and more), plus programmatic evaluation APIs.
• Prompt Hub — Centralized versioning, editing, and deployment of prompts across environments without code changes.
• OpenGround — Side-by-side comparison of LLMs by cost, duration, and response tokens.
• Vault — Centralized storage for LLM API keys, retrievable remotely without application restarts.
• Fleet Hub — OpAMP-based centralized management and monitoring of OpenTelemetry Collectors across infrastructure with secure TLS communication.
• Rule Engine — Conditional rules with AND/OR logic to match runtime trace attributes and dynamically retrieve contexts, prompts, and evaluation configs.
• Coding Agent Observability — A CLI that instruments local coding agents (Claude Code, Cursor, Codex) to emit OpenTelemetry traces for sessions, prompts, tool calls, file edits, and subagent spawns.
• GPU Monitoring — Dedicated GPU metrics collection for NVIDIA and AMD hardware.
• Custom Cost Tracking — Support for custom pricing files to estimate costs for fine-tuned or custom models.

Update: controller-0.9.0 and Fleet Hub

The latest published release is controller-0.9.0 (published June 10, 2026), reflecting active development on the OpenLIT Controller component. The introduction of Fleet Hub in version 1.15.0 was a significant architectural change: the standalone OpenTelemetry Collector container was merged into the main OpenLIT container, requiring Docker Compose users to run with --remove-orphans when upgrading. The roadmap lists upcoming features including auto-evaluation metrics based on usage, human feedback for LLM events, dataset generation from LLM events, and search over traces — signaling continued investment in the evaluation and data pipeline layers of the platform.