Langfuse

Langfuse is an open-source LLM engineering platform designed to help teams build production-grade AI applications. Founded during Y Combinator's W23 batch just as GPT-4 was being released, Langfuse addresses the unique challenges of developing and maintaining LLM-based applications that traditional software engineering tools aren''t equipped to handle.

The platform offers comprehensive observability and tracing capabilities that are essential for working with the complex, non-deterministic nature of LLM applications. Langfuse captures the full context of LLM application execution, including API calls, prompts, retrieval processes, and tool usage, allowing developers to trace the complete flow of information through their applications. This is particularly valuable for debugging complex systems like agents with tools, RAG implementations, or multi-step agentic workflows where pinpointing the root cause of issues can be challenging.

Beyond basic tracing, Langfuse provides tools for quality monitoring through various evaluation methods, including model-based evaluations, user feedback collection, and manual scoring. These capabilities allow teams to measure output quality over time, across different user segments, or across application versions. The platform also offers cost monitoring features to track model usage and expenses, latency insights to identify performance bottlenecks, and dataset creation tools to build high-quality training and evaluation data.

Langfuse is framework-agnostic but offers seamless integrations with popular LLM frameworks like LangChain, LlamaIndex, and OpenAI, as well as supporting custom implementations through SDKs for Python and JavaScript. Its open data layer approach ensures that all data collected remains accessible to users, with APIs available for extracting information for downstream use cases like usage-based billing.

Available as both self-hosted and cloud options, Langfuse offers tiered pricing from a free Hobby plan for small projects up to Enterprise plans with advanced security features and dedicated support. The MIT-licensed open-source nature of Langfuse aligns with its philosophy that observability data should be open rather than locked in proprietary ecosystems.