Traceloop

Traceloop is an open-source-backed observability and evaluation platform for LLM applications, built by a team with roots in ML production pipelines and model monitoring. The core SDK, OpenLLMetry, is released under the Apache 2.0 license and extends OpenTelemetry to give developers full visibility into prompts, responses, latency, and model quality. Traceloop announced it is joining ServiceNow, marking a significant milestone in its trajectory from Y Combinator-backed startup to enterprise acquisition.

What It Is

Traceloop sits in the LLM observability and evaluation category. It captures telemetry from LLM calls, vector database queries, and AI framework operations, then surfaces that data through monitoring dashboards, evaluation pipelines, and CI/CD quality gates. The platform is designed to close the feedback loop between what developers ship and how models actually behave in production — catching quality regressions, prompt drift, and safety issues before users feel them.

How the Feedback Loop Works

Traceloop structures its workflow in four stages:

• Connect: One line of code (Traceloop.init()) instruments your app and starts capturing live data on prompts, responses, latency, and more.
• Evaluate: Built-in metrics for faithfulness, relevance, and safety run automatically against real production data, providing a quality baseline without manual test writing.
• Train: Custom evaluators can be defined by annotating real examples, letting teams encode their own definition of quality rather than relying solely on off-the-shelf metrics.
• Automate: Standard and custom evaluations run on every pull request or in real time, enforcing thresholds and acting as quality gates before code ships.

Open Standards Architecture

OpenLLMetry is built directly on top of OpenTelemetry, meaning its trace data is compatible with any existing observability stack. The GitHub README lists 25+ supported destinations including Datadog, Dynatrace, Honeycomb, Grafana, New Relic, Splunk, IBM Instana, Sentry, and ServiceNow Cloud Observability. Instrumentation covers:

• LLM providers: OpenAI/Azure OpenAI, Anthropic, Google Gemini, AWS Bedrock, Mistral AI, Cohere, Groq, HuggingFace, Ollama, Replicate, Together AI, Vertex AI, IBM Watsonx, and more.
• Vector DBs: Pinecone, Chroma, Weaviate, Qdrant, Milvus, LanceDB, Marqo.
• Frameworks: LangChain, LlamaIndex, LangGraph, CrewAI, Haystack, LiteLLM, Langflow, OpenAI Agents, Agno, AWS Strands, and MCP.

The team also notes that their semantic conventions are now part of the OpenTelemetry project itself, contributing to the broader standardization of LLM observability.

Enterprise Deployment Model

Traceloop is designed to run in cloud, on-premises, or air-gapped environments. The platform is SOC 2 and HIPAA compliant, and supports deployment on AWS, GCP, Azure, and any Kubernetes setup. It is also available for purchase directly through the AWS, GCP, and Azure Marketplaces, which the pricing page notes simplifies legal, procurement, and enterprise discount program (EDP) spend. The SDK supports Python, TypeScript, Go, and Ruby.

Update: Joining ServiceNow

Traceloop announced it is joining ServiceNow, as noted prominently on the homepage and blog. The OpenLLMetry open-source repository remains active, with the latest release at version 0.62.1 (published June 28, 2026) and ongoing commits as of July 2026. The project has accumulated over 7,300 GitHub stars and more than 1,000 forks. Traceloop was backed by Y Combinator, Ibex Investors, Sorenson Capital, Samsung Next, Grand Ventures, and angel investors including the CEOs of Datadog, Sentry, and Elastic.