ZenithDB

ZenithDB is an open-source columnar database engine built by Polarity, Inc. and written in Rust, designed specifically for the AI agent observability workload. It stores OpenTelemetry spans, JSON-heavy traces, and versioned prompts, and exposes SQL with span-tree extensions over HTTP, gRPC, and OTLP endpoints. The project is currently in alpha status under the Apache 2.0 license, with its first release (v0.0.1) published in May 2026.

What It Is

ZenithDB is a purpose-built database for AI agent traces — the long, sparse, high-cardinality JSON span trees produced by LLM-based agents. Unlike general-purpose observability backends designed for short structured spans, ZenithDB's storage engine is optimized around five core design choices: a PAX segment format sorted by (trace_id, start_time, span_id), trace-locality enforcement at compaction time, late materialization in the scan operator, Tantivy full-text search embedded inline in segments, and a WAL on object storage with conditional PUT. The project positions itself as a database layer rather than a hosted observability product — it speaks OTLP, stores spans in an open columnar format, and exposes SQL, leaving the product layer to the user.

Architecture and Engine Design

The workspace is organized as 18 Rust crates. Five "moat" crates contain the engine's defining work:

• zen_format — PAX segment encoder/decoder with FSST, ZSTD, Gorilla, FoR, RLE, and dict codecs.
• zen_compactor — Streaming k-way merge compactor that enforces trace-locality.
• zen_query — Vectorized scan operator with late materialization and predicate pushdown.
• zen_fts — Tantivy-as-a-library embedded inline in segments for full-text search.
• zen_wal — Object-storage WAL with conditional PUT, queryable on ack.

The gateway layer uses axum (HTTP) and tonic (gRPC). The catalog can be backed by Postgres in production or an in-memory MockCatalog for development. Object storage supports local filesystem, S3, GCS, and Azure.

Query Model and Span-Tree Primitives

ZenithDB speaks both standard SQL and ZenithQL, with extensions for span-tree traversal: ANCESTORS_OF, DESCENDANTS_OF, and span-aware aggregates. JSON attributes are addressable with path expressions and can be indexed on demand. Versioned prompt templates are stored alongside the spans that ran them, enabling queries that correlate model outputs with the exact prompt version that produced them. The project homepage publishes benchmark numbers comparing ZenithDB against Postgres 16 (jsonb + pg_trgm) and ClickHouse 24.10 on a c7i.4xlarge instance using synthetic OpenTelemetry agent spans — the vendor claims sub-millisecond p95 at one million rows and states that an 8-shard configuration reaches 6.8ms p95 at one billion rows on the group-by-model benchmark.

Deployment Model

ZenithDB ships as a single binary (zen) with no JVM, no Zookeeper, and no Kafka dependency. The installer script supports macOS (Apple Silicon and Intel) and Linux (arm64 and amd64). A development profile runs entirely in-process with no external services; a production-like local stack uses Docker Compose with Postgres and MinIO. A minimal Next.js web console ships under web/ and connects to the engine's HTTP API to expose live segments, queries, compactions, WAL metrics, and a query runner. The project page also mentions a managed cloud offering in private beta.

Update: v0.0.1 Alpha Release

The first public release (v0.0.1) was published on May 13, 2026, roughly one week after the repository was created on May 7, 2026. The project badge explicitly marks status as alpha, with the note that the core engine is feature-complete and runs the full benchmark suite, but on-disk format and wire protocols may still change before 1.0. The repository had 113 stars and 4 forks as of the last recorded update on May 20, 2026.