OpenLake

OpenLake is an open-source, high-performance storage engine designed specifically for GPU workloads — LLM inference, model training, and agentic retrieval. Built in Rust on Linux io_uring, it delivers million-plus IOPS within 1ms and moves data directly into GPU VRAM via GPUDirect Storage and RDMA, eliminating the host-memory staging that bottlenecks conventional storage stacks. The project is licensed under Apache 2.0 and hosted at github.com/openlake-project/openlake.

What It Is

OpenLake is a distributed object storage system purpose-built for AI infrastructure. Unlike general-purpose S3-compatible stores, it is co-designed with the LLM inference stack: it offloads and reuses KV cache across GPU workers, supports ultra-fast checkpoint storage for RL and ML workloads, and accelerates vector index building and serving. It integrates natively with vLLM and SGLang inference engines, and with data frameworks including Apache Spark, Apache Flink, and Ray — with no code changes required for S3-compatible workflows.

Architecture: Short Path from Storage to GPU

OpenLake's design keeps the data path from NVMe or network to GPU VRAM as short and predictable as possible:

• Zero-copy transfers: GPUDirect Storage and RDMA move data directly between NVMe or RDMA NICs and GPU memory, bypassing host memory and the page cache entirely.
• Core-local async I/O: One pinned compio runtime per physical core, backed by io_uring. Requests stay on the same core throughout the hot path, avoiding work-stealing and cross-core contention.
• Burst-aware RDMA: PacedRDMA uses credit-based flow control to prevent senders from overwhelming receivers, sustaining throughput during request bursts while protecting tail latency.
• Efficient durability: SIMD Reed-Solomon erasure coding distributes data and parity across drives, providing durable storage with less capacity overhead than full replication.

KV Cache Offload and Inference Cost Reduction

The flagship use case is KV cache offload for LLM inference. OpenLake stores KV cache entries co-located on GPU hosts at petabyte scale, allowing the inference engine to write KV once and read it back in milliseconds from host RAM and NVMe — saving prefill for long and repeated prompts. The project's benchmark page reports that at a 128K context window, a prefix computed on one GPU host can be served to any other from the shared pool. The homepage benchmark for Gemma-4-E4B on H100 SXM shows OpenLake achieving 3.4× lower cost per million tokens, 3.7× higher throughput, and 19.2× faster median TTFT compared to a naive baseline at 64K context.

Setup Path

OpenLake is designed for drop-in deployment with no code changes to existing vLLM or SGLang setups:

1. Install the Python connector: pip install openlake-vllm
2. Start the storage daemon: openlaked
3. Pass the connector config to vllm serve via --kv-transfer-config

For multi-host GPU clusters with InfiniBand, each node runs openlaked with an RDMA config file, and vLLM workers point at the unified cluster. The object store mode is S3-compatible and can be built from source on Ubuntu, macOS, or Windows (WSL2) using Cargo.

Update: v0.6.0

The latest release is v0.6.0, published on 2026-07-21, with the repository last pushed on 2026-07-26. The project was created in April 2026 and has accumulated over 2,200 GitHub stars and 420 forks in its first few months, with 111 open issues indicating active development. GitHub topics include blackwell, rdma, rust, llm-training, model-serving, and throughput, signaling a roadmap that tracks the latest NVIDIA GPU generations.

Integrations and Ecosystem Fit

OpenLake works across the full data and AI stack without requiring application changes:

• Inference engines: vLLM, SGLang
• Data frameworks: Apache Spark, Apache Flink, Ray
• ML ecosystems: PyTorch, TensorFlow, Hugging Face, Databricks, Snowflake
• Orchestration: Kubernetes, Docker, Terraform
• Storage protocol: S3-compatible API (works with any S3 client, including the AWS CLI)