LMCache
LMCache is a community-driven open-source KV-cache infrastructure project for LLM inference. It turns KV cache into a reusable AI-memory layer that can be stored, compressed, searched, moved and observed across GPU, CPU memory, local disk and external backends, reducing repeated prefill computation and inference latency.
At a Glance
- AI infrastructure and platform engineering teams
- Cloud and GPU infrastructure providers
- Enterprise LLM application developers
- Research labs and open-source inference communities
- +1 more
AI Tools by LMCache
(1)
LMCache
Open Source LLM KV Cache Layer
Discussions
No discussions yet
Be the first to start a discussion about LMCache
Latest News
LMCache publishes an update on CVE-2026-105192.
LMCache documents Raw Block, a fast and recoverable NVMe tier for KV cache.
LMCache on Amazon SageMaker HyperPod for disaggregated prefill and decode.
LMCache introduces KV-cache editing with the LMCache SDK.
Products & Services
Apache-2.0 KV-cache management layer and standalone multiprocess server for LLM inference. It stores and reuses KV tensors across requests and workers, with in-process and server deployment modes.
KV-cache compression and streaming research capability for storing cache on disk or object storage and loading it efficiently.
Non-prefix KV-cache reuse that finds repeated text chunks at arbitrary prompt positions and selectively recomputes tokens to recover quality, particularly for RAG.
SDK for KV-cache editing and custom cache-edit functions, enabling research and application-specific manipulation of cached model state.
Market Position
LMCache positions itself as an open, modular KV-cache layer rather than an inference engine: it can plug into multiple serving engines, storage backends and hardware platforms. Its researched and documented alternatives/adjacent competitors include vLLM's native prefix/KV-cache facilities, NVIDIA Dynamo/KVBM, SGLang's radix cache and Mooncake; LMCache differentiates through cross-engine integrations, pluggable tiered storage, non-prefix CacheBlend reuse and standalone distributed cache management.
Leadership
Founders
Junchen Jiang
University of Chicago faculty member who led the research pivot into LLM systems and co-created LMCache, CacheGen and CacheBlend.
Yuhan Liu
University of Chicago PhD researcher whose proposal to focus on KV cache led to the CacheGen research that seeded LMCache.
Yihua Cheng
Core maintainer and architect of the open-source effort; he started the LMCache open-source effort with Junchen Jiang in April 2024 and coined the LMCache name.
Kuntai Du
University of Chicago researcher who spent a summer with the vLLM project learning LLM inference and open-source development; co-author of the LMCache research.
Executive Team
Junchen Jiang
Project founder and research lead
University of Chicago faculty member; co-creator of LMCache, CacheGen and CacheBlend.
Yihua Cheng
Core maintainer and project architect
Led the April 2024 open-source effort with Jiang; co-author of the LMCache research and expert in high-performance LLM inference.
Founding Story
In January 2023, Junchen Jiang and students at the University of Chicago pivoted toward LLM systems after the ChatGPT release. Yuhan Liu suggested treating KV cache as a system-level, streamable 3D tensor; that work produced CacheGen and CacheBlend. In April 2024, Jiang and Yihua Cheng started an independent open-source effort because they believed the industry was not yet ready for the infrastructure and wanted to build the community themselves. LMCache began without backing from a large company.
Business Model
Revenue Model
LMCache itself is an Apache-2.0 open-source project with free source code, documentation and community support. The related Tensormesh company commercializes enterprise SaaS and reserved-deployment services built by the team, but its financing and pricing are separate from LMCache.
Target Markets
- AI infrastructure and platform engineering teams
- Cloud and GPU infrastructure providers
- Enterprise LLM application developers
- Research labs and open-source inference communities
- Organizations running long-context, RAG, conversational and agent workloads
- Long-context LLM inference
- Multi-turn conversations and agentic workloads with repeated context
- Retrieval-augmented generation with reusable document chunks
- High-concurrency serving and reduced time-to-first-token
- Disaggregated prefill/decode and multi-node inference
- Multimodal inference and cross-hardware deployments
- Bloomberg
- Red Hat
- Redis
- Tencent