vLLM
vLLM is an open-source, high-throughput and memory-efficient engine/library for LLM inference and serving. Its stated goal is easy, fast and cheap LLM serving for everyone, with broad model and hardware compatibility.
At a Glance
- AI research institutions and universities
- Model developers and open-source model communities
- Cloud and GPU infrastructure providers
- Enterprises deploying LLMs in production
- +1 more
AI Tools by vLLM
(5)
Humming
JIT GEMM Kernels for NVIDIA GPUs
LLM Compressor
LLM Quantization Library for vLLM
vLLM Ascend
vLLM Plugin for Huawei Ascend
vLLM Semantic Router
LLM Request Routing Framework
vLLM
Open Source LLM Inference Library
Discussions
No discussions yet
Be the first to start a discussion about vLLM
Latest News
Products & Services
Open-source high-throughput and memory-efficient inference and serving engine for LLMs, distributed under Apache 2.0.
Major re-architecture of vLLM's core architecture intended to improve performance, scalability and extensibility.
Framework for efficient inference with omni-modality models.
High-performance, lightweight JIT-compiled GEMM kernel library for quantized inference; supports dense and MoE GEMM, sub-8-bit weights, multiple activation/scale types and NVIDIA GPUs SM75+.
Market Position
vLLM positions itself as a leading open-source, hardware-flexible inference and serving engine, differentiated by PagedAttention, continuous batching, broad model/hardware support and an OpenAI-compatible API. Its competitive set includes TensorRT-LLM, Hugging Face TGI, NVIDIA Triton, SGLang, LMDeploy and llama.cpp; unlike proprietary hosted APIs, vLLM is self-hostable and Apache-2.0 licensed.
Leadership
Founders
Woosuk Kwon
UC Berkeley computer-science researcher and PhD alumnus; original vLLM creator and current lead/core maintainer.
Zhuohan Li
UC Berkeley Sky Computing Lab researcher and original vLLM contributor; current core maintainer.
Simon Mo
LLM-serving engineer associated with Anyscale/Ray Serve; current vLLM lead/core maintainer.
Kaichao You
AI systems researcher and UC Berkeley-associated vLLM contributor; current core maintainer.
Ion Stoica
UC Berkeley computer-science professor and director of the Sky Computing Lab; Databricks co-founder and advisor to the original research effort.
Executive Team
Woosuk Kwon
Lead maintainer / core maintainer
UC Berkeley computer-science researcher and original vLLM creator.
Zhuohan Li
Core maintainer
UC Berkeley Sky Computing Lab researcher and original project contributor.
Board of Directors
Founding Story
vLLM began in 2023 as a research project in UC Berkeley's Sky Computing Lab. The initial vision was to make LLM serving substantially faster and more memory-efficient through PagedAttention and related systems optimizations, then turn that research into broadly usable open-source infrastructure.
Business Model
Revenue Model
vLLM itself is free and open source under Apache 2.0. The community project is supported through cash donations, GitHub Sponsors, OpenCollective and donated compute/Slack sponsorship; it is not documented as a conventional subscription or license-revenue business.
Target Markets
- AI research institutions and universities
- Model developers and open-source model communities
- Cloud and GPU infrastructure providers
- Enterprises deploying LLMs in production
- Developers building chat, coding, agentic and multimodal applications
- Production LLM and multimodal-model inference APIs
- Research prototypes and model evaluation
- High-concurrency chat, coding and agentic applications
- Distributed and multi-GPU/multi-node serving
- Cost-efficient quantized inference
- Embedding, retrieval, reward and classification model serving
- Amazon Rufus
- LinkedIn AI features