companies

vLLM Project

Open-source inference and serving engine for large language models, built around PagedAttention for efficient GPU memory use; originated at UC Berkeley's Sky Computing Lab.

vLLM is an open-source LLM inference and serving engine originally developed in 2023 by researchers at UC Berkeley's Sky Computing Lab, including Woosuk Kwon, Zhuohan Li, and Simon Mo. It introduced PagedAttention, a memory-management technique for transformer key-value caches that significantly improves GPU memory efficiency and throughput for serving large language models. The project grew into one of the most widely used open-source AI infrastructure projects, supporting continuous batching, distributed and multi-GPU inference, quantization, and OpenAI-compatible APIs, with contributions from thousands of developers across academia and industry. In 2025 the PyTorch Foundation announced vLLM as a Foundation-hosted project, and it was recognized as a top open-source project on GitHub for 2025. In January 2026, several vLLM creators launched a startup (reported as Inferact) to commercialize the technology, raising $150 million in seed funding, though the core vLLM project itself remains an open-source, community/foundation-governed effort rather than a conventional company.

Founded

2023

Headquarters

Berkeley, United States

Sector

infrastructure