vLLM

Easy, fast, and cost-efficient LLM serving for everyone. Deploy the widest range of open-source models on any hardware. Includes a drop-in OpenAI-compatible API for instant integration. Maximize throughput with PagedAttention. Advanced scheduling and continuous batching ensure peak GPU utilization. Slash inference costs by maximizing hardware efficiency. We make high-performance LLMs affordable and accessible to everyone. Select your preferences and run the install command. Stable represents the most currently tested and supported version of vLLM. Nightly is available if you want the latest builds. 📦 Requires Python 3.10+. Python 3.12+ recommended.