vllm
vLLM是一款大语言模型（LLM）推理与服务部署工具，可提升部署吞吐量、降低内存占用，适配多种主流硬件与大模型架构。
这个项目值得继续研究吗？
vLLM是一款大语言模型（LLM）推理与服务部署工具，可提升部署吞吐量、降低内存占用，适配多种主流硬件与大模型架构。
- 解决什么问题
- 企业部署大模型时，常遇到多用户并发访问时响应延迟高、GPU等硬件资源占用过高导致运营成本居高不下，以及不同模型、硬件适配复杂的痛点，该工具可针对性解决上述问题。
- 适合什么团队
- 有大模型私有化部署需求的企业技术团队、AI应用开发团队，以及需要对外提供大模型API服务的业务团队均可使用。
- 使用前注意
- 部署时需根据所使用的硬件（NVIDIA/AMD/Intel GPU、CPU等）匹配对应安装包，采用Apache-2.0许可无商用限制，当前项目活跃迭代，可提前核对所需模型是否在官方支持列表。
本页用于缩短初步筛选时间，不构成技术、采购或法律结论。 正式使用前请在真实业务数据上验证，并以官方说明与许可证为准。
从官方资料看清能力、部署与采用边界
以下内容依据项目公开 README 或模型卡翻译整理，代码、命令和产品名保持原样。
项目定位
vLLM最初由加州大学伯克利分校Sky Computing Lab研发，目前已成为由全球多所学术机构、企业的贡献者共同维护的活跃开源AI项目，定位为面向所有用户的简单、高效、低成本的大语言模型（Large Language Model，简称LLM）服务部署工具。
核心能力
vLLM的核心能力分为性能优化与易用性两部分：
- 性能优化层面：通过PagedAttention技术实现注意力键值内存的高效管理，配合请求连续批处理、分块预填充、前缀缓存等机制，提升服务吞吐量；支持FP8、INT8、INT4等多种量化方案，适配FlashAttention等多种优化注意力内核，可进一步降低硬件资源占用；支持推测解码、分布式推理等能力，满足大规模部署需求。
- 易用性层面：可无缝对接Hugging Face上的主流大模型，支持并行采样、波束搜索等多种解码算法；支持流式输出、结构化输出、工具调用等业务常用功能；自带兼容OpenAI规范的API服务，同时支持Anthropic Messages API与gRPC协议；支持多LoRA（LoRA即低秩适配，是一种大模型微调及部署时降低资源占用的技术）部署，适配稠密模型与混合专家（Mixture of Experts，简称MoE）层。
适配范围
vLLM目前已支持200余种Hugging Face上的模型架构，覆盖以下常见类型：
- 纯解码器架构大模型，如Llama、Qwen、Gemma等
- 混合专家（MoE）大模型，如Mixtral、DeepSeek-V3、Qwen-MoE等
- 混合注意力与状态空间模型，如Mamba、Qwen3.5等
- 多模态大模型，如LLaVA、Qwen-VL、Pixtral等
- 嵌入与检索模型，如E5-Mistral、GTE、ColBERT等
- 奖励与分类模型，如Qwen-Math等
硬件适配方面，vLLM原生支持NVIDIA GPU、AMD GPU、Intel GPU与x86/ARM/PowerPC架构CPU，同时可通过插件支持Google TPU、华为昇腾、苹果硅等多种硬件设备。
部署方式
vLLM的安装部署门槛较低，官方推荐使用uv安装，也可通过pip安装，安装命令如下：
uv pip install vllm如有二次开发需求，也可参照官方文档从源码编译安装。
官方提供了完整的部署指引，可参考对应文档：
- 安装指南
- 快速入门
- 支持模型完整列表
许可证与采用建议
vLLM采用Apache-2.0开源许可证，企业可自由商用、修改、二次分发，无授权费用。
当前项目处于活跃维护状态，建议企业采用前先核对自身使用的模型与硬件是否在官方适配范围内，小规模测试验证性能符合业务要求后再正式上线，遇到问题可通过官方用户论坛或GitHub Issues反馈。
官方资料与来源
- amd
- blackwell
- cuda
- deepseek
- deepseek-v3
- gpt
- gpt-oss
- inference
- kimi
- llama
- llm
- llm-serving
## Getting Started Install vLLM with [`uv`](https://docs.astral.sh/uv/) (recommended) or `pip`: ```bash uv pip install vllm ``` Or [build from source](https://docs.vllm.ai/en/latest/getting_started/installation/gpu/index.html#build-wheel-from-source) for development. Visit our [documentation](https://docs.vllm.ai/en/latest/) to learn more. - [Installation](https://docs.vllm.ai/en/latest/getting_started/installation.html) - [Quickstart](https://docs.vllm.ai/en/latest/getting_started/quickstart.html) - [List of Supported Models](https://docs.vllm.ai/en/latest/models/supported_models.html)
该片段来自项目 README，仅用于初步判断；实际部署请以官方文档为准。
核对上游原始说明节选
A high-throughput and memory-efficient inference and serving engine for LLMs
Easy, fast, and cheap LLM serving for everyone
| Documentation | Blog | Paper | Twitter/X | User Forum | Developer Slack |
🔥 We have built a vLLM website to help you get started with vLLM. Please visit vllm.ai to learn more. For events, please visit vllm.ai/events to join us.
---
About
vLLM is a fast and easy-to-use library for LLM inference and serving.
Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has grown into one of the most active open-source AI projects built and maintained by a diverse community of many dozens of academic institutions and companies from over 2000 contributors.
vLLM is fast with:
- State-of-the-art serving throughput
- Efficient management of attention key and value memory with PagedAttention
- Continuous batching of incoming requests, chunked prefill, prefix caching
- Fast and flexible model execution with piecewise and full CUDA/HIP graphs
- Quantization: FP8, MXFP8/MXFP4, NVFP4, INT8, INT4, GPTQ/AWQ, GGUF, compressed-tensors, ModelOpt, TorchAO, and more
- Optimized attention kernels including FlashAttention, FlashInfer, TRTLLM-GEN, FlashMLA, and Triton
- Optimized GEMM/MoE kernels for various precisions using CUTLASS, TRTLLM-GEN, CuTeDSL
- Speculative decoding including n-gram, suffix, EAGLE, DFlash
- Automatic kernel generation and graph-level transformations using torch.compile
- Disaggregated prefill, decode, and encode
vLLM is flexible and easy to use with:
- Seamless integration with popular Hugging Face models
- High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more
- Tensor, pipeline, data, expert, and context parallelism for distributed inference
- Streaming outputs
- Generation of structured outputs using xgrammar or guidance
- Tool calling and reasoning parsers
- OpenAI-compatible API server, plus Anthropic Messages API and gRPC support
- Efficient multi-LoRA support for dense and MoE layers
- Support for NVIDIA GPUs, AMD GPUs, Intel GPUs, and x86/ARM/PowerPC CPUs. Additionally, diverse hardware plugins such as Google TPUs, Intel Gaudi, IBM Spyre, Huawei Ascend, Rebellions NPU, Apple Silicon, MetaX GPU, and more.
vLLM seamlessly supports 200+ model architectures on Hugging Face, including:
- Decoder-only LLMs (e.g., Llama, Qwen, Gemma)
- Mixture-of-Expert LLMs (e.g., Mixtral, DeepSeek-V3, Qwen-MoE, GPT-OSS)
- Hybrid attention and state-space models (e.g., Mamba, Qwen3.5)
- Multi-modal models (e.g., LLaVA, Qwen-VL, Pixtral)
- Embedding and retrieval models (e.g., E5-Mistral, GTE, ColBERT)
- Reward and classification models (e.g., Qwen-Math)
Find the full list of supported models here.
Getting Started
Install vLLM with uv (recommended) or pip:
uv pip install vllmOr build from source for development.
Visit our documentation to learn more.
- Installation
- Quickstart
- List of Supported Models
Contributing
We welcome and value any contributions and collaborations. Please check out Contributing to vLLM for how to get involved.
Citation
If you use vLLM for your research, please cite our paper:
@inproceedings{kwon2023efficient,
title={Efficient Memory Management for Large Language Model Serving with PagedAttention},
author={Woosuk Kwon and Zhuohan Li and Siyuan Zhuang and Ying Sheng and Lianmin Zheng and Cody Hao Yu and Joseph E. Gonzalez and Hao Zhang and Ion Stoica},
booktitle={Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles},
year={2023}
}Contact Us
- For technical questions and feature requests, please use GitHub Issues
- For discussing with fellow users, please use the vLLM Forum
- For coordinating contributions and development, please use Slack
- For security disclosures, please use GitHub's Security Advisories feature
- For collaborations and partnerships, please contact us at collaboration@vllm.ai
Media Kit
- If you wish to use vLLM's logo, please refer to our media kit repo
优秘智能