docs: polish pass — fix stale names, duplicates, branding, and README
- Normalize 'VLLM' to 'vLLM' in prose across 6 docs (32 occurrences; wikilink targets untouched) - Remove duplicated Environment Variables block in 27B Dankclaw doc - Update stale model refs Qwen3-30B-A3B -> Qwen3.6-35B-A3B in example commands - Fix VLLM Index: -- separators to em-dashes; correct model names on links - README: document active services (ROCmFPX llama-server on Dankclaw, vLLM AWQ on Dankserver), replace broken [[docs/README.md]] link with docs/index.md
This commit is contained in:
1 parent
993b6af576
commit
f650fff0aa
8 files changed
+62
-58
No files matched your search
+12
-12
@@ -1,10 +1,10 @@
|
||||
# VLLM Overview
|
||||
# vLLM Overview
|
||||
|
||||
## What Is VLLM
|
||||
## What Is vLLM
|
||||
|
||||
VLLM is a fast and easy library for large language model inference and serving. You can use VLLM to run models on NVIDIA GPUs, AMD GPUs, Intel GPUs, and CPUs. VLLM supports over 200 model architectures from Hugging Face.
|
||||
vLLM is a fast and easy library for large language model inference and serving. You can use vLLM to run models on NVIDIA GPUs, AMD GPUs, Intel GPUs, and CPUs. vLLM supports over 200 model architectures from Hugging Face.
|
||||
|
||||
VLLM is fast with these features:
|
||||
vLLM is fast with these features:
|
||||
|
||||
- PagedAttention for efficient key and value memory management
|
||||
- Continuous batching of incoming requests
|
||||
@@ -15,7 +15,7 @@ VLLM is fast with these features:
|
||||
- Automatic kernel generation with torch.compile
|
||||
- Disaggregated prefill, decode, and encode
|
||||
|
||||
VLLM is flexible with these features:
|
||||
vLLM is flexible with these features:
|
||||
|
||||
- Seamless integration with Hugging Face models
|
||||
- High-throughput serving with parallel sampling and beam search
|
||||
@@ -28,7 +28,7 @@ VLLM is flexible with these features:
|
||||
|
||||
## Installation
|
||||
|
||||
You can install VLLM with uv (recommended) or pip:
|
||||
You can install vLLM with uv (recommended) or pip:
|
||||
|
||||
```
|
||||
uv pip install vllm
|
||||
@@ -174,15 +174,15 @@ curl http://localhost:8000/v1/chat/completions \
|
||||
|
||||
## Architecture
|
||||
|
||||
VLLM uses a multi-process architecture:
|
||||
vLLM uses a multi-process architecture:
|
||||
|
||||
- API server process handles HTTP requests, tokenization, and input processing
|
||||
- Engine core process runs the scheduler and coordinates GPU workers
|
||||
- One GPU worker process per GPU executes model forward passes
|
||||
|
||||
VLLM uses PagedAttention for memory management. PagedAttention treats KV cache as a set of memory pages. This allows efficient memory allocation and reduces fragmentation.
|
||||
vLLM uses PagedAttention for memory management. PagedAttention treats KV cache as a set of memory pages. This allows efficient memory allocation and reduces fragmentation.
|
||||
|
||||
VLLM supports multiple attention backends:
|
||||
vLLM supports multiple attention backends:
|
||||
|
||||
- FlashAttention for NVIDIA GPUs
|
||||
- FlashInfer for NVIDIA GPUs
|
||||
@@ -191,7 +191,7 @@ VLLM supports multiple attention backends:
|
||||
|
||||
## Supported Models
|
||||
|
||||
VLLM supports over 200 model architectures including:
|
||||
vLLM supports over 200 model architectures including:
|
||||
|
||||
- Decoder-only LLMs (Llama, Qwen, Gemma)
|
||||
- Mixture-of-Expert LLMs (Mixtral, DeepSeek-V3, Qwen-MoE)
|
||||
@@ -202,7 +202,7 @@ VLLM supports over 200 model architectures including:
|
||||
|
||||
## Quantization
|
||||
|
||||
VLLM supports these quantization methods:
|
||||
vLLM supports these quantization methods:
|
||||
|
||||
- FP8 for Hopper and Ada GPUs (native). Ampere runs FP8 W8A16 via Marlin; FP8 W8A8 computes via fallback
|
||||
- MXFP8/MXFP4 for Hopper GPUs
|
||||
@@ -220,7 +220,7 @@ AMD RDNA 3 / RDNA 3.5 (gfx11, Strix Halo) run FP8 via software emulation, not na
|
||||
|
||||
## Parallelism
|
||||
|
||||
VLLM supports multiple parallelism strategies:
|
||||
vLLM supports multiple parallelism strategies:
|
||||
|
||||
- Tensor Parallelism (TP) shards model parameters across GPUs
|
||||
- Pipeline Parallelism (PP) distributes model layers across GPUs
|
||||
|
||||
Reference in new issue
Block a user