docs: polish pass — fix stale names, duplicates, branding, and README
- Normalize 'VLLM' to 'vLLM' in prose across 6 docs (32 occurrences; wikilink targets untouched) - Remove duplicated Environment Variables block in 27B Dankclaw doc - Update stale model refs Qwen3-30B-A3B -> Qwen3.6-35B-A3B in example commands - Fix VLLM Index: -- separators to em-dashes; correct model names on links - README: document active services (ROCmFPX llama-server on Dankclaw, vLLM AWQ on Dankserver), replace broken [[docs/README.md]] link with docs/index.md
This commit is contained in:
1 parent
993b6af576
commit
f650fff0aa
8 files changed
+62
-58
No files matched your search
@@ -1,8 +1,8 @@
|
||||
# RAM CPU Offloading in VLLM
|
||||
# RAM CPU Offloading in vLLM
|
||||
|
||||
## Overview
|
||||
|
||||
VLLM uses CPU RAM and swap space to extend GPU memory capacity. This allows larger models to run on GPUs with limited VRAM. VLLM does not offload model weights to CPU. Instead, VLLM moves KV cache data between GPU and CPU.
|
||||
vLLM uses CPU RAM and swap space to extend GPU memory capacity. This allows larger models to run on GPUs with limited VRAM. vLLM does not offload model weights to CPU. Instead, vLLM moves KV cache data between GPU and CPU.
|
||||
|
||||
> **Current vLLM note:** Recent vLLM releases removed the `--swap-space` engine argument. CPU KV-cache offloading now uses `--kv-offloading-size`. See the parameters below for details.
|
||||
|
||||
@@ -167,10 +167,10 @@ If you see frequent preemptions:
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| VLLM_CPU_KVCACHE_SPACE | KV cache space in GiB for CPU backend | 4 |
|
||||
| VLLM_CPU_OMP_THREADS_BIND | CPU thread binding for OpenMP | auto |
|
||||
| VLLM_CPU_NUM_OF_RESERVED_CPU | Reserved CPU cores per rank | 0-1 |
|
||||
| VLLM_CACHE_ROOT | torch.compile cache directory | ~/.cache/vllm |
|
||||
| vLLM_CPU_KVCACHE_SPACE | KV cache space in GiB for CPU backend | 4 |
|
||||
| vLLM_CPU_OMP_THREADS_BIND | CPU thread binding for OpenMP | auto |
|
||||
| vLLM_CPU_NUM_OF_RESERVED_CPU | Reserved CPU cores per rank | 0-1 |
|
||||
| vLLM_CACHE_ROOT | torch.compile cache directory | ~/.cache/vllm |
|
||||
|
||||
## See Also
|
||||
|
||||
|
||||
Reference in new issue
Block a user