diff --git a/serverAppConfigs-docs/LLM/DFlash DSpark Speculative Decoding.md b/serverAppConfigs-docs/LLM/DFlash DSpark Speculative Decoding.md index bb57d90..134b851 100644 --- a/serverAppConfigs-docs/LLM/DFlash DSpark Speculative Decoding.md +++ b/serverAppConfigs-docs/LLM/DFlash DSpark Speculative Decoding.md @@ -411,7 +411,7 @@ measurement that would settle it. - Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS. - DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use. -- The old `--swap-space` flag no longer exists in current vLLM. Use `--kv-offloading-size`. +- The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by [[RAM CPU Offloading in VLLM]]. - `--speculative-config` is the preferred CLI. Legacy `--spec-method`, `--spec-model`, and `--spec-tokens` flags still exist. - Use `rejection_sample_method` default `strict` for lossless output. diff --git a/serverAppConfigs-docs/LLM/Hybrid CPU GPU Processing Dankserver.md b/serverAppConfigs-docs/LLM/Hybrid CPU GPU Processing Dankserver.md index 295cb90..017109f 100644 --- a/serverAppConfigs-docs/LLM/Hybrid CPU GPU Processing Dankserver.md +++ b/serverAppConfigs-docs/LLM/Hybrid CPU GPU Processing Dankserver.md @@ -56,7 +56,10 @@ Lower values leave more GPU memory for KV cache. This reduces swap frequency. ### Swap Space / KV Offload -Recent vLLM releases removed the `--swap-space` engine argument — `CacheConfig` on `main` has no `swap_space` field ([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/config/cache.py), read 2026-10-18). Use `--kv-offloading-size` (`kv_offloading_size`, same file) to move KV cache to CPU RAM. In Podman, mount the host cache read-write for KV offload writes. The table below shows historical `--swap-space` guidance as a sizing reference. +Current vLLM has no `--swap-space` engine argument, and the CPU KV-offload knob pair +is owned by [[RAM CPU Offloading in VLLM]] (that page carries the upstream +`CacheConfig` verification). In Podman, mount the host cache read-write for KV offload +writes. The table below shows historical `--swap-space` guidance as a sizing reference. | System RAM | Recommended Swap | |------------|------------------| diff --git a/serverAppConfigs-docs/LLM/Shared Memory Architecture Dankclaw.md b/serverAppConfigs-docs/LLM/Shared Memory Architecture Dankclaw.md index 6639ba4..c8a09d3 100644 --- a/serverAppConfigs-docs/LLM/Shared Memory Architecture Dankclaw.md +++ b/serverAppConfigs-docs/LLM/Shared Memory Architecture Dankclaw.md @@ -103,7 +103,7 @@ vllm serve model \ - `gpu_memory_utilization`: Set lower (0.8-0.85) to leave room for CPU operations - `max_model_len`: Match actual context needs -- The old `--swap-space` argument was removed from current vLLM — `CacheConfig` on `main` carries no `swap_space` field ([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/config/cache.py), read 2026-10-18). Use `--kv-offloading-size` (`kv_offloading_size`, same file) for CPU KV offload +- `--swap-space` is retired from current vLLM; the CPU KV-offload knob pair, with the upstream `CacheConfig` verification, is owned by [[RAM CPU Offloading in VLLM]] ### NUMA Binding diff --git a/serverAppConfigs-docs/LLM/VLLM Index.md b/serverAppConfigs-docs/LLM/VLLM Index.md index 87fc2b9..69d687a 100644 --- a/serverAppConfigs-docs/LLM/VLLM Index.md +++ b/serverAppConfigs-docs/LLM/VLLM Index.md @@ -60,7 +60,7 @@ vllm serve meta-llama/Llama-3.1-70B-Instruct --tensor-parallel-size 4 vllm serve Qwen/Qwen2.5-1.5B-Instruct --quantization fp8 # With GPU memory control -vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --swap-space 8 +vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --kv-offloading-size 8 --kv-offloading-backend native ``` ### Common API Endpoints @@ -79,7 +79,7 @@ vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --swap-space |-----------|-------------|---------------| | tensor_parallel_size | Number of GPUs for model sharding | 1-8 | | gpu_memory_utilization | Fraction of GPU memory for KV cache (default 0.92) | 0.7-0.95 | -| swap_space | Removed in current vLLM; use `--kv-offloading-size` | n/a | +| swap_space | Retired; live knob pair in [[RAM CPU Offloading in VLLM]] | n/a | | max_model_len | Maximum context length | auto | | max_num_batched_tokens | Max tokens per batch | 2048-8192 | | max_num_seqs | Max sequences per batch | 128-512 | diff --git a/serverAppConfigs-docs/LLM/VLLM Overview.md b/serverAppConfigs-docs/LLM/VLLM Overview.md index ae0ae2a..ecc2112 100644 --- a/serverAppConfigs-docs/LLM/VLLM Overview.md +++ b/serverAppConfigs-docs/LLM/VLLM Overview.md @@ -140,7 +140,7 @@ curl http://localhost:8000/v1/chat/completions \ | `tensor_parallel_size` | 1 | Number of GPUs for tensor parallelism | | `pipeline_parallel_size` | 1 | Number of GPUs for pipeline parallelism | | `gpu_memory_utilization` | 0.92 | Fraction of GPU memory for KV cache | -| `swap_space` | removed | Removed in current vLLM; use `--kv-offloading-size` | +| `swap_space` | — | retired; the live CPU-KV-offload knob pair is in [[RAM CPU Offloading in VLLM]] | | `enforce_eager` | false | Use eager mode (no CUDA graphs) | | `max_num_batched_tokens` | 2048 | Max tokens per batch | | `max_num_seqs` | 128 | Max sequences per batch | @@ -167,7 +167,7 @@ curl http://localhost:8000/v1/chat/completions \ | Argument | Default | Description | |----------|---------|-------------| | `gpu_memory_utilization` | 0.92 | Fraction of GPU memory for KV cache | -| `swap_space` | removed | Removed in current vLLM; use `--kv-offloading-size` | +| `swap_space` | — | retired; the live CPU-KV-offload knob pair is in [[RAM CPU Offloading in VLLM]] | | `kv_cache_dtype` | auto | KV cache data type (auto, fp8, fp8_e5m2) | | `max_model_len` | auto | Maximum context length | | `num_gpu_blocks` | auto | Number of GPU blocks | diff --git a/serverAppConfigs-docs/LLM/dankserver-vram-growth.md b/serverAppConfigs-docs/LLM/dankserver-vram-growth.md index a7a67ec..ad1d2cc 100644 --- a/serverAppConfigs-docs/LLM/dankserver-vram-growth.md +++ b/serverAppConfigs-docs/LLM/dankserver-vram-growth.md @@ -14,6 +14,10 @@ timestamp: 2026-09-03 > **SUPERSEDED 2026-08-17.** The Frank redo retired this topology. > See [[frank-redo-2026-08-17]] and [[deployment]] for the active configuration. +> The `frank-zealot` below is the 2026-08 service: llama.cpp on the RTX 3090 +> **while that card sat in Dankserver**. Today's [[frank-zealot]] is a different host +> (Dankclaw), payload and runtime build, though the same card class - this crash +> signature is not a live fault. # Dankserver VRAM and RAM Growth Under Load