diff --git a/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md b/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md index 802481a..898f266 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md +++ b/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md @@ -148,7 +148,7 @@ Both replicas, uniform except GPU index and port: |---|---|---| | `TP` | 1 | one card per replica | | `MAXLEN` | 114688 | the largest shape that boots on one card: 262144 needs 9.61 GiB of KV against 5.14 available, and 131072 needs 5.58 against 5.22 | -| `KV_MEM` | 0 | vLLM profiles this shape; the upstream pin (2x7551-32624) is a TP=2/262144 measurement and is invalid here | +| `KV_MEM` | 0 | passed through literally as `--kv-cache-memory 0`, which the image's patched KV sizing reads as "profile this shape"; the upstream pin (2x7551-32624) is a TP=2/262144 measurement and is invalid here. **Do not blank the variable instead of passing 0**: with the flag omitted vLLM sizes the cache from `GPU_UTIL` alone and the KV allocation dies with `torch.OutOfMemoryError` (measured: 2.16 GiB requested, 116 MiB free, 29.34 GiB already held) | | `MAXSEQS` | 4 | matches the per-replica share of the fleet's concurrency | | `CHUNK` | 8192 | prefill chunk (upstream default) | | `GPU_UTIL` | 0.98 | upstream default; the card is dedicated |