diff --git a/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md b/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md index 314881c..e1d8f0d 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md +++ b/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md @@ -177,6 +177,15 @@ Deployed flags: slot: the raised window is real, not just declared, and it is 22 % beyond the 114688 the vLLM predecessor could offer. +- **Nothing is replicated.** The loader does not print a per-device ledger at + this verbosity, so this is measured instead of read: raising CTX_SIZE from + 131072 to 262144 added **4.83 GB to GPU 0 and 4.84 GB to GPU 1** — both cards + carry the KV for their own attention layers, about half each — and neither + card's *total* usage (24.94 / 29.42 GB at 262144) could hold the 31.46 GB + file, so no card carries a full copy of the weights. Compare the shape this + replaced: two vLLM replicas, i.e. two complete copies of the model, one per + card, which is exactly why a 29.8 GB FP8 checkpoint could not be served there + at any usable context. ### Measured throughput (2026-09-16, deployed service, `temperature 0`) Same filler corpus for every shape, so these are shapes rather than a