From 24cdbb0e456f8e77dcf869add6467f76a61dc4ff Mon Sep 17 00:00:00 2001 From: Frank Date: Thu, 17 Sep 2026 00:29:21 -0700 Subject: [PATCH] docs: the VRAM figures settle 0.2 GiB below the post-load reading 6.44 / 6.90 is the state straight after load; after the service has served traffic it sits at 6.23 / 6.64, with and without requests in flight. Both are ~5 GiB above the floor, but the runbook should not imply a load-dependent difference that the two readings do not actually show. --- serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md b/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md index 6953404..fa9ec7a 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md +++ b/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md @@ -282,9 +282,10 @@ larger, so the question is closed rather than pending. - Both cards hold the model and they are **balanced**: at 262144 with the deployed batch and split, `rocm-smi` reports **6.44 GiB free on GPU 0 and 6.90 GiB on GPU 1** of 31.86 GiB each — reproducing the pre-deploy - measurement of the same shape to 0.01 GiB. Under load (two concurrent - requests, one per slot) they sit at **6.23 / 6.64**, and both are far above - the operator's 1.5 GiB floor. + measurement of the same shape to 0.01 GiB. That is the reading straight after + load; once the service has served traffic it settles ~0.2 GiB lower + (**6.23 / 6.64**, the same with two concurrent requests in flight and after + them). Both figures are ~5 GiB above the operator's 1.5 GiB floor. - Drafting is live on **both slots**, which is what a 2-slot deployment has to prove: `slot print_timing` reported `draft acceptance = 0.62857 (44 accepted / 70 generated), mean len = 5.40` and `0.48214 (27 / 56)` on slot 1, and