diff --git a/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md b/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md index f68c895..ba887c3 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md +++ b/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md @@ -369,6 +369,16 @@ layer-toward-ROCm0 lever is spent: further rebalancing means a ctx cut, not a layer. The seam's balance test re-derives all of this from the env file on every run. +Verified after deploy: `rocm-smi` reads **30.84 GiB used on ROCm0 +(headroom 1.02) and 28.86 GiB on ROCm1 (headroom 3.00)** — the shift +landed 1.80 / 1.97 GiB, more than the fit's 0.58 GiB gap predicted, and +the x1 card's headroom tripled. Utilization under live traffic: 60% on +the x16 card (was 40%), 98% on the x1 card — the drafter, the output +layers and the heavier compute buffers are structurally pinned to the +last-layers card, so one DeltaNet layer was all the split lever had left +to give. A chat completion answers in 1.4 s end-to-end; MTP acceptance +runs 0.51–0.91 (mean len 2.7–3.7) on the live traffic. + ## Verified serving behaviour (2026-09-16) @@ -398,13 +408,16 @@ file on every run. selector is what distinguishes it from DFlash 1. DSpark logs under the same `draft_dflash` symbol with `block_size=7` and a Markov head, so read the numbers, not the implementation name. -- Both cards hold the model and they are **balanced at the deployed shape**: - at 393216 with the deployed batch and the 34:31 split, `rocm-smi` reports - **28.91 GiB used on ROCm0 and 30.78 GiB on ROCm1** of 31.86 GiB each — the - fit predicts 29.77 / 28.32, so ROCm1 ran 2.46 GiB hotter than predicted and - the seam's MTP constant was recalibrated to the measured value (see the - 2026-09-17 part 2 section). ROCm1's 1.08 GiB of headroom clears the 1.0 GiB - floor that this shape requires. +- Both cards hold the model. The measurement below is the **34:31-era** + record of the part 2 cutover; the deployed 35:30 shape is measured in + the part 3 section (ROCm0 30.84 GiB / ROCm1 28.86 GiB, headroom + 1.02 / 3.00 GiB, both above the 1.0 GiB floor): + at 393216 with the deployed batch and the 34:31 split, `rocm-smi` + reported **28.91 GiB used on ROCm0 and 30.78 GiB on ROCm1** of 31.86 + GiB each — the fit predicted 29.77 / 28.32, so ROCm1 ran 2.46 GiB + hotter than predicted and the seam's MTP constant was recalibrated to + the measured value (see the 2026-09-17 part 2 section). ROCm1's + 1.08 GiB of headroom cleared the 1.0 GiB floor that shape required. - Drafting is live on **both slots** (the 2026-09-16 DFlash2-era numbers below are the record; MTP-era acceptance is in the part 2 section), which is what a 2-slot deployment has to