frank-medium runbook: record the verified 35:30 post-deploy state
This commit is contained in:
@@ -369,6 +369,16 @@ layer-toward-ROCm0 lever is spent: further rebalancing means a ctx cut,
|
||||
not a layer. The seam's balance test re-derives all of this from the env
|
||||
file on every run.
|
||||
|
||||
Verified after deploy: `rocm-smi` reads **30.84 GiB used on ROCm0
|
||||
(headroom 1.02) and 28.86 GiB on ROCm1 (headroom 3.00)** — the shift
|
||||
landed 1.80 / 1.97 GiB, more than the fit's 0.58 GiB gap predicted, and
|
||||
the x1 card's headroom tripled. Utilization under live traffic: 60% on
|
||||
the x16 card (was 40%), 98% on the x1 card — the drafter, the output
|
||||
layers and the heavier compute buffers are structurally pinned to the
|
||||
last-layers card, so one DeltaNet layer was all the split lever had left
|
||||
to give. A chat completion answers in 1.4 s end-to-end; MTP acceptance
|
||||
runs 0.51–0.91 (mean len 2.7–3.7) on the live traffic.
|
||||
|
||||
|
||||
## Verified serving behaviour (2026-09-16)
|
||||
|
||||
@@ -398,13 +408,16 @@ file on every run.
|
||||
selector is what distinguishes it from DFlash 1. DSpark logs under the same
|
||||
`draft_dflash` symbol with `block_size=7` and a Markov head, so read the
|
||||
numbers, not the implementation name.
|
||||
- Both cards hold the model and they are **balanced at the deployed shape**:
|
||||
at 393216 with the deployed batch and the 34:31 split, `rocm-smi` reports
|
||||
**28.91 GiB used on ROCm0 and 30.78 GiB on ROCm1** of 31.86 GiB each — the
|
||||
fit predicts 29.77 / 28.32, so ROCm1 ran 2.46 GiB hotter than predicted and
|
||||
the seam's MTP constant was recalibrated to the measured value (see the
|
||||
2026-09-17 part 2 section). ROCm1's 1.08 GiB of headroom clears the 1.0 GiB
|
||||
floor that this shape requires.
|
||||
- Both cards hold the model. The measurement below is the **34:31-era**
|
||||
record of the part 2 cutover; the deployed 35:30 shape is measured in
|
||||
the part 3 section (ROCm0 30.84 GiB / ROCm1 28.86 GiB, headroom
|
||||
1.02 / 3.00 GiB, both above the 1.0 GiB floor):
|
||||
at 393216 with the deployed batch and the 34:31 split, `rocm-smi`
|
||||
reported **28.91 GiB used on ROCm0 and 30.78 GiB on ROCm1** of 31.86
|
||||
GiB each — the fit predicted 29.77 / 28.32, so ROCm1 ran 2.46 GiB
|
||||
hotter than predicted and the seam's MTP constant was recalibrated to
|
||||
the measured value (see the 2026-09-17 part 2 section). ROCm1's
|
||||
1.08 GiB of headroom cleared the 1.0 GiB floor that shape required.
|
||||
- Drafting is live on **both slots** (the 2026-09-16 DFlash2-era numbers below
|
||||
are the record; MTP-era acceptance is in the part 2 section), which is what
|
||||
a 2-slot deployment has to
|
||||
|
||||
Reference in New Issue
Block a user