docs(log): compress the 2026-10-18 entries to incident + law
The 3-slot -> 2-slot reslice entry and the contended-rate entry carried the full measurement ledger; the numbers belong to [[frank-zealot]] (shape block + Eras rows), so the entry keeps the causal story (pool halving funds exact KV; rates taken under a live consumer are floors) and links for the figures.
This commit is contained in:
1 parent
04f4e2c6d4
commit
e7bfa82853
1 file changed
+14
-17
@@ -3,23 +3,20 @@
|
|||||||
## 2026-10-18
|
## 2026-10-18
|
||||||
- **frank-zealot resliced to two windows of exact KV (Frank's call, PR #21).**
|
- **frank-zealot resliced to two windows of exact KV (Frank's call, PR #21).**
|
||||||
`--ctx-size 786432 --parallel 3` at q8_0 K+V → **`524288 / 2` at f16 K+V**, so
|
`--ctx-size 786432 --parallel 3` at q8_0 K+V → **`524288 / 2` at f16 K+V**, so
|
||||||
both remaining conversations keep the native 262,144 window and the
|
both remaining conversations keep the native 262,144 window and the compaction
|
||||||
compaction hub reads exact K and V. Halving the pool is what funds it: the
|
hub reads exact K and V. **Halving the pool is what funds exact KV** — the law
|
||||||
f16 pair is 32,768 B/token over the 8 full-attention layers = 16,384.00 MiB,
|
that came out of it is that a KV byte rate belongs to the payload it was
|
||||||
while the old 786,432-position pool would have needed ~24,576 MiB of KV alone
|
measured on: the doc's old "f16 needs 26,112 MiB and cannot boot" figure was
|
||||||
on a card that leaves 24,576 addressable. Measured boot: KV **16,384.00**,
|
the 27B's 34,816 B/token applied to the 9B. The measured boot ledger is on
|
||||||
RS **100.50** (2 cells), compute 1,408.06 + 1,088.07, settled `nvidia-smi`
|
[[frank-zealot]] (Current shape), which is also where the 32,768 B/token
|
||||||
**21,176 used / 2,949 free**, `n_ctx_slot = 262144`, `DropInPaths` empty. The
|
derivation and the seam's fit gate live.
|
||||||
doc's old "f16 needs 26,112 MiB and cannot boot" figure was the 27B's
|
- **Rates are floors, and that is recorded as such.** The measurement window
|
||||||
34,816 B/token applied to the 9B — corrected in the runbook.
|
had another consumer on the other slot (56 prompts in three minutes, 96 %
|
||||||
- **Rates are floors, and that is recorded as such.** The endpoint had another
|
GPU-util) — the exact condition the bench window exists to exclude — so every
|
||||||
consumer for the whole measurement window (56 prompts on the other slot in
|
rate taken that evening is recorded on [[frank-zealot]] as a **FLOOR**, not as
|
||||||
three minutes, 96 % GPU-util), so single-stream decode came back 134.1–141.5
|
the shape's rate, with the instruction to re-measure with the box quiet before
|
||||||
(145.8/163.2 in a lighter minute) against the 3-slot record of 168.7,
|
a figure is carried anywhere. That is the honest shape of a number measured
|
||||||
two-stream aggregate 256.6–265.1 against ≈300 across three, and prefill
|
beside live traffic.
|
||||||
**3,159.5 at 106,282** beat the record's 2,877 while contended. Per-stream
|
|
||||||
concurrency improved (≈130 each vs 102.8); a clean re-measure needs the box
|
|
||||||
quiet.
|
|
||||||
- **Instrument correction (fleet law).** On the PrismML fork build 10777 the raw
|
- **Instrument correction (fleet law).** On the PrismML fork build 10777 the raw
|
||||||
`/completion` path answers **500 "The model produced output that does not
|
`/completion` path answers **500 "The model produced output that does not
|
||||||
match the expected Content-only format"** for a greedy `ignore_eos` run, so
|
match the expected Content-only format"** for a greedy `ignore_eos` run, so
|
||||||
|
|||||||
Reference in new issue
Block a user