Conflicts and how each was resolved: - AGENTS.md: kept the 2026-10-18 Model Swaps law (a payload change is the trio edit, the deploy, /health and one probe; no bench unless Frank orders one). The incoming Benchmarking-Policy wording it replaced is not lost: its one durable fact -- on the PrismML fork build 10777 the per-request `timings` object only reaches you on /v1/chat/completions, because a greedy ignore_eos run on the raw /completion path answers 500 "Content-only format" -- is now carried inside "When a bench IS ordered", step 2, and in [[frank-zealot]]'s Contract section. - frank-zealot-llama-prism-bonsai9-mtp.md: kept deleted. PR #22's additions (the exact-KV boot ledger, the contended floor rates, the /completion instrument trap) were ported into LLM/frank-zealot.md before the merge, which is where the one-page-per-endpoint law says they live. - log.md: auto-merged, then compressed to incident plus law -- the reslice decision, the "a KV byte rate belongs to the payload it was measured on" lesson that buried the invented 26,112 MiB figure, and the instrument law. The boot ledger and the floor table now live only on [[frank-zealot]].
This commit is contained in:
@@ -1,5 +1,32 @@
|
||||
# Bundle Change Log
|
||||
|
||||
## 2026-10-18
|
||||
- **frank-zealot resliced to two windows of exact KV (Frank's call, PR #21).**
|
||||
`--ctx-size 786432 --parallel 3` at q8_0 K+V → **`524288 / 2` at f16 K+V**, so
|
||||
both remaining conversations keep the native 262,144 window and the
|
||||
compaction hub reads exact K and V. Halving the pool is what funds it: the
|
||||
f16 pair is 32,768 B/token over the 8 full-attention layers = 16,384.00 MiB,
|
||||
while the old 786,432-position pool would have needed ~24,576 MiB of KV alone
|
||||
on a card that leaves 24,576 addressable. Measured boot: KV **16,384.00**,
|
||||
RS **100.50** (2 cells), compute 1,408.06 + 1,088.07, settled `nvidia-smi`
|
||||
**21,176 used / 2,949 free**, `n_ctx_slot = 262144`, `DropInPaths` empty. The
|
||||
doc's old "f16 needs 26,112 MiB and cannot boot" figure was the 27B's
|
||||
34,816 B/token applied to the 9B — corrected in the runbook.
|
||||
- **Rates are floors, and that is recorded as such.** The endpoint had another
|
||||
consumer for the whole measurement window (56 prompts on the other slot in
|
||||
three minutes, 96 % GPU-util), so single-stream decode came back 134.1–141.5
|
||||
(145.8/163.2 in a lighter minute) against the 3-slot record of 168.7,
|
||||
two-stream aggregate 256.6–265.1 against ≈300 across three, and prefill
|
||||
**3,159.5 at 106,282** beat the record's 2,877 while contended. Per-stream
|
||||
concurrency improved (≈130 each vs 102.8); a clean re-measure needs the box
|
||||
quiet.
|
||||
- **Instrument correction (fleet law).** On the PrismML fork build 10777 the raw
|
||||
`/completion` path answers **500 "The model produced output that does not
|
||||
match the expected Content-only format"** for a greedy `ignore_eos` run, so
|
||||
the honest per-request `timings` for this endpoint must be read off
|
||||
`/v1/chat/completions`. `AGENTS.md` said `/completion`; it now says the chat
|
||||
path.
|
||||
|
||||
## 2026-10-13
|
||||
- **frank-archon outage: a kernel knob, not the model.** The 2026-10-12 23:17
|
||||
power failure on dankserver reset `/sys/module/ttm/parameters/pages_limit`
|
||||
|
||||
Reference in New Issue
Block a user