docs(frank-zealot): re-key the exact-KV shape across the vault

AGENTS.md: topology row, the frank-zealot runbook section (shape, KV
accounting, MTP verdict, compaction-hub concurrency), and two Key
Lessons bullets -- the draft-context bill restated with the real knob
names, plus a new lesson that a quant's scale overhead belongs in its
byte rate (the 6.25 % invented by doubling 13,056.00 MiB).

LLM/frank-zealot-llama-prism-bonsai9-mtp.md: retitled to the live shape;
the 3-slot q8_0 boot ledger and its measurements are relabelled as the
arm this replaced (byte-identical numbers, honest attribution), the
corrected f16 rate derivation is added, the MTP census is labelled
q8_0-era, the fallback rungs are re-derived for the 524,288 pool, and the
boot-ledger section carries an explicit PROJECTION that must be replaced
with the measured ledger from the first f16 boot.

Pointers re-pointed (live-shape sentences only; every historical ledger
left byte-identical): LLM/index.md, vault index.md, Dankclaw/OpenWebUI.md,
and the retired-runbook "today" pointers in mimo-q5km, mimo-q6k,
qwen35-9b-q6k, swift15, sglang-mimo-v26. log.md gains the dated entry.
This commit is contained in:
Frank
2026-10-18 21:02:42 -07:00
parent 68afc6dbaf
commit 71bc1735d3
11 changed files with 300 additions and 137 deletions
+25 -17
View File
@@ -26,9 +26,10 @@ MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
200704, vision ON)** and
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
Ternary-Bonsai-2-9B at **PQ2_0** — 9B qwen35 hybrid (8 full-attention + 24
Gated DeltaNet layers + the blk.32 MTP block), THREE 262,144-token
conversations, MTP present in the file but OFF by measurement,
q8_0 K + q8_0 V (f16 KV does not fit the RTX 3090 at this window), vision via
Gated DeltaNet layers + the blk.32 MTP block), TWO slots over ONE
524,288-position pool (2 × the 262,144 window), MTP present in the file but
OFF by measurement, **exact f16 K + f16 V** — 32,768 B/token = **16,384 MiB**
of KV (three 262,144 windows at exact KV do not fit the RTX 3090), vision via
the publisher's own Q8_0 mmproj, on the RTX 3090 — and the custom
tensor type 141 + `prism.hadamard.*` metadata mean no other loader can open the
file)**.
@@ -63,7 +64,7 @@ ONE 401408-position KV pool at ctx 200704 = 2 slots x ctx, MTP always-on, image
G[llama.cpp PrismML fork CUDA build
frank-zealot, port 7771]
H[Ternary-Bonsai-2-9B PQ2_0 GGUF + Q8_0 mmproj
3 x 262144 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision]
2 slots over ONE 524288-position pool at ctx 262144, exact f16 K + f16 V, MTP off, mmproj vision]
E2 --> E --> F
E2 --> G --> H
end
@@ -132,19 +133,26 @@ frank-zealot, port 7771]
custom PQ2_0 tensor type + `prism.hadamard.*` transform, and since this swap
also the minimum build that pairs MTP with a quantized V cache) serving
prism-ml/Ternary-Bonsai-2-9B-gguf **PQ2_0 with the MTP head IN the file** on
the RTX 3090 — **THREE 262,144-token conversations**
(`--ctx-size 786432 --parallel 3`, every slot at the window the model was
trained at), **q8_0 K + q8_0 V** (17 KiB/token over the 8 full layers =
13,056 MiB ledger-exact; the f16 pair needs 26,112 MiB and dies at
`cudaMalloc`), settled **17,876 used / 6,249 free**, vision via the
publisher's **Q8_0** mmproj (Q8_0 is the publisher's requirement for
concurrent image requests, and this runs three slots), **MTP OFF by
measurement** — the draft context bills its own f16 KV over the WHOLE pool
(3,072 MiB) and at `UBATCH_SIZE=2048` that squeeze halves decode
(167.2 → 109.2 t/s); `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` is the
documented 213.5 t/s arm, declined for +17 % prefill and 6.2 GiB of headroom.
Measured: 167.2 t/s single-stream decode, 2,877 t/s prefill at 106,418,
280.2 t/s aggregate across three streams. `--metrics` rides (`METRICS=1`):
the RTX 3090 — **TWO 262,144-token conversations**
(`--ctx-size 524288 --parallel 2`, every slot at the window the model was
trained at), **unquantized f16 K + f16 V** (32,768 B/token over the 8 full
layers = **16,384 MiB** exact; the POOL halved because that rate over the
OLD 786,432 pool bills 24,576 MiB against the 24,123 the driver leaves
addressable, and the old "26,112 MiB" note was wrong — it doubled the
q8_0 figure (17,408 B/token = 13,056 MiB) *including* its 1/32 scale
overhead, so f16/q8_0 is **1.882×**, not 2×), the q8_0-era boot settled
**17,876 used / 6,249 free**, vision via the publisher's **Q8_0** mmproj
(Q8_0 is the publisher's requirement for concurrent image requests, and this
runs two slots), **MTP OFF by measurement** — the draft context bills its own
f16 KV over the WHOLE pool (4,096 B/position × 524,288 = 2,048 MiB) and at
`UBATCH_SIZE=2048` that squeeze halves decode (167.2 → 109.2 t/s);
`SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` is the documented 213.5 t/s arm,
declined for +17 % prefill and 6.2 GiB of headroom — and exact KV leaves an
f16-draft arm no room at all (a quantized draft, `--spec-draft-type-* q8_0`,
is the unmeasured door).
Measured at the q8_0-era shape: 167.2 t/s single-stream decode, 2,877 t/s
prefill at 106,418, 280.2 t/s aggregate across three streams. `--metrics`
rides (`METRICS=1`):
llama.cpp answers 501 on `/metrics` without it, which is why Prometheus
`up{job="frank-zealot"}` read 0 for its whole retained history until
2026-10-18. `contextWindow: 262144` is the **widest in the fleet**, so this