docs(frank-zealot): re-key the exact-KV shape across the vault

AGENTS.md: topology row, the frank-zealot runbook section (shape, KV
accounting, MTP verdict, compaction-hub concurrency), and two Key
Lessons bullets -- the draft-context bill restated with the real knob
names, plus a new lesson that a quant's scale overhead belongs in its
byte rate (the 6.25 % invented by doubling 13,056.00 MiB).

LLM/frank-zealot-llama-prism-bonsai9-mtp.md: retitled to the live shape;
the 3-slot q8_0 boot ledger and its measurements are relabelled as the
arm this replaced (byte-identical numbers, honest attribution), the
corrected f16 rate derivation is added, the MTP census is labelled
q8_0-era, the fallback rungs are re-derived for the 524,288 pool, and the
boot-ledger section carries an explicit PROJECTION that must be replaced
with the measured ledger from the first f16 boot.

Pointers re-pointed (live-shape sentences only; every historical ledger
left byte-identical): LLM/index.md, vault index.md, Dankclaw/OpenWebUI.md,
and the retired-runbook "today" pointers in mimo-q5km, mimo-q6k,
qwen35-9b-q6k, swift15, sglang-mimo-v26. log.md gains the dated entry.
This commit is contained in:
Frank committed 2026-10-18 21:02:42 -07:00
1 parent 68afc6dbaf
commit 71bc1735d3
11 files changed
+300 -137

No files matched your search

+34
View File
@@ -2787,3 +2787,37 @@ every restart since the cutover.
to the player, temp dir cleaned; the audible pass through the default
`pw-play` detection confirmed by Frank. Sessions open before the merge need
a restart to pick the command up.
## 2026-10-18 — frank-zealot: 2 × 262,144 over unquantized f16 KV
- Frank's instruction: **"shift frank-zealot to 2 parallels of unquantized
kv"**. The env delta is four lines: `CTX_SIZE=524288`, `PARALLEL=2`,
`CACHE_TYPE_K=f16`, `CACHE_TYPE_V=f16`. Everything else holds — batch
**4096** / ubatch **2048**, `--flash-attn on`, `--jinja`, `--metrics`,
`--presence-penalty 1.0`, `SPEC_TYPE=none`, the Q8_0 mmproj, port 7771, and
the per-slot window **262,144** — `models.yml` `contextWindow` untouched.
- **Why the pool had to halve.** Exact KV bills **32,768 B/token** (8
full-attention layers × 4 KV heads × 256 dim × 2 B × 2 caches), so the old
786,432 pool asks **24,576 MiB** against the **24,123 MiB** the driver leaves
addressable — impossible, not tight. 524,288 positions ⇒ **16,384 MiB** and
exactly two windows of 262,144.
- Arithmetic correction to the repo's long-standing figure: "the f16 pair would
need **26,112 MiB**" doubled the q8_0 number *including* its 1/32 scale
overhead — q8_0 is 17,408 B/token (1.0625 B/element), so f16/q8_0 is
**1.882×**, not 2×, and exact f16 over 786,432 is **24,576 MiB**. The
conclusion stands (three f16 windows cannot boot this card); the number was
wrong.
- The cost: **concurrency 3 → 2 conversations**, and MTP is now **doubly** out.
The 13:16 measurement still says off, and exact KV adds a second reason — the
draft context bills its own f16 KV over the WHOLE pool (**4,096 B/position**
× 524,288 = **2,048 MiB**) on a trunk projecting ~21,200 used.
- **Projection, explicitly not a measurement:** ~**21,200 MiB** used /
~**2,920 free**, derived from the 13:19 ledger's 17,922 used because only two
terms move — the KV pool (**+3,328**) and one recurrent-state cell
(**−50.25**, 3 cells → 2). The measured boot ledger and the decode/prefill
re-measure land in the follow-up commit, after the first f16 boot.
- New gate in `tests/frank_zealot_unit.bats`: dtype × pool plus the measured
non-KV terms must fit **24,123 MiB** with **≥ 1 GiB** of slack — and the same
gate proves it refuses the impossible arm (**f16 at 786,432**), not merely
passes the shipped one. Runbook re-keyed to the new shape:
`LLM/frank-zealot-llama-prism-bonsai9-mtp.md`.