docs(frank-zealot): re-key the exact-KV shape across the vault
AGENTS.md: topology row, the frank-zealot runbook section (shape, KV accounting, MTP verdict, compaction-hub concurrency), and two Key Lessons bullets -- the draft-context bill restated with the real knob names, plus a new lesson that a quant's scale overhead belongs in its byte rate (the 6.25 % invented by doubling 13,056.00 MiB). LLM/frank-zealot-llama-prism-bonsai9-mtp.md: retitled to the live shape; the 3-slot q8_0 boot ledger and its measurements are relabelled as the arm this replaced (byte-identical numbers, honest attribution), the corrected f16 rate derivation is added, the MTP census is labelled q8_0-era, the fallback rungs are re-derived for the 524,288 pool, and the boot-ledger section carries an explicit PROJECTION that must be replaced with the measured ledger from the first f16 boot. Pointers re-pointed (live-shape sentences only; every historical ledger left byte-identical): LLM/index.md, vault index.md, Dankclaw/OpenWebUI.md, and the retired-runbook "today" pointers in mimo-q5km, mimo-q6k, qwen35-9b-q6k, swift15, sglang-mimo-v26. log.md gains the dated entry.
This commit is contained in:
1 parent
68afc6dbaf
commit
71bc1735d3
11 files changed
+300
-137
No files matched your search
@@ -409,10 +409,15 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
loader for the custom PQ2_0 tensor type / `prism.hadamard.*`, and the floor
|
||||
for MTP beside a quantized V cache) serving
|
||||
prism-ml/Ternary-Bonsai-2-9B-gguf **PQ2_0 with the MTP head** on the RTX 3090
|
||||
— THREE 262,144-token conversations, q8_0 K + q8_0 V (f16 OOMs at this pool:
|
||||
26,112 vs 13,056 MiB over 8 full-attention layers), MTP **off by
|
||||
measurement** (the draft context bills 3,072 MiB of f16 KV over the whole
|
||||
pool), vision through the Q8_0 mmproj; boot 17,876 used / 6,249 free
|
||||
— TWO 262,144-token conversations (`--ctx-size 524288 --parallel 2`), exact
|
||||
f16 K + f16 V = **16,384 MiB** over 8 full-attention layers (32,768 B/token;
|
||||
the per-slot window is UNCHANGED at 262,144 — what halved is the POOL,
|
||||
because exact f16 over the OLD 786,432 pool bills **24,576 MiB** against the
|
||||
**24,123** the driver leaves addressable; the old "26,112 vs 13,056 MiB" note
|
||||
doubled the q8_0 figure with its 1/32 scale overhead included, so f16/q8_0 is
|
||||
1.882×, not 2×), MTP **off by measurement** (the draft context bills 2,048
|
||||
MiB of f16 KV over the whole pool, and exact KV leaves no room for one),
|
||||
vision through the Q8_0 mmproj; boot 17,876 used / 6,249 free (the q8_0 era)
|
||||
- [[frank-zealot-llama-prism-bonsai2-pq20]] — RETIRED 2026-10-17 → 10-18
|
||||
(rollback target, payload + build 10754 on disk): Ternary-Bonsai-2-**27B**
|
||||
PQ2_0 at TWO 200,704-token conversations, q8_0 K + q8_0 V, MTP **impossible**
|
||||
|
||||
Reference in new issue
Block a user