docs(frank-zealot): re-key the exact-KV shape across the vault

AGENTS.md: topology row, the frank-zealot runbook section (shape, KV
accounting, MTP verdict, compaction-hub concurrency), and two Key
Lessons bullets -- the draft-context bill restated with the real knob
names, plus a new lesson that a quant's scale overhead belongs in its
byte rate (the 6.25 % invented by doubling 13,056.00 MiB).

LLM/frank-zealot-llama-prism-bonsai9-mtp.md: retitled to the live shape;
the 3-slot q8_0 boot ledger and its measurements are relabelled as the
arm this replaced (byte-identical numbers, honest attribution), the
corrected f16 rate derivation is added, the MTP census is labelled
q8_0-era, the fallback rungs are re-derived for the 524,288 pool, and the
boot-ledger section carries an explicit PROJECTION that must be replaced
with the measured ledger from the first f16 boot.

Pointers re-pointed (live-shape sentences only; every historical ledger
left byte-identical): LLM/index.md, vault index.md, Dankclaw/OpenWebUI.md,
and the retired-runbook "today" pointers in mimo-q5km, mimo-q6k,
qwen35-9b-q6k, swift15, sglang-mimo-v26. log.md gains the dated entry.
This commit is contained in:
Frank
2026-10-18 21:02:42 -07:00
parent 68afc6dbaf
commit 71bc1735d3
11 changed files with 300 additions and 137 deletions
+88 -54
View File
@@ -27,7 +27,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|--------|---------|---------|-------|------|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool (= 2 × ctx) at `HALOGEN_CTX=200704`** (pool RAISED from 200,704 and DEPLOYED 2026-10-18 so BOTH slots can hold a maximum-length request at once; the parity play was frank-zealot's then-401,408 total in-flight context — zealot moved to 786,432 the same evening and dragoon's pool was NOT raised again, because every position is page cache the n-gram table loses; **11.0 GiB reserved, measured on that deploy** (the preflight formula bills 11.3) where 200,704 cost 5.5, 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; the ceiling does NOT move with it — 262,144 is the native wall and the engine refuses more without `HALOGEN_ROPE_YARN`), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770|
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10777**, commit `e3de71522`; root systemd unit, relinked — the ONLY loader for this payload, and since 2026-10-18 also the minimum build that pairs MTP with a quantized V cache) | `prism-ml/Ternary-Bonsai-2-9B-gguf` **PQ2_0 carrying the MTP head in the file** (2,675,410,720 B, Hub-LFS sha256 `c79fcdfa…` verified at fetch by `scripts/download-frank-zealot-bonsai9.sh`; `qwen35` hybrid: `block_count 33` = **32 trunk layers** (8 full-attention + 24 Gated DeltaNet at `full_attention_interval 4`) + the `blk.32` MTP block, 4 KV heads × 256 dim, native 262,144; `general.file_type 141` with **202 custom-type PQ2_0 tensors of 442** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (624,229,664 B, sha256 `e0cb6f69…`; **Q8_0 is the publisher's requirement** for concurrent image requests — the BF16 tower can crash the server, and this endpoint runs three slots) for **vision**, on the RTX 3090, **THREE 262,144-token slots** (`--ctx-size 786432 --parallel 3` = every conversation at the trained window), **q8_0 K + q8_0 V** under flash attention (17 KiB/token over the 8 full layers = 13,056 MiB pool, ledger-exact; **f16 KV needs 26,112 MiB and cannot boot this card**; +151 MiB of recurrent state for the three DeltaNet cells — the SPEC-ARMED arms print 452 MiB, because MTP depth 2 bills three seqs per slot), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF — measured, not because the head is absent**: the draft context bills its own f16 KV over the WHOLE pool (3,072 MiB) and at ubatch 2048 that squeeze halves decode (167.2 → 109.2 t/s); the documented MTP arm is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s. Production boot 17,922 used / 6,203 free; **168.7 t/s** single-stream greedy decode (three runs, 168.4–168.7) and **~300 t/s** aggregate across three concurrent streams (102.8 each); 2,877 t/s prefill at 106,418 was the trial arm. Cutover 2026-10-18 from Ternary-Bonsai-2-**27B** PQ2_0 at 2 × 200,704 (payload + build 10754 tree on disk as rollback). | 7771 |
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10777**, commit `e3de71522`; root systemd unit, relinked — the ONLY loader for this payload; the b10776 floor that gates MTP beside a quantized V cache is moot under exact KV, but the custom tensor type alone keeps the build pinned) | `prism-ml/Ternary-Bonsai-2-9B-gguf` **PQ2_0 carrying the MTP head in the file** (2,675,410,720 B, Hub-LFS sha256 `c79fcdfa…` verified at fetch by `scripts/download-frank-zealot-bonsai9.sh`; `qwen35` hybrid: `block_count 33` = **32 trunk layers** (8 full-attention + 24 Gated DeltaNet at `full_attention_interval 4`) + the `blk.32` MTP block, 4 KV heads × 256 dim, native 262,144; `general.file_type 141` with **202 custom-type PQ2_0 tensors of 442** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (624,229,664 B, sha256 `e0cb6f69…`; **Q8_0 is the publisher's requirement** for concurrent image requests — the BF16 tower can crash the server, and this endpoint runs several slots) for **vision**, on the RTX 3090, **TWO 262,144-token slots** (`--ctx-size 524288 --parallel 2` = every conversation STILL at the trained window — the per-slot contract never moved) over **unquantized f16 K + f16 V** (32 KiB/token over the 8 full layers = **16,384 MiB** pool; the POOL halved because exact KV at the old 786,432 pool is 24,576 MiB against the 24,123 MiB the driver leaves addressable — the repo's older "26,112 MiB" figure had doubled the q8_0 number including its 1/32 scale overhead; +100.5 MiB of recurrent state for the two DeltaNet cells, ~50.25 MiB a cell), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF — measured, and now doubly so**: the draft context bills its own f16 KV over the WHOLE pool (3,072 MiB at the old pool, 2,048 MiB at this one) and at ubatch 2048 that squeeze halves decode (167.2 → 109.2 t/s); the documented q8_0-era MTP arm is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s. Last MEASURED boot (the 3 × 262,144 q8_0 arm this replaces): 17,922 used / 6,203 free, **168.7 t/s** single-stream greedy decode (three runs, 168.4–168.7), **~300 t/s** aggregate across three concurrent streams (102.8 each), 2,877 t/s prefill at 106,418 — the f16 boot's ledger and rates land in the runbook once measured. Cutover 2026-10-18 from Ternary-Bonsai-2-**27B** PQ2_0 at 2 × 200,704 (payload + build 10754 tree on disk as rollback). | 7771 |
Retired endpoints (`frank-npu`, and every prior runtime) are in
the [runbook index](#runbook-index). The full swap history lives in
@@ -350,9 +350,9 @@ and `frank-zealot` through
the publisher's own **Q8_0** mmproj passed as `--mmproj` — Q8_0 is the
publisher's requirement for concurrent image requests (the card states the
BF16 tower can crash `llama-server` when several image requests run at once,
and this endpoint runs three slots). `frank-zealot`'s `contextWindow` is
**262144** — the PER-SLOT window (`--ctx-size 786432 --parallel 3` since the
2026-10-18 Bonsai-2-**9B** swap), which since that swap EQUALS the model's
and this endpoint runs several slots). `frank-zealot`'s `contextWindow` is
**262144** — the PER-SLOT window (`--ctx-size 524288 --parallel 2` since the
2026-10-18 reslice to exact f16 KV), which EQUALS the model's
native window; llama.cpp answers a longer prompt with a 400 naming the limit,
never a truncation, so the harness must compact before it.
Note also
@@ -510,7 +510,8 @@ llama.cpp refuses the custom type at load, and a loader that
ignored the Hadamard metadata would dequant WRONG — same lock class as the
ROCmFPX era. On top of that lock, **the fork gates MTP beside a quantized V
cache on build `prism-b10776-d00e252` or newer**, which is why the prebuilt
moved 10754 → 10777; the 10754 tree stays on disk as the 27B rollback runtime.
moved 10754 → 10777 (exact f16 KV makes that gate moot — the custom type alone
pins the runtime); the 10754 tree stays on disk as the 27B rollback runtime.
Retargeting the payload without the fork ships an endpoint that cannot load,
so the seam pins both paths.
@@ -520,36 +521,47 @@ attention + FFN in q8_0, which is why the draft context bills real KV), and
`qwen35.full_attention_interval = 4` ⇒ **8 full-attention + 24 Gated
DeltaNet**, 4 KV heads × 256 dim, native 262,144. **KV accounting:** the pool
scales with the 8 full layers — 8 × 4 × 256 = 8,192 elements/token per cache,
so **q8_0 K + q8_0 V bills 17,408 B/token ≈ 17 KiB** = **13,056.00 MiB** at the
786,432 pool (ledger-exact) — exactly HALF the 27B's 34,816 B/token, which is
the whole reason three native windows fit where the 27B afforded only two
shortened ones. **f16 K would bill 26,112 MiB and cannot boot on a 24,123 MiB
card.** The 24 DeltaNet layers bill a fixed recurrent state per SLOT — the
production ledger prints `RS buffer size = 150.75 MiB (3 cells, 32 layers,
3 seqs)`, while the SPEC-ARMED arms print 452.25 MiB (MTP depth 2 bills three
seqs per slot, so 9 cells) — so pool arithmetic alone under-prices the shape.
so **the deployed exact pair f16 K + f16 V bills 32,768 B/token = 32 KiB** =
**16,384 MiB** at the 524,288 pool. The q8_0 pair this endpoint shipped first
billed **17,408 B/token** (1.0625 B/element: 32 codes plus one 2-byte scale per
32-block) = **13,056.00 MiB** at the old 786,432 pool, ledger-exact — exactly
HALF the 27B's 34,816 B/token. **Exact KV at that 786,432 pool is 24,576 MiB,
more than the 24,123 MiB the driver leaves addressable, so the POOL is what
pays for f16 — the window never moved.** (The "26,112 MiB" figure this repo
carried doubled the q8_0 number *including* its scale overhead; the true ratio
is 1.882×, not 2×.) The 24 DeltaNet layers bill a fixed recurrent state per
SLOT — ~50.25 MiB a cell, so the 3-slot ledger printed `RS buffer size =
150.75 MiB (3 cells, 32 layers, 3 seqs)` and two slots bill 100.50, while the
SPEC-ARMED arms print 452.25 MiB (MTP depth 2 bills three seqs per slot, so 9
cells) — so pool arithmetic alone under-prices the shape.
**The shape (2026-10-18):** `--ctx-size 786432 --parallel 3` = THREE
conversations of 262,144, every one at the model's TRAINED window (Frank: "if
you can possibly fit a third context, do it"), `--flash-attn on`, **q8_0 K +
q8_0 V**, batch 4096 / **ubatch 2048** (the measured CUDA optimum carried from
the MiMo/Swift A/Bs on this card; it also sets the compute-buffer size, which
is what decides whether MTP fits), `--log-verbosity 4`, `--metrics`,
`--jinja`, `--mmproj` for vision, **speculative decoding OFF**
(`SPEC_TYPE=none`). Boot: **17,876 MiB used / 6,249 free**; compute 1,409 MiB
CUDA + 1,201 MiB CUDA_Host, projector 595 + 248 MiB. Measured on the same
boot: **167.2 t/s** greedy decode, 158.9 t/s single-stream chat, **280.2 t/s
aggregate across three concurrent streams**, **2,877 t/s prefill at 106,418**
tokens — the 27B record (76.6 decode / 109.4 aggregate / 691.7 prefill at 166k)
beaten on every axis. The **first request after boot is a warmup artifact**
(37.7 t/s class) — never report it.
**The shape (2026-10-18, resliced the same evening):** `--ctx-size 524288
--parallel 2` = TWO conversations of 262,144, every one still at the model's
TRAINED window (the first arm was THREE — Frank: "if you can possibly fit a
third context, do it" — and the exact-KV reslice handed one back),
`--flash-attn on`, **unquantized f16 K + f16 V**, batch 4096 / **ubatch 2048**
(the measured CUDA optimum carried from the MiMo/Swift A/Bs on this card; it
also sets the compute-buffer size, which is what decides whether MTP fits),
`--log-verbosity 4`, `--metrics`, `--jinja`, `--mmproj` for vision,
**speculative decoding OFF** (`SPEC_TYPE=none`). The MEASURED record still
belongs to the arm this replaced (3 × 262,144, q8_0 K + q8_0 V): boot **17,876
MiB used / 6,249 free**; compute 1,409 MiB CUDA + 1,201 MiB CUDA_Host, projector
595 + 248 MiB; **167.2 t/s** greedy decode, 158.9 t/s single-stream chat,
**280.2 t/s aggregate across three concurrent streams**, **2,877 t/s prefill at
106,418** tokens — the 27B record (76.6 decode / 109.4 aggregate / 691.7 prefill
at 166k) beaten on every axis. The exact-KV arm's ledger and rates go in the
runbook once booted; its projection is ~21,200 MiB used / ~2,900 free, since
only the KV pool (+3,328) and one recurrent-state cell (−50) move. The **first
request after boot is a warmup artifact** (37.7 t/s class) — never report it.
**Why MTP is off even though the head is in the file.** The drafter is GOOD —
acceptance 0.798 / 0.617 per position, mean accepted length 2.41 — and the card
recommends depth 2. It still loses AT THIS SHAPE because the draft context
bills its OWN **f16** KV over the WHOLE pool (4,096 B/position × 786,432 =
**3,072.00 MiB**, 4,830 MiB with its compute buffers) and ubatch 2048 refuses to
give it up. Booted arms, same card, same traffic (2026-10-18):
recommends depth 2. It still loses because the draft context bills its OWN
**f16** KV over the WHOLE
pool (4,096 B/position — **3,072.00 MiB** at the 786,432 pool, 4,830 MiB with
its compute buffers; **2,048.00 MiB** at the shipped 524,288 pool) and ubatch
2048 refuses to give it up. Booted arms, same card, same traffic (2026-10-18,
every row q8_0 K + q8_0 V at the 786,432 pool):
| slots | spec | ubatch | used / free | decode |
|---|---|---|---|---|
@@ -563,13 +575,17 @@ give it up. Booted arms, same card, same traffic (2026-10-18):
drafter, is what halves it at three windows. The arm that WOULD run it at three
windows is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` (213.5 t/s single-stream,
250.8 three-stream aggregate) and pays prefill −17 % and 2.7 GiB of headroom on
a card whose projector worst-case lands AFTER the engine's own fit check. This
endpoint is the fleet's compaction hub — prefill-bound, 2–3 concurrent
subagents — so the shipped arm takes the +17 % prefill (2,877 vs 2,395 t/s) and
the +12 % three-stream aggregate (280.2 vs 250.8) over single-stream speed, and
keeps 6.2 GiB free. `SPEC_TYPE=none` strips the WHOLE spec set (`--spec-type`
and every `--spec-draft-*`) — flags move as a set or the arm measures a crash
instead of an absence.
a card whose projector worst-case lands AFTER the engine's own fit check. Under
**exact KV the drafter is arithmetically out**: the trunk projects ~21,200 MiB
of the 24,123 MiB addressable before the draft context's 2,048 MiB and its
buffers are even asked for, so the doors back to speculation are ONE window per
conversation, or KV back to q8_0.
The compaction-hub load is what makes exact KV affordable: prefill is
weight-bandwidth bound and does not re-read the cache per generated token, so
the dtype is paid in decode and concurrency (two conversations now, not three),
not in the work this endpoint exists to do. `SPEC_TYPE=none` strips the WHOLE
spec set (`--spec-type` and every `--spec-draft-*`) — flags move as a set or the
arm measures a crash instead of an absence.
**Sampler**: the recipe lives in the checkpoint (`general.sampling.*` =
temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0 / repeat_penalty 1.0,
@@ -581,7 +597,7 @@ would silently re-add the flag on an explicit empty value).
**Vision**: the publisher's own **Q8_0** mmproj via `--mmproj` — Q8_0 is
MANDATORY, not a preference: the card says the BF16 tower "can crash
llama-server when several image requests run at once", and `--parallel 3` is
llama-server when several image requests run at once", and `--parallel 2` is
exactly that shape. Probe traps carry over: ≥256 px probe images or the model
answers "video", only the OpenAI `image_url` content part reaches the model,
and thinking tokens count against `max_tokens` — budget ≥512 or the probe
@@ -596,10 +612,12 @@ have to be fetched first.
**Compaction hub**: `models.yml` routes `compactionModel` for
`frank-archon`, `frank-dragoon` and `frank-zealot` (the 3090 endpoint) here —
262,144 (since the 2026-10-18 reslice) is the widest window in the fleet and
the only one that comfortably fits a worst-case compaction input (~170k
tokens). With `--parallel 3` a compaction request no longer serializes behind a
live generation (the Gemma4 era's `--parallel 1` law). Retrieval through a
262,144 (still the PER-SLOT window after the exact-KV reslice) is the widest
window in the fleet and the only one that comfortably fits a worst-case
compaction input (~170k tokens). With `--parallel 2` a compaction request still
runs beside ONE live generation, and with two already busy it waits — the Gemma4
era's `--parallel 1` law at half strength, which is the concurrency price of
exact KV. Retrieval through a
25,960-token prompt was verified exact after the swap.
**Contract law learned at verbosity 4:** a raw `/completion` request prints
@@ -1040,18 +1058,34 @@ linked runbooks.
- **A draft context bills its own f16 KV over the WHOLE POOL, and on a
single-card endpoint that squeeze can cost more than the drafter wins.** The
draft context is built at the target's `n_ctx`, so with `--parallel N` the
drafter pays the POOL, not the slot: on frank-zealot's 786,432-position
qwen35 pool, `--spec-type draft-mtp` added **3,072.00 MiB of f16 draft KV**
(4,096 B/position — the draft cache is never quantized) plus its own compute
buffers, 4,830 MiB all told. Booted arms, same card, same traffic
(2026-10-18): 3 slots + ubatch 2048 → **167.2 t/s decode with spec OFF,
109.2 with spec ON**; dropping `UBATCH_SIZE` to 1024 hands the buffers back
and spec ON gives **213.5** (+28 % over the spec-OFF arm), and at 1–2 slots
the SAME card runs MTP at ubatch 2048 and still gains ~28 %. So the MTP
question is never "does the file carry the head" — it is "what do draft KV
+ buffers cost against whatever the batch knobs keep for compute buffers",
and that answer is per-shape. Measure the arm, never the flag. See
drafter pays the POOL, not the slot: on frank-zealot's qwen35 pool
`--spec-type draft-mtp` adds **4,096 B/position of draft KV at the default
f16 dtype** (`--spec-draft-type-k/-v` can quantize the draft — untested here),
**3,072.00 MiB** at the 786,432 pool it shipped with and **2,048.00 MiB** at
the 524,288 pool it runs now, plus its own compute buffers (4,830 MiB all told
at the old pool). Booted arms, same card, same traffic (2026-10-18, every row
q8_0 KV): 3 slots + ubatch 2048 → **167.2 t/s decode with spec OFF, 109.2 with
spec ON**; dropping `UBATCH_SIZE` to 1024 hands the buffers back and spec ON
gives **213.5** (+28 % over the spec-OFF arm), and at 1–2 slots the SAME card
runs MTP at ubatch 2048 and still gains ~28 %. So the MTP question is never
"does the file carry the head" — it is "what do draft KV + buffers cost
against whatever the batch knobs keep for compute buffers", and that answer is
per-shape. Measure the arm, never the flag. Production corollary: beside
exact KV an **f16-draft** arm has no room — the trunk projects ~21,200 MiB of
the 24,123 addressable before the drafter is asked for anything; the untested
door is `--spec-draft-type-k/-v q8_0` (1,088 MiB). See
[[frank-zealot-llama-prism-bonsai9-mtp]].
- **A quant's scale overhead belongs in its byte rate, and doubling a measured
pool figure to "get the f16 number" invents 6.25 %.** This repo carried "f16
KV would bill 26,112 MiB" for weeks by doubling the q8_0 ledger figure
(13,056.00 MiB). q8_0 is 1.0625 B/element (32 codes plus one 2-byte scale per
32-block), f16 is exactly 2.0 ⇒ the true ratio is **1.882×**, so exact KV over
that pool is **24,576 MiB**. The conclusion (three f16 windows cannot boot a
card that leaves 24,123 MiB addressable) survived; the number did not — and
the honest rate is what made the resize calculable, because it shows 524,288
positions is the largest exact-KV pool that fits with room for the projector.
Derive byte rates from element count × block layout, then check them against
the ledger.
- **`--ubatch-size` has an OPTIMUM on the 2x R9700 layer split, not a plateau —
and the curve is BACKEND-SPECIFIC, so never carry it across (next bullet). Its
compute buffers land about TWICE as heavy on the card that holds the output