feat(frank-zealot): Bonsai-2-9B PQ2_0 with the MTP head at 3 x 262,144

Swap the 3090 endpoint from Ternary-Bonsai-2-27B PQ2_0 (2 x 200,704,
fork build 10754) to Ternary-Bonsai-2-9B PQ2_0 with the MTP head IN the
file, three conversations at the trained 262,144 window
(--ctx-size 786432 --parallel 3), on vendor fork build 10777 -- the
minimum build that pairs MTP with a quantized V cache. q8_0 K + q8_0 V
is the only KV that fits: 13,056 MiB against a 26,112 MiB f16 pair.
Settled boot 17,876 used / 6,249 free.

MTP is armed and then DECLINED by measurement: the draft context bills
its own f16 KV over the whole pool (3,072 MiB) and at UBATCH_SIZE=2048
that halves decode (167.2 -> 109.2 t/s). The documented MTP arm
(SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024, 213.5 t/s) is recorded in the
env header rather than left to rediscovery.

Two bugs the trial caught: ${SPEC_PRESENCE_PENALTY:=0.0} was a
default-assignment, so the env value always won and the no-presence-penalty
MTP arm was unreachable (now ${VAR=0.0}); and the Q8_0 mmproj is the
publisher's requirement for concurrent image requests, which matters at
--parallel 3.

contextWindow is now 262144 -- the widest in the fleet -- so :7771 is the
compaction hub. 27B runbook re-banned RETIRED with its bench-window
section closed; indexes, homepage card, OpenWebUI doc and the worklog
re-keyed.
This commit is contained in:
Frank
2026-10-18 13:15:18 -07:00
parent 2b127163c8
commit 466fd477d8
14 changed files with 1001 additions and 314 deletions
+13 -7
View File
@@ -399,13 +399,19 @@ The LLM inference fleet on the two hosts. Full index:
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
- [[frank-zealot-llama-prism-bonsai2-pq20]] — the **ACTIVE** frank-zealot
runbook since 2026-10-17: the PrismML vendor llama.cpp build 10754 (the only
loader for GGML type 142 / `prism.hadamard.*`) serving
prism-ml/Ternary-Bonsai-2-27B-gguf **PQ2_0** on the RTX 3090 — TWO
200,704-token conversations, q8_0 K + q8_0 V (f16 OOMs at 2×200,704: 34 KiB
vs 64 KiB per token over 16 full-attention layers), MTP **off** (no head in
the file), vision through the Q8_0 mmproj; boot 22,664 used / 1,461 free
- [[frank-zealot-llama-prism-bonsai9-mtp]] — the **ACTIVE** frank-zealot
runbook since 2026-10-18: the PrismML vendor llama.cpp build 10777 (the only
loader for the custom PQ2_0 tensor type / `prism.hadamard.*`, and the floor
for MTP beside a quantized V cache) serving
prism-ml/Ternary-Bonsai-2-9B-gguf **PQ2_0 with the MTP head** on the RTX 3090
— THREE 262,144-token conversations, q8_0 K + q8_0 V (f16 OOMs at this pool:
26,112 vs 13,056 MiB over 8 full-attention layers), MTP **off by
measurement** (the draft context bills 3,072 MiB of f16 KV over the whole
pool), vision through the Q8_0 mmproj; boot 17,876 used / 6,249 free
- [[frank-zealot-llama-prism-bonsai2-pq20]] — RETIRED 2026-10-17 → 10-18
(rollback target, payload + build 10754 on disk): Ternary-Bonsai-2-**27B**
PQ2_0 at TWO 200,704-token conversations, q8_0 K + q8_0 V, MTP **impossible**
(no head in that export), vision through the Q8_0 mmproj; boot 22,664 / 1,461
- [[frank-zealot-llama-qwen35-9b-q6k-2x200704]] — RETIRED 2026-10-17 (rollback
target, payload on disk): llama.cpp master (CUDA sm_86) serving
HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** on the RTX 3090 —