feat(frank-zealot): Bonsai-2-9B PQ2_0 with the MTP head at 3 x 262,144

Swap the 3090 endpoint from Ternary-Bonsai-2-27B PQ2_0 (2 x 200,704,
fork build 10754) to Ternary-Bonsai-2-9B PQ2_0 with the MTP head IN the
file, three conversations at the trained 262,144 window
(--ctx-size 786432 --parallel 3), on vendor fork build 10777 -- the
minimum build that pairs MTP with a quantized V cache. q8_0 K + q8_0 V
is the only KV that fits: 13,056 MiB against a 26,112 MiB f16 pair.
Settled boot 17,876 used / 6,249 free.

MTP is armed and then DECLINED by measurement: the draft context bills
its own f16 KV over the whole pool (3,072 MiB) and at UBATCH_SIZE=2048
that halves decode (167.2 -> 109.2 t/s). The documented MTP arm
(SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024, 213.5 t/s) is recorded in the
env header rather than left to rediscovery.

Two bugs the trial caught: ${SPEC_PRESENCE_PENALTY:=0.0} was a
default-assignment, so the env value always won and the no-presence-penalty
MTP arm was unreachable (now ${VAR=0.0}); and the Q8_0 mmproj is the
publisher's requirement for concurrent image requests, which matters at
--parallel 3.

contextWindow is now 262144 -- the widest in the fleet -- so :7771 is the
compaction hub. 27B runbook re-banned RETIRED with its bench-window
section closed; indexes, homepage card, OpenWebUI doc and the worklog
re-keyed.
This commit is contained in:
Frank
2026-10-18 13:15:18 -07:00
parent 2b127163c8
commit 466fd477d8
14 changed files with 1001 additions and 314 deletions
+2 -2
View File
@@ -295,13 +295,13 @@
"model": {
"unit": "zealot",
"role": "Frank's utility model",
"hf": "https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf",
"hf": "https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf",
"tagline": "HONOR GUIDE ME!",
"sfx": [
"zealot-honor.ogg"
],
"equiv": "Equivalent to last year's frontier level (2025).",
"spec": "Ternary-Bonsai-2-27B (PQ2_0 ternary weights) on an NVIDIA RTX 3090 — ~77 tokens/s; 200k-token context, two submissions at a time; sees images.",
"spec": "Ternary-Bonsai-2-9B (PQ2_0 ternary weights, MTP head in the file) on an NVIDIA RTX 3090 — ~167 tokens/s; 262k-token context, three submissions at a time; sees images.",
"inputs": [
"text",
"image"