dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(frank-zealot): carry the 3 x 200,704 shape, retire the exact-KV era row

Browse Source 
The endpoint page owns its shape: the flags row, the budget equation behind
it (25,088 B/token, 14,406.00 MiB of cache + 2,352.00 MiB of draft cache
inside 24,123 MiB addressable), the "MTP is ON" section replacing the "OFF by
measurement" one, the harness contract at the per-slot 200,704, and the
fallback rungs rewritten as deltas FROM the shipped shape.

The 2026-10-18 exact-KV boot (KV 16,384.00 / RS 100.50 / compute 1,408.06 +
1,088.07 / mmproj 868.32 charged after the fit check, settled 21,176/2,949)
and its contended rate floors move into that era's Eras row with the one-line
rollback, so the two pages that carried them collapse to one. Rates for the
new shape are deliberately absent - no bench was ordered, and the fit gate,
the health gate and the boot ledger are the evidence a swap needs.

AGENTS.md follows the same numbers: contextWindow 200704 is the per-slot
window and the trio's shape line.
This commit is contained in:
Frank
2026-10-19 03:16:48 -07:00
parent 25a8b00043
commit a7062d1cb3
2 changed files with 52 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+10 -8
View File
@@ -384,10 +384,11 @@ the publisher's own **Q8_0** mmproj passed as `--mmproj` — Q8_0 is the
publisher's requirement for concurrent image requests (the card states the
BF16 tower can crash `llama-server` when several image requests run at once,
and this endpoint runs several slots). `frank-zealot`'s `contextWindow` is
**262144** — the PER-SLOT window (`--ctx-size 524288 --parallel 2` since the
2026-10-18 reslice to exact f16 KV), which EQUALS the model's
native window; llama.cpp answers a longer prompt with a 400 naming the limit,
never a truncation, so the harness must compact before it.
**200704** — the PER-SLOT window (`--ctx-size 602112 --parallel 3` since the
2026-10-19 reslice that funded a third slot AND an armed MTP drafter),
deliberately below the model's native window; llama.cpp answers a longer prompt
with a 400 naming the limit, never a truncation, so the harness must compact
before it.
Note also
that
the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
@@ -435,10 +436,11 @@ disagrees with it is the bug.
loader ignoring that metadata dequantizes wrong); payload
`Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
tower can crash the server with concurrent image requests).
- Shape: `--ctx-size 524288 --parallel 2` = **two conversations at the
model's native 262,144**, `CACHE_TYPE_K/V=f16`, batch 4096 / ubatch 2048,
`SPEC_TYPE=none`, `--flash-attn on`, `--jinja`, `--metrics`.
- Harness/gate contract: `contextWindow: 262144` is the **per-slot** window
- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
`SPEC_TYPE=draft-mtp` (depth 2), `--flash-attn on`, `--jinja`, `--metrics`.
- Harness/gate contract: `contextWindow: 200704` is the **per-slot** window
(`n_ctx_slot = CTX_SIZE / PARALLEL`), and llama.cpp answers a longer prompt
with a 400 rather than truncating. `--metrics` is mandatory or Prometheus
reads 501 = DOWN beside a healthy service. Presence penalty has no GGUF