Merge origin/main: reconcile the frank-zealot reslice with the dedupe pass

main moved under this campaign — PR #25/#26 resliced the RTX 3090 endpoint.
The frontmatter conflict is resolved in favour of the live shape, confirmed
by re-probe the same day: `--ctx-size 602112 --parallel 3` (3 × 200,704),
`--cache-type-k f16 --cache-type-v q8_0`, `--spec-type draft-mtp`, `--jinja
--reasoning auto`, `--metrics`. The 262,144 figures that remain on the page
are era rows and the checkpoint's trained window, not the live shape.

log.md's 2026-10-19 fleet-state bullet now records both probes and the law
this taught: a docs pass that reports live shapes must re-probe before
writing the number, because main moving mid-campaign makes the first probe
stale.
This commit is contained in:
Frank
2026-10-19 05:12:41 -07:00
7 changed files with 290 additions and 208 deletions
+10 -8
View File
@@ -388,10 +388,11 @@ the publisher's own **Q8_0** mmproj passed as `--mmproj` — Q8_0 is the
publisher's requirement for concurrent image requests (the card states the
BF16 tower can crash `llama-server` when several image requests run at once,
and this endpoint runs several slots). `frank-zealot`'s `contextWindow` is
**262144** — the PER-SLOT window (`--ctx-size 524288 --parallel 2` since the
2026-10-18 reslice to exact f16 KV), which EQUALS the model's
native window; llama.cpp answers a longer prompt with a 400 naming the limit,
never a truncation, so the harness must compact before it.
**200704** — the PER-SLOT window (`--ctx-size 602112 --parallel 3` since the
2026-10-19 reslice that funded a third slot AND an armed MTP drafter),
deliberately below the model's native window; llama.cpp answers a longer prompt
with a 400 naming the limit, never a truncation, so the harness must compact
before it.
Note also
that
the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
@@ -439,10 +440,11 @@ disagrees with it is the bug.
loader ignoring that metadata dequantizes wrong); payload
`Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
tower can crash the server with concurrent image requests).
- Shape: `--ctx-size 524288 --parallel 2` = **two conversations at the
model's native 262,144**, `CACHE_TYPE_K/V=f16`, batch 4096 / ubatch 2048,
`SPEC_TYPE=none`, `--flash-attn on`, `--jinja`, `--metrics`.
- Harness/gate contract: `contextWindow: 262144` is the **per-slot** window
- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
`SPEC_TYPE=draft-mtp` (depth 2), `--flash-attn on`, `--jinja`, `--metrics`.
- Harness/gate contract: `contextWindow: 200704` is the **per-slot** window
(`n_ctx_slot = CTX_SIZE / PARALLEL`), and llama.cpp answers a longer prompt
with a 400 rather than truncating. `--metrics` is mandatory or Prometheus
reads 501 = DOWN beside a healthy service. Presence penalty has no GGUF