Merge origin/main: reconcile the frank-zealot reslice with the dedupe pass
main moved under this campaign — PR #25/#26 resliced the RTX 3090 endpoint. The frontmatter conflict is resolved in favour of the live shape, confirmed by re-probe the same day: `--ctx-size 602112 --parallel 3` (3 × 200,704), `--cache-type-k f16 --cache-type-v q8_0`, `--spec-type draft-mtp`, `--jinja --reasoning auto`, `--metrics`. The 262,144 figures that remain on the page are era rows and the checkpoint's trained window, not the live shape. log.md's 2026-10-19 fleet-state bullet now records both probes and the law this taught: a docs pass that reports live shapes must re-probe before writing the number, because main moving mid-campaign makes the first probe stale.
This commit is contained in:
@@ -388,10 +388,11 @@ the publisher's own **Q8_0** mmproj passed as `--mmproj` — Q8_0 is the
|
||||
publisher's requirement for concurrent image requests (the card states the
|
||||
BF16 tower can crash `llama-server` when several image requests run at once,
|
||||
and this endpoint runs several slots). `frank-zealot`'s `contextWindow` is
|
||||
**262144** — the PER-SLOT window (`--ctx-size 524288 --parallel 2` since the
|
||||
2026-10-18 reslice to exact f16 KV), which EQUALS the model's
|
||||
native window; llama.cpp answers a longer prompt with a 400 naming the limit,
|
||||
never a truncation, so the harness must compact before it.
|
||||
**200704** — the PER-SLOT window (`--ctx-size 602112 --parallel 3` since the
|
||||
2026-10-19 reslice that funded a third slot AND an armed MTP drafter),
|
||||
deliberately below the model's native window; llama.cpp answers a longer prompt
|
||||
with a 400 naming the limit, never a truncation, so the harness must compact
|
||||
before it.
|
||||
Note also
|
||||
that
|
||||
the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
|
||||
@@ -439,10 +440,11 @@ disagrees with it is the bug.
|
||||
loader ignoring that metadata dequantizes wrong); payload
|
||||
`Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
|
||||
tower can crash the server with concurrent image requests).
|
||||
- Shape: `--ctx-size 524288 --parallel 2` = **two conversations at the
|
||||
model's native 262,144**, `CACHE_TYPE_K/V=f16`, batch 4096 / ubatch 2048,
|
||||
`SPEC_TYPE=none`, `--flash-attn on`, `--jinja`, `--metrics`.
|
||||
- Harness/gate contract: `contextWindow: 262144` is the **per-slot** window
|
||||
- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
|
||||
200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
|
||||
B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
|
||||
`SPEC_TYPE=draft-mtp` (depth 2), `--flash-attn on`, `--jinja`, `--metrics`.
|
||||
- Harness/gate contract: `contextWindow: 200704` is the **per-slot** window
|
||||
(`n_ctx_slot = CTX_SIZE / PARALLEL`), and llama.cpp answers a longer prompt
|
||||
with a 400 rather than truncating. `--metrics` is mandatory or Prometheus
|
||||
reads 501 = DOWN beside a healthy service. Presence penalty has no GGUF
|
||||
|
||||
Reference in New Issue
Block a user