docs: LLM index + work log for frank-primary shape D, harness window
index.md intro and mermaid nodes still described 4 x 262144 fp8/bf16 and frank-small at 2 x 262144; the active-deployment list had no frank-primary entry at all. Fixed both, plus a malformed GLM list item. log.md gains the shape-D entry (the SEQS=2 constraint, the arena-lever measurement, the 2x2 probe table, the measured 1.99x), and the frank-small entry's stale consequence bullet now points at the resolution. frank-small runbook: the compaction-fit note is corrected - the 200704 window moved frank-primary's trigger to 140,492, which fits a 180,224 slot again.
This commit is contained in:
1 parent
295d5784d3
commit
656012a0fa
4 files changed
+115
-26
No files matched your search
@@ -427,8 +427,12 @@ digest-pinned) serving **`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8`**
|
||||
**fp8 KV**, **no speculative decode**, **expert prefetch OFF**, and since
|
||||
2026-10-02 the **shape-D** geometry: `--max-model-len 200704` × 3 slots with
|
||||
`--kv-cache-memory-bytes 3,150,000,000`/rank, so the pool admits **~2
|
||||
full-length conversations (measured: 398,658 tokens = 1.99×)** while the argv keeps the only slot count
|
||||
that compiles (below).
|
||||
full-length conversations** (measured live: `398,658 tokens, Maximum
|
||||
concurrency for 200,704 tokens per request: 1.99×`) while the argv keeps the
|
||||
only slot count that compiles (below). `models.yml` carries
|
||||
`contextWindow: 200704` for this endpoint — the served window, not the model's
|
||||
native 262144, so the harness compacts before vLLM answers a longer prompt
|
||||
with a 400.
|
||||
|
||||
**`--max-num-seqs 2` is impossible on this model.** Measured 2026-10-02 by
|
||||
single-variable boot probe against the then-deployed shape: SEQS 2 fails at
|
||||
|
||||
Reference in new issue
Block a user