frank-primary: 3 slots x 200704, fp8, pool sized for 2 contexts

Re-applies the desired shape incrementally on the proven-good fp8 3x262144
base, after bisecting why the first attempt (30904ba) crash-looped.

Root cause: `--max-num-seqs 2` cannot compile on this model. AOT
build_guards(strict_error=True) inside profile_run -> _dummy_run specializes
`query_start_loc`'s width to SEQS + 1 (3 at SEQS 2, 4 at SEQS 3) and the
ConstraintViolationError kills engine init ~7 min in. MNBT is orthogonal:
SEQS 2 failed at MNBT 1024 AND 2048; SEQS 3 passed at both. KV dtype was a
red herring. Measured by four single-variable boot probes.

Arena lever measured: the expert arena is sized from `max_model_len` via the
registry-priced KV reserve (9190 B/token x max_model_len), NOT from
--kv-cache-memory-bytes. 262144 -> 14.30 GiB / 6,248 slots; 200704 ->
14.87 GiB / 6,496 slots. The 6.1e9 -> 3.15e9 pool cut bought zero arena and
is a runtime-headroom lever.

Shape: MAXLEN 200704, SEQS 3 (unchanged - the only slot count that compiles),
KV_BYTES 3.15e9/rank = 2.04x full-length concurrency at 7,701 B/token fp8,
MNBT 1024, gather 32x64, no spec decode, no prefetch, no precise-memory.

AGENTS.md

@@ -24,7 +24,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 
 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
-| Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **fp8 KV** (`--kv-cache-dtype fp8`), **no speculative decode**, MNBT 1024, MoE gather 32×64, **2 seqs × 200704 ctx** at KV_BYTES 3,150,000,000/rank (fp8 measures 7,701 B/token), pool deliberately small so the expert arena keeps what it reclaims (~7,500 slots vs 6,248 at the 6.1e9 shape) — util 0.98, and the client contract wired: `--reasoning-parser …
+| Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **fp8 KV** (`--kv-cache-dtype fp8`), **no speculative decode**, MNBT 1024, MoE gather 32×64, **3 slots × 200704 ctx** at KV_BYTES 3,150,000,000/rank — the **pool, not the slot count, caps resident conversations at ~2 full-length** (fp8 measures 7,701 B/token, so one 200704 request costs 1.546e9 B; 2.04×), and shorter requests use the spare blocks. **`--max-num-seqs` must stay 3: SEQS=2 cannot boot this model** — torch.fx `ConstraintViolationError` on `query_start_loc`, measured at BOTH MNBT 1024 and 2048. The pool cut buys runtime **headroom, NOT arena**: the arena is sized by `max_model_len` via the registry KV reserve (262144 → 6,248 slots; 200704 → 6,496) — util 0.98, and the client contract wired: `--reasoning-parser …
 | Dankclaw | `frank-large` | **ROCmFPX llama.cpp fork** (`kingjones30/ROCmFPX` @ `36e9acd`, gfx1151, built by `scripts/build-rocmfpx.sh`; bare `llama-server`, root systemd unit) | `kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF` **`Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`** (26.28 GiB, ftype 115 ≈ 8.39 bpw: 353 F32 + 187 Q8_0 + 311 `GGML_TYPE_Q8_0_ROCMFPX` tensors — only this fork can read them), **MTP spec decode from the repo's own `mtp-Qwen3.8-27B-Q4_0.gguf` (n-max 4, pinned to ROCm0)**, **1 session × 262,144 (native window)** with **f16 KV**, the bundled BF16 mmproj for vision, thinking on with the card's recipe (temp 1.0 / top_p 0.95 / top_k 20). Measured 2026-10-02: decode **17.0–17.1 t/s** with MTP vs **7.6 without (+123 %)**, draft acceptance 0.462, prefill 325–351 t/s, prompt-cache replay 0.5 s, boot→healthy 16 s, GTT 64.5 GiB. Cut over 2026-10-02 from gufo; rollback tag `frank-large-gufo-q4kxl-20261001` | 7770 |
 | Dankclaw | `frank-small` | llama.cpp master, CUDA sm_86 build (root systemd unit, relinked) | bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** (9B dense qwen3_5 hybrid, 6.4 GiB GGUF, no MTP weights so no spec decode) + the repo's f16 mmproj for **vision**, on the RTX 3090, **3 × 180,224** (pool 540,672 — NOT the native window; three contexts cost 81,920 tokens per conversation) with **asymmetric KV f16 K + q8_0 V** under flash attention, 21,684 used / **2,441 free** of 24,576 MiB, 106.8 tok/s decode, 3,814 t/s prefill. Resized twice 2026-10-02 (Q6_K → Q5_K_M → 3 slots + vision → this pool); rollback `models/MiMo-Q6K/` or `PARALLEL=2 CTX_SIZE=524288` | 7771 |
 
@@ -419,30 +419,49 @@ Rollback: [[frank-small-llama-mimo-q6k]] (Q6_K file on disk at
 (venv on disk at `/home/dank/GitPackages/sglang-cu128/`; its INT8 payload was
 staged to `~dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02).
 
-### frank-primary — tcclaviger vLLM, Qwen3.8-Flash-Next FP8 + MTP4 (Dankserver, ACTIVE)
+### frank-primary — tcclaviger vLLM, Qwen3.8-Flash-Next FP8 (Dankserver, ACTIVE)
 
 The **tcclaviger RDNA4 vLLM fork** (`docker.io/tcclaviger/vllm@sha256:da4029666…`,
 digest-pinned) serving **`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8`**
 (~172.8 GiB, 131 shards) on **both** Radeon AI PRO R9700s (gfx1201), **TP=2**,
-**MTP4** speculative decoding, **expert prefetch OFF**, 200,000 ctx x 3 seqs,
-**fp8 KV**. Deployed 2026-10-01 by rollback from commit `2339fae` after the
-GLM/DFlash2 experiment was declared failed (same day).
+**fp8 KV**, **no speculative decode**, **expert prefetch OFF**, and since
+2026-10-02 the **shape-D** geometry: `--max-model-len 200704` × 3 slots with
+`--kv-cache-memory-bytes 3,150,000,000`/rank, so the pool admits **~2
+full-length conversations (2.04×)** while the argv keeps the only slot count
+that compiles (below).
 
-Measured 2026-09-29 tuning campaign: **decode 37.4-40.6 tok/s, acceptance
-~54 %** (MTP4 + prefetch off — the fleet's only endpoint that meets the
-35 tok/s target). 256k ctx arm measured 34.1-36.5; 2-way concurrency sums
-BELOW parity (25.49 vs 26.8 single) — slots are a capacity feature, not a
-throughput one. V-only q8 KV is NOT possible via config: stock
-`--kv-cache-dtype` is K+V together; a V-only quant needs a fork patch.
+**`--max-num-seqs 2` is impossible on this model.** Measured 2026-10-02 by
+single-variable boot probe against the then-deployed shape: SEQS 2 fails at
+BOTH MNBT 1024 and 2048 with `torch.fx.experimental.symbolic_shapes.Constraint
+ViolationError: Constraints violated (L['query_start_loc'].size()[0]) …
+specialized it to be a constant (3)`; SEQS 3 passes at both. `model_runner.py`
+sizes that input buffer `max_num_reqs + 1` and `_dummy_run` sets
+`num_reqs = min(max_num_batched_tokens, max_num_reqs)`, so the width is
+`SEQS + 1` for any `MNBT ≥ SEQS` — MNBT only moves the compile RANGE
+((1,1024) vs (1,2048)). `--enforce-eager` bypasses the path but drops CUDA
+graphs. The same failure class killed the 2026-10-01 bf16 2×262144 shape.
 
-The engine auto-sets the attention block size (mamba-page floor); the
-expert plan's registry prices KV per token; the expert arena (6,574 slots
-x 2.344 MiB = 15.05 GiB/rank) is the decode-critical region — cutting it
-for KV headroom collapses the offload hit rate.
+**The expert arena's lever is `max_model_len`, not `--kv-cache-memory-bytes`.**
+The fork's planner sizes expert cache as `budget − non-expert − KV reserve −
+graphs − overhead`, and the KV reserve is registry-priced at 9190 B/token ×
+`max_model_len`: 262144 → 2.48 GiB reserve → arena 14.30 GiB / **6,248 slots**;
+200704 → 1.91 GiB → 14.87 GiB / **6,496 slots**. Δreserve 0.57 GiB = Δarena
+0.57 GiB. The 6.1e9 → 3.15e9 pool cut bought **zero** arena; it buys runtime
+headroom (the shape it replaced died with `0 bytes free` on a real 995-token
+request). Grow the arena by lowering `max_model_len`. The arena (2.344 MiB per
+slot) is the decode-critical region — cutting it for KV headroom collapses the
+offload hit rate.
+
+Measured 2026-09-29 tuning campaign (fp8 + MTP4 era, since retired): decode
+**37.4-40.6 tok/s, acceptance ~54 %**; the 256k ctx arm measured 34.1-36.5;
+2-way concurrency sums BELOW parity (25.49 vs 26.8 single) — slots are a
+capacity feature, not a throughput one. V-only q8 KV is NOT possible via
+config: stock `--kv-cache-dtype` is K+V together; a V-only quant needs a fork
+patch.
 
 Unit and self-healing: root systemd unit (`scripts/frank-primary.service`,
 `User=dank`, `LimitMEMLOCK=infinity`), `ExecStop=podman stop -t 120
-frank-primary-glm`. The serve script drives `podman run` directly, gates
+frank-primary-qwen`. The serve script drives `podman run` directly, gates
 the digest-pinned image, the payload's `config.json` and its exact
 131-shard count and the GPU device nodes, clears stale container
 identities, then blocks on `podman wait`. `deploy_llm_primary` gates the
@@ -849,7 +868,7 @@ Active:
 |---------|---------|
 | frank-large (ROCmFPX llama.cpp fork, Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP4 + vision, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-rocmfpx-q8agent-mtp.md` |
 | frank-small (llama.cpp CUDA, MiMo-V2.6-Distill-Qwen-9B Q5_K_M, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-llama-mimo-q5km.md` |
-| frank-primary (tcclaviger vLLM, Qwen3.8-Flash-Next FP8 + MTP4, Dankserver) | `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md` |
+| frank-primary (tcclaviger vLLM, Qwen3.8-Flash-Next FP8, Dankserver) | `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md` |
 | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
 | Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |