frank-primary: roll back to qwen FP8 + MTP4 (2339fae trio)

DFlash2 experiment declared failed 2026-10-01: the drafter serves and
drafts (12.8 tok/s proposed) but acceptance is pinned at 1.0-1.38 on ALL
text classes including verbatim repetition — the aux-hidden-state capture
semantics are wrong (single-stream entering-state vs post-layer/comb is
the untested alternative). At ~1.2 acceptance the 8-token verify tax makes
decode 5x SLOWER than the 9.9-10.0 baseline (1.6-1.8 tok/s measured).

Restores the verified 37-40 tok/s shape: qwen3.8-flash-next FP8, MTP4,
prefetch off, 200k x 3 seqs, fp8 KV.

scripts/serve-frank-primary.sh

@@ -1,30 +1,27 @@
 #!/usr/bin/env bash
 # serve-frank-primary.sh — frank-primary endpoint (tcclaviger RDNA4 vLLM fork,
-# Intel/GLM-5.3-Flash-W4A16-AutoRound, 2x R9700, dankserver :7770).
-#
-# 2026-09-30 cutover from orcarouter Qwen3.8-Flash-Next-Uncensored-FP8 + MTP4.
-# Rollback: commit 2339fae carries the whole pre-cutover scripts/ tree; restore
-# it and deploy. The qwen payload and the R9V state dirs stay on disk.
+# orcarouter Qwen3.8-Flash-Next-Uncensored-FP8 + MTP4 spec decode, 2x R9700,
+# dankserver :7770; 2026-09-29 cutover from the R9V radiance fork — git tag
+# frank-primary-r9v-q8-20260929 is the pre-cutover state).
 #
 # The engine is driven by `podman run` DIRECTLY: the image's entrypoint execs
-# `vllm serve "$@"`, so the run line below IS the engine argv.
+# `vllm serve "$@"`, so the run line below IS the engine argv. This replaces
+# the R9V launcher/shim machinery (that fork needed a docker->podman shim and
+# a 104-key on-host setup state; this stack is fully expressed by flags, and
+# every flag here was qualified by boot probe on 2026-09-29 — see
+# frank-primary.env for the qualification record).
 #
-# Why this payload runs the offload path at all: 170 GiB of int4 experts cannot
-# fit in 64 GiB of VRAM, so the engine keeps 12,096 experts per rank in pinned
-# host memory and serves them through a 2,383-slot VRAM LRU arena. The MoE
-# kernel is libr4d's grouped W4A16 GEMM (R4dW4a16MoEMethod), reached because the
-# checkpoint's quant_method "auto-round" is overridden to "inc" by the fork.
-#
-# Shape: MAXLEN 131,072 x 2 seqs, bf16 KV, util 0.98, MTP OFF, prefetch OFF.
-# Measured 2026-09-30: decode 9.9-10.0 tok/s, prefill 918 tok/s warm. The
-# 35 tok/s target is NOT met — the bound is the fork's expert-gather read path
-# (8.2 GB/s effective vs the link's 22.74 GB/s); see frank-primary.env and the
-# runbook's arithmetic section.
+# Shape: MAXLEN 200000 x 3 sessions (600k total), KV q8/q8 (--kv-cache-dtype
+# fp8: e4m3 for both K and V — the stock flag quantizes them together, which
+# is the requested q8/q8), KV_BYTES 7.2 GiB PER RANK (the engine measured
+# 729,545 tokens of capacity / 3.65x concurrency at 200k per request), MTP
+# depth 4, prefix caching, expert prefetch OFF (telemetry measured it at 36%
+# precision — the tuned incumbent deletes it), TunableOp off.
 #
 # The root unit (scripts/frank-primary.service, User=dank,
-# LimitMEMLOCK=infinity) grants the pin budget for the 73.64 GiB/rank of pinned
-# host expert rows. A plain shell session also works, but the unit is the
-# fleet-parity boot-persistent shape.
+# LimitMEMLOCK=infinity) grants the pin budget for the tiered-expert host
+# slabs (~20.5 GiB/rank of pinned anonymous memory). A plain shell session
+# also works, but the unit is the fleet-parity boot-persistent shape.
 set -euo pipefail
 
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -47,9 +44,8 @@ set +a
 : "${FRANK_PRIMARY_PUBLISH:?FRANK_PRIMARY_PUBLISH must be host:container}"
 : "${FRANK_PRIMARY_MAXLEN:?FRANK_PRIMARY_MAXLEN must be set}"
 : "${FRANK_PRIMARY_SEQS:?FRANK_PRIMARY_SEQS must be set}"
-: "${FRANK_PRIMARY_NBATCH:?FRANK_PRIMARY_NBATCH must be set}"
-: "${FRANK_PRIMARY_TOOLPARSER:?FRANK_PRIMARY_TOOLPARSER must be set}"
-: "${FRANK_PRIMARY_REASONINGPARSER:?FRANK_PRIMARY_REASONINGPARSER must be set}"
+: "${FRANK_PRIMARY_KV_BYTES:?FRANK_PRIMARY_KV_BYTES must be set (PER RANK)}"
+: "${FRANK_PRIMARY_KV_DTYPE:?FRANK_PRIMARY_KV_DTYPE must be set (fp8 = q8/q8 KV)}"
 # Capture podman's output BEFORE grepping: under `set -o pipefail`, the
 # classic `podman images | grep -q` pattern races — grep exits on its first
 # match, podman dies SIGPIPE (141) writing the remaining lines, and pipefail
@@ -104,34 +100,20 @@ podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
   --cap-add SYS_PTRACE \
   --device /dev/kfd --device /dev/dri --device /dev/mem \
   -e HIP_VISIBLE_DEVICES=0,1 -e VLLM_ROCM_USE_AITER=0 \
-  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
   -v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \
   "$FRANK_PRIMARY_IMAGE" /models \
   --served-model-name "$FRANK_PRIMARY_SERVED_NAME" --port 7770 \
   --tensor-parallel-size 2 \
   --max-model-len "$FRANK_PRIMARY_MAXLEN" \
   --max-num-seqs "$FRANK_PRIMARY_SEQS" \
-  --max-num-batched-tokens "$FRANK_PRIMARY_NBATCH" \
   --gpu-memory-utilization "$FRANK_PRIMARY_UTIL" \
-  --enable-expert-offload \
-  --expert-precise-memory \
+  --kv-cache-memory-bytes "$FRANK_PRIMARY_KV_BYTES" \
+  --kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
   --enable-prefix-caching \
-  --enable-auto-tool-choice \
-  --tool-call-parser "$FRANK_PRIMARY_TOOLPARSER" \
-  --reasoning-parser "$FRANK_PRIMARY_REASONINGPARSER" \
+  --speculative-config "{\"method\":\"mtp\",\"num_speculative_tokens\":$FRANK_PRIMARY_SPEC}" \
   --trust-remote-code \
   > "$LOG" 2>&1
 
-# NOT passed, deliberately (each measured on this payload — see the env file):
-#   --kv-cache-dtype             fp8 disables the sparse MLA backend's rope-free
-#                                Triton path, which is the only MLA decode on
-#                                gfx12 (aiter's MLA op is gated on CDNA>2).
-#   --speculative-config         MTP head loads and accepts well, but buys zero
-#                                decode and collapses prefill on an
-#                                offload-bound shape.
-#   --expert-prefetch*           36% precision; deleting it bought +40-51%.
-#   --kv-cache-memory-bytes      the plan's own reserve covers this shape.
-
 # From here on, this script's lifetime IS the container's: wait returns the
 # container's exit code, so a crash is a nonzero ExecStart and the unit's
 # Restart=on-failure is real. (systemd stop goes through ExecStop in the