frank-primary: shape F — MNBT 2048, util 0.982, CUDA-graph plan term pinned to the measured capture

The arena is budget - non-expert - KV reserve - graphs - overhead. With util
capped at 0.982 (never 1.00) the graph estimate is the only plan term this repo
can shrink: the fork charges an analytic 256 MiB + 52 MiB/graph = 0.55 GiB where
the engine's own post-capture line reports 0.31 GiB, so pricing it at
256 + 12/graph returns ~0.24 GiB to the expert arena. MNBT 1024 -> 2048 is
arena-free: the registry overhead is base + 0.22 GiB per 1024 ABOVE its nbt 4096
anchor, and the fitted 5.25 GiB was fitted AT 4096.

scripts/frank-primary.env

@@ -1,5 +1,15 @@
 # frank-primary.env — tcclaviger RDNA4 vLLM production config (dankserver :7770).
 #
+# 2026-10-03 shape F: MNBT 2048 and the CUDA-graph plan term pinned to the
+# measured capture cost. The expert arena is `budget - non-expert - KV reserve
+# - graphs - overhead`, and with UTIL capped (see FRANK_PRIMARY_UTIL) the graph
+# estimate is the only plan term this repo can reach: the fork charges an
+# analytic 0.55 GiB (256 MiB base + 52 MiB/graph) where the engine's own
+# post-capture line reports 0.31 GiB on the heavier rank. Pricing it honestly
+# returns 0.24 GiB to the arena. The rest of the idle free VRAM (~3.3 GiB/card)
+# stays unreachable from flags: it is locked inside the registry's overhead
+# (5.25 GiB) and KV-price (9,190 B/token vs the measured 7,892) terms, which
+# live in the image, not in our argv or env.
 # 2026-10-02 shape E: fp8 KV, MAXLEN 200704 x SEQS 3, pool 3.15e9/rank, and
 # --expert-precise-memory ON. The window shrink from the native 262144 and the
 # --max-num-seqs 3 lock both come from shape D (SEQS=2 CANNOT boot this model —
@@ -80,13 +90,18 @@ FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-r9v qwen-fp8"
 # GATHER 32x64 stays: R4D_LRU_GATHER_CHUNKS=32 / R4D_LRU_GATHER_LANES=64
 # measured +18.7% decode vs default geometry and the win grows with depth.
 # Do not drop it.
-# MNBT 1024 is Frank's lock: MNBT 512 doubles TTFT at 4064 ctx (2.46 s vs
-# 1.36 s); 1024-4096 are interchangeable on decode; 12288 costs 2.5% and
-# arena. 1024 keeps the workspace floor so the arena stays max. (Side note
-# measured 2026-10-02: MNBT 2048 compiles the first graph in 26 s vs 172 s at
-# 1024 — a startup-cost lever, not a correctness one. The compile RANGE is part
-# of the cache key, so FRANK_PRIMARY_CACHE_DIR pays for whichever rung
-# is chosen exactly once.)
+# MNBT 2048 (Frank's call, 2026-10-03; the lock was 1024). The measured law
+# still holds: MNBT 512 doubles TTFT at 4064 ctx (2.46 s vs 1.36 s), 1024-4096
+# are interchangeable on decode, 12288 costs 2.5% plus arena. 2048 is ARENA-FREE
+# by construction: the registry's overhead model is
+# base + 0.22 GiB x max(0, (nbt - 4096)/1024) with the anchor at nbt 4096, so
+# every rung up to and including 4096 prices the SAME 5.25 GiB overhead — and
+# the fitted 5.25 was fitted at nbt 4096, i.e. our 2048 sits UNDER the anchor,
+# so the activation term is covered by a figure measured at a bigger batch.
+# SEQS 3 boots at both 1024 and 2048 (measured 2026-10-02; the SEQS=2
+# ConstraintViolationError is a SEQS property, not an MNBT one). Costs exactly
+# one recompilation: the compile RANGE is part of the cache key, so
+# FRANK_PRIMARY_CACHE_DIR pays for whichever rung is chosen once.
 # PREFETCH IS ABSENT BY LAW: expert-cache telemetry measured the prefetcher at
 # 36% precision (36,720 slabs prefetched, 13,369 served) — roughly half the
 # gather traffic was waste. Never re-add --expert-prefetch without a fresh A/B.
@@ -149,10 +164,30 @@ FRANK_PRIMARY_SEQS=3
 # as the shipped value; it remains the conservative arm if free VRAM ever binds.
 FRANK_PRIMARY_KV_BYTES=3150000000
 FRANK_PRIMARY_KV_DTYPE=fp8
-FRANK_PRIMARY_MNBT=1024
+FRANK_PRIMARY_MNBT=2048
 FRANK_PRIMARY_GATHER_CHUNKS=32
 FRANK_PRIMARY_GATHER_LANES=64
-FRANK_PRIMARY_UTIL=0.98
+# UTIL is Frank's cap: 0.982 maximum, never 1.00. It is an ARENA lever — the
+# fork plans `budget = device_total x util` — but a weak one here: 0.980 -> 0.982
+# moves the budget 31.22 -> 31.28 GiB, i.e. +28 slots. Raising it to 1.00 would
+# add 0.64 GiB and Frank vetoed it: this endpoint has twice died at the ceiling
+# (the 2x262144 bf16 boot at 0.13 GiB free, the 5x262144 boot at "0 bytes free"
+# on a real 995-token request), so the margin stays.
+FRANK_PRIMARY_UTIL=0.982
+# CUDA-graph plan term, pinned to the measured capture cost. expert_plan.py
+# charges estimate_cudagraph_memory() = BASE_MB + PER_GRAPH_MB x n_graphs with
+# defaults 256/52, and the estimator's own docstring says to tune these against
+# the post-capture line. MEASURED on shape E (model_runner.py:1094, same
+# 6-graph plan, same shape): "Graph capturing finished in 3 secs, took 0.31 GiB"
+# on the heavier rank (TP0) and 0.31/0.17 across ranks — vs the 0.55 GiB the
+# plan charged. 12 MiB/graph keeps ~10 MiB of margin over the fitted marginal
+# cost (~10 MiB/graph) and, unlike VLLM_CUDAGRAPH_MEM_ESTIMATE_MB (an explicit
+# TOTAL), scales correctly if a future MNBT or spec-decode change grows the
+# ladder. Verify every deploy against the "Estimated CUDA graph memory" and
+# "Graph capturing finished" lines; if capture exceeds the estimate, raise
+# FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB back toward 52.
+FRANK_PRIMARY_CUDAGRAPH_BASE_MB=256
+FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB=12
 
 # --- optional engine knobs (unset == the flag is absent) ---------------------
 # FRANK_PRIMARY_SPEC holds the WHOLE --speculative-config JSON; setting it adds

scripts/serve-frank-primary.sh

@@ -11,7 +11,8 @@
 # every flag here was qualified by boot probe on 2026-09-29 — see
 # frank-primary.env for the qualification record).
 #
-# Shape (2026-10-02 shape E): MAXLEN 200704 x 3 slots, fp8 KV, KV_BYTES
+# Shape (2026-10-03 shape F, on the 2026-10-02 shape E base): MAXLEN 200704 x 3
+# slots, fp8 KV, KV_BYTES
 # 3,150,000,000 PER RANK — the POOL, not the slot count, caps resident
 # conversations at ~2 full-length (measured: 398,658 tokens = 1.99x);
 # shorter requests use the spare blocks. --max-num-seqs stays 3 because SEQS=2
@@ -21,12 +22,17 @@
 # is OFF today by ENV KNOB, not by measurement: FRANK_PRIMARY_SPEC / _COMPCONFIG
 # / _GEN_CONFIG hold the whole JSON value and unset means no flag. The 2026-10-02
 # retirement of MTP compared the wrong pair — see frank-primary.env.
-# --max-num-batched-tokens 1024 (Frank's lock), MoE gather geometry
+# --max-num-batched-tokens 2048 (Frank's call 2026-10-03; arena-free because the
+# registry prices overhead flat up to its nbt 4096 anchor), MoE gather geometry
 # R4D_LRU_GATHER_CHUNKS=32 / R4D_LRU_GATHER_LANES=64 (+18.7% decode, grows
 # with depth), prefix caching, expert prefetch OFF (36% precision measured),
 # TunableOp off. Shape E adds --expert-precise-memory (fitted 5.25 GiB overhead
 # instead of the 7.60 GiB default -> expert arena 17.22 GiB / 7,522 slots, +16%)
-# and GPU_MAX_HW_QUEUES=2. The native-262144 and fp8+MTP4 shapes are retired to
+# and GPU_MAX_HW_QUEUES=2; shape F pins the plan's CUDA-graph term to the
+# measured capture cost (VLLM_CUDAGRAPH_MEM_BASE_MB / _PER_GRAPH_MB, container
+# env — the analytic default charges 0.55 GiB where capture took 0.31 GiB, so
+# the 0.24 GiB difference goes to the arena) and raises util 0.98 -> 0.982,
+# Frank's cap. The native-262144 and fp8+MTP4 shapes are retired to
 # git tag frank-primary-fp8-mtp-20261001.
 #
 # THE COMPILE CACHE IS MOUNTED, and that is a boot-cost fix, not a tuning knob:
@@ -138,6 +144,8 @@ podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
   ${FRANK_PRIMARY_GPU_MAX_HW_QUEUES:+-e GPU_MAX_HW_QUEUES="$FRANK_PRIMARY_GPU_MAX_HW_QUEUES"} \
   ${FRANK_PRIMARY_GATHER_CHUNKS:+-e R4D_LRU_GATHER_CHUNKS="$FRANK_PRIMARY_GATHER_CHUNKS"} \
   ${FRANK_PRIMARY_GATHER_LANES:+-e R4D_LRU_GATHER_LANES="$FRANK_PRIMARY_GATHER_LANES"} \
+  ${FRANK_PRIMARY_CUDAGRAPH_BASE_MB:+-e VLLM_CUDAGRAPH_MEM_BASE_MB="$FRANK_PRIMARY_CUDAGRAPH_BASE_MB"} \
+  ${FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB:+-e VLLM_CUDAGRAPH_MEM_PER_GRAPH_MB="$FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB"} \
   ${CACHE_DIR:+-v "$CACHE_DIR":/root/.cache} \
   -v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \
   "$FRANK_PRIMARY_IMAGE" /models \