frank-primary: shape E — --expert-precise-memory (arena 6,496->7,522 slots), GPU_MAX_HW_QUEUES=2, and a mounted /root/.cache

Measured by boot probe on the shape-D base (production stopped, candidate
booted AS the service), plan numbers read off the engine's own log:

  runtime overhead   7.60 -> 5.25 GiB (precise gfx1201 TP2 registry)
  expert cache      14.87 -> 17.22 GiB
  slots              6,496 -> 7,522 (+1,026, +16%)
  swap region                13.7 GiB / 5,986 slots
  boot                       healthy in 420 s, no OOM

'--expert-precise-memory': True also appears in non-default args, so a silent
argparse drop is visible. The pool is NOT cut (option A) because the pool is
not an arena lever: 3.15e9/rank keeps shape D's measured 1.99x concurrency,
and a 2.2e9 arm probed in the same window (277,686 tokens, 1.38x) is recorded
as the conservative fallback.

Boot-cost bug found while diffing Frank's 3xR9700 recipe: the image ships no
/root/.cache at all, our boot writes 1.1 GB of vLLM AOT/torch_compile
artifacts (4,665 files) plus 229 MB of comgr JIT kernels into the ephemeral
layer, and podman rm -f discards them — so every deploy/restart/reboot re-pays
'torch.compile took 181.20 s in total' (173.19 s of it one graph). Mounting
FRANK_PRIMARY_CACHE_DIR at /root/.cache persists both and hides nothing.

GPU_MAX_HW_QUEUES=2 added at Frank's direction (real HSA knob); not A/B
measured. Retracted my own recipe-diff claim that cudagraph_capture_sizes
needed pinning: capture is 3 PIECEWISE + 2 FULL graphs, 3 s, 0.31 GiB.

Tests: 367/367.

scripts/serve-frank-primary.sh

@@ -11,7 +11,7 @@
 # every flag here was qualified by boot probe on 2026-09-29 — see
 # frank-primary.env for the qualification record).
 #
-# Shape (2026-10-02 shape D): MAXLEN 200704 x 3 slots, fp8 KV, KV_BYTES
+# Shape (2026-10-02 shape E): MAXLEN 200704 x 3 slots, fp8 KV, KV_BYTES
 # 3,150,000,000 PER RANK — the POOL, not the slot count, caps resident
 # conversations at ~2 full-length (measured: 398,658 tokens = 1.99x);
 # shorter requests use the spare blocks. --max-num-seqs stays 3 because SEQS=2
@@ -21,8 +21,18 @@
 # decode, --max-num-batched-tokens 1024 (Frank's lock), MoE gather geometry
 # R4D_LRU_GATHER_CHUNKS=32 / R4D_LRU_GATHER_LANES=64 (+18.7% decode, grows
 # with depth), prefix caching, expert prefetch OFF (36% precision measured),
-# TunableOp off. The native-262144 and fp8+MTP4 shapes are retired to git tag
-# frank-primary-fp8-mtp-20261001.
+# TunableOp off. Shape E adds --expert-precise-memory (fitted 5.25 GiB overhead
+# instead of the 7.60 GiB default -> expert arena 17.22 GiB / 7,522 slots, +16%)
+# and GPU_MAX_HW_QUEUES=2. The native-262144 and fp8+MTP4 shapes are retired to
+# git tag frank-primary-fp8-mtp-20261001.
+#
+# THE COMPILE CACHE IS MOUNTED, and that is a boot-cost fix, not a tuning knob:
+# the image ships no /root/.cache at all, our first boot writes 1.1 GB of vLLM
+# AOT/torch_compile artifacts plus 229 MB of comgr JIT kernels into the
+# container's ephemeral layer, and `podman rm -f` deletes them — so every
+# deploy/restart/reboot re-pays `torch.compile took 181.20 s in total` (173.19 s
+# of it one graph). Mounting FRANK_PRIMARY_CACHE_DIR at /root/.cache makes all
+# later boots load the compiled artifact instead, and hides nothing.
 #
 # The root unit (scripts/frank-primary.service, User=dank,
 # LimitMEMLOCK=infinity) grants the pin budget for the tiered-expert host
@@ -98,6 +108,15 @@ done
 LOG=/home/dank/logs/frank_primary.log
 mkdir -p "$(dirname "$LOG")"
 
+# Persistent compile cache (see the header). Create it before the mount so even
+# the first, cold boot has a target; unset -> no mount, exactly the old
+# ephemeral behaviour. It carries /root/.cache/vllm (AOT + torch_compile) and
+# /root/.cache/comgr (HIP kernel JIT).
+CACHE_DIR="${FRANK_PRIMARY_CACHE_DIR:-}"
+if [ -n "$CACHE_DIR" ]; then
+    mkdir -p "$CACHE_DIR"
+fi
+
 # --device /dev/dri is the WHOLE DIRECTORY (individual renderD nodes enumerate
 # without the card nodes and HIP refuses contexts — "No CUDA GPUs are
 # available" while device_count() says 2). --ipc=host for the engine's shared
@@ -109,8 +128,10 @@ podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
   --cap-add SYS_PTRACE \
   --device /dev/kfd --device /dev/dri --device /dev/mem \
   -e HIP_VISIBLE_DEVICES=0,1 -e VLLM_ROCM_USE_AITER=0 \
+  ${FRANK_PRIMARY_GPU_MAX_HW_QUEUES:+-e GPU_MAX_HW_QUEUES="$FRANK_PRIMARY_GPU_MAX_HW_QUEUES"} \
   ${FRANK_PRIMARY_GATHER_CHUNKS:+-e R4D_LRU_GATHER_CHUNKS="$FRANK_PRIMARY_GATHER_CHUNKS"} \
   ${FRANK_PRIMARY_GATHER_LANES:+-e R4D_LRU_GATHER_LANES="$FRANK_PRIMARY_GATHER_LANES"} \
+  ${CACHE_DIR:+-v "$CACHE_DIR":/root/.cache} \
   -v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \
   "$FRANK_PRIMARY_IMAGE" /models \
   --served-model-name "$FRANK_PRIMARY_SERVED_NAME" --port 7770 \