frank-primary: record the measured shape-E and compile-cache numbers

Live after the 1ac4d48 deploy: 'expert_precise_memory': True, runtime overhead
5.25 GiB, expert cache space 17.22 GiB / 7522 slots, swap region 13.7 GiB /
5986 slots, GPU KV cache 398,658 tokens / 1.99x, pool 3,150,000,000, MNBT 1024
-- the probe's projection is now production.

Compile cache proven on consecutive boots of that shape: cold 'Compiling a
graph for compile range (1, 1024) takes 179.24 s' / 'torch.compile took
188.26 s'; warm 'Directly load the compiled graph(s) ... from the cache, took
2.143 s' / 'torch.compile took 3.97 s in total' / 'init engine ... took 41.13 s
(compilation: 3.98 s)'. Host dir 1.4 GB / 4,707 files. Time to /health stays
payload-bound (490 s warm), so the claim is ~184 s of inductor/AOT build per
restart, not a proportional boot cut.

Tests: 367/367.

AGENTS.md

@@ -433,11 +433,13 @@ The **tcclaviger RDNA4 vLLM fork** (`docker.io/tcclaviger/vllm@sha256:da4029666
 digest-pinned) serving **`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8`**
 (~172.8 GiB, 131 shards) on **both** Radeon AI PRO R9700s (gfx1201), **TP=2**,
 **fp8 KV**, **no speculative decode**, **expert prefetch OFF**, and since
-2026-10-02 the **shape-D** geometry: `--max-model-len 200704` × 3 slots with
+2026-10-02 the **shape-E** geometry: `--max-model-len 200704` × 3 slots with
 `--kv-cache-memory-bytes 3,150,000,000`/rank, so the pool admits **~2
 full-length conversations** (measured live: `398,658 tokens, Maximum
 concurrency for 200,704 tokens per request: 1.99×`) while the argv keeps the
-only slot count that compiles (below). `models.yml` carries
+only slot count that compiles (below), plus `--expert-precise-memory`
+(`runtime overhead 5.25 GiB`, `expert cache space 17.22 GiB / 7,522 slots`),
+`GPU_MAX_HW_QUEUES=2` and a mounted `/root/.cache`. `models.yml` carries
 `contextWindow: 200704` for this endpoint — the served window, not the model's
 native 262144, so the harness compacts before vLLM answers a longer prompt
 with a 400.
@@ -485,7 +487,14 @@ torch_compile tree (`VLLM_CACHE_ROOT`, whose default is
 nothing in the image is hidden because the image has no such directory. The
 compile RANGE is part of the cache key, so an MNBT change costs one
 recompilation, not a permanent tax. Same class of fix as the recipe's
-`/cache/{triton,vllm,inductor}` volumes.
+`/cache/{triton,vllm,inductor}` volumes. **Proven live 2026-10-02 on
+consecutive boots:** cold `Compiling a graph for compile range (1, 1024) takes
+179.24 s` / `torch.compile took 188.26 s`; warm `Directly load the compiled
+graph(s) for compile range (1, 1024) from the cache, took 2.143 s` /
+`torch.compile took 3.97 s in total` / `init engine (profile, create kv cache,
+warmup model) took 41.13 s (compilation: 3.98 s)`. Time to `/health` is
+dominated by the 172.8 GiB payload load, so the visible win is the ~184 s of
+inductor/AOT build, not a proportional boot cut.
 
 Measured 2026-09-29 tuning campaign (fp8 + MTP4 era, since retired): decode
 **37.4-40.6 tok/s, acceptance ~54 %**; the 256k ctx arm measured 34.1-36.5;
@@ -501,8 +510,9 @@ the digest-pinned image, the payload's `config.json` and its exact
 131-shard count and the GPU device nodes, clears stale container
 identities, then blocks on `podman wait`. `deploy_llm_primary` gates the
 same, waits out :7770, relinks/enables/restarts the unit and health-gates
-with the MainPID check. Boot cost ~7 min warm; health returns 503 while
+with the MainPID check. Boot cost ~7 min warm (the 172.8 GiB payload load; the
-loading.
+~184 s inductor/AOT build inside it is what the mounted cache now skips — see
+the compile-cache lesson below); health returns 503 while loading.
 
 Runbook: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
 Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`.

scripts/frank-primary.env

@@ -127,6 +127,16 @@ FRANK_PRIMARY_GPU_MAX_HW_QUEUES=2
 # compiled function to /root/.cache/vllm/torch_compile_cache/…'. Mounting a
 # host dir at /root/.cache persists the vLLM AOT/torch_compile tree AND the
 # comgr HIP-kernel JIT cache, and hides nothing (the image has no such dir).
+# PROVEN LIVE 2026-10-02, two boots of this exact shape:
+#   cold  `Compiling a graph for compile range (1, 1024) takes 179.24 s`
+#         `torch.compile took 188.26 s in total`
+#   warm  `Directly load the compiled graph(s) for compile range (1, 1024)
+#          from the cache, took 2.143 s`
+#         `torch.compile took 3.97 s in total`
+#         `init engine (profile, create kv cache, warmup model) took 41.13 s
+#          (compilation: 3.98 s)`
+# (Total time to /health is dominated by the 172.8 GiB payload load, not the
+# compile — the saving is the ~184 s of inductor/AOT build.)
 # Safe to grow/prune: entries are keyed by version + config hash, so a stale
 # entry is a miss, never a wrong answer.
 FRANK_PRIMARY_CACHE_DIR=/home/dank/.cache/frank-primary

scripts/serve-frank-primary.sh

@@ -32,7 +32,11 @@
 # container's ephemeral layer, and `podman rm -f` deletes them — so every
 # deploy/restart/reboot re-pays `torch.compile took 181.20 s in total` (173.19 s
 # of it one graph). Mounting FRANK_PRIMARY_CACHE_DIR at /root/.cache makes all
-# later boots load the compiled artifact instead, and hides nothing.
+# later boots load the compiled artifact instead, and hides nothing. PROVEN
+# LIVE 2026-10-02 on consecutive boots of this shape: cold `torch.compile took
+# 188.26 s`, warm `Directly load the compiled graph(s) for compile range
+# (1, 1024) from the cache, took 2.143 s` / `torch.compile took 3.97 s in
+# total` / `init engine … took 41.13 s (compilation: 3.98 s)`.
 #
 # The root unit (scripts/frank-primary.service, User=dank,
 # LimitMEMLOCK=infinity) grants the pin budget for the tiered-expert host

serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md

@@ -338,6 +338,19 @@ includes the compile range, so the MNBT choice costs exactly one compilation.
 Entries are version- and config-hashed: a stale entry is a miss, never a wrong
 answer.
 
+**Proven live on the deployed endpoint, 2026-10-02, consecutive boots of this
+exact shape:**
+
+| boot | engine log |
+|---|---|
+| cold (deploy) | `Compiling a graph for compile range (1, 1024) takes 179.24 s` → `torch.compile took 188.26 s in total` |
+| warm (restart) | `Directly load the compiled graph(s) for compile range (1, 1024) from the cache, took 2.143 s` → `torch.compile took 3.97 s in total` → `init engine (profile, create kv cache, warmup model) took 41.13 s (compilation: 3.98 s)` |
+
+The host directory held **1.4 GB in 4,707 files** after the cold boot. Time to
+`/health` is dominated by the 172.8 GiB payload load (the warm boot reached
+health in 490 s), so the win is the ~184 s of inductor/AOT build rather than a
+proportional cut in boot time.
+
 `GPU_MAX_HW_QUEUES=2` caps the HSA hardware queues per GPU. Verified a real knob
 rather than a no-op - the literal is read out of
 `/opt/rocm/core-10.0/lib/libhsa-runtime64.so` - and shipped deliberately

serverAppConfigs-docs/log.md

@@ -1187,4 +1187,20 @@ every restart since the cutover.
   The flag was fine both times.
 - Live: `health=200`, `NRestarts=0`, arena 17.22 GiB / 7,522 slots, KV 398,658
   tokens / 1.99x. Docs: [[dankserver-qwen38-fp8-expert-offload-tuning]].
+- **Deployed `1ac4d48`, e2e 33/33, and the live plan matched the probe exactly:**
+  `'expert_precise_memory': True`, `runtime overhead … 5.25 GiB`, `expert cache
+  space 17.22 GiB / 7522 slots`, `swap region 13.7 GiB / 5986 slots`, `GPU KV
+  cache size: 398,658 tokens … 1.99x`, pool 3,150,000,000, MNBT 1024. The
+  precise overhead and the arena growth are now production numbers, not a
+  probe's projection.
+- **The compile-cache fix was then proven on the same endpoint, two consecutive
+  boots of the deployed shape:** cold `Compiling a graph for compile range
+  (1, 1024) takes 179.24 s` → `torch.compile took 188.26 s`; warm `Directly load
+  the compiled graph(s) for compile range (1, 1024) from the cache, took
+  2.143 s` → `torch.compile took 3.97 s in total` → `init engine (profile,
+  create kv cache, warmup model) took 41.13 s (compilation: 3.98 s)`. The host
+  dir held 1.4 GB in 4,707 files after the cold boot. **Time to `/health` is
+  dominated by the 172.8 GiB payload load** (the warm boot still needed 490 s),
+  so the honest claim is ~184 s of inductor/AOT build recovered per restart —
+  not a proportional boot cut.