feat(frank-archon): move the endpoint to radiance 1.8.1 and its budget model

The decode stall is step contention (measured: 18.7 ms decode-only steps against 317 ms steps that carry a prefill chunk, decode at 13 t/s median while prefill runs), and the two engine knobs that fix it arrived after this endpoint's pin. Moving to 1.8.1 (2026-10-18) brings both, and brings two CLI breaks with them: - --vram-weights-mib / --vram-kv-mib were REMOVED at 1.3.0. The per-card split between the expert slab and the KV pool is now derived by the engine's budget resolver from --gpu-headroom-mib, the cache bound (--max-model-len, --max-num-seqs) and --host-pool-mib, and the cache may lend idle blocks to the expert slab. The slab size and the pool's token count are boot-ledger lines from here on, so the capacity contract (3 x 200,000 fp8) is verified live by the probe's capacity leg instead of computed from an env number. - The prefix cache has no disk tier any more: --prefix-cache-dir and --prefix-cache-disk-mib are refusals and --prefix-cache-host-mib is --prefix-cache-mib. 64 GiB of parked KV on NVMe becomes re-prefill, so prefix_hit_rate is the thing to watch after this boot. env: image re-pinned to the 1.8.1 digest; the pair and the disk tier dropped; FRANK_ARCHON_PREFIX_CACHE_MIB replaces the host/disk pair; new FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto (explicit, because auto vs off is the difference between decode waiting one chunk and waiting a whole step) and FRANK_ARCHON_PREFILL_INTERLEAVE=0 (opt-in through a guarded argv array). serve: argv follows the new CLI, /data mount and the pre-created kvcache path go with the disk tier, and the retired env keys are no longer required (they would abort ExecStart). autoresearch-frank-archon.sh takes the same CLI, with a cold prefill now coming from --prefix-cache-mib 0 instead of a per-run disk directory. run-swift15-bf16.sh is untouched: it pins its own engines by design. tests: the two budget-arithmetic seams asserted knobs that no longer exist and are replaced by one seam that asserts the retired knobs are ABSENT from env and argv - a stale line is a boot refusal, not a no-op. The container-contract seam now asserts the tune mount and the absence of /data.
Merge pull request 'docs(frank-zealot): boot ledger of ba69ea5, corrected ubatch and q8_0 credits' (#26 ) from chore/frank-zealot-mtp-reslice into main
2026-10-19 10:18:24 -07:00 · 2026-10-19 03:38:40 -07:00 · 2026-10-19 03:38:03 -07:00 · 2026-10-19 03:19:04 -07:00 · 2026-10-19 03:16:48 -07:00 · 2026-10-19 03:16:35 -07:00
46 changed files with 907 additions and 848 deletions
@@ -342,7 +342,11 @@ file is a defect: it is the next thing to go stale.
  serve script that consumes it, and in the endpoint page. Every other page
  cites it by link.
 - Technique pages (spec decode, KV math, offload budgets, PP bandwidth) stay
-  technique pages; they stop carrying per-endpoint throughput tables.
+  technique pages: they stop carrying a per-endpoint throughput table or a live
+  shape, but a campaign ledger that exists nowhere else is owned by the page that
+  measured it. The defect is two pages carrying one figure, not a number living on a
+  technique page — move a figure only when an endpoint page or an env header is its
+  home; otherwise leave it in place and name the era it measured.

 ## Model Config (harness)

@@ -384,10 +388,11 @@ the publisher's own **Q8_0** mmproj passed as `--mmproj` — Q8_0 is the
 publisher's requirement for concurrent image requests (the card states the
 BF16 tower can crash `llama-server` when several image requests run at once,
 and this endpoint runs several slots). `frank-zealot`'s `contextWindow` is
-**262144** — the PER-SLOT window (`--ctx-size 524288 --parallel 2` since the
-2026-10-18 reslice to exact f16 KV), which EQUALS the model's
-native window; llama.cpp answers a longer prompt with a 400 naming the limit,
-never a truncation, so the harness must compact before it.
+**200704** — the PER-SLOT window (`--ctx-size 602112 --parallel 3` since the
+2026-10-19 reslice that funded a third slot AND an armed MTP drafter),
+deliberately below the model's native window; llama.cpp answers a longer prompt
+with a 400 naming the limit, never a truncation, so the harness must compact
+before it.
 Note also
 that
 the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
@@ -435,10 +440,11 @@ disagrees with it is the bug.
  loader ignoring that metadata dequantizes wrong); payload
  `Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
  tower can crash the server with concurrent image requests).
- Shape: `--ctx-size 524288 --parallel 2` = **two conversations at the
-  model's native 262,144**, `CACHE_TYPE_K/V=f16`, batch 4096 / ubatch 2048,
-  `SPEC_TYPE=none`, `--flash-attn on`, `--jinja`, `--metrics`.
- Harness/gate contract: `contextWindow: 262144` is the **per-slot** window
+- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
+  200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
+  B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
+  `SPEC_TYPE=draft-mtp` (depth 2), `--flash-attn on`, `--jinja`, `--metrics`.
+- Harness/gate contract: `contextWindow: 200704` is the **per-slot** window
  (`n_ctx_slot = CTX_SIZE / PARALLEL`), and llama.cpp answers a longer prompt
  with a 400 rather than truncating. `--metrics` is mandatory or Prometheus
  reads 501 = DOWN beside a healthy service. Presence penalty has no GGUF
@@ -938,6 +944,14 @@ linked runbooks.
  already does all three; ad-hoc fetches tend not to. Gated repos (e.g.
  `orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF`, same 104.09 GiB Q4_K_S)
  return 403 even with the host's HF token until that account accepts the gate.
+- **A scout verdict is a lead, never a finding.** In the 2026-10-19 vault de-dup
+  campaign, four read-only scouts worked the same pages: one classified era pages as
+  stale *because* they document retired payloads (that is an era page's content), one
+  reported files that do not exist, one reported duplications that grep disproved
+  (`457.9 t/s` and the ping-RTT table are solo-held by `Ethernet Bandwidth Math`),
+  and one cited a line number past the end of the file it judged. Re-verify every
+  claimed duplicate and every claimed staleness against the file and today's fleet
+  state before editing; a scout yield buys no evidence.

 ## Runbook Index

@@ -29,10 +29,10 @@
 # The scratch mirrors the production container contract exactly (serve-frank-
 # primary.sh): --network host, --init, seccomp=unconfined (io_uring for the
 # n-gram table), memlock unbounded (the ~132 GiB pinned ledger), /dev/kfd +
-# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
-# differences: the container name (arbench, so prod's identity is untouched)
-# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
-# prefill is cold — the prod tier at kvcache/frank-archon is left intact.
+# /dev/dri, the .rad ro, the tune cache mount. The ONLY differences: the
+# container name (arbench, so prod's identity is untouched) and the prefix cache
+# switched OFF (--prefix-cache-mib 0), so every prefill is cold — the tier is
+# host RAM now, there is no per-run disk directory to wipe.
 set -uo pipefail

 ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-archon.env}"
@@ -60,9 +60,8 @@ set -a; . "$ENV_FILE"; set +a
 sudo systemctl stop frank-archon >/dev/null 2>&1 || true
 podman rm -f "$SCRATCH" >/dev/null 2>&1 || true

-# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
-FRESH_KV="$FRANK_ARCHON_STATE_DIR/kvcache/$SCRATCH"
-rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
+# --- the prefix cache is switched off below, so there is no scratch KV tree ---
+# to wipe; the cold prefill comes from --prefix-cache-mib 0.
 mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"

 # --- boot the scratch: the production run line, scratch name + fresh KV ------
@@ -73,7 +72,6 @@ podman run -d --name "$SCRATCH" \
  --ulimit memlock=-1:-1 \
  --device /dev/kfd --device /dev/dri \
  -v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-  -v "$FRANK_ARCHON_STATE_DIR":/data \
  -v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
  -v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
  "$FRANK_ARCHON_IMAGE" \
@@ -87,12 +85,9 @@ podman run -d --name "$SCRATCH" \
  --placement "$FRANK_ARCHON_PLACEMENT" \
  --host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
  --gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
-  --vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
-  --vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
  --kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
-  --prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
-  --prefix-cache-dir "/data/kvcache/$SCRATCH" \
-  --prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
+  --prefix-cache-mib 0 \
+  --busy-prefill-chunk "${FRANK_ARCHON_BUSY_PREFILL_CHUNK:-auto}" \
  --ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
  --num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
  --max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
@@ -1459,7 +1459,7 @@ LLM_ZEALOT_RUNTIME="${LLM_ZEALOT_RUNTIME:-/home/dank/GitPackages/llama-prism-b10
 # image path — a deploy that starts without it silently ships a text-only
 # endpoint. Checked unconditionally, like frank-dragoon's. Q8_0 is MANDATORY,
 # not a preference: the card says the BF16 tower can crash llama-server when
-# several image requests run at once, and this endpoint runs --parallel 2. Its
+# several image requests run at once, and this endpoint runs --parallel 3. Its
 # 868.32 MiB worst-case VRAM (595.29 model + 248.10 compute in the boot ledger)
 # lands AFTER the engine's fit check, so it is not inside the q8_0 budget.
 LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf}"
@@ -7,10 +7,15 @@
 # only the reason a reader editing THIS line cannot recover from the page.
 # Where the two disagree, this file wins on values; the boot ledger arbitrates
 # the engine shape.
-FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
-# 1.0.13 is pinned by digest on purpose: 1.2.0 is a DEAD END on this payload
-# (acceptance 0.7123 vs 0.7549 — the page's Dead ends section names the suspect).
-# The .rad file mounted under /models; /data holds the prefix-cache disk tier.
+FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed
+# 1.8.1 (2026-10-18), pinned by digest. Two CLI breaks sit between it and the
+# 1.0.13 line this endpoint ran for two eras, and both are boot refusals, not
+# warnings: --vram-weights-mib/--vram-kv-mib were REMOVED at 1.3.0 (the card is
+# now split by the engine's budget resolver — see below), and the prefix cache
+# lost its disk tier. Draft acceptance has to be re-scored on any pin past 1.2.0:
+# 1.2.0's MoE weight rotation measured 0.7123 against the 0.7549 control, and the
+# suspect is untangled (the page's Dead ends section names it).
+# The .rad file mounts under /models; the tune cache mounts separately below.
 # The model dir holds ONLY .rad files — the HF download and staging stay beside it.
 FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
 FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad
@@ -52,21 +57,25 @@ FRANK_ARCHON_MAX_SEQS=3
 # sequence.
 FRANK_ARCHON_MAX_MODEL_LEN=200000
 FRANK_ARCHON_PLACEMENT=expert_tiered
-# Pinned host pool for expert rows that do not fit VRAM. It moves WITH
-# VRAM_WEIGHTS: the pair and this pool serve the per-rank resident set together,
-# so cutting the slab spills here — under-sizing it is a boot failure (36,864
+# Pinned host pool for expert rows that do not fit VRAM. Under the budget
+# resolver this is ALSO the lever that decides which experts stay on the card:
+# the card's leftover goes to the expert plane first and the KV cache takes the
+# rest, so growing the pool moves experts off the card and shrinking it spills
+# them onto VRAM the cache wanted. Under-sizing it is a boot failure (36,864
 # refused a boot). The ledger's coverage line is the referee, not this file.
 FRANK_ARCHON_HOST_POOL_MIB=44928
 # VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
 # floor is built from it.
 FRANK_ARCHON_GPU_HEADROOM_MIB=96
-# ONE budget per card: VRAM_WEIGHTS + VRAM_KV <= 30,969 MiB, the ceiling the
-# engine's own budget table prints (derivation and the slab/pool price: the
-# page's VRAM laws). Idle KV VRAM is ALSO the expert cache's spillover, so a
-# bigger pool pays twice — which is why the pool sits at the contract and the
-# slab takes the rest.
-FRANK_ARCHON_VRAM_WEIGHTS_MIB=25600
-FRANK_ARCHON_VRAM_KV_MIB=5120
+# There is no VRAM pair any more. --vram-weights-mib/--vram-kv-mib were removed
+# at 1.3.0: the engine takes capacity - already-held - GPU_HEADROOM - arena,
+# puts the static (non-expert) weights off the top, then divides the rest between
+# the expert plane and the KV cache — experts first when the cache has no idle
+# blocks to lend, and never less than ONE MAX_MODEL_LEN sequence for the cache.
+# What moves the split from here is HOST_POOL_MIB (which experts stay resident),
+# MAX_MODEL_LEN/MAX_SEQS (the cache bound) and GPU_HEADROOM_MIB. The slab size
+# the prefill-slope law cares about is a boot-ledger line now, not a knob: read
+# it after every resize instead of asserting it here.
 # fp8 is the ONLY rung below bf16 on this engine: one kv_dtype for the whole
 # cache, so llama.cpp's exact-K/quantized-V split (frank-zealot) is unservable
 # here. bf16 costs 1.89x fp8, not 2x — the QSA block-key plane is dtype-fixed.
@@ -75,11 +84,25 @@ FRANK_ARCHON_KV_DTYPE=fp8
 # 2, and 3 x (1+2) = 9 rows stays inside the wht6 25-row exactness bound.
 FRANK_ARCHON_SPEC_TOKENS=2
 FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
-# Prefix cache tiers. The host tier is sized in ENTRIES and one entry tracks the
-# cache dtype (58.50 KiB at fp8), so a dtype change re-sizes the parked set
-# without touching these numbers.
-FRANK_ARCHON_PREFIX_HOST_MIB=8192
-FRANK_ARCHON_PREFIX_DISK_MIB=65536
+# Step sharing. A prefill chunk that shares a step with decoding sequences holds
+# the device for the chunk's whole time, and every decode row in that step waits
+# that long for its next token (the census: [[LLM Performance Diagnosis]] §9).
+# `auto` leaves each decode row behind the chunk a row plus its draft window and
+# keeps a share for whatever is waiting to be admitted; a number pins the chunk's
+# cap; `off` is the pre-1.4 behaviour this endpoint ran for two eras, where the
+# chunk took the step and admission never ran. Interleave inserts N decode-only
+# steps after a chunk step taken beside decoders: it protects their cadence and
+# holds a waiting request out of those steps, so it stays 0 until the post-migration
+# decode-vs-prefill buckets say it is wanted.
+FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto
+FRANK_ARCHON_PREFILL_INTERLEAVE=0
+# Prefix cache: pinned host memory only. The disk tier was removed outright
+# (--prefix-cache-dir and --prefix-cache-disk-mib are refusals), so what used to
+# be restored from NVMe is now re-prefilled. The tier is sized in ENTRIES and one
+# entry tracks the cache dtype (58.50 KiB at fp8). Watch prefix_hit_rate after
+# this change: a drop is extra prefill, and prefill is exactly the traffic that
+# costs decode its cadence.
+FRANK_ARCHON_PREFIX_CACHE_MIB=8192
 # Frank's pin law: the PLE n-gram table serves from pinned host RAM the cards
 # read. `ram` REFUSES the start if memlock is too small — the unit's
 # LimitMEMLOCK=infinity is the other half of this law.
@@ -11,35 +11,51 @@ LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server
 # metadata dequantizes wrong, silently.
 MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf
 # Q8_0 IS MANDATORY: the publisher states the BF16 tower can crash llama-server
-# under concurrent image requests, which is exactly --parallel 2.
+# under concurrent image requests, which is exactly --parallel 3.
 MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
 # Served id — the harness/family name, not a fact about the payload.
 # Id-keyed consumers: OpenWebUI (model row, access_grant, api_configs), models.yml.
 ALIAS=frank-zealot
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7771
-# CTX_SIZE is the POOL llama.cpp divides across slots: 524,288 / 2 = 262,144 a
-# conversation, each still at the trained window. MUST stay divisible by
-# PARALLEL (the seam enforces it) and models.yml contextWindow quotes the
-# PER-SLOT number — a longer prompt is a 400, never a truncation. The pool
-# cannot go back to 786,432 under exact f16 KV: 24,576 MiB of cache against the
-# 24,123 MiB the driver leaves addressable, before a weight loads.
-CTX_SIZE=524288
-PARALLEL=2
-BATCH_SIZE=4096
+# CTX_SIZE is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
+# conversation. MUST stay divisible by PARALLEL (the seam enforces it) and
+# models.yml contextWindow quotes the PER-SLOT number — a longer prompt is a
+# 400, never a truncation. The pool is bought, not granted: K+V at 25,088
+# B/token is 14,406.00 MiB here and the MTP draft context bills its own f16 KV
+# over the WHOLE pool (4,096 B/position ⇒ 2,352.00 MiB), so the old 786,432
+# pool cannot carry a drafter at any KV dtype — 18,816 + 3,072 of cache alone
+# against the 24,123 MiB the driver leaves addressable.
+CTX_SIZE=602112
+PARALLEL=3
+# The logical batch is VRAM- and throughput-neutral at a fixed ubatch (the law
+# on [[frank-zealot]]); 1024×6 keeps it a clean multiple of the ubatch.
+BATCH_SIZE=6144
 # Ubatch is a prefill knob (decode never moves with it) and it sets the
-# compute-buffer size — which is what decides whether a drafter fits.
-# UBATCH_SIZE=1024 is the documented MTP arm.
-UBATCH_SIZE=2048
-# Exact K and V: the retrieval-fidelity ask, paid with one conversation of
-# concurrency. Rate law and the arithmetic: [[frank-zealot]] → Laws.
+# compute-buffer size, which decides whether a drafter fits. 1024 is the rung
+# every measured MTP arm here ran on (213.5 t/s against 109.2 at 2048). Its
+# VRAM credit is SHAPE-specific: the 786,432-pool arm census showed ~1,306 MiB
+# back, this boot measured 2,089.88 MiB of target compute against 2,496.13 at
+# ubatch 2048 — 406 MiB back, with the split moving (CUDA0 +257, CUDA_Host
+# −663). The drafter is funded by CACHE_TYPE_V below, not by this knob.
+UBATCH_SIZE=1024
+# K exact, V in the 8-bit class: 16,384 + 8,704 = 25,088 B/token (8,192
+# elements per token per cache × 2 B for f16, × 1.0625 for q8_0 — there is no
+# true fp8 KV dtype in this build). That 7,680 B/token saved against exact V is
+# what funds BOTH the third conversation and the drafter. Quantized V is
+# illegal without --flash-attn, which the serve script pins. Rate law and the
+# arithmetic: [[frank-zealot]] → Laws.
 CACHE_TYPE_K=f16
-CACHE_TYPE_V=f16
-# Speculation OFF — measured, not assumed: the draft context bills its own f16
-# KV over the WHOLE pool, and there is no room beside exact KV (the arm census
-# is on the page). SPEC_TYPE=none strips the WHOLE set (--spec-type and every
-# --spec-draft-*); half a set has segfaulted at load.
-SPEC_TYPE=none
+CACHE_TYPE_V=q8_0
+# Speculation ON (Frank, 2026-10-19): the head is in the file
+# (qwen35.nextn_predict_layers = 1, blk.32.nextn.*) and rides the MAIN model's
+# weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of
+# draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for
+# by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V
+# beside MTP needs fork build >= b10776; LLAMA_BIN is 10777.
+# SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and
+# every --spec-draft-*); half a set has segfaulted at load.
+SPEC_TYPE=draft-mtp
 SPEC_DRAFT_N_MAX=2
 SPEC_DRAFT_N_MIN=0
 SPEC_DRAFT_P_MIN=0
@@ -15,12 +15,13 @@
 # log states every budget before anything is allocated (weights vs KV
 # split, the expert host pool's coverage of the worst case, the PLE pin,
 # the P2P answer per pair) — read it after every resize.
-# Deployed shape (2026-10-13): THREE decode slots x 200,000 tokens, fp8
-# (quantized) KV in a 5,120 MiB pool (contract 3 x 200,000 + 1.3 %) and a
-# 25,600 MiB resident expert slab — 30,720 MiB of the 30,969 MiB the engine's
-# own budget table makes claimable, spent on the slab because prefill is
-# slab-limited. 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache host
-# 8 GiB + disk 64 GiB, PLE pinned in RAM.
+# Deployed shape (2026-10-19, radiance 1.8.1): THREE decode slots x 200,000
+# tokens, fp8 KV, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache in
+# pinned host RAM, PLE pinned in RAM. The per-card split between the expert slab
+# and the KV pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair
+# and the budget resolver derives it from headroom, the cache bound and the host
+# pool, so the slab size and the pool's token count are LEDGER lines: after every
+# boot read them out of the log and check the pool still backs 3 x 200,000.
 #
 # The root unit (scripts/frank-archon.service, User=dank,
 # LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
@@ -56,13 +57,11 @@ set +a
 : "${FRANK_ARCHON_PLACEMENT:?FRANK_ARCHON_PLACEMENT must be set}"
 : "${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
 : "${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
-: "${FRANK_ARCHON_VRAM_WEIGHTS_MIB:?FRANK_ARCHON_VRAM_WEIGHTS_MIB must be set}"
-: "${FRANK_ARCHON_VRAM_KV_MIB:?FRANK_ARCHON_VRAM_KV_MIB must be set}"
+: "${FRANK_ARCHON_BUSY_PREFILL_CHUNK:?FRANK_ARCHON_BUSY_PREFILL_CHUNK must be set (auto|off|N)}"
 : "${FRANK_ARCHON_KV_DTYPE:?FRANK_ARCHON_KV_DTYPE must be set}"
 : "${FRANK_ARCHON_SPEC_TOKENS:?FRANK_ARCHON_SPEC_TOKENS must be set}"
 : "${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
-: "${FRANK_ARCHON_PREFIX_HOST_MIB:?FRANK_ARCHON_PREFIX_HOST_MIB must be set}"
-: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
+: "${FRANK_ARCHON_PREFIX_CACHE_MIB:?FRANK_ARCHON_PREFIX_CACHE_MIB must be set}"
 : "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
 : "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
 : "${FRANK_ARCHON_TTM_TARGET_MIB:?FRANK_ARCHON_TTM_TARGET_MIB must be set}"
@@ -103,19 +102,12 @@ if [ ! -f "$FRANK_ARCHON_TEMPLATE" ]; then
    exit 1
 fi

-mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
-# The engine's create is NOT recursive: with only the state dir mounted, its
-# "cannot create /data/kvcache/frank-archon: No such file or directory"
-# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
-# the full path on the host so the tier opens on first boot.
-
 mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
 # rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
 # the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
 # $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
 # the cache must ride a host mount or it dies with the --rm container
-# that wrote it. The dir is pre-created for the same reason as kvcache
-# (the engine's create is not recursive).
+# that wrote it. The engine's create is not recursive, so the dir is pre-created.

 if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
    echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
@@ -148,6 +140,15 @@ done
 LOG=/home/dank/logs/frank_archon.log
 mkdir -p "$(dirname "$LOG")"

+# Interleave is opt-in: 0 is the engine's default and passing it anyway would put
+# a flag in argv that changes nothing but the record. The sharing cap is always
+# explicit because `auto` vs `off` is the difference between decode waiting one
+# chunk and waiting the whole step.
+PREFILL_ARGS=()
+if [ "${FRANK_ARCHON_PREFILL_INTERLEAVE:-0}" -gt 0 ]; then
+    PREFILL_ARGS=(--prefill-interleave "$FRANK_ARCHON_PREFILL_INTERLEAVE")
+fi
+
 # Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
 # --network host (the server binds the host's addresses directly, no NAT),
 # --init (the engine releases the cards on SIGTERM), seccomp=unconfined
@@ -156,8 +157,9 @@ mkdir -p "$(dirname "$LOG")"
 # makes for host memory), memlock unbounded (the pinned ledger — legal
 # because the root unit grants LimitMEMLOCK=infinity to this dank process),
 # /dev/kfd + /dev/dri (render/video groups are moot for a root container).
-# The template rides a single-file mount; the .rad dir is ro; /data holds
-# the prefix-cache disk tier.
+# The template rides a single-file mount and the .rad dir is ro. The prefix cache
+# has no disk tier any more, so the only host state this container writes is the
+# rad-tune cache under /opt/radiance/share/radiance/tune.
 podman run -d --name "$FRANK_ARCHON_CONTAINER" \
  --network host \
  --init \
@@ -165,7 +167,6 @@ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
  --ulimit memlock=-1:-1 \
  --device /dev/kfd --device /dev/dri \
  -v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-  -v "$FRANK_ARCHON_STATE_DIR":/data \
  -v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
  -v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
  "$FRANK_ARCHON_IMAGE" \
@@ -179,15 +180,13 @@ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
  --placement "$FRANK_ARCHON_PLACEMENT" \
  --host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
  --gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
-  --vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
-  --vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
  --kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
-  --prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
-  --prefix-cache-dir /data/kvcache/frank-archon \
-  --prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
+  --prefix-cache-mib "$FRANK_ARCHON_PREFIX_CACHE_MIB" \
  --ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
  --num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
  --max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
+  --busy-prefill-chunk "$FRANK_ARCHON_BUSY_PREFILL_CHUNK" \
+  "${PREFILL_ARGS[@]}" \
  ${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
  > "$LOG" 2>&1

@@ -5,33 +5,45 @@
 #
 # 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export, then the same
 # evening TWO 262,144-token conversations (--ctx-size 524288 --parallel 2) over
-# an UNQUANTIZED f16 KV cache. The 9B bills 8,192 elements per token per cache
-# (8 full-attention layers instead of the 27B's 16), so exact K and V costs
-# 32,768 B/token — 16,384.00 MiB at the 524,288 pool, which is what a 24,123 MiB
-# addressable card can carry beside the trunk. Verbosity 4 ships so the VRAM-fit
-# forensics (KV layout, buffer sizes, graph capture) land in the journal.
+# an UNQUANTIZED f16 KV cache.
+#
+# 2026-10-19 (Frank): THREE 200,704-token conversations over a 602,112 pool
+# (--ctx-size 602112 --parallel 3), K left at the engine's own default (f16),
+# V dropped to q8_0, batch 1024×6 / ubatch 1024, and the MTP drafter ARMED
+# (--spec-type draft-mtp). The 9B bills 8,192 elements per token per cache (8
+# full-attention layers instead of the 27B's 16), so f16 K is 16,384 B/token and
+# q8_0 V is 8,704: 25,088 B/token = 14,406.00 MiB at the 602,112 pool. That
+# 7,680 B/token against exact V is exactly what funds the third conversation
+# AND a draft context that bills f16 KV over the whole pool. Verbosity 4 ships
+# so the VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in
+# the journal.
 #
 # Architecture (engine ledger, not the marketing page): qwen35 hybrid, 32
 # layers with qwen35.full_attention_interval = 4 ⇒ 8 full-attention + 24 Gated
 # DeltaNet, 4 KV heads × 256 dim, native 262,144. The KV pool scales with the 8
-# full layers (16,384.00 MiB of f16 at the 524,288 pool); the DeltaNet layers
-# cost a fixed recurrent state per SLOT — ~50.25 MiB a cell, so 100.50 MiB for
-# two. The runtime is a hard lock, not a preference: custom GGML type 142 plus
-# prism.hadamard.* metadata, which only the vendor build registers (the b10776
-# gate that moved the prebuilt 10754 → 10777 was MTP beside quantized V, which
-# exact KV now makes moot — the type-142 lock alone keeps the build pinned).
+# full layers (14,406.00 MiB of f16 K + q8_0 V at the 602,112 pool); the
+# DeltaNet layers cost a fixed recurrent state per SLOT — ~50.25 MiB a cell, so
+# 150.75 MiB for three, and an armed depth-2 drafter triples that again. The
+# runtime is a hard lock, not a preference: custom GGML type 142 plus
+# prism.hadamard.* metadata, which only the vendor build registers — and the
+# b10776 gate that moved the prebuilt 10754 → 10777 (MTP beside a quantized V)
+# is LOAD-BEARING again on this shape, not moot as it was under exact KV.
 # Vision comes from the publisher's Q8_0 projector (the BF16 one can crash the
-# server on concurrent images, and this endpoint runs two slots), so --mmproj is
+# server on concurrent images, and this endpoint runs three slots), so --mmproj is
 # what gives this endpoint eyes; MMPROJ_FILE="" is the documented text-only
 # opt-out.
 #
-# Speculative decoding ships OFF, measured: the head loads and accepts well
-# (0.798/0.617 per position, mean length 2.41) but the draft context bills its
-# own f16 KV over the WHOLE pool — 4,096 B/position × 524,288 = 2,048 MiB plus
-# buffers — and the trunk already projects 21,234 MiB with exact KV. Exact KV
-# and MTP cannot share this card; frank-zealot.env carries the q8_0-era arm table
-# and the two doors back to the drafter (one window, or KV back to q8_0 at
-# --ubatch-size 1024 for 213.5 t/s).
+# Speculative decoding ships ON (Frank, 2026-10-19). The head is IN the file and
+# accepts well (0.798/0.617 per position, mean length 2.41, measured 2026-10-18);
+# what made it lose that evening was the bill, not the drafts — the draft context
+# is built against the target's WHOLE pool, so blk.32 adds 4,096 B/position
+# (2,352.00 MiB at 602,112) plus 480.97 MiB of its own compute buffers on the
+# card, and it triples the recurrent state per slot (452.25 MiB for three cells
+# against 50.25 a cell with the drafter off). CACHE_TYPE_V=q8_0 is what pays
+# that: 7,680 B/token cheaper than exact V, 4,410 MiB at this pool. The ubatch
+# drop to 1024 contributes ~406 MiB here, NOT the ~1,306 the 786,432-pool census
+# showed — that figure does not transfer between shapes. SPEC_TYPE=none is
+# the rollback and strips the whole flag set.
 #
 # Sampling: the checkpoint carries the publisher's thinking recipe as GGUF
 # metadata (general.sampling.* = temp 1.0 / top_p 0.95 / top_k 20 / min_p 0 /
@@ -58,49 +70,46 @@ fi
 : "${ALIAS:=frank-zealot}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7771}"
-# ctx-size is the POOL llama.cpp divides across slots: 524,288 / 2 = 262,144
-# per conversation — every slot still at the model's trained window. The harness
-# contextWindow MUST quote the PER-SLOT number — a longer prompt is answered
-# with a 400, never truncated. Under exact KV the pool cannot stay at 786,432:
-# 32,768 B/token × 786,432 = 24,576 MiB, more than the card's whole addressable
-# 24,123 MiB before a single weight loads.
-: "${CTX_SIZE:=524288}"
-: "${PARALLEL:=2}"
+# ctx-size is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
+# conversation. The harness contextWindow MUST quote the PER-SLOT number — a
+# longer prompt is answered with a 400, never truncated. The pool is priced by
+# the KV bill, not by the trained window: at 25,088 B/token (f16 K + q8_0 V) the
+# 602,112 pool costs 14,406.00 MiB and the armed drafter adds its own f16 KV over
+# the whole pool (2,352.00 MiB). Exact V at this pool (18,816 MiB) or the old
+# 786,432 pool at any dtype cannot boot inside the 24,123 MiB the driver leaves
+# addressable.
+: "${CTX_SIZE:=602112}"
+: "${PARALLEL:=3}"
 # Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
-# 2048 is the measured CUDA optimum (MiMo era, same arch/shape: +4.6 % prefill
-# at 8.9k depth, decode flat, 616 MiB). The logical batch is VRAM- and
-# throughput-neutral at a fixed ubatch. It ALSO sets the compute-buffer size,
-# which is the term that decides whether an MTP draft context fits beside a KV
-# pool: 1024 hands back 1,306 MiB and was the arm that ran the drafter at three
-# q8_0 windows (frank-zealot.env carries the table).
-: "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=2048}"
-# Exact KV: f16 K + f16 V — no quantization error anywhere in the attention
-# state. 32 KiB/token at this geometry (8 full-attention layers × 4 KV heads ×
-# 256 dim × 2 B × 2 caches); 16,384.00 MiB at the 524,288 pool. That is the whole
-# reason the slot count is 2: the same dtype at the old 786,432 pool is 24,576
-# MiB and dies at cudaMalloc. q8_0 K + q8_0 V (17,408 B/token, 13,056.00 MiB at
-# 786,432) is the documented fallback rung and funds three windows; CACHE_TYPE_V
-# alone is the halfway door. Explicit either way so the contract is visible in
-# the argv, and --flash-attn stays pinned because quantized V — the fallback —
-# is illegal without it (no true fp8 KV dtype exists in this build).
+# The logical batch is VRAM- and throughput-neutral at a fixed ubatch, so 6144
+# (1024×6) is free; the ubatch is what sets the compute-buffer size, and 1024 is
+# the rung every measured MTP arm here ran on (213.5 t/s at 1024 against 109.2
+# at 2048) — worth ~406 MiB at THIS shape, not the ~1,306 the old pool showed.
+: "${BATCH_SIZE:=6144}"
+: "${UBATCH_SIZE:=1024}"
+# Asymmetric KV: f16 K (the engine's own default, spelled out so the contract is
+# visible in argv) + q8_0 V. 8,192 elements per token per cache (8 full-attention
+# layers × 4 KV heads × 256 dim) ⇒ 16,384 B/token for K and 8,704 for V (q8_0 is
+# 1.0625 B/element — llama.cpp has no true fp8 KV dtype), 25,088 B/token and
+# 14,406.00 MiB at the 602,112 pool. That is the halfway door between exact KV
+# (32,768 B/token, which costs the third conversation and the drafter both) and
+# the fully quantized pair (17,408 B/token, which costs retrieval fidelity in the
+# keys); --flash-attn stays pinned because quantized V is illegal without it.
 : "${CACHE_TYPE_K:=f16}"
-: "${CACHE_TYPE_V:=f16}"
-# Speculative decoding defaults OFF, measured on this card and shape: the head
-# IS in the checkpoint (qwen35.nextn_predict_layers = 1 — MTP is a property of
-# the FILE, so the plain PQ2_0 sibling could not run it at any flag value), it
-# loads, and it accepts well (mean accepted length 2.41) — but the draft context
-# bills its own f16 KV over the WHOLE pool (4,096 B/position, 2,048 MiB at the
-# 524,288 pool) and at ubatch 2048 that squeeze halves decode: 167.2 → 109.2 t/s
-# on the q8_0 3-slot shape. The draft cache dtype IS a knob (--spec-draft-type-k
-# / --spec-draft-type-v, both default f16 — verified in this build's --help), so
-# an armed drafter can halve its own bill; there is no --spec-draft-model because
-# the draft rides the main model's weights. Card gate: quantized V alongside MTP
-# needs fork build >= prism-b10776, which is why LLAMA_BIN moved to the 10777
-# prebuilt; exact KV removes that gate but NOT the type-142 loader lock.
-# SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024 is the documented MTP arm (213.5 t/s);
-# SPEC_TYPE=none strips the whole set (flags move together).
-: "${SPEC_TYPE:=none}"
+: "${CACHE_TYPE_V:=q8_0}"
+# Speculative decoding defaults ON: the head IS in the checkpoint
+# (qwen35.nextn_predict_layers = 1 — MTP is a property of the FILE, so the plain
+# PQ2_0 sibling could not run it at any flag value) and the draft context rides
+# the MAIN model's weights, so there is no --spec-draft-model. Its bill is the
+# target's whole pool at 4,096 B/position plus ~0.45 GiB of buffers at this
+# ubatch and a tripled recurrent state; CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024
+# above are what pay for it. The draft cache dtype IS a further knob
+# (--spec-draft-type-k / --spec-draft-type-v, both default f16 — verified in this
+# build's --help) and stays at its default here. Card gate: quantized V alongside
+# MTP needs fork build >= prism-b10776, which is why LLAMA_BIN is the 10777
+# prebuilt; the type-142 loader lock keeps it pinned regardless.
+# SPEC_TYPE=none is the rollback and strips the whole set (flags move together).
+: "${SPEC_TYPE:=draft-mtp}"
 : "${SPEC_DRAFT_N_MAX:=2}"
 : "${SPEC_DRAFT_N_MIN:=0}"
 : "${SPEC_DRAFT_P_MIN:=0}"
@@ -14,7 +14,7 @@ timestamp: 2026-09-03

 DFlash and DSpark are the two newest speculative decoding methods. They draft many tokens in one parallel pass instead of one token at a time. This reduces decode latency on memory-bound workloads.

-DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both have draft models for the Qwen3.6 models that Dankserver and Dankclaw run.
+DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both publish draft models for the Qwen3.5/Qwen3.6 families (table below); what any of them pairs with on this fleet is an era on [[frank-dragoon]] or [[frank-zealot]].

 ## How DFlash Works

@@ -115,12 +115,12 @@ unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
 - Internal order is fixed (ngram types first, then drafters); CLI order is
  irrelevant.
 - **A legal chain can still lose on a recurrent model.** `draft-mtp,ngram-mod`
-  on the qwen35 hybrid (frank-dragoon era) measured −15 % prose / −7 %
-  block-copy (2026-10-03): adding the ngram layer logs `recurrent state
-  rollback is not compatible with 'ngram-mod' - disabling rollback support`,
-  and MTP verification depends on that rollback, so acceptance drops on ALL
-  traffic even where the matcher fires. Measure chains per architecture,
-  never by analogy.
+  on the qwen35 hybrid loses on both traffic shapes (2026-10-03): adding the
+  ngram layer logs `recurrent state rollback is not compatible with
+  'ngram-mod' - disabling rollback support`, and MTP verification depends on
+  that rollback, so acceptance drops on ALL traffic even where the matcher
+  fires. The measured size of each loss is [[frank-dragoon]]'s (Laws). Measure
+  chains per architecture, never by analogy.
 - ngram-mod prints NO startup banner — the load log shows only the drafter's
  `adding speculative implementation` line. Never conclude "chain not
  engaged" from the load log; identify behaviourally (`draft_n` > 0 only
@@ -198,7 +198,10 @@ Compatible MTP-only draft GGUFs on Hugging Face:

 Speculative decoding is lossless under rejection sampling: the target verifies every draft token. A draft head from the base model works with a derivative target; wrong proposals just lower the acceptance rate. The accepted MTP draft lengths are 2-3 tokens; the "MTP Defaults Are a Trap" writeup (thefrontierlab.ai) documents the tuning pitfalls.

-On Dankclaw, `draft-dflash` remains the benchmarked winner (20.2 t/s at 4k vs 16.9 for DFlash+ngram and 8.7 for ngram-only). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
+On the mid-August 2026 Dankclaw shape `draft-dflash` was the benchmarked winner
+at 4k — 20.2 t/s, against 16.9 with ngram layered on and 8.7 ngram-only; those
+three figures are that retired llama.cpp era's property ([[frank-dragoon]] owns
+the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).

 ## Draft Models

@@ -213,13 +216,12 @@ DFlash draft model sizes for the models on these servers:

 The DFlash draft is not a standalone model. It must pair with its target model. The model cards recommend block size 8 for higher concurrency and 16 for single-stream throughput.

-## Deployment on Dankserver (NVIDIA RTX 3090)
+## DFlash on an RTX 3090 (Ampere) — the archived vLLM path

-The Dankserver llama.cpp deployment (`frank-zealot`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
-
-> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-zealot` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
-
-The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
+DFlash was tried on the 3090 line, lost there to the ngram chain, and was never
+kept deployed; that card's payloads, shapes and rates are [[frank-zealot]]'s
+(its Eras rows are the llama.cpp arms the draft was tested against). What
+transfers is the Ampere arithmetic — compute capability 8.6, so:

 - FP8 KV cache is valid on CUDA 12.
 - FP8 weights compute through a fallback path; no native FP8 tensor cores.
@@ -248,11 +250,15 @@ docker run --rm --name vllm-qwen36-dflash \
    --gpu-memory-utilization 0.95
 ```

-## Deployment on Dankclaw (AMD Strix Halo gfx1151)
+## DFlash and DSpark on Strix Halo (gfx1151) — what killed them

-`frank-dragoon` on Dankclaw ran the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec was the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. This page is a historical record of that era (mid-August 2026); for the current deployment see [[frank-dragoon]].
-
-History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only.
+Both context-backed drafters died the same way on the iGPU: under parallel load.
+DSpark took `ROCm error: an illegal memory access` inside the shared draft
+context with two concurrent 16k prompts; DFlash corrupted multi-slot into
+premature EOS and was retired first. The timeline, the crash class, the
+"concurrency is the trigger, not size" finding and the `PARALLEL=1` workaround
+are on [[qwen3.8-spec-matrix]]; the shape they were measured on is
+[[frank-dragoon]]'s mid-August 2026 era.

 ## A/B protocol on a live llama.cpp endpoint

@@ -359,16 +365,18 @@ pay for K sequential forwards.
 Trained heads are target-specific: EAGLE/DFlash consume the TARGET's hidden
 states, so a dense-27B DFlash2 is useless for a 512-expert MoE trunk.

-**Step 4 — acceptance reality checks.** Fleet datum: a trained z-lab DFlash2
-at depth 7 accepted ~41 % on its matched target (`draft-dflash 7 → 43.5
-tok/s`); treat 40–60 % as the honest band for a matched pair. Cheapest proxy
-for `p` at temp 0 when both models are greedy and share a tokenizer: greedy
-continuation agreement on identical prompts, two API calls, no weights moved.
-llama.cpp can also synthesize the projection without any draft artifact:
+**Step 4 — acceptance reality checks.** Treat 40–60 % as the honest band for a
+matched pair; the arm that anchors the bottom of it on this fleet is the
+depth-7 `draft-dflash` row of [[frank-medium-llamacpp-dflash2]]'s spec table
+(accepted share, rate and VRAM for every depth are in that table). Cheapest
+proxy for `p` at temp 0 when both models are greedy and share a tokenizer:
+greedy continuation agreement on identical prompts, two API calls, no weights
+moved. llama.cpp can also synthesize the projection without any draft artifact:
 `--spec-synth-rates P0,P1,...` / `--spec-synth-len L`. Speculation pays IN
-PROPORTION TO HOW SLOW THE TARGET IS — the same 41 % gave 2.3x on an
-18.8 tok/s layer-split target and a loss on a 25 tok/s one; fast targets are
-usually sparse MoE, the worst case for drafting.
+PROPORTION TO HOW SLOW THE TARGET IS — that band's midpoint was worth 2.3× on
+the slow two-card layer-split target and a LOSS on the 25 tok/s target of the
+worked example above; fast targets are usually sparse MoE, the worst case for
+drafting.

 **Step 5 — check the attachment surface BEFORE the arithmetic.** Closed
 engines refuse external drafters by design: halogen-flash-server's
@@ -403,7 +411,7 @@ measurement that would settle it.

 - Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
 - DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
- The old `--swap-space` flag no longer exists in current vLLM. Use `--kv-offloading-size`.
+- The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by [[RAM CPU Offloading in VLLM]].
 - `--speculative-config` is the preferred CLI. Legacy `--spec-method`, `--spec-model`, and `--spec-tokens` flags still exist.
 - Use `rejection_sample_method` default `strict` for lossless output.

@@ -17,66 +17,59 @@ Dates: researched 2026-08. Hardware is verified on the live machine unless
 marked "verify first". Re-verified 2026-08-15/16; corrections marked below.

 **Live today (2026-10-18).** The era described below — llama.cpp on the 8060S
-iGPU behind :7770 — is retired. Dankclaw now serves
-[[frank-dragoon]] (halogen-flash-server 0.16.2, port
-7770) and [[frank-zealot]] (llama.cpp PrismML fork on
-the RTX 3090, port 7771 — drift today: that unit is `failed` since 2026-10-18
-03:40 and a hand-run 9B-MTP process answers the port), and boots `iommu=pt` since 2026-10-18
-([[dankclaw-iommu-translation-off]]). The numbers below are the measurements of
-the era they were taken in; the `ttm` ceiling and the NVMe scheduler rows were
-re-checked 2026-10-18 and still hold.
+iGPU — is retired; what runs on this box now (both endpoints, their engines and
+ports) is [[frank-dragoon]]'s and [[frank-zealot]]'s pages, and both run under their
+repo units today — no drift open as of 2026-10-19 (`frank-zealot` active since
+2026-10-18 21:06, the deployer's 2-slot reslice; `frank-dragoon` active). The OS facts
+are this page's: `iommu=pt` since 2026-10-18 ([[dankclaw-iommu-translation-off]]), and
+the `ttm` ceiling and NVMe scheduler rows below were re-checked 2026-10-18 and
+still hold. Every benchmark number below is a measurement of the era it was
+taken in.

 ## Corrections (re-verified 2026-08-15/16)

- **Backend**: that era's production was **Vulkan0**, not ROCm. Measured on this box
-  (b10433): Vulkan 25.13 t/s vs ROCm 24.33 t/s decode at 16k; at 262k ctx
-  Vulkan decodes 48% faster (5.68 vs 3.83 t/s). ROCm wins prefill (410 vs
-  302 t/s @64k). The "HIP out-performs Vulkan for decode" claim did not
-  hold on the 8060S. See [[frank-dragoon]].
+- **Backend**: that era's production was **Vulkan0**, not ROCm. On this box ROCm
+  wins prefill and Vulkan wins decode, and the decode gap widens with context:
+  the 64k A/B is [[qwen3.8-strix-optimizations]] §4, the 262k A/B is
+  [[qwen3.8-spec-matrix]] §High-Context Backend Comparison. The "HIP
+  out-performs Vulkan for decode" claim did not hold on the 8060S.
 - **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
  does not exist on this machine (checked 2026-08-15). GPU DPM is already
  `performance`, forced level `auto`. This item is N/A.
- **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). The law
-  still holds on `frank-zealot` (`CACHE_TYPE_K=q8_0`, `CACHE_TYPE_V=q8_0` —
-  `scripts/frank-zealot.env:55-56`).
+- **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). What
+  `frank-zealot` runs now is its own Current shape — this row is not it.
 - **Spec**: DFlash/DSpark are retired. That era's production was `--spec-default`
-  (ngram-mod) with draft-mtp first. See [[qwen3.8-spec-matrix]]. Today `frank-zealot`
-  is configured with speculation **off** (`SPEC_TYPE=none` — `scripts/frank-zealot.env:67`).
- **TTM**: raised to 114 GiB on 2026-08-25, set live (no reboot) and
-  persisted in `/etc/modprobe.d/ttm.conf`. The effective VRAM ceiling that
-  once held the service at ~15 GiB is gone.
+  (ngram-mod) with draft-mtp first — timeline on [[qwen3.8-spec-matrix]]; the
+  spec setting each endpoint runs today is on its own page.
+- **TTM**: the `pages_limit` raise that removed the ~15 GiB ceiling is recorded
+  once on [[Dankclaw System Specs]] (Memory).

 ## Verified Memory Picture (measured 2026-08-11)

 | Figure | Value | Source |
 | ------ | ----- | ------ |
 | OS physical RAM | 30.5 GiB | `/proc/meminfo` |
-| TTM pages_limit | 114 GiB (29,883,536 x 4KiB) — raised 2026-08-25 | `/sys/module/ttm/parameters/pages_limit` |
 | ROCm VRAM total | 96.0 GiB | `rocm-smi --showmeminfo vram` under the then firmware VGM carve |
 | ROCm VRAM used | 76.2 GiB | `rocm-smi` while serving 3x200700 |

 That 2026-08 reading is the firmware-VGM era: the BIOS carved ~96 GiB as
 "dedicated" iGPU VRAM, so the kernel saw only ~30 GiB (the same split is
-recorded in [[ling-3.0-flash-trial]] §Why Vulkan + GTT). The OS now accounts
-**121 GiB** (`MemTotal 127108080 kB`, 2026-10-18) and `rocm-smi
--showmeminfo vram` reports **2.0 GiB** (2,147,483,648 B) of dedicated VRAM —
-the carve is gone and the GPU budget is the GTT aperture, not the SMI figure
-([[Dankclaw System Specs]]).
-The TTM row above is still live: `pages_limit = 29883536` (114 GiB) and
-`/etc/modprobe.d/ttm.conf` still carries the `options ttm` line (both read
-2026-10-18).
+recorded in [[ling-3.0-flash-trial]] §Why Vulkan + GTT; the 76.2 GiB row is an
+era [[frank-dragoon]] load reading). Every live number on this box — `MemTotal`,
+the ~114 GiB GTT aperture that is the real GPU budget, the 2.0 GiB dedicated
+carve and the `ttm` page cap — belongs to [[Dankclaw System Specs]], re-read
+2026-10-18.

 This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
 limits documented in [[frank-dragoon]].

 ## 1. Self-Hosted Agentic Tools

-All of these connect to the local OpenAI-compatible endpoint
-(`http://127.0.0.1:7770/v1`). Today that port is **halogen-flash-server**, not a
-llama.cpp server, and the address must be the IPv4 literal: `curl
-http://localhost:7770/v1/models` returns nothing while
-`http://127.0.0.1:7770/v1/models` answers 200 (checked 2026-10-18 — the rootless
-podman publish is IPv4-only). None need NVIDIA.
+All of these connect to the local OpenAI-compatible endpoint on
+`:7770` (which engine owns that port is [[frank-dragoon]]'s to say), and the
+address must be the IPv4 literal: `curl http://localhost:7770/v1/models` returns
+nothing while `http://127.0.0.1:7770/v1/models` answers 200 (checked 2026-10-18
+— the rootless podman publish is IPv4-only). None need NVIDIA.

 | Tool | Use | Endpoint config | Priority |
 | ---- | --- | --------------- | -------- |
@@ -90,12 +83,13 @@ podman publish is IPv4-only). None need NVIDIA.
 | Qdrant | Vector store for RAG | Standalone, CPU-native | Low |

 The pragmatic first installs are Open WebUI, Cline, and Aider. They all work
-with the endpoint on 7770 and need no extra GPU. The 7770 endpoint today is
-halogen-flash-server, not llama.cpp.
+with the endpoint on 7770 and need no extra GPU.

 ## 2. Inference Runtime Tuning (llama.cpp on gfx1151)

-Verified against the running service and llama.cpp source.
+Verified against that era's running service and llama.cpp source; the era is
+retired ([[frank-dragoon]]), so this table is history and a method, not a live
+TODO list.

 | Change | Recommendation | Priority |
 | ------ | -------------- | -------- |
@@ -104,8 +98,8 @@ Verified against the running service and llama.cpp source.
 | Flash attention | Leave `--flash-attn auto`. It helps prompt processing, not decode. | Keep |
 | Load mode | Start with mmap (default). Enable `--mlock` only if page-fault stutter appears. | Optional |
 | Threads | Use `--threads 12 --threads-batch 16 --threads-http 1` for 16 Zen5 cores + 3 slots. Frees cores for the ROCm driver. | Medium |
-| DFlash n_max | Current 8 (block 16) is high. Try `--spec-draft-n-max 5`; it may reduce rejection cost at the same throughput. | Verify |
-| ngram layering | Measured slower than DFlash alone at 4k (16.9 vs 20.2 t/s). Not enabled. | Keep off |
+| DFlash n_max | Depth 8 (block 16) was high; a shallower draft cuts rejection cost at the same throughput. Sweep depth per the A/B protocol in [[DFlash DSpark Speculative Decoding]]. | Verify |
+| ngram layering | Layering ngram on the DFlash draft measured slower than the draft alone at 4k — figures on [[DFlash DSpark Speculative Decoding]]. | Keep off |

 ## 3. Server / OS / Hardware Settings

@@ -113,7 +107,7 @@ All require root and a reboot for the modprobe change.

 | Setting | Action | Effect | Priority |
 | ------- | ------ | ------ | -------- |
-| TTM GPU pool | `/etc/modprobe.d/ttm.conf` → `options ttm pages_limit=29883536 page_pool_size=29883536` (applied 2026-08-25, set live then persisted) | Lifts the 15 GiB cap to 114 GiB; the full 262k KV + Q8 AGENT payload fit in one slot | Done |
+| TTM GPU pool | Raised 2026-08-25; the knob, its value and its persistence file are [[Dankclaw System Specs]]'s (Memory row) | Lifted the ~15 GiB kernel-page ceiling so a full 262k KV plus a ~30 GB payload fits one slot | Done |
 | Power profile | **N/A on this machine** (2026-08-15; re-checked 2026-10-18: no `power_profile` under `/sys/class/drm/card0/device/` or `card1/device/`); GPU DPM already `performance` | none | ~~Medium~~ |
 | NVMe scheduler | Already `none` (`[none] mq-deadline` in `/sys/block/nvme0n1/queue/scheduler`, re-read 2026-10-18) | No change | Done |
 | BIOS VGM | Set Variable Graphics Memory / UMA to "High" in the EVO-X2 BIOS | Max dedicated VRAM for the GPU carve-out; verify in the live BIOS | Verify |
@@ -66,6 +66,7 @@ The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name
 | System RAM | 121 GiB (shared CPU + GPU; `free -g` total 121, `MemTotal 127108080 kB`, 2026-10-18) |
 | GPU-Visible (ROCm) | ~114 GiB GTT aperture (`mem_info_gtt_total` 122,402,963,456 B — the budget) |
 | Swap | 64 GB `/swap.img` (`swapon`, 2026-10-18; `free` reports 63 GiB) |
+| TTM page cap | `ttm.pages_limit = 29883536` (114 GiB of 4 KiB pages) and `page_pool_size=29883536`, persisted in `/etc/modprobe.d/ttm.conf`, applied live 2026-08-25 and re-read 2026-10-18 (`/sys/module/ttm/parameters/pages_limit`). The kernel default is half of RAM, so in the firmware-VGM era (~30 GiB visible) the cap was ~15 GiB and a 28 GB payload plus a 262k KV cache could not load — this is the row every other page should cite |
 | /dev/shm | 61 GiB tmpfs (half of RAM, `df /dev/shm`, 2026-10-18) |

 ### Storage
@@ -147,10 +147,10 @@ LAN/SATA/WiFi:
  they were in separate IOMMU groups (15 and 20 — re-read 2026-10-18: still 15
  and 20), and both root ports had ACS `ReqRedir+`, so this page inferred P2P
  would hairpin at the root complex. **That inference is superseded:** the live
-  engine says the opposite about the same pair — "I device: the 2 cards reach
-  each other's memory directly (peer to peer)" (`podman logs frank-archon`,
-  2026-10-15 boot). Plain P2P traffic measured 45.27 GB/s bidirectional on the
-  *old* board — DMA was never the gate, atomics were.
+  engine reports the two cards reaching each other's memory directly, which
+  [[frank-archon]] and [[Dankserver System Specs]] record. Plain P2P traffic
+  measured 45.27 GB/s bidirectional on the *old* board — DMA was never the gate,
+  atomics were.
 - Kernel 7.0.0-31-generic with `iommu=pt` then; this host runs 7.0.0-38-generic
  with `iommu=pt` today (`/proc/cmdline`, read 2026-10-18). Both were ruled out
  as causes ([above](#ruled-out-empirically-do-not-re-derive)) and this host's
@@ -163,10 +163,9 @@ launched since the swap. Until the 2-rank probe runs — or a `--split-mode row`
 boot that does *not* log `internal AllReduce init failed ... falling back to
 meta-backend butterfly` — "collectives work here" was a strong expectation, not a
 measurement. **RCCL never got that measurement**; what settled the operational
-question instead is production traffic: radiance's TP=2 wire moves cross-rank
-all-reduces over the direct P2P path and reports its own precision — "tp wire:
-LOSSY (--tp-wire wht6) at 128 KiB a message and above … Steps of at most 25 rows
-are exact" (`podman logs frank-archon`, 2026-10-15 boot).
+question instead is production traffic — radiance's TP=2 wire carries cross-rank
+all-reduces over the direct P2P path and prints its own precision, and the
+measured `wht6` banner with its exact-step bound is on [[frank-archon]].

 ## Measured matrix

@@ -233,35 +232,34 @@ landed, at x8/x8, clears the arithmetic with room to spare.

 Both cooperation modes were run on this host on 2026-09-20 — the same salted
 10-prompt battery (`scripts/bench-conversation.py`), the service stopped and
-masked between arms. Decode/prefill are geometric means over the ten rows;
-burst-4 is four concurrent copies of one prompt, reported as aggregate tokens/s.
+masked between arms. The arm table is [[frank-medium-radiance-mxfp4]]'s bench of
+record (arms R = radiance TP=2 + DFlash2 as deployed, I = the llama.cpp layer
+split that preceded it, L = llama.cpp `--split-mode tensor` against RCCL); this
+page keeps only the verdict. **TP=2 won on every axis — roughly 2.1× decode and
+3.6× at 4-way concurrency over the layer split — and replaced the layer split the
+day of measurement.**

-| Shape | Cross-device path | Decode | TTFT | Prefill | burst-4 |
-|---|---|---|---|---|---|
-| **radiance TP=2 + DFlash2 — deployed** | R4D one-shot allreduce | **95.8 t/s** | **0.21 s** | **1493.6 t/s** | **259.4 t/s (65.6/stream)** |
-| llama.cpp layer split 32:33 + DFlash2 — predecessor | none, boundary activation copy | 45.7 t/s | 0.72 s | ~530 t/s | 71.7 t/s |
-| llama.cpp `--split-mode tensor` 1:1 | RCCL allreduce | 30.5 t/s | 0.71 s | n/r | 54.3 t/s |
-
-Row 3 carries a caveat that no flag can remove: llama.cpp's tensor mode could not
-load the DFlash2 drafter (it aborts at `ggml-backend.cpp:941` — a draft weight
+Arm L carries a caveat no flag can remove: llama.cpp's tensor mode could not load
+the DFlash2 drafter (it aborts at `ggml-backend.cpp:941` — a draft weight
 pre-allocated on the non-primary device has no backend once `n_copies == 1`), so
-it decoded **without** speculation while the other two rows decoded with it. The
-comparison that decides anything is row 1 against row 2, and some of that gap is
-the 4-bit MXFP4 checkpoint rather than the parallelism.
+it decoded **without** speculation while the other arms decoded with it; and part
+of R's margin over I is the 4-bit MXFP4 checkpoint, not the parallelism.

 Two numbers elsewhere on this page are therefore not interchangeable:

 - The 272 t/s recorded 2026-09-16 is **prefill**, on a **butterfly** allreduce,
-  with RCCL off, on the old board. It was never a TP decode number.
- 26.3 t/s from the same table is that butterfly tensor's decode. Row 3's 30.5 is
-  an RCCL tensor decode a day later: same shape family, different collective path.
+  with RCCL off, on the old board. It was never a TP decode number, and its
+  paired decode figure is in "The third path" below.
+- The butterfly tensor arm and the RCCL tensor arm a day later are the same shape
+  family on different collective paths; the measured pair lives with
+  [[frank-medium-radiance-mxfp4]]'s arms.

 RCCL needs PCIe atomics on *every* participating GPU — the gate the old board
 failed. vLLM's R4D one-shot allreduce dispatches no collective at all, which is
 why TP=2 runs today rather than merely "being unblocked": the image ships TheRock
 RCCL, and TheRock carries no gfx1201 device code (`invalid device function`), so
 radiance could not have used RCCL on this host even with atomics routed. Ubuntu's
-RCCL 2.27 does carry gfx1201 kernels, and that is what row 3 linked against.
+RCCL 2.27 does carry gfx1201 kernels, and that is what arm L linked against.

 **Rule for any future multi-GPU collective experiment: stop and mask the
 production service first**, per the benchmarking policy, or run it on a host that
@@ -292,12 +290,11 @@ on the second card, so the atomics gate never comes up.

 That is what `frank-medium` ran from 2026-09-16 to 09-20
 ([[frank-medium-llamacpp-dflash2]]): one process, one model, both cards,
-262144 context. Measured against tensor split
-on this host: 1991 t/s prefill / 18.8 tok/s decode layer-split, against 272 /
-26.3 tensor (butterfly fallback, no RCCL). **That tensor number is not a
-prediction for an RCCL-backed build** — it is what tensor split costs when the
-allreduce degrades to a meta-backend butterfly, and it is the number the
-board-swap path exists to retire.
+262144 context. Its measured layer-split rates are that page's; this host's
+tensor arm under the butterfly fallback measured 272 t/s prefill / 26.3 tok/s
+decode. **That tensor number is not a prediction for an RCCL-backed build** — it
+is what tensor split costs when the allreduce degrades to a meta-backend
+butterfly, and it is the number the board-swap path exists to retire.

 The ranking therefore changed twice. On 2026-09-16: software already gets both
 cards onto one model, so the hardware fix stopped being a capability question
@@ -57,7 +57,10 @@ Key properties, per the project README and docs:

 ## NPU Hardware on Dankclaw

-Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU rated at 50 TOPS (vendor spec: "NPU TOPS — Up to 50 TOPS", [amd.com product page](https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html)). It sits beside the Radeon 8060S iGPU and shares the same unified memory pool — 30 GiB exposed to the OS in the 2026-08 reading below; the box presents **121 GiB** today (`free -g` total 121, read 2026-10-18).
+Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU
+with 50 TOPS of compute and, crucially, **no dedicated VRAM** - it shares system memory
+via the IOMMU SVA mapping. The budget this page reasons about - 29 GiB usable at
+4800 MT/s LPDDR5X - is the 2026-08 host state; the live figure is [[Dankclaw System Specs]].

 Live verification on Dankclaw, 2026-08-16:

@@ -89,10 +92,9 @@ Budget math for this fleet:
 | Dankserver (Ryzen 7 7700X) | 62 GB | no NPU |
 | FLM bench box (Kraken Point) | 32 or 96 GB | ~16 or ~48 GB |

-2026-10-18 re-read: Dankclaw's OS now reports **121 GiB** (`free -g`) and
-Dankserver **246 GiB** (`free -g`), so the half-of-RAM rule would price the NPU
-budget at ~60 GiB and the 7700X box still has no NPU. The 30 GiB / 62 GB rows
-are the 2026-08-16 state; the table is moot while no NPU endpoint is deployed.
+The half-of-RAM gate scales with host RAM - live figures in [[Dankclaw System Specs]]
+and [[Dankserver System Specs]]. The live gate on Dankclaw is still `flm validate`
+(it owns the current number).

 Weight sizes (approximate, Q4-class):

@@ -159,7 +161,10 @@ Qwen3.6-35B-A3B (Q4_KS), FLM v0.9.45, Kraken Point **96 GB DRAM**:
 | Decode t/s | 13.65 | 13.41 | 13.09 | 12.51 | 11.24 | 9.51 |
 | Prefill t/s | 78.98 | 118.04 | 156.43 | 197.93 | 218.84 | 221.96 |

-Context: the current PP split (head + 3090 worker) decodes at 14-18 t/s for the 27B dense model. A 35B-A3B at ~10-14 t/s on an NPU with 48 GB of RAM budget is not a speed win over the existing topology.
+- PP and NPU together cost 2.2 GiB + NPU model + ~2 GiB of headroom. The then-current
+  PP split (head + 3090 worker, 2026-08 era - see [[frank-redo-2026-08-17]]) decoded at
+  14-18 t/s for the 27B dense model, so an NPU offload would free the head or upgrade
+  the worker's model.

 The vendor also claims "over 10x more power-efficient" versus GPU. That is a marketing claim. We have not measured it.

@@ -219,7 +224,12 @@ Notes:

 ## Promotion (2026-08-16)

-`frank-zealot` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
+`frank-zealot` then ran the NPU endpoint on Dankclaw (2026-08; the name was reused
+twice after this window - see the banner - and the NPU unit today is `frank-npu`,
+masked). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`),
+wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`.
+`LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap
+`FLM_MODEL` in the env to switch.

 ### Boot policy (2026-08-17)

@@ -256,11 +266,13 @@ The 4-stream wall times form a near-perfect 10.7 s ladder. Each stream self-repo
 Consequence: the NPU endpoint is a single lane. It suits low-frequency background work (triage, summarization, agent side-tasks). It does not suit interactive chat for several simultaneous users. The 27B llama.cpp service, by contrast, runs PARALLEL=2 with true concurrent slots.

 ## Test Runbook
-
 This runbook targets a retired endpoint; keep it as the procedure if the NPU is
-ever revived. Do not run large model loads while the production service holds
-most of the unified pool — 30 GiB exposed to the OS in the 2026-08 reading,
-**121 GiB** today (`free -g`, read 2026-10-18). Check free RAM first, or stop
+ever revived.
+
+**Host RAM availability.** The gate is half of system RAM by default - a relic of the
+install: the endpoint ran on a machine that presented 30 GiB of system RAM; on today's
+box that ceiling is far larger (live figure: [[Dankclaw System Specs]]). Check free RAM
+first, or stop
 the service the way the bench policy requires (`sudo systemctl stop
 frank-dragoon && sudo systemctl mask frank-dragoon` — self-healing would
 otherwise restart it). The NPU shares the unified pool with everything else on
@@ -119,5 +119,6 @@ re-download.

 - [[Llama Cpp Runtimes]] — which fork ships which custom types
 - [[Llama Cpp Code Diff]] — the enum tables and code-level differences
- [[frank-dragoon]] — the ftype-115 lock-in in production
+* [[kingjones-q8agent-rocmfpx]] — the era that served the ftype-115 file, and
+  [[frank-dragoon]] for what runs there now
 - [[Llama Cpp Serving Sizing]] — pricing what a payload costs once it loads
@@ -56,7 +56,10 @@ Lower values leave more GPU memory for KV cache. This reduces swap frequency.

 ### Swap Space / KV Offload

-Recent vLLM releases removed the `--swap-space` engine argument — `CacheConfig` on `main` has no `swap_space` field ([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/config/cache.py), read 2026-10-18). Use `--kv-offloading-size` (`kv_offloading_size`, same file) to move KV cache to CPU RAM. In Podman, mount the host cache read-write for KV offload writes. The table below shows historical `--swap-space` guidance as a sizing reference.
+Current vLLM has no `--swap-space` engine argument, and the CPU KV-offload knob pair
+is owned by [[RAM CPU Offloading in VLLM]] (that page carries the upstream
+`CacheConfig` verification). In Podman, mount the host cache read-write for KV offload
+writes. The table below shows historical `--swap-space` guidance as a sizing reference.

 | System RAM | Recommended Swap |
 |------------|------------------|
@@ -20,8 +20,8 @@ Sweep decode across context depth. **Flat** ⇒ attention over history is not
 the cost; the problem is per-token expert movement and KV/attention tuning is
 ruled out entirely. A genuinely context-bound server shows a clear downward
 slope. Record cold vs warm separately — a small gap means the cache is
-working. Measured on the fp8 expert-offload shape: 12.11 tok/s FLAT from 304
-to 16,095 tokens.
+working. The canonical flat run (fp8 expert-offload shape, a few hundred to
+sixteen thousand tokens of context) is in [[Tcclaviger Fork Ops]]'s profile.

 ## 2. Duty-cycle arithmetic (the "GPU not fully utilized" answer)

@@ -161,7 +161,7 @@ of the same RAM, not extra memory.
 ## 7. IBS model-access attribution (AMD CPUs)

 Real CPU access patterns for mmap'd weights, live, no stop (established on
-frank-archon, 2026-09-26, ~2.7M attributed samples in 75 s). Reference:
+[[frank-archon]], 2026-09-26, ~2.7M attributed samples in 75 s). Reference:
 `scripts/frank-archon-expert-activity.py` (self-contained dump + sample +
 analyze; `--analyze-only`).

@@ -23,8 +23,9 @@ guesswork.
   each near the target; record `nvidia-smi` used/total plus the boot log's
   `n_slots`/`n_ctx_slot`: shape A = current/cheap, shape B = new demand.
   `non-KV = used − (tokens × KiB/token)`; the two rows separate
-   weights/buffers/graphs from the new demand (a 918 MB projector bills
-   ~1.4 GiB).
+   weights/buffers/graphs from the new demand; a projector is billed after the
+   fit at roughly 1.5x its file size ([[frank-zealot]] Contract has that
+   measurement).
 3. **Solve the band:** `KV_tokens ≤ (card − non-KV − headroom_low) /
   KiB/token`, round the pool DOWN to a multiple of `PARALLEL × 4096` so every
   slot gets an equal integer window. Boot and measure; if outside the band,
@@ -54,8 +55,8 @@ First decide which kind of memory is in question:
 | device | kind | read it with |
 |---|---|---|
 | RTX 3090 (Dankclaw) | discrete GDDR6X | `nvidia-smi --query-gpu=memory.total,memory.used,memory.free` |
-| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` (114 GiB) vs `mem_info_vram_*` (2 GiB carve-out) |
-| R9700s (Dankserver) | discrete | `rocm-smi --showmeminfo vram` → 34,208,743,424 B = 32,624 MiB (31.86 GiB) per card, read 2026-10-18 — **not** the 64 GiB "SVM-backed" figure this row once carried ([[Dankserver System Specs]]) |
+| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` vs `mem_info_vram_*`; the aperture and carve-out values are [[Dankclaw System Specs]]'s |
+| R9700s (Dankserver) | discrete | `rocm-smi --showmeminfo vram`; the per-card byte/MiB value (and the stale "64 GiB SVM-backed" figure it replaced) is [[Dankserver System Specs]]'s |

 Per-process truth on NVIDIA:
 `nvidia-smi --query-compute-apps=pid,process_name,used_memory`. Never add the
@@ -63,9 +64,10 @@ iGPU's GTT to host RAM as a separate pool; `free -g` is the arbiter for
 co-tenants.

 - **Engine ledger first.** Halogen prints weights pinned (+ remaining 2 MiB
-  contiguous blocks — the honest fragmentation signal: 240 blocks/12.3 GiB
-  after reboot vs 475/14.7 GiB before means the model did not come back into
-  the same hole), device budget, KV pool/working split, "left for everything
+  contiguous blocks — the honest fragmentation signal: a block census that drops
+  across a restart means the model did not come back into the same hole; the
+  before/after census this was learned on is that era's, on [[frank-dragoon]]),
+  device budget, KV pool/working split, "left for everything
  else". llama.cpp prints them at `llama-model.cpp:1827` (per-backend model buffer)
  and `llama-context.cpp:692` (compute buffer) on the era's build; on today's
  `master` the same `LLAMA_LOG_INFO` lines sit at `llama-model.cpp:2009`
@@ -83,13 +85,13 @@ co-tenants.
  graph/compute pool).
 - **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
  "move layers to the card" converts reclaimable cache into device memory and
-  frees nothing for co-tenants. A served model's footprint ≠ weight size
-(a retired frank-zealot shape: 22.6 of 24.1 GiB for an 11.29 GiB model; the card
-today reads 24,576 MiB = 24.0 GiB total, `nvidia-smi --query-gpu=memory.total`,
-2026-10-18, and the deployed payload is the PQ2_0 pair
-[[frank-zealot]]). Do the capacity
-  subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
-  28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
+  frees nothing for co-tenants. A served model's footprint is not its weight
+  size: the MiMo Q5_K_M era booted a 6.40 GiB payload to **20,530 MiB used** of the
+  3090's addressable 24,576 — the Eras row and its addressable ≠ advertised law are on
+  [[frank-zealot]]. What the card reports used is weights + KV + compute buffers +
+  projector, never weights alone. Do
+  the capacity subtraction before designing, on the card number the owner page
+  quotes, not the advertised one.
 - **Context lengths: derive, never remember.** `llama-bench` computes
  `n_ctx = n_prompt + n_gen + n_depth` (defaults 512/128/0, type_k/v F16,
  flash_attn AUTO, split_mode LAYER). `-p 512,16384 -n 64` = contexts
@@ -112,10 +114,10 @@ them from the binary (`llama-server --help`) before sizing on them.
 **State law** (compute before trusting any cap): a state is the serialized KV
 of the whole sequence + constant recurrent state + the draft context's KV:
 `per_token = 2 × n_full_attn_layers × n_kv_heads × head_dim × bytes (K and V) +
-draft_per_token`. Measured on frank-medium (qwen3.5 hybrid, f16, MTP draft):
-16×4×256×2×2 = 64 KiB/token + 4 KiB draft = **68 KiB/token**; a full
-196,608-token slot state ≈ 12.8 GiB. Cross-check against the journal's
-`prompt state size` lines — the two must agree.
+draft_per_token`. The worked qwen35-hybrid instance of that formula (full layer
+count, f16 KV, MTP draft) and the slot-size table it prices are on
+[[frank-medium-llamacpp-dflash2]]; cross-check any such arithmetic against the
+journal's `prompt state size` lines — the two must agree.

 **The journal is the instrument:** `exceeds cache size limit … skipping` =
 cap binding in production; `making room … removing oldest` / `cache size
@@ -162,13 +164,13 @@ Only the charlie12345/ROCmFPX build-qwen38 (PR #27742 qwen4exp merged) takes
 it; CIRU, upstream and stock builds do not. Validate live: load with
 `--flash-attn on`, health 200, a real completion finishing `"stop"`, journal
 clean of `GGML_ASSERT`/`cudaMalloc`/`missing tensor`, and flash-attn engaged
-in the timings. Context cap for full-model residency on that shape stayed
-32,768.
+in the timings. The context cap that build reached for full-model residency is
+part of that era's record on [[frank-dragoon]].

 ## Related

 - [[GGUF Compatibility]] — what a payload is before it is priced
- [[frank-zealot]] — the two-boot priced shape of the retired MiMo era (live shape today: [[frank-zealot]])
- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
+- [[frank-zealot]] — the endpoint whose Eras rows are the two-boot priced shapes this method produced
+- [[frank-medium-llamacpp-dflash2]] — the state-law worked example behind section 3
 - [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
 - [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side
@@ -40,9 +40,9 @@ curl -s <health>                                      # then a REAL completion
 ```

 Deploy caveat: "healthy after Ns" = systemd ACTIVE, not model LOADED —
-confirm with a real `/v1/chat/completions`. RTX 3090 (Ampere) has no
-FP8/FP4 tensor cores: FP8 weights run via upcast but must still fit; choose
-quant by size fit, then accept the compute path the card supports. Before
+confirm with a real `/v1/chat/completions`. On Ampere an FP8 file buys memory
+and no compute ([[Quantization by Hardware Architecture]]): choose the quant by
+size fit, then accept the compute path the card supports. Before
 choosing a smaller quant, state WHICH of the two you are buying — headroom,
 or a design change (a smaller quant does not change what the runtime can
 express; per-tensor cross-device placement stays refused under
@@ -60,8 +60,8 @@ express; per-tensor cross-device placement stays refused under
 ## 3. Fetch and verify (HF payloads)

 **Inventory by QUANT NAME, never by extension** — large quants ship as
-`.partNofM` fragments (Q4_K_M = 37.00+37.00+36.97 GiB), so an
-`endswith(".gguf")` filter makes Q4_K_S look nonexistent. Check both
+`.partNofM` fragments, so an `endswith(".gguf")` filter makes a whole rung look
+nonexistent (AGENTS.md carries the worked fragment ledger). Check both
 mradermacher repos (`…-i1-GGUF` imatrix, `…-GGUF` plain) and the model's own
 repo; the tree API's `blobs=true` gives `lfs.oid` per file. Gated mirrors
 return 403 even with a valid token until the account accepts the gate —
@@ -150,7 +150,7 @@ recoverable, drift is not.
   entry in `serverAppConfigs-docs/log.md`.
 2. `~/.omp/agent/models.yml`: `contextWindow` must equal the REAL per-slot
   context — llama.cpp logs `n_ctx_slot` per prompt (`--parallel N` divides
-   the pool; e.g. 262144 native / 2 slots = 131072 per slot); vLLM reports
+  the pool; e.g. a 524288 pool over 2 slots = 262144 per slot); vLLM reports
   `max-model-len` and its concurrency line. Overselling causes silent
   client-side overflow or a discovered-wall 400. Also `maxTokens` vs the
   engine's cap, `input` capabilities (text/image), and
@@ -252,12 +252,10 @@ small/fast + image roles at the new endpoint; move the vision role WITH the
 vision tower (one owner per capability). `shared/homepage/services.json`: a
 card per endpoint, probed from the monitor's own host. Prometheus: only add
 a job if the runtime actually exposes Prometheus text — and re-measure rather
-than repeat, because that answer moves. This bullet once said halogen serves
-`/health`, `/cache`, `/v1/*` "and nothing else"; measured 2026-10-18 on
-Dankclaw, `curl -s http://127.0.0.1:7770/metrics` returns Prometheus text
-(`# HELP llamacpp:prompt_tokens_total …`, content-type
-`text/plain; version=0.0.4`) while an invented path 404s — so the surface has
-grown and a scrape target is expressible again.
+than repeat, because that answer moves: halogen's HTTP surface has grown twice
+since this page was written, and the endpoint list that matters for a scrape
+target — including whether `/metrics` answers — is [[frank-dragoon]]'s wire
+contract.

 **4. Cutover, then verify by driving it.** `deploy.sh <target> --service
 <large> --service <small>` (order matters); per endpoint: `/health`,
@@ -265,10 +263,10 @@ grown and a scrape target is expressible again.
 `reasoning_content`, a `tools` request returns `tool_calls`, an image
 request is answered (or refused with a named reason), the runtime's own
 speculation counters move, and the unit is `enabled` and `active`. All three LLM
-endpoints are **root** units today (`User=dank` + `WantedBy=multi-user.target` —
-`scripts/frank-archon.service:30,71`, `scripts/frank-dragoon.service:8,31`,
-`scripts/frank-zealot.service:8,26`), so linger is not what gates them; linger
-matters for rootless quadlets. Read the engine's boot ledger for a silent clamp. Finally spend one
+endpoints are **root** units today (`scripts/frank-archon.service`,
+`scripts/frank-dragoon.service`, `scripts/frank-zealot.service`), so linger is
+not what gates them; linger matters for rootless quadlets. Read the engine's boot
+ledger for a silent clamp. Finally spend one
 real request through the harness per changed role — routing labels lie; the
 servers' own journals do not.

@@ -285,4 +283,5 @@ predecessor "stopped" (the port race above).
 - [[Llama Cpp Serving Sizing]] — pricing the shape once it loads
 - [[RDNA4 MoE Expert Offload VRAM Budget]] — vLLM-side fit arithmetic
 - [[LLM Bench Windows and Measurement]] — measuring the new deployment
- [[frank-archon]] — the retired tcclaviger vLLM endpoint (2026-09-29 → 10-05); the live Dankserver endpoint is [[frank-archon]]
+* [[frank-archon]] — the live Dankserver endpoint; the tcclaviger vLLM era
+  (2026-09-29 → 10-05) is a row in its Eras table
@@ -65,13 +65,11 @@ LPDDR5X on the Strix Halo — 128 GB LPDDR5x-8000 on a 256-bit bus,
 streaming loses by construction and no tuning rescues it; doubling the link
 only doubles a losing number.**

-Worked example (Dankclaw 2026-09-16): Gen4 x4 measured H2D **6.72 / D2H 6.60
-GB/s** (85 % of spec) — pinned H2D ≈ async, pageable within 1 %, i.e.
-link-limited not staging-limited. A 3 MiB slab (the expert-streaming
-granularity) = 474 µs. A3B MoE at Q4_K_M (~1.70 GiB active/token) → 272
-ms/token ⇒ **3.7 tok/s** (4.6 with 20 % resident) vs measured **20.46 tok/s**
-with experts computed on the CPU, 25–26.5 on the iGPU. The streaming design
-loses 3–5x.
+Worked example: the Dankclaw Gen4 x4 run of 2026-09-16 — measured H2D/D2H
+rates, the 3 MiB slab cost, and the streaming-vs-compute comparison it decided
+— is recorded on [[x4-pair-experiment]] §The link, measured. Read it once and
+apply the rule above: on that link the streaming design lost by 3–5x, and the
+x8 ceiling would only have halved the loss.

 ## 3. Measure the link: two probes, same mechanics

@@ -150,9 +148,9 @@ int main(int argc, char **argv) {
  (the usual "pageable costs 2x" rule does not apply at this link).
 - Small-transfer penalty (2.345 MiB: ~106 µs vs ~78 µs ideal, ~21 %) is
  per-transfer overhead — recoverable only by BATCHING transfers, not by a
-  faster link. (Dankserver 2026-09-29: bulk 22.7 GB/s at the 2.344 MiB slab =
-  91 % of Gen5 x8, while the engine's effective read path ran ~9.5 GB/s — the
-  gap is the fork's gather software path, not the wire.)
+  faster link. The engine-vs-wire version of that reading on Dankserver
+  (bulk slab rate against the engine's effective gather rate, same
+  granularity) is [[frank-archon]]'s tcclaviger-era record.
 - Both GPUs symmetric ⇒ no cross-device rebalance is available; measure both.
 - Don't quote a derived line from inside a probe without hand-checking it —
  a printf format/argument mismatch in one of these printed a nonsense
@@ -186,7 +184,8 @@ sudo lspci -vv -s <bridge> | grep AtomicOpsCap                   # want Routing+
 `Routing-` is a read-only hardware capability. Typical AM5 layout: one x16
 slot on CPU root ports (atomics fine), the rest behind the chipset (usually
 not). Bandwidth for TP is small anyway: ~10 MB per token-step for a 27B at
-batch 4 ≈ 1.4 ms at Gen4 x4 — compute-bound, not interconnect-bound.
+batch 4 is a millisecond-scale figure at x4 — compute-bound, not
+interconnect-bound (full derivation: [[Dual-R9700-Cooperation]] §Bandwidth).

 **Probe matrix (bare 2-rank `torch.distributed`, one process per GPU, 25 s PG
 timeout) — one probe answers several design questions:**
@@ -215,13 +214,12 @@ reporting an outage (`gitea` vs `gitea-server`).

 1. CPU-attached M.2 → PCIe x4 riser (~$30) — puts the card on the CPU root
   complex; x4 suffices for TP.
-2. Board whose CPU x16 bifurcates x8/x8 (done 2026-09-20 on dankserver — both
-   R9700s on root ports `00:01.1`/`00:01.3`, every bridge `Routing+`, both
-   endpoints `ReqEn+`). Re-checked 2026-10-18 without root: both root ports
-   report `current_link_width=8` at `32.0 GT/s`
+2. Board whose CPU x16 bifurcates x8/x8 (done 2026-09-20 on dankserver — the
+   per-device chain census, including both endpoints reading `ReqEn+`, is
+   [[Dual-R9700-Cooperation]]'s). Re-checked 2026-10-18 without root: both root
+   ports report `current_link_width=8` at `32.0 GT/s`
   (`/sys/bus/pci/devices/0000:00:01.{1,3}/current_link_width`), and the live
-   engine states direct P2P between the cards
-   ([[Dual-R9700-Cooperation]]).
+   engine states direct P2P between the cards.
 3. llama.cpp `--split-mode layer` — no collectives at all ([[Llama Cpp Multi-Device Splits]]); costs the GGUF conversion plus paged attention /
   continuous batching / spec decode.
 4. Independent TP=1 replicas + routing proxy — cards never share one model.
@@ -43,8 +43,16 @@ state, verified against the live journal and output smoke tests
  the mismatch (`resolve_fused_ops: Flash Attention enabled`) via the
  GGML_CUDA_FA_ALL_QUANTS rebuild — see "FA-all-quants rebuild" below.
 - Pool `262144`, `PARALLEL=1`, live `n_ctx_slot = 262144`.
- MTP: standalone draft head, acceptance 0.83 then 0.75, mean len
-  4.0-4.3, per-position ~0.86/0.71/0.71/0.71.
+* MTP: standalone draft head, acceptance 0.83 then 0.75, mean len 4.0-4.3,
+  per-position ~0.86/0.71/0.71/0.71 — stronger at every position than the old bundled
+  head, which measured 0.739/0.554/0.427/0.340.
+- Spec chain as deployed: `--spec-draft-n-max 4` on the standalone draft, with
+  the `--spec-default` ngram-mod fallback at n_match 24 / n_min 48 / n_max 64
+  (the preset that survived the DSpark/DFlash retirements —
+  [[qwen3.8-spec-matrix]]).
+- Batch 8192 / ubatch 256, `--direct-io`, `--fit off`, `--gpu-layers all`. The
+  pi `models.json` declared 262144 (`contextWindow = maxTokens`, the house
+  convention of that era).
 - Head GPU side: **ROCm0** (no `--device` flag; HIP registers before
  Vulkan).
 - Worker steady state: 23854/24576 MiB used, ~273 MiB free (tight but
@@ -215,8 +223,11 @@ head env ([[frank-dragoon]]).
   faster prefill, ~3% slower decode (measured on this box).
 3. **Pin `--tensor-split 1,1`.** The default split is proportional to
   *reported free memory*: the Strix reports up to 96 GiB (GTT) vs the
-   3090's 24 GB, so the default lands ~20/80 on the slow device (verified
-   with a 4B smoke test). 1,1 = 50/50 = balanced per-layer latency.
+   3090's 24 GB, so the default lands ~20/80 on the slow device. 1,1 = 50/50 =
+   balanced per-layer latency. The 2026-08-15 4B smoke test that proved it
+   (production untouched): at a 0.2/0.8 split only 4 of 36 layers reached the
+   3090, and at a real split worker VRAM rose ~819 MiB with 95 t/s prefill /
+   58.8 t/s decode on the tiny model; link RTT ~0.3–0.5 ms.
 4. **`-dio` on the head.** Large model + RPC hangs at load without direct
   I/O.
 5. **RPC is unauthenticated and unencrypted.** LAN only. The worker binds
@@ -147,12 +147,14 @@ cut to GPTQ-w4 for the 3090 is a bench experiment that needs no new
 hardware; EXL3 (`convert.py` on the 3090, exllama serve) is the
 quality-per-bit extreme of the same menu.
 Caveats before any swap: the serving contract (OpenAI server,
-`reasoning_content`, vision mmproj, compaction-hub window
-200,704 ×2 slots) must be met by the challenger runtime, and the
-24 GB card runs ONE server — benching follows the stop-and-mask law.
+`reasoning_content`, vision mmproj, and the compaction-hub window the endpoint
+declares in its Current shape — [[frank-zealot]]) must be met by the challenger
+runtime, and the 24 GB card runs ONE server — benching follows the stop-and-mask
+law.

 #### Worked example — the frank-archon w4dn8 recipe ported to the 3090
-(audit 2026-10-08, target `Qwen3.6-35B-A3B`)
+(audit 2026-10-08, target `Qwen3.6-35B-A3B`; sized at that date's hub shape —
+the window history is [[frank-zealot]]'s Eras table)
 **The methodology ports; the encoding must be translated.** llama.cpp
 decodes no GPTQ — the recipe re-expresses on GGUF-native terms:
 imatrix on our corpora (replaces the radiance grams; runs the 65 GiB
@@ -253,8 +255,6 @@ top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
 - Repo: `scripts/convert-swift15-w4dn8.sh`,
  `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`,
  `scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`,
-  `scripts/build-kairic-edge-gfx1151.sh`, [[frank-dragoon]],
-  [[frank-dragoon]], [[qwen3.8-strix-optimizations]],
-  [[frank-archon]],
-  [[frank-zealot]],
-  [[frank-dragoon]], [[GGUF Compatibility]].
+  `scripts/build-kairic-edge-gfx1151.sh`; endpoints [[frank-archon]],
+  [[frank-dragoon]], [[frank-zealot]]; method [[qwen3.8-strix-optimizations]]
+  and [[GGUF Compatibility]].
@@ -108,9 +108,10 @@ Use ROCmFP4 only with the ROCmFPX fork of llama.cpp. Upstream llama.cpp cannot r

 ## Choices for This Project

-### Dankserver (RTX 3090, 24 GB, Ampere)
+### The RTX 3090 (24 GB, Ampere) — frank-zealot's card

-The RTX 3090 has no native FP8 or FP4 tensor cores. Use:
+The 3090 has no native FP8 or FP4 tensor cores. It has sat in Dankclaw since
+2026-09-16; [[frank-zealot]]'s Eras table records the move off Dankserver. Use:

 | Ranking | Format | Why |
 |---------|--------|-----|
@@ -121,13 +122,17 @@ The RTX 3090 has no native FP8 or FP4 tensor cores. Use:

 ### Dankclaw (Strix Halo gfx1151, RDNA 3.5)

-RDNA 3.5 has no native FP8 tensor cores. The APU shares memory. The active Ling-3.0-flash service runs vLLM-ROCm with the AMD Quark INT4 (W4A16) format (the jcbtc CIRU Strix-native bundle). INT4 weights keep the 124B model inside the 96 GiB frame buffer with a full-precision KV cache.
+RDNA 3.5 has no native FP8 tensor cores and the APU shares its memory, so the
+GPU budget is the GTT aperture and every live figure on the box is
+[[Dankclaw System Specs]]. What serves here today is [[frank-dragoon]]: a
+vendor `.hgn` checkpoint, a format this page cannot rank because no public
+quantizer exists for it ([[Quantization Production]]).

 | Ranking | Format | Why |
 |---------|--------|-----|
-| 1 | INT4 W4A16 (Quark/vLLM) | active; fits 96 GiB; native ROCm kernels; concurrency |
-| 2 | MXFP4 (Quark) | same-bit, higher accuracy; beta on gfx1151 (A/B experiment) |
-| 3 | GGUF Q4_K_M / Q8_0 | works with upstream llama.cpp only |
+| 1 | `.hgn` (deployed) | the halogen engine's own format; production door is closed |
+| 2 | GGUF K/I quants, K_P blends | the only self-produced option on this card ([[Quantization Production]]) |
+| 3 | INT4 W4A16 (Quark) / MXFP4 (Quark) | the retired Ling-3.0-flash vLLM era, measured under the firmware-VGM frame buffer: [[ling-3.0-flash-trial]] |
 | 4 | FP8 (vLLM) | software emulation; memory savings only; no compute win |

 ## See Also
@@ -11,9 +11,19 @@ timestamp: 2026-08-28

 ## Overview

-vLLM uses CPU RAM and swap space to extend GPU memory capacity. This allows larger models to run on GPUs with limited VRAM. vLLM does not offload model weights to CPU. Instead, vLLM moves KV cache data between GPU and CPU.
+Stock vLLM offloads **KV pages**, not weights: the scheduler moves KV cache between GPU
+and CPU. That is the upstream behaviour and it is NOT the whole story on our hosts — the
+tcclaviger/R4D fork ran a real **weight** offload (`--enable-expert-offload`, tiered
+expert placement, a pinned host pool per rank), which is how a 147 GiB MoE served from a
+246 GiB box: [[RDNA4 MoE Expert Offload VRAM Budget]] and the tcclaviger era row of
+[[frank-archon]]. First-hand in-repo evidence outranks a generic upstream claim about our
+own hosts ([[Model Swap and Rollback]] §10).

-> **Current vLLM note:** Recent vLLM releases removed the `--swap-space` engine argument. CPU KV-cache offloading now uses `--kv-offloading-size`. See the parameters below for details.
+> **Verified 2026-10-19 against `vllm/config/cache.py` on `main`:** there is no
+> `swap_space` field — `--swap-space` is gone from current engine args. CPU KV offload is
+> `--kv-offloading-size`, and it activates **only** together with `--kv-offloading-backend`
+> (default `native`); both fields live in that one file. This page is the fleet's owner of
+> that knob contract — other pages link here instead of restating it.

 ## How CPU Offloading Works

@@ -42,9 +52,12 @@ This parameter controls the fraction of GPU memory used for KV cache.

 Lower values leave more GPU memory for model weights. Higher values allow more concurrent requests but increase swap frequency.

-### swap_space (removed) / kv-offloading-size (current)
+### swap_space (removed) / kv-offloading-size + kv-offloading-backend (current)

-The `--swap-space` argument was removed from current vLLM engine args. Use `--kv-offloading-size` instead. The table below describes the historical `--swap-space` values for reference.
+`--swap-space` no longer exists in current engine args (verified above); the live pair is
+`--kv-offloading-size` with `--kv-offloading-backend`, and offload stays off until the size
+is set. The table below is the historical `--swap-space` sizing guidance, kept as a sizing
+reference only.

 | Value | Description |
 |-------|-------------|
@@ -103,6 +116,9 @@ This parameter sets the KV cache data type.

 ## Memory Configuration Examples

+All three examples carry the retired `--swap-space` syntax and are kept as sizing
+references; the live knob pair is named above.
+
 ### Small Model on Single GPU

 ```
@@ -157,7 +173,7 @@ WARNING scheduler.py: Sequence group 0 is preempted by PreemptionMode.RECOMPUTE
 If you see frequent preemptions:

 1. Increase `gpu_memory_utilization`
-2. Increase `swap_space`
+2. Raise the KV offload budget (`--kv-offloading-size`, with `--kv-offloading-backend`)
 3. Decrease `max_num_seqs`
 4. Increase `tensor_parallel_size`
 5. Increase `max_model_len` to match actual usage
@@ -169,7 +185,8 @@ If you see frequent preemptions:
 - Reduce `max_num_seqs` to limit concurrent requests
 - Disable CUDA graphs with `--enforce-eager` to save memory
 - Set `compilation_config.cudagraph_capture_sizes` to smaller values
- Use `--kv-cache-memory` to skip memory profiling
+- Set `--kv-cache-memory-bytes` to pin the KV budget instead of letting the profiler guess
+  (the flag our own `scripts/patches/gpu_worker-kv-cap.patch` reads)
 - Limit multi-modal inputs with `limit_mm_per_prompt`

 ## Environment Variables
@@ -8,8 +8,8 @@ timestamp: 2026-10-05T00:00:00-07:00

 # RDNA4 Container Serving

-Bring-up path for Dankserver (2x Radeon AI PRO R9700, gfx1201, 31.86 GiB each =
-**63.7 GiB real VRAM**, 246 GiB RAM, 7700X) with
+Bring-up path for Dankserver (2x Radeon AI PRO R9700, gfx1201 — card count,
+per-card VRAM and host RAM are [[Dankserver System Specs]]'s) with
 `docker.io/tcclaviger/vllm` (vLLM 0.29.0.dev, GLM/Qwen-specialised RDNA4 build
 with native MoE quant kernels and a first-class expert offload planner). The
 tuning history and current deployed shape live in
@@ -155,7 +155,9 @@ podman run -d --name <ctr> --ipc=host --ulimit memlock=-1:-1 \

 ## 5. Memory budgeting: the levers are not where they look

-The planner builds `budget = total × util`, subtracts a KV reserve, and gives
+Every figure in this section is a tcclaviger-era boot on the port that is
+[[frank-archon]]'s today; the laws they teach are general. The planner builds
+`budget = total × util`, subtracts a KV reserve, and gives
 the REMAINDER to the expert arena. Consequences, all measured:

 - **`--gpu-memory-utilization` grows the ARENA, not the KV pool** (KV stayed
@@ -163,11 +165,14 @@ the REMAINDER to the expert arena. Consequences, all measured:
 - **Lowering `--max-model-len` is the WRONG lever and backfires:** a smaller
  context shrinks the reserve, handing the space to the arena — measured 65536
  → 0.64 GiB KV, 32768 → 0.34 GiB KV.
- **Hybrid-attention undercount:** the planner's per-token KV figure (9190
-  B/token for Qwen4Exp) misses the large fixed per-sequence Mamba state
+- **Hybrid-attention undercount:** the planner's per-token KV figure for
+  Qwen4Exp is an undercount — the printed figure and the true one are both on
+  [[frank-archon]] — because it misses the large fixed per-sequence Mamba state
  (~2x), so the reserve is always short and the arena silently covers it.
-  Fix by pinning the pool: `--kv-cache-memory-bytes <BYTES>` (2 GiB →
-  113,595 tokens / 3.47x concurrency at 32k). For long context,
+  Fix by pinning the pool: `--kv-cache-memory-bytes <BYTES>` (era reading: a
+  2 GiB pin = 113,595 tokens ≈ 3.47x concurrency at 32k — that pair is this
+  page's record of the tcclaviger boots, not [[frank-archon]]'s). For long
+  context,
  `--kv-offloading-size` (GiB) spills KV to host.
 - **`--expert-offload-mem` caps the HOST side, not the VRAM arena**
  (`cap = int(x * GIB)` against `host_memory_available()`). Do not reach for
@@ -201,18 +206,21 @@ host RAM = `host-backed set + layers still loading`, not the whole model. At
 runtime `apply_hot_cold` sends arena-resident experts down a hot GEMM and the
 rest straight over PCIe via UVA views — no staging copy in the critical path.

-**Emulation check:** a dequantize-to-compute path doubles real host RAM (118
-GiB offload request → ~238 GiB) and is invisible in the weight count. Read the
-log for `Using 'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit".
+**Emulation check:** a dequantize-to-compute path doubles real host RAM — the
+measured pair (offload request vs host footprint) is on [[frank-archon]] — and
+the doubling is invisible in the weight count. Read the log for `Using
+'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit".

 ## 6. Performance profile and honest measurement

-Decode on this pair is **PCIe-gather-bound, context-independent**: 12.11
-tok/s FLAT from 304 to 16,095 context tokens — KV attention is not the cost.
-With ~29 % of experts arena-resident, ~0.84 GB crosses PCIe per token at the
-engine's effective read rate; cold-vs-warm differing ~10 % confirms the LRU
-works; growth is linear in hit rate, and only speculative decode cuts
-sequential STEPS rather than making each cheaper.
+Decode on this pair is **PCIe-gather-bound, context-independent**: the rate is
+flat from a few hundred to sixteen thousand context tokens, so KV attention is
+not the cost (the flat figure and its MTP/gather deltas are
+[[Tcclaviger Fork Ops]]'s profile of the same shape). With ~29 % of experts
+arena-resident, under a gigabyte crosses PCIe per token at the engine's
+effective read rate; cold-vs-warm differing ~10 % confirms the LRU works;
+growth is linear in hit rate, and only speculative decode cuts sequential STEPS
+rather than making each cheaper.

 **Benchmark traps that produced fake numbers here:**

@@ -8,10 +8,12 @@ timestamp: 2026-10-05T00:00:00-07:00

 # RDNA4 MoE Expert Offload VRAM Budget

-For VRAM-bound vLLM on the tcclaviger RDNA4 fork (frank-archon: Qwen3.8-
-Flash-Next FP8, TP=2, 2x R9700 at 31.86 GiB each, expert offload to host RAM).
-Tuning history and the deployed shape live in
-[[frank-archon]]; boot-failure capture lives in
+For VRAM-bound vLLM on the tcclaviger RDNA4 fork. Every boot ledger, arena
+figure and rate below is from [[frank-archon]]'s tcclaviger era (2026-09-29 →
+10-05: Qwen3.8-Flash-Next FP8, TP=2, 2x R9700 at 31.86 GiB each, expert offload
+to host RAM) — that page owns the endpoint's shape, tuning history and era
+record; this page owns only the plan arithmetic, the ladder method and the audit
+procedure. Boot-failure capture lives in
 [[vLLM Boot Forensics and Config Interrogation]].

 ## 1. The plan law
@@ -7,6 +7,12 @@ timestamp: 2026-09-03
 ---

 # Running Qwen3.6 27B FP8 on Dankclaw
+> **RETIRED - vLLM era (before the 2026-08-17 redo).** Nothing on this page serves
+> today: no vLLM or ROCm process runs on Dankclaw, and the iGPU endpoint at
+> `192.168.0.143:7770` is [[frank-dragoon]] (halogen flash-server). The page stays as
+> the era record of the Qwen3.6-27B-FP8 vLLM bring-up and the Strix Halo ROCm
+> recipes ([[frank-redo-2026-08-17]]). The hardware table below carries the era's
+> host figures; live ones are [[Dankclaw System Specs]].

 ## Model Information

@@ -29,6 +35,7 @@ Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. It uses 27
 | Vision Encoder | Yes (image-text-to-text) |

 ## Hardware: Dankclaw
+*Era figures (2026-08). The host has changed since - [[Dankclaw System Specs]].*

 | Property | Value |
 |----------|-------|
@@ -55,7 +62,11 @@ The FP8 variant requires approximately 16-17 GB. ThinkingCap-FP8 requires ~29 GB

 ## Prerequisites

-> **Production note:** Dankclaw's primary service now runs vLLM-ROCm with the Ling-3.0-flash CIRU INT4 model (see `scripts/frank-dragoon.service`). This page documents the alternative vLLM FP8 deployment of Qwen3.6-27B.
+> **Era note (2026-08, superseded twice):** at writing time Dankclaw's primary
+> service was the vLLM-ROCm Ling-3.0-flash CIRU INT4 trial ([[ling-3.0-flash-trial]],
+> retired 2026-08-17), and this page documented the alternative vLLM FP8 deployment
+> of Qwen3.6-27B. Both paths are dead; `scripts/frank-dragoon.service` today is the
+> halogen flash-server unit ([[frank-dragoon]]).

 Install vLLM for ROCm. Version 0.24.0+ is required for Qwen3.6 with speculative decoding:

@@ -36,7 +36,7 @@ In unified memory:

 ### ROCm Version

-vLLM supports AMD GPUs with ROCm 6.3 or above *(unverified 2026-10-18)*. Dankclaw has ROCm 7.2.4 from AMD repos (`/etc/apt/sources.list.d/rocm.list` → `deb https://repo.radeon.com/rocm/apt/7.2.4 noble main`; `rocm-core 7.2.4.70204-93~24.04`, read 2026-10-18). No vLLM endpoint runs on this host today — 7770 is halogen-flash-server and 7771 is llama.cpp ([[Dankclaw System Specs]]) — so this page is reference material, not the serving path.
+vLLM supports AMD GPUs with ROCm 6.3 or above *(unverified 2026-10-18)*. Dankclaw's ROCm version and which engine owns which port today are [[Dankclaw System Specs]]'s — so this page is reference material, not the serving path.

 Check ROCm version:

@@ -103,7 +103,7 @@ vllm serve model \

 - `gpu_memory_utilization`: Set lower (0.8-0.85) to leave room for CPU operations
 - `max_model_len`: Match actual context needs
- The old `--swap-space` argument was removed from current vLLM — `CacheConfig` on `main` carries no `swap_space` field ([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/config/cache.py), read 2026-10-18). Use `--kv-offloading-size` (`kv_offloading_size`, same file) for CPU KV offload
+- `--swap-space` is retired from current vLLM; the CPU KV-offload knob pair, with the upstream `CacheConfig` verification, is owned by [[RAM CPU Offloading in VLLM]]

 ### NUMA Binding

@@ -166,7 +166,7 @@ Unified memory has lower bandwidth than dedicated VRAM:

 ### ROCm Overcommit

-ROCm reports ~114 GiB of GPU memory on Strix Halo APU (the GTT aperture: `mem_info_gtt_total` = 122402963456 bytes = 114 GiB, read 2026-10-18 under `/sys/class/drm/card1/device/`; `llama-server --list-devices` shows 116,732 MiB — the 2026-08-17 "96 GiB" figure was stale). Physical RAM visible to the OS is **121 GiB** today (`free -g` total 121, read 2026-10-18; 64 GiB in the earlier reading on this page, upgraded from 32 GB). The GPU can address more than physical RAM exists because the GTT aperture maps over the shared LPDDR. vLLM will see ~114 GiB and try to allocate accordingly. Set `--gpu-memory-utilization` to 0.80 or lower to prevent OOM.
+ROCm reports the GTT aperture as "GPU memory" on this APU; the aperture size, the physical `MemTotal` behind it and the firmware carve-out are [[Dankclaw System Specs]]'s numbers, and that page also carries the 2026-08-17 "96 GiB" reading this page used to quote (stale). One reporting quirk worth keeping here: `llama-server --list-devices` prints 116,732 MiB for the same device the sysfs file calls 114 GiB — engines round the aperture differently, so read the sysfs value when sizing. The GPU can address more than physical RAM exists because the GTT maps over the shared LPDDR. vLLM will see the aperture and try to allocate accordingly; set `--gpu-memory-utilization` to 0.80 or lower to prevent OOM.

 Lower bandwidth means slower inference. Plan for lower throughput.

@@ -232,9 +232,8 @@ FP8 quantization depends on GPU architecture:
  the matmul claim as the settled half.)

 For FP4 on Dankclaw the ROCmFPX llama.cpp fork (`Q2_0_ROCMFP4`) provided native
-ROCm FP4 kernels; that runtime line is **retired** (CIRU/Kairic eras,
-[[frank-dragoon]], [[frank-dragoon]]) and no
-ROCmFPX build serves this host today.
+ROCm FP4 kernels; that runtime line is **retired** — the CIRU and Kairic eras on
+[[frank-dragoon]] — and no ROCmFPX build serves this host today.

 ## Example Commands

@@ -60,6 +60,8 @@ TUNABLE=0`, stock gather 8×16 → 26.80/25.15/26.81 tok/s @ 166/1087/4064 ctx.

 Decode on this stack is PCIe-gather-bound (~30% GPU duty cycle), so
 batch-shaped compute knobs move ~nothing.
+Measured with no spec: **12.11 tok/s FLAT from 304 to 16,095 context tokens** — KV
+attention is not the cost, so a context-slope hunt on this stack is null by construction.

 The `R4D_*/CLAV_*` knob family (QSA_*, MHC, ROUTE, ACT_QUANT, AR_QUANT,
 SILU_QUANT) are kill-switches defaulting ON, not perf dials; real levers are
@@ -120,95 +122,36 @@ exactly SEQS×(n+1) (`suboptimal` warning) → try 8192.

 ## Serving a model larger than VRAM (dankserver recipe)

-Dankserver: 2× R9700 (gfx1201), 31.86 GiB each = 63.7 GiB real VRAM, 246 GiB
-RAM. `torch.cuda.device_count()` returns 3 and summing `total_memory` gives
-a bogus 186.7 GiB (the Raphael iGPU is in the list — see
-[[Llama Cpp Serving Sizing]] for the reading discipline).
+Dankserver's card count, per-card VRAM and host RAM are in [[Dankserver System Specs]];
+`torch.cuda.device_count()` returns 3 on this box and summing `total_memory` overstates it
+badly (the Raphael iGPU is in the list — reading discipline: [[Llama Cpp Serving Sizing]]).

-**Kernel gate — this decides which checkpoint can work at all.**
-`/app/r4dhip/r4d.so` exports `gemm_moe_mxfp4a8_*` and `gemm_moe_dsfp4a8_*`
-but NO `gemm_moe_w4a16_*`. Expert offload requires an experts class with
-BOTH a compiled kernel AND an `offload_recipe`; the planner aborts startup
-otherwise. Usable: MXFP4, DSFP4, FP8-HIP (`R4dMxfp4MoEExperts`,
-`R4dDsfp4MoEExperts`, `Fp8HipMoEExperts`). `R4dW4a16MoEExperts` has no
-kernel (the class exists, the r4d.so entry does not); RFA has neither. Check
-the checkpoint's quant against this list BEFORE downloading anything.
+Everything else about running this image — the quant-kernel gate that decides
+which checkpoints can serve at all, the entrypoint contract, the known-good
+invocation, the `/dev/dri` and memlock traps, the util and max-model-len arena
+traps, the `--ipc=host` / `-d` stdout traps and the gated-fetch discipline — is
+written once on [[RDNA4 Container Serving]], and the plan arithmetic behind the
+arena/KV traps is [[RDNA4 MoE Expert Offload VRAM Budget]]'s. Three findings
+stay here because this harness produced them:

-Entrypoint is `/app/tools/image_entrypoint.sh`, ending `exec vllm serve
-"$@"` — pass the model path as the FIRST argument and do NOT pass `serve`.
-Known-good invocation:
+- **The KV reserve undercounts hybrid-attention models ~2×** (the planner's
+  B/token figure against ~18,000 real: the Mamba state is a fixed per-sequence
+  cost). Fix by pinning the pool: `--kv-cache-memory-bytes`.
+- **`--expert-offload-mem` caps the HOST side**, not the VRAM arena —
+  structurally orthogonal to `--expert-precise-memory` (verified in-container
+  2026-10-05: precise-memory only swaps the GPU overhead term,
+  expert_offload_mem only sets the host bound; they rejoin only inside
+  `place_experts`, and the plan rewrites `expert_offload_mem` back to the cap
+  the plan met, so plan and allocation cannot drift).
+- **Check whether the stack supports the model natively before porting an
+  overlay.** The image already registered `Glm5NextForConditionalGeneration`,
+  `Qwen4ExpForCausalLM` and both MTP variants — chasing four missing symbols
+  through two image tags to bring an overlay up was entirely wasted work.

-```bash
-sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'
-podman run -d --name <ctr> --ipc=host --ulimit memlock=-1:-1 \
-  --cap-add SYS_PTRACE \
-  --device /dev/kfd --device /dev/dri --device /dev/mem \
-  -e HIP_VISIBLE_DEVICES=0,1 -e VLLM_ROCM_USE_AITER=0 \
-  -v /home/dank/models/<model>:/models:ro \
-  docker.io/tcclaviger/vllm:latest /models \
-  --served-model-name <name> --port 7770 \
-  --tensor-parallel-size 2 \
-  --max-model-len 32768 \
-  --kv-cache-memory-bytes 2147483648 \
-  --max-num-seqs 4 \
-  --gpu-memory-utilization 0.95 \
-  --enable-prefix-caching --trust-remote-code
-```
-
-`-p 7770:7770` is not included, so the API is only reachable via
-`podman exec` — add `-p` if host-side curl is needed. Load ~5 min;
-`/health` answers connection-refused the whole time; watch for
-`Application startup complete` in `podman logs <ctr>`. Verified working:
-`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (gated; 172.8 GiB, 131
-shards, `Qwen4ExpForConditionalGeneration`, 48 layers, 512 experts top-10,
-multimodal, native 262,144).
-
-Traps that each cost a boot:
-
-1. **Mount the whole `/dev/dri`, never individual render nodes.** Render
-   nodes without the primary `card*` nodes → HIP enumerates two agents and
-   refuses a context on either (`RuntimeError: No CUDA GPUs are available`)
-   while `device_count()` cheerfully returns 2. Confirmed by direct A/B.
-2. **memlock must be raised** (done on Dankserver via
-   `/etc/security/limits.d/99-memlock.conf`). Host default 8 MiB makes any
-   large pinned allocation fail as a misleading
-   `torch.AcceleratorError: CUDA error: out of memory` from `pin_memory()`;
-   it cannot be raised from inside a container or root's own shell — only
-   PAM applies it at login. Matters because `--expert-host-mlock` defaults
-   True.
-3. **`--gpu-memory-utilization` does NOT buy KV under expert offload** —
-   the planner sizes the arena against the budget and leaves KV the
-   remainder; raising util 0.95→0.98 grew the arena and the KV pool stayed
-   at 0.64 GiB.
-4. **Lowering `--max-model-len` is the WRONG lever and backfires** — the
-   budget subtracts the KV reserve first, so a smaller context shrinks the
-   reserve and hands the space to the arena (measured: 65536 → 0.64 GiB KV;
-   32768 → 0.34 GiB).
-5. **The KV reserve undercounts hybrid-attention models ~2×** (9190 B/token
-   planned vs ~18,000 real: the Mamba state is a fixed per-sequence cost).
-   Fix by pinning the pool: `--kv-cache-memory-bytes`.
-6. **`--expert-offload-mem` caps the HOST side**, not the VRAM arena —
-   structurally orthogonal to `--expert-precise-memory` (verified
-   in-container 2026-10-05: precise-memory only swaps the GPU overhead term,
-   expert_offload_mem only sets the host bound; they rejoin only inside
-   `place_experts`, and the plan rewrites `expert_offload_mem` back to the
-   cap the plan met, so plan and allocation cannot drift).
-7. `--ipc=host` and `--shm-size` are mutually exclusive in podman; pick one.
-8. `podman run -d` without `-a` sends no stdout to a redirect — the log file
-   only receives the container ID; use `podman logs`.
-
-Status reads: use `scripts/dankstatus.sh`, never hand-rolled
-`/proc/meminfo` parsing — `/proc/meminfo` separates key from value with a
-TAB, so `grep MemAvailable | tr -s " " | cut -d" " -f2` returns empty and
-awk prints `0.0` — a false "host out of memory" that invites an unnecessary
-abort. Gated checkpoints: token via a mode-600 `curl -K` config, never argv
-(`pgrep -a curl` prints it); always `--fail`; compare each file's byte count
-to the Hub's declared size; `-C -` resumes truncated shards.
-
-**Check whether the stack supports the model natively before porting an
-overlay.** The image already registered `Glm5NextForConditionalGeneration`,
-`Qwen4ExpForCausalLM` and both MTP variants — chasing four missing symbols
-through two image tags to bring an overlay up was entirely wasted work.
+The checkpoint this recipe was verified on is
+`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (gated; 172.8 GiB, 131 shards,
+`Qwen4ExpForConditionalGeneration`, 48 layers, 512 experts top-10, multimodal,
+native 262,144) — an [[frank-archon]] era payload.

 ## Compile-cache persistence (survives `podman rm -f`)

@@ -240,20 +183,15 @@ cold, of which 173 s was one graph).

 ## Plan-line verification checklist (after any fork deploy)

-```
-ssh dankserver 'sudo journalctl CONTAINER_NAME=frank-archon-qwen -b -o cat --no-pager \
-  | grep -aoE "(budget \(total x util\) +[0-9.]+|expert cache space +[0-9.]+ GiB +[0-9,]+ slots|Estimated CUDA graph memory.{0,80}|Graph capturing finished.{0,40}|GPU KV cache size.{0,90})"'
-```
-
-`grep -oE` with a digit-anchored tail — plain `grep`/journalctl truncate
-these columns to `17.`. Verify: new budget/graphs/arena in the plan;
-`GPU KV cache size` matches the requested token count; `Estimated CUDA graph
-memory` vs the capture line (raise `_PER_GRAPH_MB` toward 52 if capture
-exceeds the charge); `rocm-smi --showmeminfo vram` high-water per card = the
-real margin; `systemctl show frank-archon -p NRestarts --value` = 0;
-`VLLM_CUDAGRAPH_*` present in `podman exec … env`; then a real chat
-completion from a PEER — `curl localhost:7770/health` inside an ssh session
-on the host can return `000` and looks like an outage that isn't.
+The journal one-liner that reads the plan lines (digit-anchored `grep -oE`,
+because plain grep truncates those columns to `17.`) and the post-deploy
+verification order — plan vs captured graph memory, `GPU KV cache size` vs the
+ask, per-card `rocm-smi` high-water, `NRestarts` = 0, `VLLM_CUDAGRAPH_*` present
+in the container env, then a real completion from a PEER because
+`curl localhost:7770/health` inside an ssh session on the host can return `000`
+and look like an outage — are step 3 of
+[[RDNA4 MoE Expert Offload VRAM Budget]] §7. Run that; do not re-derive it
+here.

 ## Related

@@ -60,7 +60,7 @@ vllm serve meta-llama/Llama-3.1-70B-Instruct --tensor-parallel-size 4
 vllm serve Qwen/Qwen2.5-1.5B-Instruct --quantization fp8

 # With GPU memory control
-vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --swap-space 8
+vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --kv-offloading-size 8 --kv-offloading-backend native
 ```

 ### Common API Endpoints
@@ -79,7 +79,7 @@ vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --swap-space
 |-----------|-------------|---------------|
 | tensor_parallel_size | Number of GPUs for model sharding | 1-8 |
 | gpu_memory_utilization | Fraction of GPU memory for KV cache (default 0.92) | 0.7-0.95 |
-| swap_space | Removed in current vLLM; use `--kv-offloading-size` | n/a |
+| swap_space | Retired; live knob pair in [[RAM CPU Offloading in VLLM]] | n/a |
 | max_model_len | Maximum context length | auto |
 | max_num_batched_tokens | Max tokens per batch | 2048-8192 |
 | max_num_seqs | Max sequences per batch | 128-512 |
@@ -140,7 +140,7 @@ curl http://localhost:8000/v1/chat/completions \
 | `tensor_parallel_size` | 1 | Number of GPUs for tensor parallelism |
 | `pipeline_parallel_size` | 1 | Number of GPUs for pipeline parallelism |
 | `gpu_memory_utilization` | 0.92 | Fraction of GPU memory for KV cache |
-| `swap_space` | removed | Removed in current vLLM; use `--kv-offloading-size` |
+| `swap_space` | — | retired; the live CPU-KV-offload knob pair is in [[RAM CPU Offloading in VLLM]] |
 | `enforce_eager` | false | Use eager mode (no CUDA graphs) |
 | `max_num_batched_tokens` | 2048 | Max tokens per batch |
 | `max_num_seqs` | 128 | Max sequences per batch |
@@ -167,7 +167,7 @@ curl http://localhost:8000/v1/chat/completions \
 | Argument | Default | Description |
 |----------|---------|-------------|
 | `gpu_memory_utilization` | 0.92 | Fraction of GPU memory for KV cache |
-| `swap_space` | removed | Removed in current vLLM; use `--kv-offloading-size` |
+| `swap_space` | — | retired; the live CPU-KV-offload knob pair is in [[RAM CPU Offloading in VLLM]] |
 | `kv_cache_dtype` | auto | KV cache data type (auto, fp8, fp8_e5m2) |
 | `max_model_len` | auto | Maximum context length |
 | `num_gpu_blocks` | auto | Number of GPU blocks |
@@ -14,6 +14,10 @@ timestamp: 2026-09-03

 > **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
 > See [[frank-redo-2026-08-17]] and [[deployment]] for the active configuration.
+> The `frank-zealot` below is the 2026-08 service: llama.cpp on the RTX 3090
+> **while that card sat in Dankserver**. Today's [[frank-zealot]] is a different host
+> (Dankclaw), payload and runtime build, though the same card class - this crash
+> signature is not a live fault.

 # Dankserver VRAM and RAM Growth Under Load

@@ -241,6 +241,11 @@ Before 2026-09-26 the port belonged to the **`frank-medium`** service: [[frank-m
  seqs 2 and seqs 3 are statistically tied (30.9-33.1 t/s, inside the ±3 % floor). The "seqs 2
  cannot boot" law belonged to the retired tcclaviger fork's `torch.compile` shape
  specialization, and radiance booted seqs 2 on its first fp8-native start.
+  **Same era, separate law: a dequantize-to-compute path bills host RAM twice.** On the
+  EMULATION backend an offload request of **118 GiB** landed as **~238 GiB** of real host RAM —
+  the dequantized copy rides on top of the pinned weights, and the doubling is invisible in the
+  weight count. Read the engine log for `Using 'EMULATION' Mxfp4 MoE backend` before concluding
+  "doesn't fit"; the container-side procedure and the kernel gate are [[RDNA4 Container Serving]].
 - **One encoding per expert table** (`rad_arch_declare`): a whole expert kept bf16 is the only
  thing legal beside the table, so a split tier (down=fp8, gate_up=w4) is unservable.
 - **Descriptor-selection law.** The plugin is picked by the exact `(arch_id, quant descriptor)`
@@ -25,6 +25,12 @@ what each buys. The answer is that `--ubatch-size` has an **optimum** on this
 stack, that the logical batch does not matter at all, and that the optimum is
 much cheaper than what was deployed — which then moved the layer split.

+The deployment this sweep tuned (`frank-medium`, 2x R9700, llama.cpp layer
+split) is retired; its shape and boot ledger are
+[[frank-medium-llamacpp-dflash2]]'s, and the endpoint lineage continues on
+[[frank-archon]]. What this page owns is the sweep, the harness traps and the
+coupling law between batch and split.
+
 ## The shape of the measurement

 A cell is a **64k prefill measured with 64k of KV already resident**, plus a
@@ -104,7 +110,7 @@ kill — no requests):
 |---|---|---|---|---|
 | 36,29 — deployed before | 1536 | 4.71 GiB | 4.91 GiB | 0.20 GiB |
 | 36,29 | **512** | 6.01 GiB | 7.32 GiB | 1.31 GiB |
-| **35,30 — deployed now** | **512** | **6.44 GiB** | **6.90 GiB** | **0.46 GiB** |
+| **35,30 — deployed after** | **512** | **6.44 GiB** | **6.90 GiB** | **0.46 GiB** |

 Two things follow.

@@ -153,7 +159,7 @@ other (measured 2.41 GiB vs 1.30 GiB going 1536 → 512). Balance by
 measurement only: one `-ts` step is one layer, worth ~0.43 GiB on a DeltaNet
 boundary and ~1.43 GiB on an attention one, so the reachable minimum gap is
 about one DeltaNet layer, never zero. Reference: 36,29 @1536 → 4.71/4.91;
-36,29 @512 → 6.01/7.32; **35,30 @512 → 6.44/6.90 (deployed)**.
+36,29 @512 → 6.01/7.32; **35,30 @512 → 6.44/6.90 (the shape that deployed)**.

 Re-fit both ubatch coefficients in `tests/frank_medium_unit.bats` at the new
 deployed point and re-anchor the per-card constants; check neighbouring
@@ -1,7 +1,7 @@
 ---
 type: runbook
 title: frank-medium Model Swap
-description: How to swap the checkpoint behind the frank-medium endpoint (Dankserver :7770, vllm-radiance TP=2) — the streamlining toolchain (hf-manifest.py, hf-fetch.sh, swap-frank-medium.sh), the quantization-aware entrypoint contract, the quadlet edit, the deploy, the capability battery, and the rollback path.
+description: RETIRED-era procedure (frank-medium is a retired endpoint; `:7770` on Dankserver is [[frank-archon]] today and the live swap law is [[Model Swap and Rollback]]) — how a checkpoint swap behind frank-medium worked: the streamlining toolchain (hf-manifest.py, hf-fetch.sh, swap-frank-medium.sh), the quantization-aware entrypoint contract, the quadlet edit, the deploy, the capability battery, and the rollback path.
 tags:
  - frank-medium
  - dankserver
@@ -14,12 +14,13 @@ timestamp: 2026-09-20

 # frank-medium Model Swap

-Swapping the checkpoint behind **frank-medium** (Dankserver `:7770`,
-vllm-radiance TP=2) is a four-step process, and the toolchain below drives all
-of it. The design goal (2026-09-20, after the MXFP4 → FP8 swap) is that a swap
-is **one command**: generate a manifest, download it verified, point the
-quadlet at it, deploy. The entrypoint is quantization-aware, so the only repo
-change a swap makes is the quadlet's four env vars.
+**Historical.** frank-medium is a retired endpoint — `:7770` on Dankserver is
+[[frank-archon]] today, and the live procedure is [[Model Swap and Rollback]].
+This page records how a swap worked in that era: a four-step process whose design
+goal (2026-09-20, after the MXFP4 → FP8 swap) was that a swap be **one command** —
+generate a manifest, download it verified, point the quadlet at it, deploy. The
+entrypoint was quantization-aware, so the only repo change a swap made was the
+quadlet's four env vars.

 ## Why this exists

@@ -1,7 +1,7 @@
 ---
 type: configuration
 title: frank-medium vllm-radiance MXFP6 Runbook
-description: ACTIVE frank-medium runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era.
+description: RETIRED era runbook (endpoint renamed frank-archon 2026-10-10; this shape is a row in [[frank-archon]]'s Eras table) — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era.
 tags:
  - frank-medium
  - dankserver
@@ -19,7 +19,8 @@ timestamp: 2026-09-21

 # frank-medium vllm-radiance MXFP6 Runbook

-The ACTIVE frank-medium deployment: the `ggz14/vllm-radiance-mxfp4:0.13.0-e060895`
+**RETIRED era (2026-09-21 → 10-10; the endpoint is [[frank-archon]] today and this
+shape is one row in its Eras table.)** The deployment was the `ggz14/vllm-radiance-mxfp4:0.13.0-e060895`
 image (built from PR #51's head, tagged locally as
 `localhost/radiance-vllm:0.13.0-e060895`) plus the `radiance-vllm-mxfp4`
 patches at e060895, serving `hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6`
@@ -14,14 +14,10 @@ The pipeline-parallel split and the FastFlowLM NPU endpoint were retired.
 The old scripts moved to `old/scripts/` in this repository.

 > **HISTORY — the topology below is two fleet generations dead.** Today the
-> fleet is three endpoints: [[frank-archon]]
-> (`192.168.0.235:7770`, radiance 1.0.13, 2×R9700 TP=2),
-> [[frank-dragoon]] (`192.168.0.143:7770`, halogen
-> flash-server 0.16.2 on the iGPU) and
-> [[frank-zealot]] (`192.168.0.143:7771`, llama.cpp
-> PrismML fork build 10754 on the RTX 3090 — the card that sat in Dankserver
-> until 2026-09-15). No vLLM and no ROCm endpoint serves anything; there is no
-> pipeline parallelism and no NPU endpoint.
+> fleet is three endpoints, each owned by its page: [[frank-archon]]
+> (`192.168.0.235:7770`), [[frank-dragoon]] (`192.168.0.143:7770`) and
+> [[frank-zealot]] (`192.168.0.143:7771`). No vLLM and no ROCm endpoint serves
+> anything; there is no pipeline parallelism and no NPU endpoint.

 ## New Topology

@@ -1,6 +1,6 @@
 ---
 type: reference
-title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 2 × 262144, exact f16 KV"
+title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 3 × 200704, f16 K + q8_0 V, MTP armed"
 description: "The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env ships it, every era with its rollback path, the deduplicated laws of this card, the wire contract, and the owning files."
 tags:
  - llm
@@ -12,10 +12,11 @@ tags:
 timestamp: 2026-10-18
 ---

-# frank-zealot — llama.cpp (PrismML) Bonsai-2-9B PQ2_0-MTP, 2 × 262,144, exact f16 KV
+# frank-zealot — llama.cpp (PrismML) Bonsai-2-9B PQ2_0-MTP, 3 × 200,704, f16 K + q8_0 V, MTP armed

 Dankclaw's RTX 3090 endpoint — `http://192.168.0.143:7771`, wire id `frank-zealot`, the fleet's **compaction hub**
-(widest window it has). One page per endpoint: eras are rows below, not pages. Siblings: [[frank-archon]],
+(three conversations at 200,704 each, the window the other two endpoints also cap at). One page per
+endpoint: eras are rows below, not pages. Siblings: [[frank-archon]],
 [[frank-dragoon]]. Deploy with `./scripts/deploy.sh dankclaw --service frank-zealot`; swap mechanics in
 [[Model Swap and Rollback]], sizing math in [[Llama Cpp Serving Sizing]]. `scripts/frank-zealot.env` and
 `scripts/serve-frank-zealot.sh` are the authority for the shape; where any prose disagrees with them, the prose
@@ -31,72 +32,87 @@ is the bug.
 | Runtime | `/home/dank/GitPackages/llama-prism-b10777/llama-server` — PrismML **fork**, `0.2.0-dev (build 10777, commit e3de71522)`; the only loader for this payload |
 | Trio | `scripts/frank-zealot.service` (root, relinked, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`, `TimeoutStopSec=180`) · `scripts/frank-zealot.env` · `scripts/serve-frank-zealot.sh` |
 | Address / alias | `BIND_ADDR=0.0.0.0`, `LISTEN_PORT=7771`, `ALIAS=frank-zealot` (a flag, not a fact) |
-| Flags | `--ctx-size 524288 --parallel 2` ⇒ 262,144 per slot · `--cache-type-k f16 --cache-type-v f16` (exact, **16,384 MiB**) · `--flash-attn on` · `--batch-size 4096 --ubatch-size 2048` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · `--log-verbosity 4` · `--presence-penalty 1.0` · **`SPEC_TYPE=none`** |
+| Flags | `--ctx-size 602112 --parallel 3` ⇒ 200,704 per slot · `--cache-type-k f16 --cache-type-v q8_0` (asymmetric, **14,406.00 MiB**) · `--flash-attn on` · `--batch-size 6144 --ubatch-size 1024` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · `--log-verbosity 4` · `--presence-penalty 1.0` · **`--spec-type draft-mtp`** with `--spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0` |

 **Why this shape.** `general.architecture qwen35`: 33 blocks (32 trunk + `blk.32` = the MTP block),
 `full_attention_interval 4` ⇒ 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, `nextn_predict_layers 1`
 (the head **is in the file**), trained window 262,144, `general.sampling.*` 1.0/0.95/20. Only the 8 full layers bill
-per token (32,768 B at exact f16 ⇒ 16,384 MiB over the pool) and the DeltaNet layers add ~50 MiB of recurrent state
-**per slot**: exact KV costs the third conversation and nothing else.
+per token and the DeltaNet layers add ~50 MiB of recurrent state **per slot**, so the shape is a single budget
+equation at 25,088 B/token (f16 K 16,384 + q8_0 V 8,704): **14,406.00 MiB** of cache over a 602,112 pool +
+**2,352.00 MiB** of draft cache (blk.32 at its own f16 rate, billed over the WHOLE pool) + ~7.7 GiB of weights,
+projector, buffers and states, inside the 24,123 MiB the driver leaves addressable. Exact V (32,768 B/token ⇒
+18,816 MiB here) buys neither the third conversation nor the drafter; the fully quantized pair (17,408 B/token)
+buys both and costs the keys their precision.

-**MTP is OFF by measurement, not by absence of a head.** The drafter is *good* — 2026-10-18 measured
-acceptance 0.798 / 0.617 per position, mean accepted length 2.41, and the publisher's card says depth 2 works
-best — and it still loses here, because the draft context is built against the *target's* pool: `blk.32` bills
-4,096 B/position at its default f16 ⇒ 2,048 MiB at 524,288 positions on a trunk that projects 21,234 MiB of the
-24,123 MiB addressable (that projection predicts 2,889 MiB free; replace it with the measured pair). Exact KV and a
-drafter cannot share this card. `--spec-draft-type-k`/`--spec-draft-type-v` (default f16, verified in this build's
-`--help`) would cut the draft to 1,088 MiB — legal on b10777, **never booted beside exact KV**. There is no
-`--spec-draft-model`: the MTP draft rides the main model's weights. The env carries the recipe
-(`SPEC_DRAFT_N_MAX=2`, `SPEC_DRAFT_N_MIN=0`, `SPEC_DRAFT_P_MIN=0`); `SPEC_TYPE=none` never sends it.
+**MTP is ON (Frank, 2026-10-19).** The drafter was always good — 2026-10-18 measured acceptance 0.798 / 0.617 per
+position, mean accepted length 2.41, and the publisher's card says depth 2 works best — and arming it is a
+budget decision, not a flag: the draft context is built against the *target's* pool, so `blk.32` bills 4,096
+B/position (2,352.00 MiB at 602,112) plus 480.97 MiB of its own CUDA0 buffers, and it triples the recurrent
+state (452.25 MiB for three cells against 150.75 with the drafter off). `CACHE_TYPE_V=q8_0` is what pays that:
+7,680 B/token cheaper than exact V = **4,410.00 MiB** at this pool. `UBATCH_SIZE=1024` — the rung every measured
+MTP arm here ran on — contributes only ~406 MiB at THIS shape, not the ~1,306 the 786,432-pool census showed,
+and `UBATCH_SIZE=2048` beside an armed drafter is still the shape that halved decode (167.2 → 109.2 t/s).
+`--spec-draft-type-k`/`--spec-draft-type-v` (default f16, verified in this build's `--help`) would cut the draft's bill again and stay at their default; there is no
+`--spec-draft-model` — the draft rides the main model's weights, so a draft file would load the trunk twice.
+`SPEC_TYPE=none` is the rollback and strips the whole set.

 **Harness contract** (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw-3090)`, id `frank-zealot`):
-`contextWindow: 262144` — **the per-slot window**, because llama.cpp divides `--ctx-size` across slots and answers a
+`contextWindow: 200704` — **the per-slot window**, because llama.cpp divides `--ctx-size` across slots and answers a
 longer prompt with a 400, never a truncation — plus `maxTokens: 16384`, `input: [text, image]`,
 `compat.reasoningContentField: reasoning_content`, `compat.reasoningEffortMap` (**load-bearing**: the template takes
 only `xhigh|medium|low` and raises on anything else), `extraBody` 1.0 / 0.95 / 20 / `min_p 0`, and the
 `compactionModel` of [[frank-archon]], [[frank-dragoon]] **and** frank-zealot all pointing here.

-**The exact-KV boot, measured** (2026-10-18 evening, under the unit, verbosity 4). KV **16,384.00 MiB** =
-32,768 B/token × 524,288 positions, to the MiB — rate derivation and ledger agree with no slop, which is what
-buries the doc's old "f16 needs 26,112 MiB and cannot boot" figure for good: that was the 27B's 34,816 B/token
-applied to the 9B. RS **100.50 MiB** (2 cells, one 50.25 MiB cell lighter than the q8_0 era), compute
-1,408.06 + 1,088.07 CUDA_Host, mmproj worst case **868.32 MiB** charged *after* the engine's own fit check,
-settled `nvidia-smi` **21,176 used / 2,949 free** of 24,123, `n_slots = 2`, `n_ctx_slot = 262144`,
-`DropInPaths` empty, no clamp line, no `cudaMalloc` error — within 24 MiB of the pre-flight projection. That is
-a desktop margin: the ~2 GiB of slack is what an image storm has to live inside.
+**No rates, by law.** Frank ordered the reslice, not a number: the evidence a swap needs is
+the seam's fit gate, the deployer's health gate, and the boot ledger below. The previous era's
+exact-KV boot and its rate floors live in that era's row; those floors were measured while
+another consumer drove the other slot (56 prompts in three minutes, 96 % GPU-util), the exact
+condition a bench window exists to exclude.

-**Rates on this shape are FLOORS, not the shape's rates** — measured the same evening greedy with `ignore_eos`
-at 512 tokens, first request discarded, while another consumer drove the other slot (56 new prompts in three
-minutes, 96 % GPU-util): the exact condition the bench window exists to exclude, and one contended deep arm
-even came back truncated (a 512 KB body accepted as `prompt_n = 4`).
+**The 2026-10-19 boot, measured** — verbosity-4 ledger of `ba69ea5`, started 03:21:56 PDT.
+Recorded here because projections drift, and nowhere else:

-| Arm | q8_0 3-slot record | f16 2-slot, contended |
-|---|---|---|
-| single-stream decode | 168.7 | **134.1 – 141.5** (145.8 and 163.2 in a lighter minute) |
-| concurrent aggregate | ≈300 across 3 (102.8 each) | **256.6 / 262.4 / 265.1** across 2 (≈128–133 each) |
-| prefill at ~106k | 2,877 at 106,418 | **3,159.5 at 106,282** |
-| decode at 106k depth | never measured | **71.5** |
+| Term | Ledger |
+|---|---|
+| Target KV over the 602,112 pool | **14,406.00 MiB** — K (f16) 9,408.00 + V (q8_0) 4,998.00, `200704 cells, 8 layers, 3/3 seqs` |
+| Draft KV (`blk.32`, its own f16 context) | **2,352.00 MiB** — 1,176.00 + 1,176.00, `200704 cells, 1 layers, 3/3 seqs`: the whole pool, again |
+| Recurrent state | **452.25 MiB** — `3 cells, 32 layers, 3 seqs, 2 rs_seq`, R (f32) 20.25 + S (f32) 432.00 |
+| Compute buffers | target 1,665.00 CUDA0 + 424.88 CUDA_Host; draft context 480.97 + 424.86 |
+| Weights | 2,283.29 CUDA0 + 257.66 CPU-mapped |
+| Projector | 595.29 model + 248.10 compute + 24.93 CPU (its 868.32 worst case is still charged AFTER the engine's fit check) |
+| Engine's own verdict | projected 21,639 vs 23,404 free, then **"cannot meet free memory target of 1,867 MiB, need to reduce device memory by 102 MiB"** |
+| Settled | **22,870 used / 1,255 free**, `n_slots = 3`, `n_ctx_slot = 200704`, `kv_unified = false` |

-What the floors already support: per-conversation concurrency **improved** (two streams hold ~130 t/s each where
-three held 102.8), endpoint aggregate sits ~12 % under the 3-slot record, prefill beat the record while
-contended, and single-stream decode is between parity and −20 %. The 1.882× KV byte bill is real; its true size
-on this card is still unmeasured, so quoting 134 or 163 t/s as "the" number is a guess in either direction —
-re-run the same arms with the box quiet before a figure is carried anywhere else.
+This is the first boot on this card that does **not** meet llama.cpp's own free-memory target,
+and 1,255 MiB settled sits under the ~2 GiB band every previous era held. It boots, `/health`
+answers, and the seam's gate still passes — it priced 22,428 MiB, conservative by 442 against
+this ledger. The cheapest rungs back into the band are `CTX_SIZE=401408` (two conversations) or
+`SPEC_TYPE=none`; both are one line and neither changes the payload.
+
+**The drafter is engaged, measured on the same boot** (three ordinary requests, not a bench):
+acceptance 0.706 and 0.763 accepted-per-generated, mean accepted length 2.41 / 2.53,
+per-position rates (0.882, 0.765) and (0.871, 0.699) — depth 2 earning its keep at 0.3–1.5 MiB
+of draft state per saved prompt. The three probes that prove a swap: chat → `"zealot alive"`
+with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY content with
+`finish_reason: length` — the thinking-budget trap, not a fault); tools →
+`finish_reason: tool_calls` with `{"a":24,"b":7}`; vision → a 512 px solid-red PNG answers
+"Red" at `prompt_tokens` 280.

 **Fallback rungs**, cheapest first — one line of `frank-zealot.env` each, and each must keep the seam's fit gate green:

-| Knob | Effect |
+| Knob | Effect from the shipped shape |
 |---|---|
-| `UBATCH_SIZE=1024` | hands back 1,306 MiB of compute buffers; prefill −17 %; it is also the arm that ran the drafter at three windows (213.5 t/s) |
-| `CACHE_TYPE_V=q8_0` | exact K + quantized V: KV 16,384 → **12,544 MiB**, freeing 3,840 (llama.cpp has no true fp8 KV dtype — q8_0 is the 8-bit class) |
-| `CTX_SIZE=401408` | two 200,704 conversations, exact KV at 12,544 MiB (same 3,840 freed) — `contextWindow` MUST follow the per-slot number |
-| `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` | the era below this one: 13,056 MiB of cache, the third conversation back |
-| `CACHE_TYPE_K/V=q4_0` | 10,240 B/token ⇒ 5,120 MiB (the 27B era measured q4_0 at 16,392 used / 7,733 free at 2 × 200,704) |
+| `SPEC_TYPE=none` | the drafter off: hands back its 2,352.00 MiB of cache, 480.97 MiB of CUDA0 buffers and the 301.50 MiB of rollback planes it adds to the recurrent state (~3.1 GiB), at the cost of the ~+28 % decode the 2026-10-18 census measured whenever the card has room |
+| `UBATCH_SIZE=2048` | +4.6 % prefill and only ~406 MiB of compute buffers at THIS shape (the ~1,306 MiB the 786,432-pool census showed does not transfer) — and the rung that HALVED decode beside an armed drafter (167.2 → 109.2 t/s) |
+| `CTX_SIZE=401408` | two 200,704 conversations: frees 4,802 MiB of cache — `contextWindow` MUST follow the per-slot number |
+| `CACHE_TYPE_V=f16` | exact V: +4,410 MiB at this pool with the drafter armed, which is a `cudaMalloc` death the seam refuses — legal only once the drafter or a slot is gone |
+| `CACHE_TYPE_K=q8_0` | the fully quantized pair (17,408 B/token): frees 4,410 MiB and moves the quantization error into the keys |

 ## Eras

 | Window | Payload | Runtime | Shape | Replaced because | Rollback |
 |---|---|---|---|---|---|
+| 2026-10-18 21:06 → 2026-10-19 | the same 9B MTP file and projector | PrismML fork 10777 | 2 × 262,144 (pool 524,288), exact f16 K+V = 16,384.00 MiB, `SPEC_TYPE=none`, batch 4096 / ubatch 2048; boot ledger KV 16,384.00 / RS 100.50 / compute 1,408.06 + 1,088.07 CUDA_Host / mmproj worst case 868.32 charged after the fit check, settled **21,176 used / 2,949 free**, `n_ctx_slot = 262144`, within 24 MiB of the projection; floors 134.1–141.5 t/s single-stream, 256.6–265.1 aggregate over two, 3,159.5 t/s prefill at 106,282, 71.5 t/s decode at 106k depth (contended — floors, not the shape's rates) | Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352 | one line each — `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` — and redeploy; no payload move |
 | 2026-10-18 13:19 → same evening (the boot `68afc6d` replaced) | the same 9B MTP file and projector | PrismML fork 10777 | 3 × 262,144 (pool 786,432), q8_0 K+V = 13,056 MiB, settled 17,956 / 6,169 MiB; 168.7 t/s decode (three runs 168.4–168.7), ~300 t/s over three streams (102.8 each), 2,877 t/s prefill at 106,418 | Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable | one env line — `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` — and redeploy; no payload move |
 | 2026-10-18, 24 min (arm under the unit) | `Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf` 5,946,648,928 B + hardlinked Q8 mmproj | PrismML fork 10777 | 2 × 200,704, batch/ubatch 3072/1536 | Frank scratched the 27B plan — ~48 t/s decode vs 76.6 for PQ2_0 on the same weights | `/home/dank/models/Bonsai2-PTQ1_0/` + `scripts/download-frank-zealot-bonsai2-ptq1.sh` |
 | 2026-10-17 → 10-18 | Ternary-Bonsai-2-**27B** PQ2_0 7,206,168,928 B `3907dc16…` + Q8_0 mmproj 629,246,976 B | PrismML fork **10754** (`~/GitPackages/llama-prism/`) | 2 × 200,704 (pool 401,408), q8_0 K+V = 13,328 MiB, settled 22,664 / 1,461; 76.6 t/s decode, 109.4 aggregate over two streams, 691.7 t/s prefill at 166,027; MTP impossible — no head in the file | the 9B bills half the KV per token (8 full layers, not 16), which funded three slots at the native window and beat it on every rate | `/home/dank/models/Bonsai2-PQ2_0/` + the 10754 fork tree, `scripts/download-frank-zealot-bonsai2.sh`, revert the swap commit |
@@ -136,6 +152,10 @@ re-run the same arms with the box quiet before a figure is carried anywhere else
  `tests/frank_zealot_unit.bats`'s fit gate can see a boot that would die at `cudaMalloc`, so every retune keeps it
  green — and the first verbosity-4 ledger plus settled `nvidia-smi` of a new dtype belongs on this page, because
  projections drift.
+  The same trap one knob down: the ubatch 2048 → 1024 step handed back **~1,306 MiB** at the 786,432 / q8_0-pair
+  arm and **406 MiB** at the 602,112 / f16-K + q8_0-V boot (target compute 2,496.13 → 2,089.88, with CUDA0 going
+  UP 257 while CUDA_Host went down 663). Changing the dtype or the pool moves the compute buffers too, so a
+  credit measured at one shape is not a budget line at the next.
  The two boots that priced non-KV: 2 × 262,144 **without** the projector — 20,190 used, KV 12,800 ⇒ non-KV
  **7,390 MiB**; 3 × 196,608 **with** it — 22,972 used, KV 14,113 ⇒ non-KV **8,859 MiB**. The difference prices the
  vision tower at **~1.4 GiB for a 918 MB file**, because the encoder's activation and compute buffers ride with it.
@@ -160,7 +180,7 @@ re-run the same arms with the box quiet before a figure is carried anywhere else

 ## Contract

- `/health` → `{"status":"ok"}`; `/props` reports `n_ctx` and `presence_penalty`; `/slots` shows occupancy; a third
+- `/health` → `{"status":"ok"}`; `/props` reports `n_ctx` and `presence_penalty`; `/slots` shows occupancy; a fourth
  concurrent conversation **waits** for a slot — the slot count is the concurrency contract.
 - `/metrics` answers 200 with 15 `llamacpp:*` families **only with `--metrics`**; without it llama.cpp answers **501**, which Prometheus reads as DOWN while `/health` stays 200. The scrape target is `host.containers.internal:7771/metrics` and the oldest retained sample of `up{job="frank-zealot"}` (2026-10-11) is already 0 — this target was never successfully scraped, across every payload era. `METRICS=0` is the documented opt-out and re-breaks it; the counters cost nothing on the request path.
 - Reasoning arrives in **`reasoning_content`** (`--reasoning-format deepseek`) and **counts against `max_tokens`**: a short budget yields empty `content` with `finish_reason: "length"`, which reads as a broken model, not a mid-thought one. The slot prefix cache is live (`cached_tokens` in `usage`).
@@ -168,9 +188,9 @@ re-run the same arms with the box quiet before a figure is carried anywhere else
  (1.0 / 0.95 / 20). `presence_penalty` is the one recipe value with no GGUF metadata key, so it must reach argv;
  the serve script uses `${PRESENCE_PENALTY=1.0}`, deliberately **not** `:=`, because `""` is the documented opt-out.
 - Vision: only the OpenAI **`image_url`** content part reaches the model — the legacy top-level `images` field bumps `usage.prompt_tokens` while the model invents an answer. Probe with a **≥ 256 px** solid color (a postage stamp answers "video") and budget **≥ 512** tokens; a 512 px image bills ~280 prompt tokens.
- **Q8_0 mmproj, not BF16**: the publisher warns the BF16 tower "can crash llama-server when several image requests run at once", which is exactly the `--parallel 2` shape. `MMPROJ_FILE=` (explicitly empty) is the text-only opt-out and returns ~1.4 GiB; a path at a missing file is a boot **refusal**, never silent blindness.
+- **Q8_0 mmproj, not BF16**: the publisher warns the BF16 tower "can crash llama-server when several image requests run at once", which is exactly the `--parallel 3` shape. `MMPROJ_FILE=` (explicitly empty) is the text-only opt-out and returns ~1.4 GiB; a path at a missing file is a boot **refusal**, never silent blindness.
 - Tools: native via `--jinja`; `finish_reason: tool_calls` with valid JSON arguments.
- Two non-faults: a `400` naming 262,144 is one conversation outgrowing its **slot**, not pool exhaustion; and
+- Two non-faults: a `400` naming 200,704 is one conversation outgrowing its **slot**, not pool exhaustion; and
  the raw `/completion` path is a **dead instrument on this fork build** — a greedy `ignore_eos` run answers
  **500 "The model produced output that does not match the expected Content-only format"**, because build 10777
  guards the raw path the moment reasoning markup leaks (older builds only logged `failed to parse reasoning
@@ -19,8 +19,8 @@ each of those owns exactly that one thing. The incidents behind the laws are in
 |---|---|
 | [[frank-medium-batch-ubatch]] | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… |
 | [[frank-medium-llamacpp-dflash2]] | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… |
-| [[frank-medium-model-swap]] | How to swap the checkpoint behind the frank-medium endpoint (Dankserver :7770, vllm-radiance TP=2) — the streamlining… |
-| [[frank-medium-radiance-mxfp4]] | ACTIVE frank-medium runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
+| [[frank-medium-model-swap]] | RETIRED-era swap procedure for the frank-medium endpoint (`:7770` is now [[frank-archon]]) — the… |
+| [[frank-medium-radiance-mxfp4]] | RETIRED frank-medium era runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
 | [[frank-medium-rocmfpx]] | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… |
 | [[frank-npu]] | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… |
 | [[frank-redo-2026-08-17]] | This page records the full redo of the inference fleet on 2026-08-17. |
@@ -15,6 +15,8 @@ This page was the plan for a vLLM-ROCm deployment of
 Dankclaw. The deployment never shipped. The actual Ling work used the
 llama.cpp Q6_K fork trial instead: [[ling-3.0-flash-trial]].

-The trial service (`frank-dragoon-ling`) is retired and disabled under the
-boot policy (2026-08-17). `frank-dragoon` is the Qwen3.8-27B PP
-head, not the Ling path. See git history for the full decision table.
+The trial service (`frank-dragoon-ling`) was retired and disabled under the boot
+policy on 2026-08-17, in the same redo that renamed `frank-dragoon` to the
+Qwen3.8-27B PP head ([[frank-redo-2026-08-17]]). Two fleet generations later the
+name belongs to the halogen flash-server endpoint ([[frank-dragoon]]). See git
+history for the full decision table.
@@ -14,54 +14,28 @@ timestamp: 2026-09-03
 # Qwen3.8-27B Speculative-Decoding Matrix

 This page records the speculative-decoding benchmark history and the
-production tuning for the Qwen3.8-27B service (`frank-dragoon`),
-now a pipeline-parallel head with a Dankserver worker.
+production tuning for the then-Qwen3.8-27B service (`frank-dragoon`), at the
+time a pipeline-parallel head with a Dankserver worker.

-## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68)
+## Production state (2026-08-17)

-Current production, verified against the live process and journal
-(2026-08-17, load 02:06 + smoke tests). Topology detail:
-[[Pipeline Parallelism/Qwen3.8 PP Deployment]].
-
- Devices: `RPC0` (Dankserver RTX 3090) + `ROCm0` (Strix iGPU). The head
-  passes no `--device` flag. Default selection puts RPC first and picks
-  ROCm for the iGPU (HIP registers before Vulkan). The Vulkan0 side stays
-  an option via `HEAD_BIN=.../build-vulkan-rpc`.
- Model: `Qwen3.8-27B-Uncensored-Q6_K.gguf` (JonathanColetti, 22.43 GB,
-  imatrix, 65 blocks, 866 tensors, NO bundled MTP head). The repo's
-  standalone draft sidecar `Qwen3.8-27B-Uncensored-draft-Q8_0.gguf`
-  (3.16 GB) is loaded via `--spec-draft-model`; it runs on ROCm0 only
-  (1718 MiB, zero worker impact).
- Spec: **MTP** (standalone draft) `--spec-draft-n-max 4` + the
-  `--spec-default` ngram-mod fallback (48/64/24). Measured acceptance
-  0.83 then 0.75, mean len 4.0-4.3, per-position ~0.86/0.71/0.71/0.71 —
-  stronger than the old bundled head (0.739/0.554/0.427/0.340).
- KV: K **q8_0** + V **f16** (mismatched). 3136 B/token per FA layer =
-  784 MiB at 262144 (K 1088 + V 2048). RPC0 8624 MiB (11 FA), ROCm0
-  3920 MiB (5 FA); total 12544 = 16 x 784. **Flash Attention stays
-  enabled with the mismatch** (`resolve_fused_ops: Flash Attention
-  enabled`) via the GGML_CUDA_FA_ALL_QUANTS rebuild of both binaries
-  at 9b05354ec — see the PP doc.
- Split: `--tensor-split 0.68,0.32` = 45 of 65 blocks (0-44) on the
-  3090, 69% of the layer weight. The maximum: 46 blocks -> ~0.3 GiB
-  margin; 48 blocks (f >= 0.7197) adds FA block 47 -> KV 9408 -> OOM by
-  ~0.85 GiB. Boundary math: the PP doc.
- Context: pool 262144, 1 slot, `n_ctx_slot = 262144`. The pi
-  `models.json` declares 262144 (contextWindow = maxTokens, house
-  convention).
- Worker steady state: 23854/24576 MiB used on the 3090, ~273 MiB free
-  (tight but stable; all allocations happen at load).
- Batch 8192 / ubatch 256, `--direct-io`, `--fit off`,
-  `--gpu-layers all`.
+The pipeline-parallel shape this matrix was tuned for — Q6_K target, standalone
+Q8_0 draft on ROCm0 only, `--tensor-split 0.68,0.32` (45 of 65 blocks), K `q8_0`
+/ V `f16`, pool 262144 at one slot, worker steady at 23854/24576 MiB — is
+recorded once, in [[Pipeline Parallelism/Qwen3.8 PP Deployment]] §Production
+state, together with the split-limit arithmetic (why 45 blocks is the maximum),
+the KV-per-layer bill and the FA-all-quants rebuild the mismatch needed. That
+era's endpoint identity is an [[frank-dragoon]] era. This page keeps only the
+speculative-decoding measurements.

 ## How MTP works on Qwen3.8 (bundled-head era)

 The unsloth `UD-*` GGUFs keep the MTP (Multi-Token Prediction) head inside
 the model file. llama.cpp b10411+ auto-detects it and creates the draft
 context from the target model. Do not pass `--spec-draft-model` for those
-files. This is superseded: the production `Qwen3.8-27B-Uncensored-Q6_K.gguf`
-has NO bundled head, and the serve script passes the standalone draft
-sidecar with `--spec-draft-model` (see Production state above).
+files. This is superseded: the production target of 2026-08-17 carried no
+bundled head, and the serve script passed the standalone draft sidecar with
+`--spec-draft-model` (shape: [[Pipeline Parallelism/Qwen3.8 PP Deployment]]).

 The stock llama.cpp build must be b10411 or newer. The 2026-08-14 rebuild
 on Dankclaw is b10433 (`9b05354ec`) with `GGML_HIP` + `gfx1151` + Vulkan.
@@ -83,14 +57,14 @@ standalone configuration only; the PP head runs ROCm0.
  6/ABRT, systemd restart, ~503 during reload). DSpark reuses the DFlash
  block-diffusion machinery, which shares one draft context across
  parallel slots; concurrent block drafting faults in HIP stream sync.
-  This is the same multi-slot bug class that retired `draft-dflash`
-  (premature EOS). The crash shows on 16k concurrent streams, not on a
-  67k pair — concurrency is the trigger, not size. Fix: PARALLEL=1
-  (requests queue safely). Retired the same day.
+  This is the same multi-slot bug class that retired `draft-dflash` (premature
+  EOS, retired 2026-08-14). The crash shows on 16k concurrent streams, not on a
+  67k pair — concurrency is the trigger, not size. Fix: PARALLEL=1 (requests
+  queue safely). Retired the same day.
 - **2026-08-15 to 08-16: `--spec-default` (ngram-mod), Vulkan0** — n_match
  24 / n_min 48 / n_max 64, draftless, PARALLEL=2 (2 x 32k parallel stream
  test passes). Chosen for parallel safety after DSpark; kept as the
-  fallback under MTP in the current config.
+  fallback under MTP in the 2026-08-17 shape.

 ## Benchmark Method

@@ -146,19 +120,19 @@ ubatch 256, single slot, on scratch port 7892.

 Verdict: at very high context, Vulkan0 decodes **48% faster** (5.68 vs
 3.83 t/s); ROCm0 prefills 12% faster.  Generation is the interactive cost
-(users stream tokens). This verdict applies to the standalone
-configuration; the PP head runs ROCm0 (HIP registers before Vulkan).
+(users stream tokens). This verdict applies to the standalone configuration; the PP
+head runs ROCm0 — HIP registers before Vulkan and the iGPU dedup rule (the llama.cpp
+PR #23897 workaround) keeps the first backend's device.
 Both `tg_3s` values were rock-steady across the run (3.83 and 5.67 t/s).
 High-context decode collapses on both backends versus ~25 t/s at 16k ctx;
 that is the long-context attention penalty, not a spec artifact.

-## Production Configuration
+## Where the tunables live

-The live tunables live in `scripts/frank-dragoon.env` (head) and
-`scripts/frank-pp-worker.env` (worker) in this repo. The Production
-state section at the top of this page records the current values: Q6_K
-model + standalone MTP draft, split 0.68/0.32, 262144 ctx, PARALLEL=1,
-q8_0/f16 KV.
+That era's head and worker env/units are listed (and marked deleted) in
+[[Pipeline Parallelism/Qwen3.8 PP Deployment]] §Files. `scripts/frank-dragoon.env`
+is no longer a PP-head env at all — it is the halogen-flash-server trio of the
+current era, which [[frank-dragoon]] owns.

 ## See Also

@@ -28,7 +28,7 @@ Related pages:
 | 2 | MTP `n_max` explicit 4–5 + Q4_0 draft head | +5–15% on the spec share | restart, 1.56 GB file | test 08-16 |
 | 3 | Plain `Q8_0` A/B vs UD-Q8_K_XL (same 8 bits) | dequant ALUs may differ | restart, 29 GB file | test 08-16 |
 | 4 | Prefill path: ROCm backend + tuned ubatch/threads-batch | 2–3× prefill for agentic turns | per-workload backend choice | partial data |
-| 5 | TTM `pages_limit` raised to 114 GiB | removes the hard VRAM ceiling; the full 262k f16 KV plus the Q8 AGENT payload, MTP draft, and mmproj fit in one slot | modprobe, set live 2026-08-25 | applied (29,883,536 pages) |
+| 5 | TTM `pages_limit` raised | removes the hard VRAM ceiling; the full 262k f16 KV plus the Q8 AGENT payload, MTP draft, and mmproj fit in one slot | modprobe, set live 2026-08-25 | applied — value and file: [[Dankclaw System Specs]] (Memory) |
 | 6 | Vulkan ICD pinning + loader-layer disable | robustness; small | service env vars | test 08-16 |

 Everything else (quantization downgrade, NVFP4, larger drafts) is documented
@@ -62,32 +62,32 @@ Consequences of the hybrid design:
   cannot be prefix-cached the way attention KV can. Every agentic turn
   re-prefills the whole conversation. **Prefill is the hidden cost of
   agentic use**, and it is the dominant cost we have not yet optimized.
-3. **MTP is native and strong.** The model ships a nextn head; our matrix
-   measured 13.3 t/s vs 7.2 t/s (no spec) at 16k on Vulkan.
+3. **MTP is native and strong.** The model ships a nextn head; on Vulkan at 16k
+   it roughly doubled the no-spec rate (the pair of figures is
+   [[qwen3.8-spec-matrix]]'s realistic-prompt table).

-## Current production state (baseline)
+## Baseline this page optimized (2026-08-15)

-`frank-dragoon` (Dankclaw, Vulkan0, 2026-08-15):
-
- Model: UD-Q8_K_XL, 31.5 GB, bundled MTP head.
- `--spec-default` (ngram-mod) with draft-mtp first in the chain.
- 2 parallel slots × 262144 ctx, KV q8_0, batch 8192, ubatch 256,
-  threads 12, `-dio` off, `--fit off`.
- Measured: ~25 t/s at 16k ctx; 5.68 t/s at 262k ctx; prefill 302 t/s @64k
-  (Vulkan) vs 410 t/s @64k (ROCm, same build).
- GPU: 28 GiB of 96 GiB GTT used. **74 GiB sits idle.**
+`frank-dragoon` then ran llama.cpp on the iGPU with the UD-Q8_K_XL target
+(bundled MTP head), `--spec-default` ngram-mod under a draft-mtp first link, two
+262144 slots, q8_0 KV, batch 8192 / ubatch 256, threads 12. That era's ledger is
+[[frank-dragoon]]'s; what this page keeps is the shape of the problem: the
+service occupied **28 GiB of the then 96 GiB firmware VRAM carve, ~74 GiB
+idle**, decode held ~25 t/s at 16k and collapsed at 262k depth
+([[qwen3.8-spec-matrix]] §High-Context Backend Comparison), and prefill was the
+unoptimized agentic cost (section 4).

 ## The levers, in detail

 ### 1. Pipeline-parallel to the RTX 3090 (biggest win)

-The 3090 on Dankserver is idle (serves a 22 GB MoE small-model service with
-~2.4 GB free). llama.cpp RPC splits the model by **layers**: each device
-runs its layer range, hidden states cross the 2.5 GbE ethernet once per
-token. The per-token transfer is ~41 KiB (5120-wide f32 hidden state, both
-directions); at ~0.5 ms LAN RTT the bandwidth cost is negligible — the split
-is latency-bound, not bandwidth-bound (see
-[Ethernet Bandwidth Math](Pipeline%20Parallelism/Ethernet%20Bandwidth%20Math.md)).
+The 3090 on Dankserver had headroom that night (it carried the day's small-model
+service with a couple of GiB free — that card's eras are [[frank-zealot]]'s).
+llama.cpp RPC splits the model by **layers**: each device runs its layer range,
+hidden states cross the 2.5 GbE ethernet once per token. The per-position wire
+bill is measured in [[Pipeline Parallelism/Ethernet Bandwidth Math]] §0; at
+~0.5 ms LAN RTT the bandwidth cost is negligible — the split is latency-bound,
+not bandwidth-bound.

 Why this is a real speedup and not just "two GPUs": the 3090 does
 matmuls roughly 2–3× faster per layer than the 8060S (936 vs 512 GB/s
@@ -95,58 +95,32 @@ memory, 80 vs 40 CUs). A 50/50 layer split balances per-layer latency, so
 each token step takes ≈ max(half-on-3090, half-on-Strix) + one RTT instead
 of all-64-layers-on-Strix.

-Design decisions (verified in b10433 source, 2026-08-15):
-
- **Head on Dankclaw, worker on Dankserver.** The 31.5 GB model file stays
-  on the head; the worker receives its tensor share over the wire at load
-  (with `-c` it caches the tensors locally after the first load).
- **Head must NOT pass `--device`.** When `params.devices` is set,
-  `llama_prepare_model_devices` builds the device list from exactly those
-  devices and the RPC device is silently dropped (`--rpc` becomes a no-op).
-  Default device selection inserts RPC servers **at the front** of the list.
- **The local iGPU side runs on ROCm, not Vulkan.** Default selection walks
-  registered backends in order; HIP registers before Vulkan, and the iGPU
-  dedup rule (PR #23897 workaround) keeps the first backend's device.
-  Verified live 2026-08-15: device list `[RPC0, ROCm0]`, Vulkan0 never
-  listed. Consequence: the Strix half runs on the ROCm backend — better
-  prefill, ~3% slower decode than Vulkan0 (measured on this box).
-  A second head build (`/home/dank/llama.cpp/build-vulkan-rpc`, HIP off,
-  Vulkan on, RPC on) was built the same night for an A/B; with it the
-  device list becomes `[RPC0, Vulkan0]`.
- **Default split is proportional to *reported free memory*, which is wrong
-  here.** The Strix reports up to 96 GiB (GTT ceiling) vs the 3090's 24 GB,
-  so the default lands ~20/80 in favor of the slow device (verified with a
-  4B smoke test: 4 of 36 layers on the 3090). We pin `--tensor-split 1,1`
-  (50/50) explicitly.
- **`-dio` (direct I/O) on the head.** Large model + RPC load hangs without
-  it (visorcraft Strix finding; matches our 28 GiB load-stall history).
- **Version skew hangs the handshake.** Worker built at the identical
-  commit (9b05354ec / b10433) in a worktree
-  `/home/dank/llama-cpp-rpc` on Dankserver — production b10121 build
-  untouched.
-
-Smoke test (2026-08-15, 4B model, 0.2/0.8 split, production untouched):
-worker VRAM rose ~819 MiB (real split), 95 t/s prefill, 58.8 t/s decode on
-the tiny model. RTT on the link: ~0.3–0.5 ms.
-
-Full design, scripts, and morning runbook:
-[Qwen3.8 PP Deployment](Pipeline%20Parallelism/Qwen3.8%20PP%20Deployment.md).
+Every decision this led to is written once, in
+[[Pipeline Parallelism/Qwen3.8 PP Deployment]] §Gotchas (with the 4B smoke test
+that proved each): head on the box that already holds the model file, **no
+`--device` on the head** (setting it silently drops RPC), the fixed
+HIP-before-Vulkan device selection, the reported-free-memory split trap, the
+identical-build requirement, and `-dio` on the head — the last one visorcraft's
+Strix finding, matching our own 28 GiB load-stall history. Scripts and the
+morning runbook are on the same page.

 ### 2. Speculative-decoding tuning (same bits, free tokens)

- **MTP `n_max`.** Production leaves `--spec-draft-n-max` unset. Our matrix
-  (Vulkan, 16k) found 13.3 t/s at the tested setting vs 7.2 without spec;
-  the published Strix tuning curve peaks at n_max 3–4 (30.3 t/s at 4 vs
-  28.9 at 3 vs 29.9 at 5). The PP env sets **4** explicitly; the matrix
-  re-tests 3/4/5 on the real split.
+- **MTP `n_max`.** Production left `--spec-draft-n-max` unset; the fleet matrix
+  (Vulkan, 16k, realistic prompt) found the depth-5 peak against the no-spec
+  baseline — both figures are [[qwen3.8-spec-matrix]]'s realistic-prompt table.
+  The published Strix tuning curve peaks at n_max 3–4 (30.3 t/s at 4 vs 28.9 at
+  3 vs 29.9 at 5). The PP env set **4** explicitly; the matrix re-tested 3/4/5
+  on the real split.
 - **Q4_0 draft head.** The published Strix card measured a standalone
  Q4_0 MTP head ~6–10% faster than Q8_0 for the same acceptance (the draft
  head is a 248k-vocab output projection; cheaper dequant, same accuracy).
  The bundled head inside UD-Q8_K_XL is Q8-class; the standalone
  `ggml-org/mtp-Qwen3.8-27B-Q4_0.gguf` (1.56 GB) is being downloaded for
  an A/B via `--spec-draft-model`.
- **Ngram-mod fallback stays.** It is the only spec type that survived
-  parallel load (DSpark crashed ROCm, DFlash corrupts multi-slot).
+- **Ngram-mod fallback stays.** It is the only spec type that survived parallel
+  load; the DSpark ROCm fault and the DFlash multi-slot corruption are timed and
+  explained on [[qwen3.8-spec-matrix]].

 ### 3. Same-bit quant A/B: UD-Q8_K_XL vs plain Q8_0

@@ -162,10 +136,10 @@ zero quality cost.
 Every agentic turn re-prefills the whole conversation (no prefix caching for
 hybrid state, upstream). Levers:

- **Backend choice per workload.** Measured @64k on this box (b10433):
-  ROCm 410 t/s vs Vulkan 302 t/s prefill; Vulkan 25.1 vs ROCm 24.3 t/s
-  decode. For chat-heavy use: Vulkan. For agentic/batch-heavy use: ROCm.
-  The PP head (ROCm side) gets this for free.
+- **Backend choice per workload.** Measured @64k on this box (b10433, the era
+  [[frank-dragoon]] owns): ROCm 410 t/s vs Vulkan 302 t/s prefill; Vulkan 25.1
+  vs ROCm 24.3 t/s decode. For chat-heavy use: Vulkan. For agentic/batch-heavy
+  use: ROCm. The PP head (ROCm side) gets this for free.
 - **ubatch/threads-batch.** Production runs ubatch 256. Larger ubatches
  (512/1024) fill the iGPU better during prefill; `--threads-batch` for
  the CPU-bound tail. Untested at prefill scale on this box — matrix item.
@@ -175,14 +149,14 @@ hybrid state, upstream). Levers:

 ### 5. OS level (from the Improvements doc, re-verified 2026-08-15)

- **TTM `pages_limit` is now 114 GiB (29,883,536 pages)** — applied 2026-08-25.
-  Written live to `/sys/module/ttm/parameters/pages_limit` (no reboot required)
-  and persisted in `/etc/modprobe.d/ttm.conf`. The full 262k f16 KV cache plus
-  the ~28 GB Q8 AGENT payload, MTP draft, and mmproj now fit in one slot. The
-  load-stall risk from the old 15 GiB cap is resolved. Note: 96 GiB is the
-  driver-reported VRAM carve-out (VGM/SVM pool); the TTM pages_limit is the
-  kernel page map that was the real ceiling.
- `iommu=pt` — verified present on the kernel cmdline. Done.
+- **TTM `pages_limit`** — the raise itself (value, live write, modprobe file,
+  date) is one row on [[Dankclaw System Specs]] (Memory). What matters to this
+  page is the mechanism: the driver-reported 96 GiB VRAM carve-out (VGM/SVM
+  pool) was never the ceiling — the TTM kernel page map was, and until it was
+  raised a 262k f16 KV cache plus a ~28 GB payload could not load in one slot.
+- **Boot line** — IOMMU policy on this host is [[dankclaw-iommu-translation-off]]'s
+  and [[Dankclaw System Specs]]'s; it has changed twice since this lever was
+  written, so no era-scoped reading of `/proc/cmdline` belongs here.
 - **Vulkan ICD pinning** — `radeon_icd.json` exists; no `VK_ICD_FILENAMES`
  or loader-layer env is set. Pinning the radeon ICD and disabling loader
  layers (`VK_LOADER_LAYERS_DISABLE=all`) is the recommended Strix
@@ -240,8 +214,8 @@ Per config: `pp-test.sh bench 4` (4k prompt / 256 out) and `bench 16`
 (16k / 256). Report: decode t/s (server print_timing), prefill t/s, MTP
 acceptance, worker VRAM, head GTT usage.

-Success bar: **≥ 30 t/s at 4k and ≥ 20 t/s at 16k decode** (vs 25 t/s
-production @16k) with the same 8-bit model.
+Success bar: **≥ 30 t/s at 4k and ≥ 20 t/s at 16k decode**, against the
+production @16k bar in the Baseline section, with the same 8-bit model.

 ## Sources

@@ -8,8 +8,9 @@ timestamp: 2026-10-05T00:00:00-07:00

 # vLLM Boot Forensics and Config Interrogation

-For serve-script vLLM endpoints (frank-archon/qwen-fp8 on dankserver, any
-`podman rm -f` + restart service).
+For serve-script vLLM endpoints (the vLLM eras of [[frank-archon]] — that page's
+Eras table is which ones; any endpoint that runs `podman rm -f` + a restart
+service).

 ## 1. Capture the true cause — the journal, not `podman logs`

@@ -41,7 +41,7 @@ From the 111 GiB GGUF (`mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M`,
 - 48 layers, of which **12 are full attention** (indices `[3,7,11,15,19,23,27,31,35,39,43,47]`,
  every 4th — `full_attention_interval = 4`) and 36 are GDN/linear.
 - Only the 12 attention layers carry KV. At the engine's measured 28 KiB per
-  position, a 200k-token conversation reads **5.5 GiB of KV per token**.
+  position, a 200k-token conversation holds **5.5 GiB of KV**.

 - A layer split moves **whole layers**, so it moves ~1.62 GiB of experts with
  every layer (75 GiB / 48). Attention layers are spread across the whole depth
@@ -82,10 +82,12 @@ Real placement in the split runs, sampled from `nvidia-smi` during the run:
 split is genuine, not a mislabelled single-device run (see Finding 5 for why
 that check is mandatory here).

-Reference points on the same silicon: halogen (deployed) ≈ **25 tok/s** at real
-contexts; the retired ik hybrid 3.8–4.6 tok/s; mainline with the trunk on the
-3090 and CPU experts 1.01 tok/s (2026-09-15); the CPU expert path's prefill
-ceiling 192 tok/s.
+Reference points, all era readings rather than measurements of this run: the
+deployed halogen shape at ≈ **25 tok/s** at real contexts and the retired ik
+hybrid at 3.8–4.6 tok/s (both are [[frank-dragoon]] eras, and that page's Eras
+rows are where those shapes live); mainline with the trunk on the 3090 and CPU
+experts at 1.01 tok/s (2026-09-15, this page's own earlier pass); the CPU expert
+path's prefill ceiling at 192 tok/s.

 ## Finding 1 — per-tensor placement across devices is killed by one flag: `-ot`

@@ -248,15 +250,16 @@ distinguishes a real split from a mislabelled sweep.
 ## Conclusion

 The fleet's answer is unchanged — **two independent endpoints**
-([[frank-dragoon]] on the iGPU via halogen,
-[[frank-zealot]] holding a whole 27B in the 3090's 24 GB) — but two
-claims from the first round of this experiment are now corrected:
+([[frank-dragoon]] on the iGPU, [[frank-zealot]] alone on the 3090; each page
+owns what its card holds) — but two claims from the first round of this
+experiment are now corrected:

 1. **Cross-vendor layer split is possible** (measured: CUDA0 + Vulkan1, 27.23
   tok/s and 268.5 t/s prefill at 64k). It is not a regime change (+7 % decode),
-   and it costs the 3090 ~15 GiB — which this fleet spends on the vision endpoint
-   (`frank-zealot` holds 22.4 of the card's 24.6 GiB). An operator who wants the
-   7 % must give up the fleet's only image-capable endpoint.
+   and it costs the 3090 ~15 GiB — memory this fleet instead spends on the
+   vision endpoint, whose settled footprint is an Eras row on
+   [[frank-zealot]]. An operator who wants the 7 % must give up the fleet's only
+   image-capable endpoint.
 2. **Per-tensor placement across two devices is impossible while `-ot` is used**,
   for a scheduler-mechanical reason, regardless of driver or vendor mix.

@@ -316,11 +319,13 @@ Operational notes for the next run:
 - Restarting frank-dragoon needs the quadlet source back in
  `~/.config/containers/systemd/` plus `daemon-reload`; the generator re-creates
  the unit from that file only. After the restore (and the host reboot, see the
-  landmines) the ledger read `400384 positions, ~27.2 GiB` against a 33.5 GiB
-  budget — no clamp — with 12.3 GiB left for everything else (14.7 GiB before
-  the reboot: the same model, a more fragmented host afterwards — the engine
-  reported 240 free contiguous 2 MiB blocks against 475 earlier). Both endpoints
-  answered `/health` and served `/v1/models` under their names.
+  landmines) its boot ledger read `400384 positions, ~27.2 GiB` against a 33.5
+  GiB budget — no clamp — with 12.3 GiB left for everything else (14.7 GiB
+  before the reboot: the same model, a more fragmented host afterwards — the
+  engine reported 240 free contiguous 2 MiB blocks against 475 earlier). That
+  ledger is an era reading of [[frank-dragoon]]; the block-census lesson it
+  teaches is on [[Llama Cpp Serving Sizing]]. Both endpoints answered `/health`
+  and served `/v1/models` under their names.


 ## Smaller-quant check: Q4_K_S against the recorded Q4_K_M numbers
@@ -393,10 +398,10 @@ expert mass) — 580 × 3 MiB slabs:
 | *the same bytes computed in host RAM at 256 GB/s* | 7.1 ms | the link is **31× slower** than the memory behind it |

 Against this host's measured alternatives — **20.46 tok/s** for llama.cpp with
-the experts *computed* on the CPU (link carries activations only), 25 for
-halogen, 26.5 for mainline on the iGPU — an engine that *streams* experts across
-this link loses by 3–5×, and the port's x8 ceiling would only double it (≈7–9
-tok/s).
+the experts *computed* on the CPU (link carries activations only, config B′
+above), 25 for the halogen shape deployed that day ([[frank-dragoon]]), 26.5 for
+mainline on the iGPU — an engine that *streams* experts across this link loses
+by 3–5×, and the port's x8 ceiling would only double it (≈7–9 tok/s).

 For the specific engine that raised the question: FreeToken v0.1.2 keeps expert
 weights in host RAM with an LRU GPU cache — exactly this streaming shape — and
@@ -26,8 +26,8 @@ that endpoint's page, not a page.
 |---|---|
 | [[frank-medium-batch-ubatch]] | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… |
 | [[frank-medium-llamacpp-dflash2]] | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… |
-| [[frank-medium-model-swap]] | How to swap the checkpoint behind the frank-medium endpoint (Dankserver :7770, vllm-radiance TP=2) — the streamlining… |
-| [[frank-medium-radiance-mxfp4]] | ACTIVE frank-medium runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
+| [[frank-medium-model-swap]] | RETIRED-era swap procedure for the frank-medium endpoint (`:7770` is now [[frank-archon]]) — the… |
+| [[frank-medium-radiance-mxfp4]] | RETIRED frank-medium era runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
 | [[frank-medium-rocmfpx]] | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… |
 | [[frank-npu]] | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… |
 | [[frank-redo-2026-08-17]] | This page records the full redo of the inference fleet on 2026-08-17. |
@@ -3,23 +3,20 @@
 ## 2026-10-18
 - **frank-zealot resliced to two windows of exact KV (Frank's call, PR #21).**
  `--ctx-size 786432 --parallel 3` at q8_0 K+V → **`524288 / 2` at f16 K+V**, so
-  both remaining conversations keep the native 262,144 window and the
-  compaction hub reads exact K and V. Halving the pool is what funds it: the
-  f16 pair is 32,768 B/token over the 8 full-attention layers = 16,384.00 MiB,
-  while the old 786,432-position pool would have needed ~24,576 MiB of KV alone
-  on a card that leaves 24,576 addressable. Measured boot: KV **16,384.00**,
-  RS **100.50** (2 cells), compute 1,408.06 + 1,088.07, settled `nvidia-smi`
-  **21,176 used / 2,949 free**, `n_ctx_slot = 262144`, `DropInPaths` empty. The
-  doc's old "f16 needs 26,112 MiB and cannot boot" figure was the 27B's
-  34,816 B/token applied to the 9B — corrected in the runbook.
- **Rates are floors, and that is recorded as such.** The endpoint had another
-  consumer for the whole measurement window (56 prompts on the other slot in
-  three minutes, 96 % GPU-util), so single-stream decode came back 134.1–141.5
-  (145.8/163.2 in a lighter minute) against the 3-slot record of 168.7,
-  two-stream aggregate 256.6–265.1 against ≈300 across three, and prefill
-  **3,159.5 at 106,282** beat the record's 2,877 while contended. Per-stream
-  concurrency improved (≈130 each vs 102.8); a clean re-measure needs the box
-  quiet.
+  both remaining conversations keep the native 262,144 window and the compaction
+  hub reads exact K and V. **Halving the pool is what funds exact KV** — the law
+  that came out of it is that a KV byte rate belongs to the payload it was
+  measured on: the doc's old "f16 needs 26,112 MiB and cannot boot" figure was
+  the 27B's 34,816 B/token applied to the 9B. The measured boot ledger is on
+  [[frank-zealot]] (Current shape), which is also where the 32,768 B/token
+  derivation and the seam's fit gate live.
+  - **Rates are floors, and that is recorded as such.** The measurement window
+    had another consumer on the other slot (56 prompts in three minutes, 96 %
+    GPU-util) — the exact condition the bench window exists to exclude — so every
+    rate taken that evening is recorded on [[frank-zealot]] as a **FLOOR**, not as
+    the shape's rate, with the instruction to re-measure with the box quiet before
+    a figure is carried anywhere. That is the honest shape of a number measured
+    beside live traffic.
 - **Instrument correction (fleet law).** On the PrismML fork build 10777 the raw
  `/completion` path answers **500 "The model produced output that does not
  match the expected Content-only format"** for a greedy `ignore_eos` run, so
@@ -9,9 +9,9 @@
 # rollback). It replaces the FreeToken seam of
 # 2026-10-05 (git tag frank-archon-freetoken-pr587-20261005) and pins what
 # flags alone do not say: the served window clears the harness worst case,
-# the KV pool backs seqs x max-model-len at the declared cache dtype (an
-# engine HARD limit, not a ratio), the per-card VRAM pair stays inside the
-# the pinned-host ledger has a ttm pages cap floor, the serve script ENFORCES
+# the retired per-card budget knobs are ABSENT (the pinned engine refuses
+# them at ExecStart), the pinned-host ledger has a ttm pages cap floor, the
+# serve script ENFORCES
 # it and RAISES the knob itself when a reboot reset it (the cap's default is
 # half of RAM), the served window is what models.yml must
 # quote, the host-networking contract (no --publish), the image gate on a
@@ -146,20 +146,23 @@ nosudo_bin() {
    grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
    grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
    grep -q '^FRANK_ARCHON_SPEC_TOKENS=2$' "$ENV_FILE"
-    # expert_tiered placement: vram_weights + the pinned host pool serve the
-    # per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
-    # KV pool moves the pool with it. KV is QUANTIZED fp8 (the engine takes
-    # only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table is pinned
-    # in host RAM per Frank's law.
+    # expert_tiered placement: the pinned host pool plus whatever expert VRAM the
+    # budget resolver leaves serve the per-rank resident set TOGETHER, so the pool
+    # size is also the lever on which experts stay on the card. KV is QUANTIZED
+    # fp8 (the engine takes only bf16|fp8 — no per-tensor K/V split), the PLE
+    # n-gram table is pinned in host RAM per Frank's law. The per-card VRAM pair
+    # is NOT asserted here: 1.3.0 removed it and the resolver derives the split,
+    # so the pool's token count is a boot-ledger line — verified live by the
+    # capacity leg of scripts/probe-frank-archon-kv.py at every cutover.
    grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
    grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
-    grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
-    grep -q '^FRANK_ARCHON_VRAM_KV_MIB=[1-9]' "$ENV_FILE"
    grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
    grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
-    # Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
-    grep -q '^FRANK_ARCHON_PREFIX_HOST_MIB=8192$' "$ENV_FILE"
-    grep -q '^FRANK_ARCHON_PREFIX_DISK_MIB=65536$' "$ENV_FILE"
+    # Prefix cache: pinned host RAM only (the disk tier was removed), and the
+    # step-sharing cap is stated explicitly — auto vs off is the difference
+    # between decode waiting one chunk and waiting a whole step.
+    grep -q '^FRANK_ARCHON_PREFIX_CACHE_MIB=8192$' "$ENV_FILE"
+    grep -q '^FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto$' "$ENV_FILE"
    # The pinned ledger (2 x pool + 47.7 GiB PLE + prefix host) must be
    # under the ttm pages cap, and the env states the floor the serve
    # script enforces (>= 140,000 MiB — the kernel's own default cap, half of
@@ -170,7 +173,7 @@ nosudo_bin() {
    ttmt=$(awk -F= '/^FRANK_ARCHON_TTM_TARGET_MIB=/{print $2}' "$ENV_FILE")
    [ "$ttmt" -gt "$ttmm" ]
    pool=$(awk -F= '/^FRANK_ARCHON_HOST_POOL_MIB=/{print $2}' "$ENV_FILE")
-    ph=$(awk -F= '/^FRANK_ARCHON_PREFIX_HOST_MIB=/{print $2}' "$ENV_FILE")
+    ph=$(awk -F= '/^FRANK_ARCHON_PREFIX_CACHE_MIB=/{print $2}' "$ENV_FILE")
    # 2 pools + PLE 47.7 GiB (48,824 MiB) + prefix host, rounded up.
    ledger=$(( 2 * pool + 48824 + ph + 2048 ))
    [ "$ttmm" -ge "$ledger" ]
@@ -179,47 +182,21 @@ nosudo_bin() {
    [ "$(awk -F= '/^FRANK_ARCHON_RAD_MIN_MIB=/{print $2}' "$ENV_FILE")" -gt 0 ]
 }

-@test "the KV pool backs seqs x max-model-len at the declared cache dtype" {
-    # The engine's own hard limit: a request longer than the pool's token
-    # count is "accepted by the API and then refused as too large to ever fit
-    # -- this is a hard limit, not a backing ratio". Rates are measured, not
-    # derived: per rank, kv_attn (13 layers) pages 26.00 KiB per 4 tokens at
-    # fp8 and 52.00 KiB at bf16, while kv_qsa_bkey (13 layers) pages 3.25 KiB
-    # at BOTH — the QSA block key is an index, not attention state, and does
-    # not follow --kv-cache-dtype — so the rate is 7,488 B/token at fp8 and
-    # 14,144 at bf16 (1.89x, not 2x). The dtype-independent terms also live in
-    # the pool: ~110 MiB a sequence of recurrent state (kv_gdn_state 54 MiB x 2
-    # copies + conv/tail/ple) and 448 MiB of prefix checkpoints (8 x 55.99).
-    # Verified against BOTH live boots: fp8 3543-330-448 = 2765 MiB -> 387,240
-    # tokens (ledger 386,904); bf16 6912-220-448 = 6244 MiB -> 462,946
-    # (ledger 462,752). Both within 0.1%.
-    kv=$(awk -F= '/^FRANK_ARCHON_VRAM_KV_MIB=/{print $2}' "$ENV_FILE")
-    seqs=$(awk -F= '/^FRANK_ARCHON_MAX_SEQS=/{print $2}' "$ENV_FILE")
-    maxlen=$(awk -F= '/^FRANK_ARCHON_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE")
-    case "$(awk -F= '/^FRANK_ARCHON_KV_DTYPE=/{print $2}' "$ENV_FILE")" in
-        fp8)  bpt=7488 ;;
-        bf16) bpt=14144 ;;
-        *) fail "unknown FRANK_ARCHON_KV_DTYPE" ;;
-    esac
-    tokens=$(( (kv - 110 * seqs - 448) * 1048576 / bpt ))
-    [ "$tokens" -ge $(( seqs * maxlen )) ]
-}
-
-@test "the per-card VRAM pair stays inside the measured card ceiling" {
-    # The engine's OWN budget table, not a hand-picked reserve. Boot of
-    # 2026-10-13 at the pair 24000/5120: card 32,624 MiB (31.86 GiB), budgets
-    # claimed 29,120, `3.42 GiB is neither budgeted nor accounted — the
-    # driver's context, the activation arena and --gpu-headroom-mib come out of
-    # it; anything past those is throughput on the floor`, and `card 0 settled
-    # at 2.01 GiB free against 209.19 MiB (the 96.00 MiB --gpu-headroom-mib and
-    # 113.19 MiB kept for the static region)`. Claimable reserve above the
-    # engine's floor: 2,058 - 209 = 1,849 MiB, so the ceiling is 30,969 MiB.
-    # The 2026-10-11 bound of 29,120 ("keep >= 1 GiB free", card read as 31,856
-    # MiB) was MY reserve, which the engine never asked for; the shipped pair
-    # spends the difference and lands ~458 MiB above the floor.
-    w=$(awk -F= '/^FRANK_ARCHON_VRAM_WEIGHTS_MIB=/{print $2}' "$ENV_FILE")
-    kv=$(awk -F= '/^FRANK_ARCHON_VRAM_KV_MIB=/{print $2}' "$ENV_FILE")
-    [ $(( w + kv )) -le 30969 ]
+@test "the retired budget knobs are absent from the env and the argv" {
+    # Not unused leftovers: the pinned engine REFUSES to start on any of them
+    # (--vram-weights-mib/--vram-kv-mib went away at 1.3.0, the prefix-cache disk
+    # tier outright), and a unit that refuses at ExecStart sits in
+    # `activating (auto-restart)` while the port still looks alive from whatever
+    # held it last. A stale line here is an outage, so it is asserted away.
+    for key in FRANK_ARCHON_VRAM_WEIGHTS_MIB FRANK_ARCHON_VRAM_KV_MIB \
+               FRANK_ARCHON_PREFIX_HOST_MIB FRANK_ARCHON_PREFIX_DISK_MIB; do
+        ! grep -q "^${key}=" "$ENV_FILE"
+        ! grep -q "$key" "$SERVE"
+    done
+    for flag in --vram-weights-mib --vram-kv-mib --prefix-cache-host-mib \
+                --prefix-cache-dir --prefix-cache-disk-mib; do
+        ! grep -q -- "$flag" "$SERVE"
+    done
 }

@test "the serve script wires the engine flags from the env" {
@@ -232,11 +209,13 @@ nosudo_bin() {
    grep -q -- '--max-model-len "$FRANK_ARCHON_MAX_MODEL_LEN"' "$SERVE"
    grep -q -- '--placement "$FRANK_ARCHON_PLACEMENT"' "$SERVE"
    grep -q -- '--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB"' "$SERVE"
-    grep -q -- '--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB"' "$SERVE"
-    grep -q -- '--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB"' "$SERVE"
    grep -q -- '--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE"' "$SERVE"
-    grep -q -- '--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB"' "$SERVE"
-    grep -q -- '--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB"' "$SERVE"
+    grep -q -- '--prefix-cache-mib "$FRANK_ARCHON_PREFIX_CACHE_MIB"' "$SERVE"
+    grep -q -- '--busy-prefill-chunk "$FRANK_ARCHON_BUSY_PREFILL_CHUNK"' "$SERVE"
+    # Interleave is opt-in: it reaches argv only through the guarded array, so a
+    # 0 in the env leaves the engine's own default in place.
+    grep -q -- 'PREFILL_ARGS=(--prefill-interleave "$FRANK_ARCHON_PREFILL_INTERLEAVE")' "$SERVE"
+    grep -qF -- '"${PREFILL_ARGS[@]}"' "$SERVE"
    grep -q -- '--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT"' "$SERVE"
    grep -q -- '--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS"' "$SERVE"
    # The boot-ladder escape hatch is opt-in: unset renders NO extra argv.
@@ -261,7 +240,10 @@ nosudo_bin() {
    grep -q -- '--device /dev/kfd' "$SERVE"
    grep -q -- '--device /dev/dri' "$SERVE"
    grep -q -- '-v "$FRANK_ARCHON_MODEL_DIR":/models:ro' "$SERVE"
-    grep -q -- '-v "$FRANK_ARCHON_STATE_DIR":/data' "$SERVE"
+    grep -q -- '-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune' "$SERVE"
+    # No /data mount: the prefix cache lives in pinned host RAM now, so the only
+    # host state the container writes is the rad-tune cache above.
+    ! grep -q -- ':/data ' "$SERVE"
    grep -q -- '-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro' "$SERVE"
 }

@@ -4,15 +4,15 @@
 # qwen35 hybrid with 32 layers (8 full-attention + 24 Gated DeltaNet) and the
 # MTP draft head INSIDE the file — via the PrismML fork build of llama.cpp
 # (vendor build 10777), the ONLY loader that registers the payload's custom
-# tensor type 142 and honours its prism.hadamard.* metadata: TWO 262,144-token
-# conversations (--ctx-size 524288 --parallel 2) over an UNQUANTIZED f16 KV
-# cache. Exact KV bills 32,768 B/token here, so the pool is what pays for it —
-# the same dtype at the old 786,432 pool is 24,576 MiB against the 24,123 MiB
-# the driver leaves addressable, and the slot count is the term that moved. The
-# publisher's own Q8_0 vision projector rides along (the BF16 one can crash on
-# concurrent images), and MTP stays OFF: the head is present, but exact KV
-# leaves no room for a draft context that bills f16 KV over the whole pool. Log
-# verbosity 4 ships so the VRAM-fit forensics land in the journal.
+# tensor type 142 and honours its prism.hadamard.* metadata: THREE 200,704-token
+# conversations (--ctx-size 602112 --parallel 3) over an f16 K + q8_0 V cache —
+# 25,088 B/token, 14,406.00 MiB at this pool — with the MTP drafter ARMED
+# (--spec-type draft-mtp). The asymmetry IS the shape: exact V (32,768 B/token)
+# leaves nothing for a draft context that bills its own f16 KV over the WHOLE
+# pool (4,096 B/position, 2,352.00 MiB here), and the 786,432 pool this shape
+# replaced cannot carry a drafter at any dtype inside the 24,123 MiB the driver
+# leaves addressable. The publisher's own Q8_0 vision projector rides along (the
+# BF16 one can crash on concurrent images). Log
 # This seam runs the serve script against a stub engine and asserts on the
 # argv it actually execs, so what is pinned is the command the operator gets.

@@ -143,7 +143,7 @@ run_stage() {
    done
 }

-@test "two conversations share the pool at a 262144 window each" {
+@test "three conversations share the pool at a 200704 window each" {
    serve_argv
    [ "$status" -eq 0 ]
    [ "$(argv_value --alias)" = "frank-zealot" ]
@@ -152,30 +152,36 @@ run_stage() {
    [ "$(argv_value --model)" = "$MODEL_DIR/$MODEL_STUB" ]
    [ "$(argv_value --n-gpu-layers)" -gt 0 ]
    # llama.cpp divides --ctx-size across slots, so the observable contract is
-    # the per-slot window. Exact KV costs the SLOT COUNT, not the window: every
-    # conversation still gets the model's TRAINED 262,144, and the harness
-    # contextWindow must quote that number — the pool and the slot count are
-    # neither, and a longer prompt is a 400, never a truncation.
-    [ "$(argv_value --parallel)" -eq 2 ]
-    [ "$(argv_value --ctx-size)" -eq 524288 ]
-    local pool slots
+    # the per-slot window: harness contextWindow, Prometheus dashboards and every
+    # consumer's compaction trigger read THAT number, and a longer prompt is a
+    # 400, never a truncation. Two invariants the pool/slot split has to keep:
+    # it divides evenly (llama.cpp floors it otherwise) and a slot never exceeds
+    # the model's TRAINED window (there is no YaRN here, so the engine would be
+    # quoting a window it cannot fill).
+    [ "$(argv_value --parallel)" -eq 3 ]
+    [ "$(argv_value --ctx-size)" -eq 602112 ]
+    local pool slots window
    pool="$(argv_value --ctx-size)"; slots="$(argv_value --parallel)"
    [ $((pool % slots)) -eq 0 ]
-    [ $((pool / slots)) -eq 262144 ]
+    window=$((pool / slots))
+    [ "$window" -eq 200704 ]
+    [ "$window" -le 262144 ]
 }

@test "the KV dtype pair follows the env file and flash-attn stays pinned on" {
    # Asserted against the env rather than a literal so a retune moves one
-    # file, not two. The shipped pair is f16/f16: exact attention state, and the
-    # slot count is what makes it affordable — this payload bills 8,192 elements
-    # per token per cache (8 full-attention layers × 4 KV heads × 256 dim), so
-    # exact KV is 32,768 B/token = 16,384.00 MiB at the 524,288 pool, where the
-    # q8_0 pair this file used to ship billed 17,408 B/token = 13,056.00 MiB at
-    # the old 786,432 pool (ledger-exact both times). The 24 DeltaNet layers
+    # file, not two. The shipped pair is ASYMMETRIC — f16 K (the engine's own
+    # default) plus q8_0 V: this payload bills 8,192 elements per token per cache
+    # (8 full-attention layers × 4 KV heads × 256 dim), so K costs 16,384 B/token
+    # and V 8,704 (q8_0 is 1.0625 B/element; there is no true fp8 KV dtype in
+    # this build) = 25,088 B/token = 14,406.00 MiB at the 602,112 pool. Exact V
+    # would bill 32,768 B/token and the all-q8_0 pair this file used to ship
+    # billed 17,408 — the halfway rung is what pays for the third slot AND the
+    # armed drafter (the fit gate below prices both). The 24 DeltaNet layers
    # bill a fixed recurrent state per slot (~50.25 MiB a cell), which is why
    # pool arithmetic alone under-prices the shape. flash-attn is not
-    # decoration: the quantized-V fallback is only legal with it, so it stays
-    # explicit and a dtype retune cannot silently fall back.
+    # decoration: quantized V is only legal with it, so it stays explicit and a
+    # dtype retune cannot silently fall back.
    local env_k env_v
    env_k="$(sed -nE 's/^CACHE_TYPE_K=(.+)$/\1/p' "$ENV_FILE")"
    env_v="$(sed -nE 's/^CACHE_TYPE_V=(.+)$/\1/p' "$ENV_FILE")"
@@ -192,30 +198,47 @@ run_stage() {
    # The fit is arithmetic off the payload geometry plus the non-KV terms
    # measured off the boot ledger (weights 2,036.71 + compute buffers 1,409.02
    # + projector 595 + 248.10 + CUDA context ≈ 460 = 4,749 MiB, dtype- and
-    # slot-independent at this ubatch), plus one ~51 MiB recurrent-state cell
-    # per slot. 24,123 MiB is what the driver leaves addressable (nvidia-smi
-    # reports it as used + free; its total column reads 24,576).
-    local env_k env_v pool slots rate_k rate_v kv total
+    # slot-independent at ubatch 2048), plus one ~51 MiB recurrent-state cell per
+    # slot. That constant is deliberately NOT credited with the ~1,306 MiB the
+    # shipped ubatch 1024 hands back: the gate stays conservative.
+    # An ARMED drafter is billed, because it is the term this shape pays
+    # CACHE_TYPE_V=q8_0 to afford: the draft context is built against the
+    # TARGET's whole pool at its own f16 rate (measured 4,096 B/position at
+    # blk.32) plus buffers and its own recurrent cells — the measured spec-ON
+    # against spec-OFF delta at the 786,432 / ubatch-1024 arm was 3,524 MiB
+    # beside 3,072 MiB of draft KV, i.e. ~452 MiB, doubled here for the third
+    # slot's extra cells. 24,123 MiB is what the driver leaves addressable
+    # (nvidia-smi reports it as used + free; its total column reads 24,576).
+    local env_k env_v pool slots rate_k rate_v kv total spec f16 exact_pool exact_v
    env_k="$(sed -nE 's/^CACHE_TYPE_K=(.+)$/\1/p' "$ENV_FILE")"
    env_v="$(sed -nE 's/^CACHE_TYPE_V=(.+)$/\1/p' "$ENV_FILE")"
    pool="$(sed -nE 's/^CTX_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")"
    slots="$(sed -nE 's/^PARALLEL=([0-9]+)$/\1/p' "$ENV_FILE")"
+    spec="$(sed -nE 's/^SPEC_TYPE=(.+)$/\1/p' "$ENV_FILE")"
    rate_k="$(kv_bytes_per_token "$env_k")"
    rate_v="$(kv_bytes_per_token "$env_v")"
+    f16="$(kv_bytes_per_token f16)"
    kv="$(( pool * (rate_k + rate_v) / 1048576 ))"
    total="$(( kv + 4749 + 51 * slots ))"
+    if [ "$spec" != "none" ]; then
+        total="$(( total + pool * 4096 / 1048576 + 768 ))"
+    fi
    # The shipped arm has to leave real room: the projector's worst case is
    # charged AFTER the engine's own fit check, so a projection that lands on
    # the ceiling is a crash waiting for the first image request.
    [ "$total" -le $(( 24123 - 1024 )) ]

-    # And the gate must be able to REFUSE: the same exact KV at the pool this
-    # shape left behind is 24,576 MiB of cache alone — a boot that dies before
-    # /health ever answers. Proving the gate bites is what makes passing one
-    # meaningful.
-    local over
-    over="$(( 786432 * (rate_k + rate_v) / 1048576 + 4749 + 51 * 3 ))"
-    [ "$over" -gt 24123 ]
+    # And the gate must be able to REFUSE, or passing above means nothing. Two
+    # boots this shape can walk straight into: the retired dtype pair at the
+    # retired pool (exact f16 at 786,432 = 24,576 MiB of cache alone, which died
+    # at cudaMalloc), and the shipped pool with V moved back to f16 while the
+    # drafter stays armed — 18,816 MiB of cache plus the draft's 2,352. Both die
+    # before /health ever answers, and no health gate can see either.
+    exact_pool="$(( 786432 * (rate_k + f16) / 1048576 + 4749 + 51 * 3 ))"
+    [ "$exact_pool" -gt 24123 ]
+    exact_v="$(( pool * (rate_k + f16) / 1048576 + 4749 + 51 * slots
+                    + pool * 4096 / 1048576 + 768 ))"
+    [ "$exact_v" -gt 24123 ]
 }

@test "log verbosity follows the env file" {
@@ -277,48 +300,54 @@ run_stage() {
    argv_value --ubatch-size | grep -qE '^[0-9]+$'
 }

-@test "speculation is off: no drafter flag reaches the argv" {
-    # Frank, 2026-10-18: this file DOES carry the MTP head (the boot arms prove
-    # it loads and drafts, acceptance 0.798/0.617 per position, mean accepted
-    # length 2.41) and it is still off, because the draft context bills its own
-    # f16 KV over the WHOLE pool — 4,096 B/position × 524,288 = 2,048 MiB plus
-    # buffers — on a trunk that already projects 21,234 MiB with exact KV. The
-    # MTP arms in frank-zealot.env are all q8_0-era measurements; the doors back
-    # to the drafter are one window per conversation, or KV back to q8_0 with
-    # UBATCH_SIZE=1024 (213.5 t/s). The flags move as a set:
-    # SPEC_TYPE=none strips --spec-type, --spec-draft-n-max AND
-    # --spec-draft-model together, so a no-spec boot cannot carry half a spec
-    # pair (the ROCmFPX-era baseline arm segfaulted at load doing that).
+@test "the armed drafter ships the whole MTP set and no second copy of the trunk" {
+    # Frank, 2026-10-19: the drafter is ON. The head is IN this file
+    # (qwen35.nextn_predict_layers = 1, blk.32.nextn.*) and it accepted well
+    # when it was measured off (0.798/0.617 per position, mean accepted length
+    # 2.41); what it now pays for is CACHE_TYPE_V=q8_0 plus UBATCH_SIZE=1024,
+    # which is why the dtype pair and the batch pair are pinned next door.
+    # The observable contract is the SET: --spec-type plus every --spec-draft-*
+    # the env names, and NO --spec-draft-model — the MTP draft rides the MAIN
+    # model's weights, so a draft file would load the trunk a second time.
+    local n_max
+    n_max="$(sed -nE 's/^SPEC_DRAFT_N_MAX=([0-9]+)$/\1/p' "$ENV_FILE")"
    serve_argv
    [ "$status" -eq 0 ]
+    [ "$(argv_value --spec-type)" = "draft-mtp" ]
+    [ "$(argv_value --spec-draft-n-max)" = "$n_max" ]
+    [ "$(argv_value --spec-draft-n-min)" = "0" ]
+    [ "$(argv_value --spec-draft-p-min)" = "0" ]
+    ! grep -qxF -- '--spec-draft-model' <<<"$output"
+}
+
+@test "the rollback knob strips the entire spec set" {
+    # SPEC_TYPE=none is the documented retreat (and the shape this endpoint
+    # shipped before 2026-10-19). The law it exists to obey: the flags move as a
+    # SET — a no-spec boot that keeps --spec-draft-n-max, or a baseline arm that
+    # hands over --model-draft with no --spec-type, has segfaulted at load with
+    # no diagnostic. So the rollback is proven by ABSENCE of all of them.
+    serve_argv
+    sed -i 's/^SPEC_TYPE=.*/SPEC_TYPE=none/' "$STAGE/test.env"
+    run_stage
+    [ "$status" -eq 0 ]
    ! grep -qxF -- '--spec-type' <<<"$output"
    ! grep -qxF -- '--spec-draft-n-max' <<<"$output"
+    ! grep -qxF -- '--spec-draft-n-min' <<<"$output"
+    ! grep -qxF -- '--spec-draft-p-min' <<<"$output"
    ! grep -qxF -- '--spec-draft-model' <<<"$output"
-    grep -qE '^SPEC_TYPE=none$' "$ENV_FILE"
 }

-@test "naming a drafter ships the MTP set, with or without a separate draft file" {
-    # The documented MTP arm (frank-zealot.env): SPEC_TYPE=draft-mtp ALONE is
-    # legal here because the head is IN the checkpoint and the draft context
-    # rides the main model's weights — so --spec-draft-model must NOT appear,
-    # or the trunk would be loaded twice.
+@test "an external drafter still reaches the engine as a complete set" {
+    # The other arm the ROCmFPX-era crash was about: a named draft FILE must
+    # arrive alongside --spec-type and the depth recipe, never alone. This
+    # endpoint does not use it (the head is in the trunk), so the knob ships
+    # empty and only this test sets it.
    serve_argv
-    {
-        echo 'SPEC_TYPE=draft-mtp'
-        echo 'SPEC_DRAFT_N_MAX=2'
-    } >> "$STAGE/test.env"
-    run_stage
-    [ "$status" -eq 0 ]
-    [ "$(argv_value --spec-type)" = "draft-mtp" ]
-    [ "$(argv_value --spec-draft-n-max)" = "2" ]
-    ! grep -qxF -- '--spec-draft-model' <<<"$output"
-
-    # And an external drafter still reaches the engine as a complete set — the
-    # pair the ROCmFPX-era crash was about.
    echo "SPEC_DRAFT_FILE=$MODEL_DIR/$DRAFT_STUB" >> "$STAGE/test.env"
    run_stage
    [ "$status" -eq 0 ]
    [ "$(argv_value --spec-draft-model)" = "$MODEL_DIR/$DRAFT_STUB" ]
+    [ "$(argv_value --spec-type)" = "draft-mtp" ]
 }

@test "vision ships by default and MMPROJ_FILE empty opts out of the image path" {
Author	SHA1	Message	Date
Frank	9de58ed6fd	feat(frank-archon): move the endpoint to radiance 1.8.1 and its budget model The decode stall is step contention (measured: 18.7 ms decode-only steps against 317 ms steps that carry a prefill chunk, decode at 13 t/s median while prefill runs), and the two engine knobs that fix it arrived after this endpoint's pin. Moving to 1.8.1 (2026-10-18) brings both, and brings two CLI breaks with them: - --vram-weights-mib / --vram-kv-mib were REMOVED at 1.3.0. The per-card split between the expert slab and the KV pool is now derived by the engine's budget resolver from --gpu-headroom-mib, the cache bound (--max-model-len, --max-num-seqs) and --host-pool-mib, and the cache may lend idle blocks to the expert slab. The slab size and the pool's token count are boot-ledger lines from here on, so the capacity contract (3 x 200,000 fp8) is verified live by the probe's capacity leg instead of computed from an env number. - The prefix cache has no disk tier any more: --prefix-cache-dir and --prefix-cache-disk-mib are refusals and --prefix-cache-host-mib is --prefix-cache-mib. 64 GiB of parked KV on NVMe becomes re-prefill, so prefix_hit_rate is the thing to watch after this boot. env: image re-pinned to the 1.8.1 digest; the pair and the disk tier dropped; FRANK_ARCHON_PREFIX_CACHE_MIB replaces the host/disk pair; new FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto (explicit, because auto vs off is the difference between decode waiting one chunk and waiting a whole step) and FRANK_ARCHON_PREFILL_INTERLEAVE=0 (opt-in through a guarded argv array). serve: argv follows the new CLI, /data mount and the pre-created kvcache path go with the disk tier, and the retired env keys are no longer required (they would abort ExecStart). autoresearch-frank-archon.sh takes the same CLI, with a cold prefill now coming from --prefix-cache-mib 0 instead of a per-run disk directory. run-swift15-bf16.sh is untouched: it pins its own engines by design. tests: the two budget-arithmetic seams asserted knobs that no longer exist and are replaced by one seam that asserts the retired knobs are ABSENT from env and argv - a stale line is a boot refusal, not a no-op. The container-contract seam now asserts the tune mount and the absence of /data.	2026-10-19 10:18:24 -07:00
dank	b9d21ae947	Merge pull request 'docs(frank-zealot): boot ledger of `ba69ea5`, corrected ubatch and q8_0 credits' (#26 ) from chore/frank-zealot-mtp-reslice into main	2026-10-19 03:38:40 -07:00
Frank	30e8ed55a0	docs(frank-zealot): record the `ba69ea5` boot ledger and correct the carried credits The boot of record for the new shape, from the verbosity-4 journal plus settled nvidia-smi: target KV 14,406.00 (K f16 9,408.00 + V q8_0 4,998.00), draft KV 2,352.00 over the same 200,704 cells x 3 seqs, recurrent state 452.25, compute 1,665.00 + 424.88 (draft 480.97 + 424.86), weights 2,283.29 + 257.66, projector 595.29 + 248.10 + 24.93, settled 22,870 used / 1,255 free with n_slots = 3 / n_ctx_slot = 200704. Two inherited numbers did not survive it, and both are corrected in the env, the serve rationale and the page: - The ubatch 2048 -> 1024 step was quoted at "-~1,306 MiB". That was the 786,432 / q8_0-pair arm. At THIS shape the target's compute buffers went 2,496.13 -> 2,089.88: 406 MiB, with CUDA0 going UP 257 while CUDA_Host went down 663. q8_0 V (4,410 MiB, not the 3,840 previously written) is what funds the drafter, alone. - The fit gate priced 22,428 against a real 22,870: conservative by 442 MiB, so it stays honest as written. And one finding stated plainly: this is the first boot on this card that misses llama.cpp's own free-memory target - the engine printed "cannot meet free memory target of 1,867 MiB, need to reduce device memory by 102 MiB" and settled at 1,255 MiB free, under the ~2 GiB band every previous era held. Cheapest rungs back are CTX_SIZE=401408 or SPEC_TYPE=none, both one line. Probes on the same boot go in too: chat turn (448 tokens, 1,755 chars of thinking - empty content at max_tokens 64 is the documented trap), tools (finish_reason tool_calls, {"a":24,"b":7}), vision (512 px solid red -> "Red", 280 prompt tokens), and the drafter's own statistics (acceptance 0.706 / 0.763, mean accepted length 2.41 / 2.53).	2026-10-19 03:38:03 -07:00
dank	ba69ea563e	Merge pull request 'feat(frank-zealot): 3 x 200,704, q8_0 V, MTP armed' (#25 ) from chore/frank-zealot-mtp-reslice into main	2026-10-19 03:19:04 -07:00
Frank	a7062d1cb3	docs(frank-zealot): carry the 3 x 200,704 shape, retire the exact-KV era row The endpoint page owns its shape: the flags row, the budget equation behind it (25,088 B/token, 14,406.00 MiB of cache + 2,352.00 MiB of draft cache inside 24,123 MiB addressable), the "MTP is ON" section replacing the "OFF by measurement" one, the harness contract at the per-slot 200,704, and the fallback rungs rewritten as deltas FROM the shipped shape. The 2026-10-18 exact-KV boot (KV 16,384.00 / RS 100.50 / compute 1,408.06 + 1,088.07 / mmproj 868.32 charged after the fit check, settled 21,176/2,949) and its contended rate floors move into that era's Eras row with the one-line rollback, so the two pages that carried them collapse to one. Rates for the new shape are deliberately absent - no bench was ordered, and the fit gate, the health gate and the boot ledger are the evidence a swap needs. AGENTS.md follows the same numbers: contextWindow 200704 is the per-slot window and the trio's shape line.	2026-10-19 03:16:48 -07:00
Frank	25a8b00043	feat(frank-zealot): reslice to 3 x 200,704 with q8_0 V and MTP armed Frank's order, 2026-10-19: --parallel 3 at 200,704 a conversation (pool 602,112), K at the engine default (f16), V to q8_0, batch 1024x6 / ubatch 1024, and the MTP drafter armed explicitly. The three requests are one budget equation. At 8,192 elements per token per cache the pair now bills 25,088 B/token = 14,406.00 MiB over the 602,112 pool, and the armed draft context adds its own f16 KV over that whole pool (4,096 B/position = 2,352.00 MiB) plus buffers and recurrent cells. Dropping V to the 8-bit class frees 4,410 MiB against exact V and the ubatch drop hands back ~1,306 - that is what pays for the third slot AND the drafter, neither of which exact V could fund. The serve script's written-down fallbacks move with the env so a missing env file cannot boot the retired shape, and the seam now: asserts the per-slot contract (divides evenly, <= the trained 262,144), bills an armed drafter in the VRAM fit gate, proves the gate still REFUSES both boots this shape can walk into, pins the shipped MTP set (and the absence of --spec-draft-model, which would load the trunk twice), and pins SPEC_TYPE=none as the rollback that strips the whole set.	2026-10-19 03:16:35 -07:00
dank	a5fe8db877	Merge pull request 'docs(vault): one fact, one owner - de-stale the technique and era pages (C1-C6)' (#24 ) from wt/20261017-181502 into main	2026-10-19 02:06:12 -07:00
Frank	c195c839b4	docs(agents): say a scout verdict is a lead, not a finding The 2026-10-19 vault de-dup campaign sent four read-only scouts over the same pages and had to parent-verify every claim: one inverted era-page law (flagged pages stale for documenting the retired payloads they exist to record), one listed files that do not exist, one reported duplications grep disproved, one cited a line past the end of the file it judged. Record the rule beside the other evidence laws so the next agent does not integrate a yield unchecked. Refs: vault de-dup campaign, PR #24.	2026-10-19 02:04:40 -07:00
Frank	500ac2b871	docs(vault): consolidate the vLLM offload-knob claim to one owner (C2, C6) The "`--swap-space` is removed; use `--kv-offloading-size`" claim was carried by six pages, two of them with their own copy of the upstream source citation. It now lives only in [[RAM CPU Offloading in VLLM]], which carries the 2026-10-19 verification against `vllm/config/cache.py` (no `swap_space`; `kv_offloading_size` activates only with `kv_offloading_backend`): - `VLLM Overview` (two argument tables), `VLLM Index` (parameter table), `DFlash DSpark Speculative Decoding`, `Shared Memory Architecture Dankclaw`, `Hybrid CPU GPU Processing Dankserver` -> cite the owner. - `VLLM Index`'s example command used the dead flag; it now shows the live pair. - `dankserver-vram-growth`: banner disambiguates the 2026-08 `frank-zealot` (llama.cpp on the 3090 when that card was in Dankserver) from today's endpoint of the same name on Dankclaw, so a retired crash signature does not read as a live fault on a same-card-class service. Refs: vault de-dup campaign, clusters C2 and C6.	2026-10-19 02:01:14 -07:00
Frank	6a8cbde260	docs(vault): retire the ACTIVE labels on frank-medium era pages (cluster C5) The root and LLM index pages still advertised two retired endpoints' pages as live procedure: - `frank-medium-radiance-mxfp4`: frontmatter said "ACTIVE frank-medium runbook" and the body opened "The ACTIVE frank-medium deployment". Both re-keyed to the era it ran (2026-09-21 -> 10-10), with [[frank-archon]]'s Eras table named as the owner of that shape. - `frank-medium-model-swap`: read as live how-to for a retired endpoint. Re-keyed to historical, pointing at [[Model Swap and Rollback]] for the current swap law. - Both index pages' rows re-keyed to match (rows are hand-copied from frontmatter - no generator - so both sites move together). Refs: vault de-dup campaign, cluster C5.	2026-10-19 02:00:21 -07:00
Frank	73fb27254e	docs(vault): era-correct the vLLM/Ling era pages (cluster C3) - `Running Qwen3.6 27B FP8 on Dankclaw`: had no banner at all and opened as a live how-to. Added the RETIRED banner (iGPU endpoint today is [[frank-dragoon]]), era-tagged the hardware table, and rewrote the "primary service now runs vLLM-ROCm with Ling CIRU INT4" production note, which was false by three fleet generations. - `ling-3.0-flash-vllm-plan`: "frank-dragoon is the Qwen3.8-27B PP head" -> past tense with the 2026-08-17 redo as the anchor and the live owner named. - `frank-redo-2026-08-17` banner: stripped restated live shapes that had already drifted (radiance 1.0.13, PrismML b10754 - the fleet runs 10777) and the 3090-provenance clause owned by the PP deployment page; banner is now locator-only (names + ports + links). Verified before cutting: 457.9 t/s and the ping-RTT figures are solo-held by Ethernet Bandwidth Math.md (no duplicate), so both scout-reported "duplicates" were kept. Refs: vault de-dup campaign, cluster C3.	2026-10-19 01:59:13 -07:00
Frank	e32df1b04f	docs(NPU page): era-tag host-RAM and PP-rate restatements One fact, one owner (de-dup campaign C4): - `FastFlowLM NPU Inference.md` carried the host RAM figures three times (29 GiB budget input kept as era data; the 2026-10-18 `free -g` re-reads of 121/246 GiB cut -> [[Dankclaw System Specs]]/[[Dankserver System Specs]]). - The 14-18 t/s PP-split decode is this page's own solo-held era measurement: kept, tagged to the 2026-08 era with [[frank-redo-2026-08-17]]. - Promotion section: `frank-zealot` "now runs the NPU endpoint" era-tagged (the name has been reused twice since; the NPU unit is `frank-npu`, masked). - Repaired the Test Runbook splice introduced by the range edit above. Refs: vault de-dup campaign, cluster C4.	2026-10-19 01:58:03 -07:00
Frank	a518fc7c22	docs(vault): de-stale the quant and swap method pages Parent-written (scout summaries were corroborated against the files, not trusted): every restatement of an endpoint shape now cites the endpoint page, and four claims contradicted by today's fleet state are corrected. - Quantization Production: the challenger-contract line quoted the retired compaction-hub window; now cites [[frank-zealot]]'s Current shape. The 3090 worked example is era-tagged. Sources listed [[frank-dragoon]] three times. - Quantization by Hardware Architecture: said the 3090 lives on Dankserver (it moved to Dankclaw 2026-09-16; the Eras table owns the move) and ranked the retired Ling-3.0 Quark INT4 service as the ACTIVE Dankclaw workload. The active service is [[frank-dragoon]] on a closed `.hgn` format. - GGUF Compatibility: Related claimed the ftype-115 ROCmFPX file was "in production"; that is an era now, so it points at [[kingjones-q8agent-rocmfpx]]. - Model Swap and Rollback: dropped the fragment byte-count ledger (AGENTS.md owns it), the Ampere FP8 hardware claim ([[Quantization by Hardware Architecture]] owns it), halogen's HTTP surface ([[frank-dragoon]]'s wire contract) and three brittle unit-file line refs (accepted loss: line numbers are derivable, the root-unit law stays). Fixed the per-slot example, which described the retired 3-slot arithmetic, and a Related bullet that called [[frank-archon]] retired while linking the same page twice. Census over the diff: 1 orphan line, the deleted unit line-number refs above.	2026-10-19 01:52:30 -07:00
Frank	9f1a4e4e51	docs(dankclaw): drop the stale frank-zealot drift note The page claimed the unit had been failed since 2026-10-18 03:40 with a hand-run process on the port. Verified live: both units are active under their repo units (zealot since 21:06, the deployer's 2-slot reslice).	2026-10-19 01:39:54 -07:00
Frank	4a13dcc5b7	docs(agents): say who owns a campaign ledger The dedupe passes kept trying to port single-owner experiment figures into endpoint Eras tables. The law now states the test explicitly: two pages carrying one figure is the defect; a figure with exactly one holder is fine wherever it lives.	2026-10-19 01:39:54 -07:00
Frank	e7bfa82853	docs(log): compress the 2026-10-18 entries to incident + law The 3-slot -> 2-slot reslice entry and the contended-rate entry carried the full measurement ledger; the numbers belong to [[frank-zealot]] (shape block + Eras rows), so the entry keeps the causal story (pool halving funds exact KV; rates taken under a live consumer are floors) and links for the figures.	2026-10-19 01:34:06 -07:00
Frank	04f4e2c6d4	docs(vault): strip restated fleet facts from 17 technique pages Enforces "one fact, one owner": technique pages keep mechanism, laws, traps and verdicts; per-endpoint shapes, digests and era measurements are cited to the endpoint page that owns them. Also repairs the four pointers the pass left empty (each verified against the cited owner before landing): - EMULATION dequant doubles host RAM (118 GiB -> ~238 GiB) -> [[frank-archon]] - 12.11 tok/s flat 304 -> 16,095 tokens -> [[Tcclaviger Fork Ops]] - bundled-head acceptance ladder 0.739/0.554/0.427/0.340 -> [[Qwen3.8 PP Deployment]] - llama.cpp PR #23897 dedup provenance -> [[qwen3.8-spec-matrix]] Replaces the retired-shape illustration in Llama Cpp Serving Sizing with the owned MiMo Q5_K_M row (6.40 GiB payload -> 20,530 MiB used), and unwraps a hard-wrapped [[Dankserver System Specs]] link that failed link integrity. Orphan census over the diff: 0 numbers lost (comma-insensitive match against the whole vault plus AGENTS.md).	2026-10-19 01:33:47 -07:00
dank	129b953843	Merge pull request 'docs: one fact, one owner — endpoint pages own every fleet fact' (#23 ) from wt/20261017-181502 into main	2026-10-19 01:15:56 -07:00