frank-primary: radiance native-FP8 TP=2 cutover (engine, payload build, rollback)

Replace the FreeToken PR587 endpoint (single-GPU: its TP=2 dies in the
pynccl.cu ROCm JIT) with the radiance engine on both R9700s, TP=2 via
libr4d's fused one-shot all-reduce. Official Qwen/Qwen3.8-Flash-Next-FP8
payload at native block-FP8 precision:

- scripts/build-frank-primary-rad.sh: staging (quantization_config drop =
  plugin SELECTION law) -> checkpoint reshape -> rad-convert with ONE
  --quant rule (PLE rtn fixed-scale); plan-only first.
- scripts/stack-qwen38-official-fp8.py: the two value-preserving reshapes
  (experts byte-stacked into the plugin's stacked layout; PLE's 128
  scaleless fp8 shards decoded to one bf16 shard with the checkpoint's own
  scale; the rtn rule re-encodes at that same scale - codes round-trip
  exactly and land the one-element scale plane the plugin declares).
- scripts/hfget.sh + verified download (131/131 shards sha256-match).
- froggeric v22.5 chat template vendored (--override-chat-template).
- serve script (pinned ledger: 2x40 GiB pools + 47.7 GiB PLE + prefix host),
  TTM preflight fixed for kernels with pages_limit only (no pages_max -
  an absent knob is skipped, not read as zero); pages_limit raised to
  52,428,800 pages = 200 GiB on the host pre-cutover.
- root unit, relinked; deployer gates image digest + .rad floor 158,000 MiB
  + health; prometheus re-adds the /metrics job.
- rollback: tag frank-primary-freetoken-pr587-20261005 (FreeToken trio +
  Dockerfile); payload + image stay on disk until the window closes.

deploy-remote.sh also carries the parallel session's frank-small payload
gates (swift15) - committed here per Frank's direction to commit the tree.

scripts/frank-primary.env

@@ -1,82 +1,135 @@
-# frank-primary.env — FreeToken (PR #587 ROCm) serving the orcarouter
-# Qwen3.8-Flash-Next-Uncensored-FP8 payload on dankserver :7770.
+# frank-primary.env — radiance (StillDeadcode, Apache-2.0) serving the
+# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at NATIVE precision
+# (modelopt block-FP8 throughout, no requantisation) on BOTH R9700s (TP=2),
+# dankserver :7770.
 #
-# 2026-10-05 cutover from the tcclaviger RDNA4 vLLM fork (git tag
-# frank-primary-vllm-shapeG-20261003 is the pre-cutover state; the 362-line
-# vLLM-era env is in that tag). The engine is FreeToken at PR head
-# 77a7201f98f00d93f618c6cf8b81ffe8360e0899 (branch rocm714_torch214), built
-# on the host into a local image by
-# scripts/docker/frank-primary-freetoken.Dockerfile — the PR's own
-# docs/install_amd.md stack (ROCm 7.14 base image, torch 2.11.0+rocm7.14,
-# triton 3.7) for gfx1201.
+# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
+# pinned host RAM; git tag frank-primary-freetoken-pr587-20261005 is the
+# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
+# tag and rebuildable). Before that: tcclaviger vLLM fork (tag
+# frank-primary-vllm-shapeG-20261003).
 #
-# WHY FREETOKEN: the vLLM fork's expert offload planned VRAM/host tiers per
-# boot from an estimator (util 0.982 law, arena 17.77 GiB / 7,762 slots,
-# ~40k-token GDN state faults) and 1.0 was unbootable. FreeToken keeps ALL
-# expert weights in pinned HOST RAM (~120 GiB) with an LRU GPU cache, sizes
-# KV from an explicit token pool, and serves on ONE R9700 (32 GB) with the
-# second card idle. RAM fit: 246 GiB total, service stopped frees the vLLM
-# container's ~170 GiB; weights ~120 GiB + PLE 47.7 GiB + overhead ≈ 190 GiB.
+# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
+# gfx1201, drives `podman run` directly from the digest-pinned
+# stilldeadcode/radiance image (287 MiB; host needs only the amdgpu driver),
+# and its qwen4exp plugin reads the modelopt FP8 encoding NATIVELY
+# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
+# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
+# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
+# The 4-bit-codebook publisher shape (qwen3.8-next-flash-fp8-iq4r-moe.rad,
+# --host-pool-mib 12288) remains the documented fallback if the native
+# container proves unbootable.
 #
-# PAYLOAD: unchanged — /home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8,
-# 131 safetensors shards. Quant is standard modelopt block-FP8: 75,264
-# weight_scale_inv tensors (block 128x128), zero input_scale (FreeToken #462
-# covers exactly this export shape), one weight_scale on the PLE ngram
-# embedding. FreeToken loads HF safetensors directly; no FTW conversion.
+# PAYLOAD: /home/dank/models/rad/qwen4exp-fp8-native.rad — a rad-convert
+# NO-RECIPE pure copy of the official 131-shard checkpoint (every weight
+# keeps its block-FP8 encoding; ~161 GiB). The convert runs against a
+# staging dir whose config.json drops `quantization_config`: the descriptor
+# string is only the plugin SELECTION key (spec §2.4 — exact match, no
+# fallback), and no shipped plugin claims (qwen4exp, "fp8"); the shipped
+# qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
+# containers carry. Per-weight encodings are data, not the descriptor —
+# removing the key changes nothing about what the weights hold. Built by
+# scripts/build-frank-primary-rad.sh (staging + convert + rad-info).
 #
-# SERVING FACTS (docs/cli.md, verified against main+PR587):
-# - `ft serve` loads the checkpoint directly; `--served-model-name` keeps the
-#   harness id; reasoning/tool parsers are AUTO (reasoning lands in
-#   reasoning_content — models.yml flips reasoningContentField to match).
-# - KV: --page-size must stay 1 for GDN/SWA models (default); --cache-type
-#   radix auto-picks the GDN-aware variant.
-# - MoE: --moe-strategy auto -> offload (fp8 experts CANNOT use
-#   --moe-cpu-layers; only bf16/nvfp4/mxfp4 have a CPU executor).
-#   --moe-cache-rate holds the GPU LRU fraction of all experts.
-# - `ft ctl cache --kv --moe` grows the pools LIVE on a running server
-#   (shrinking needs a restart); `ft ctl stats` reports pool occupancy.
-# - KV MATH: 12 full-attention layers x 2 KV heads x 256 head_dim x 2 (K+V)
-#   x 2 B = 24 KiB/token (f16 KV; the 36 GDN layers use fixed per-seq state,
-#   not per-token KV). See FRANK_PRIMARY_FT_NUM_TOKENS below for the pool
-#   size and its measured page-size law.
-FRANK_PRIMARY_IMAGE=localhost/frank-primary-freetoken:pr587-77a7201
-FRANK_PRIMARY_MODEL_DIR=/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8
-FRANK_PRIMARY_MODEL_SHARDS=131
+# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
+# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
+#   then engine.serve()) — during load the port refuses connections, so a
+#   bare curl /health gate is honest. /health == /ping: 200 JSON while
+#   serving, 503 only while stopping (core/server/admin.cpp:385).
+# - Reasoning: --reasoning-format defaults to auto, emitting thinking in
+#   `reasoning_content` (config.cpp). Tool calls: the qwen3.8 dialect, from
+#   the chat format. Both need no parser flags.
+# - Sampling: the OpenAI endpoint accepts temperature, top_p, top_k, min_p,
+#   typical_p, presence_penalty, frequency_penalty, repetition_penalty,
+#   repeat_last_n, chat_template_kwargs, reasoning_effort,
+#   enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
+#   models.yml's extraBody set is fully inside it. Unknown params are
+#   refused (400), which is why nothing unverified rides the wire.
+# - MTP: the official checkpoint carries the mtp.* head (3,101 tensors,
+#   mtp_num_hidden_layers=1) and the plugin probes draft depth 3 — the
+#   publisher's throughput peak for this model. Stated explicitly with
+#   --num-speculative-tokens 3.
+# - The fused TP all-reduce is declared over max-num-seqs x
+#   (1 + --num-speculative-tokens) rows; 2 x 4 = 8 rows fits either wire.
+FRANK_PRIMARY_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
+# The .rad container FILE (mounted under /models) and the state dir for the
+# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
+# .rad files; the raw HF download and its staging dir live beside it and
+# are never mounted.
+FRANK_PRIMARY_MODEL_DIR=/home/dank/models/rad
+FRANK_PRIMARY_MODEL_FILE=qwen4exp-fp8-native.rad
+FRANK_PRIMARY_STATE_DIR=/home/dank/radiance-state
 # Container identity is the SERVICE name (Frank's call, 2026-10-05): one
-# name across systemctl and podman ps. The vLLM-era name frank-primary-qwen
-# moved into the legacy list so every deploy cleans it up.
+# name across systemctl and podman ps. The runtime changes; the name does
+# not. The legacy list retires every predecessor identity that could hold
+# the GPUs or the port.
 FRANK_PRIMARY_CONTAINER=frank-primary
 FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe"
 FRANK_PRIMARY_SERVED_NAME=frank-primary
-FRANK_PRIMARY_PUBLISH=7770:7770
+FRANK_PRIMARY_PORT=7770
+# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
+# repo's root chat_template.jinja, vendored here; the checkpoint's own
+# template is baked into the .rad by rad-convert and this overrides it).
+# Radiance derives the reasoning/tool-call reply format from it.
+FRANK_PRIMARY_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
 
-# --- FreeToken engine knobs (first-boot conservative, resize live) -----
-# Single GPU: the R9700 at index 0. HIP_VISIBLE_DEVICES=0 hides card 1 so
-# the in-container index 0 IS the right card; --gpu 0 selects it.
-FRANK_PRIMARY_FT_GPU=0
-# KV pool in TOKENS (not bytes — vLLM-era FRANK_PRIMARY_KV_BYTES is gone).
-# 299,968 tokens x 24 KiB = 7.0 GiB; clears the harness worst case
-# (177,356 tokens) and one NATIVE 262,144-token conversation + full budget.
-# MEASURED FIRST BOOT (2026-10-05): the auto attention backend is
-# qsa_sparse, which OVERRIDES page size to 64, so --num-tokens must be a
-# MULTIPLE OF 64 — 300,000 was refused ("nearest valid values: 299968 or
-# 300032"). Grows live with `ft ctl cache --kv <n>`; shrink needs a
-# restart.
-FRANK_PRIMARY_FT_NUM_TOKENS=299968
-# Concurrent decode slots. 2 is the first-boot conservative value; the
-# radix pool admits ~2 full-length 200,704-token conversations. Raise with
-# --max-running-requests only after a real-traffic look at slot occupancy.
-FRANK_PRIMARY_FT_MAX_RUNNING=2
-# Per-request output budget ceiling: the harness's maxTokens (36,864).
-# Requests that omit max_tokens get this default instead of the engine's
-# 32,768.
-FRANK_PRIMARY_FT_MAX_OUTPUT_TOKENS=36864
-# GPU expert LRU cache as a FRACTION of all experts (~120 GiB fp8 total ->
-# ~12 GiB VRAM at 0.10). First-boot conservative; resize live with
-# `ft ctl cache --moe` after reading the boot ledger + ft ctl stats.
-FRANK_PRIMARY_FT_MOE_RATE=0.10
-# Escape hatch for boot-ladder experiments (YaRN, --max-prefill-length,
-# --moe-cpu-threads, --sampling-defaults none, ...). Unset -> no extra
-# argv. Keep multi-flag values as one string; the serve script expands it
-# unquoted on purpose.
-FRANK_PRIMARY_FT_EXTRA_ARGS=
+# --- engine shape (first-boot conservative; the boot ledger arbitrates) --
+# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
+# x8/x8 on root ports) made collectives legal on this host; radiance's TP
+# wire is libr4d's own, not RCCL.
+FRANK_PRIMARY_TP=2
+# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
+# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
+# engine default.
+FRANK_PRIMARY_TP_WIRE=wht6
+# Decode slots. 2 is Frank's first-boot value ("use parallelism = 2 to
+# start"); raise only after a real-traffic look at slot occupancy and the
+# KV pool's headroom line. The recurrent-state cost scales with it (54
+# MiB/copy/card, 2 copies with speculation).
+FRANK_PRIMARY_MAX_SEQS=2
+# Served window. Native is 262,144; 200,000 is the publisher's production
+# value (compose) and what models.yml's contextWindow must quote — a longer
+# prompt is refused by the engine, never truncated.
+FRANK_PRIMARY_MAX_MODEL_LEN=200000
+FRANK_PRIMARY_PLACEMENT=expert_tiered
+# Pinned host pool for the expert rows that do not fit VRAM. Native FP8
+# experts are 112.8 GiB total (56.4 GiB per rank slice) vs the 4-bit
+# publisher shape's ~29 GiB — this pool is the cost of native precision.
+# The boot ledger prints the pool's coverage ("the pool backs N% of the
+# worst case ... M full-length sessions"); resize from that line, not from
+# arithmetic.
+FRANK_PRIMARY_HOST_POOL_MIB=40960
+# VRAM left unclaimed per card (publisher default 96 MiB).
+FRANK_PRIMARY_GPU_HEADROOM_MIB=96
+# Weights share of claimed VRAM (publisher production value; higher = more
+# prefix-cache hit rate, less KV headroom).
+FRANK_PRIMARY_EXPERT_VS_CACHE=0.82
+# KV cache in FP8 (7.32 KiB/token at this model's 12 full-attention layers,
+# 2 KV heads, 256 head_dim, K+V).
+FRANK_PRIMARY_KV_DTYPE=fp8
+# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
+# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
+# flag states it so the ledger records the intent).
+FRANK_PRIMARY_SPEC_TOKENS=3
+FRANK_PRIMARY_MAX_BATCHED_TOKENS=2048
+# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
+FRANK_PRIMARY_PREFIX_HOST_MIB=4096
+FRANK_PRIMARY_PREFIX_DISK_MIB=65536
+# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
+# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
+# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
+FRANK_PRIMARY_NGRAM_PLACEMENT=ram
+# TTM preflight: the pinned ledger (2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
+# prefix host) is ~132 GiB. ttm's pages_limit/pages_max cap the GPU-mappable
+# pinned pages; the current cap (32,252,917 pages = ~123 GiB) is BELOW the
+# plan, so the serve script refuses to start until both are raised. This is
+# the floor it enforces (in MiB), with ~8 GiB of margin.
+FRANK_PRIMARY_TTM_MIN_MIB=140000
+# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
+# multi-flag values as one string; the serve script expands it unquoted on
+# purpose.
+FRANK_PRIMARY_EXTRA_ARGS=
+# The .rad's minimum size in MiB (deployer gate; a truncated convert is a
+# guaranteed long-timeout boot instead of a fast failure). The pure copy of
+# the 172.8 GB checkpoint should land ~161 GiB.
+FRANK_PRIMARY_RAD_MIN_MIB=158000