dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs(frank-archon): reconcile the runbook with the live 1.8.1 two-slot shape

Browse Source 
The endpoint moved to radiance 1.8.1 with MAX_SEQS=2 and the
--vram-weights-mib/--vram-kv-mib pair removed (PR #30); env, serve script and
bats carried the new shape while the page and the AGENTS.md block still
described the 1.0.13 3-slot era, so every point-of-use number was wrong.

Re-derived from the live checkout and the 2026-10-19 11:10 boot ledger, not
from the stale pre-#31 draft: runtime block (digest, version oracle, the two
CLI breaks that are boot refusals), shape table (demand-signal rows, derived
VRAM split, restored spec/batch row, host-only prefix tier, step sharing),
container contract (no /data mount), the measured 2-slot ledger, a new Eras row
for the retired 1.0.13 era whose rollback is a digest plus a trio rewrite, and
the laws: pool sizing and the per-card budget re-scoped to the pair era, a new
resolver law (funds ~90% of the worst case, host RAM below a slab step is inert,
MAX_SEQS is a demand signal), the sludge law marked measured-at-3-slots with the
fix ordered-not-verified, the knob-availability law flipped, and the slots law
now pricing the third slot in KV (4.20 -> 2.76 GiB).
This commit is contained in:
Frank
2026-10-19 14:55:43 -07:00
parent 3d10a4ca4f
commit d9b82b372e
2 changed files with 112 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/LLM/frank-archon.md
+103 -57
View File
@@ -1,6 +1,6 @@
---
type: reference
title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)"
title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.8.1, publisher Qwen3.8-Flash-Next fp8-iq4r-moe .rad, 2× R9700 TP=2 (ACTIVE)"
description: "The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught."
tags:
- llm
@@ -12,7 +12,7 @@ tags:
- tp2
- expert-offload
- endpoint
timestamp: 2026-10-18
timestamp: 2026-10-19
---
# frank-archon — radiance (StillDeadcode), 2× R9700 TP=2, Dankserver :7770
@@ -20,8 +20,9 @@ timestamp: 2026-10-18
The radiance engine (C++/HIP, `libr4d` kernels built for gfx1201, no Python in the serving
path) driving `podman run` straight from the digest-pinned image under a **root** systemd
unit; container name = unit name. Host Dankserver (192.168.0.235), port 7770, both Radeon AI
PRO R9700s. Capacity contract **3 conversations × 200,000 tokens at fp8 KV**; a payload swap
is a `MODEL_FILE` line plus a re-download — see Eras.
PRO R9700s. Capacity contract **2 conversations × 200,000 tokens at fp8 KV** — at 1.8.1 that pair
is the **demand signal** the budget resolver sizes the cache against, not a KV allocation (Laws).
A payload swap is a `MODEL_FILE` line plus a re-download — see Eras.
## Current shape
@@ -44,14 +45,20 @@ declared size + sha256) as the transient unit `iq4r-fetch` (log
`/home/dank/models/StillDeadcode/…` keeps card, LICENSE and recipe as provenance, and only
`.rad` files are mounted.
**Runtime** — `stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e`
= pin **1.0.13** (Codeberg `v1.0.13`, commit `53282fa`): a 287 MiB image in dank's
**rootless** storage, host needs nothing but the amdgpu driver. Pin the INDEX manifest digest,
never the image ID (`69d05724…` is the config digest and sends `podman run` to the network).
The serve script's gate is that digest ref itself — `podman image exists name@sha256:…` in one
command (the 2026-09-30 no-pipe/grep-race lesson) — so a retagged look-alike or a newer
`latest` cannot satisfy it. 1.2.0 is a rejected serve pin (Laws); the
`--vram-weights-mib`/`--vram-kv-mib` pair is accepted by both pins, which is the rollback symmetry.
**Runtime** — `stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed`
= pin **1.8.1** (adopted 2026-10-18). The version oracle is the boot's own `STARTUP REPORT --
radiance 1.8.1` line, because `podman inspect` reports only the digest. The image lives in dank's
**rootless** storage and the host needs nothing but the amdgpu driver. Pin the INDEX manifest
digest, never the image ID (`69d05724…` is the config digest and sends `podman run` to the
network). The serve script's gate is that digest ref itself — `podman image exists name@sha256:…`
in one command (the 2026-09-30 no-pipe/grep-race lesson) — so a retagged look-alike or a newer
`latest` cannot satisfy it. 1.2.0 is a rejected serve pin (Laws). **Two CLI breaks ride this pin,
and both are boot refusals, never silently-ignored flags:** the `--vram-weights-mib`/`--vram-kv-mib`
pair died at 1.3.0 (the resolver derives the split) and `--prefix-cache-dir` /
`--prefix-cache-disk-mib` died with the disk tier. A rollback to 1.0.13 (`sha256:4c3f4845…`) is
therefore a trio rewrite, not a digest swap — the Eras row carries it, and
`tests/frank_archon_unit.bats` asserts the retired knobs are ABSENT from env and argv so a stale
line cannot reach a boot.
**Trio** — `scripts/frank-archon.env` (every number, with its arithmetic in the header)
`scripts/serve-frank-archon.sh`, `scripts/frank-archon.service`. Gates run in order: image
@@ -64,8 +71,10 @@ the name registered, and every one of those identities holds this GPU. Container
(publisher-vetted, `docs/DOCKER.md`): `--network host`, `--init` (so the engine releases the
cards on SIGTERM), `--security-opt seccomp=unconfined` (the default profile refuses io_uring —
the PLE table's reads — and the NUMA calls the ROCm runtime makes for host memory), `--ulimit
memlock=-1:-1`, `/models` ro + `/data` rw + the tune cache at `/opt/radiance/share/radiance/tune`
template as a single-file mount, then `exec podman wait`: the script's lifetime IS the
memlock=-1:-1`, `/models` ro + the rad-tune cache at
`/opt/radiance/share/radiance/tune` + the template as a single-file mount (there is **no `/data`**
any more: it left with the prefix disk tier, and `FRANK_ARCHON_STATE_DIR` is now only the host-side
directory that holds `tune/`), then `exec podman wait`: the script's lifetime IS the
container's, so a crash is a nonzero `ExecStart` and the restart policy is real.
Unit: `After=network-online.target`, `WantedBy=multi-user.target`, `User=dank`
`LimitMEMLOCK=infinity`, `StartLimitIntervalSec=0`, `OOMScoreAdjust=-500` (the last process the
@@ -79,27 +88,33 @@ on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_p
| Knob (env name, `FRANK_ARCHON_` dropped) | Live value |
|---|---|
| TP / wire | `TP=2`, `TP_WIRE=wht6` — libr4d one-shot all-reduce, not RCCL; `min-kb` stays the engine default |
| Slots / window | `MAX_SEQS=3`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) |
| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds 3 slots (Laws) |
| VRAM pair | `VRAM_WEIGHTS_MIB=25600` + `VRAM_KV_MIB=5120` = 30,720 of the 30,969 ceiling; `GPU_HEADROOM_MIB=96` |
| Slots / window | `MAX_SEQS=2`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**, not a KV allocation |
| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) |
| VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) |
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=44928` **per rank** |
| PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
| Spec / batch | — |
| Prefix cache | `PREFIX_HOST_MIB=8192` + `PREFIX_DISK_MIB=65536` under `STATE_DIR=/home/dank/radiance-state` (mounted rw at `/data`) |
| Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) |
| Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache |
| Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
| Step sharing | `BUSY_PREFILL_CHUNK=auto` (a shared chunk reserves a row plus the draft window per waiting decoder) · `PREFILL_INTERLEAVE=0` — held at zero until the decode-vs-prefill buckets ask for it |
| TTM gate | `TTM_MIN_MIB=150000`, `TTM_TARGET_MIB=204800` |
| Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |
Boot ledger for this shape (2026-10-15, pin 1.0.13): pool 5.00 GiB backs **607,740 tokens**
(`kv_attn` fp8 page 26.00 KiB, `kv_qsa_bkey` 3.25 KiB), `kv_gdn_state` 54 MiB × 6 slots, slab
**17,548 slots** with 4.22 GiB of KV lent to it, sampler **9 rows**, template 28,234 B, vision
live (≤ 4,194,304 px, 16,384 patches/pass).
Boot ledger for the live shape (2026-10-19 11:10, pin 1.8.1, `MAX_SEQS=2`, `HOST_POOL_MIB=44928`):
the resolver prints `elastic 26.73 GiB` → **experts 23.73 GiB (89 %) / kv cache 2.76 GiB (10 %)**,
`vram_weights 27.31 GiB`, `host_pool 44.47 GiB`. `kv_attn` (13 layers, 4-token blocks) pages
**26.00 KiB** → 91,301 blocks = **365,204 tokens = 1.8 full-length sessions = 91 % of the
2 × 200,000 worst case** (the resolver never funds 100 %, see Laws); `mover` lends 2,146 slots
(2.54 GiB) out of the cache to the expert slab; sampler **9 rows**, template 28,234 B, vision live
(≤ 4,194,304 px, 16,384 patches/pass).
The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before
anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin,
the P2P answer per pair) — read it after every resize, and unlike every predecessor there is no
per-boot VRAM planner to second-guess. For comparison, the card's own suggested run line on 32
GB cards is `--host-pool-mib 12288 --expert-vs-cache-ratio 0.82` with seqs 8 and spec 3; the
per-boot VRAM planner to second-guess. For comparison, the publisher's own suggested run line on 32
GB cards was `--host-pool-mib 12288 --expert-vs-cache-ratio 0.82` with seqs 8 and spec 3 — a
1.0.x-era line, and `--expert-vs-cache-ratio` was removed at 1.2.0, so today it is a capacity floor
quoted from the card, not a run line this pin accepts; the
shape above is the fleet's measured one. `FRANK_ARCHON_EXTRA_ARGS` is the boot-ladder escape
hatch (unset → no extra argv; it is expanded unquoted on purpose, so keep multi-flag values as
one string).
@@ -130,6 +145,7 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---|
| 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) |
| 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
| 2026-10-05 → 10-08 | `qwen4exp-fp8-native.rad`, 177,109 MiB (173.0 GiB) — native modelopt block-FP8, no requantisation | radiance 1.0.7 line, same digest | — | native FP8 experts (56.4 GiB/rank) ate a 40 GiB pool and left KV at 3,543 MiB = 386,904 tokens, **1.93 sessions** — context economics, not quality; and the host pool could not be trimmed to pay for it (36,864 refused the boot) | rebuild-only: `scripts/download-frank-archon.sh` (official `Qwen/Qwen3.8-Flash-Next-FP8`, 131 shards, 172.8 GB, ungated) + `scripts/build-frank-archon-rad.sh` (no-recipe convert under the descriptor law); raw **and** `.rad` were `rm`'d 2026-10-15 (Frank's call, to free the volume for the iq4r download) |
| 2026-10-05 (same day) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` safetensors, 131 shards | FreeToken PR #587 head `77a7201` (`rocm714_torch214`), image `localhost/frank-primary-freetoken:pr587-77a7201` | — | TP=2 cannot boot on ROCm: the per-rank `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201, so the second card idled | git tag `frank-archon-freetoken-pr587-20261005` (trio + `scripts/docker/frank-primary-freetoken.Dockerfile`, which exists only there); image still in rootless storage; payload needs a re-fetch, and that repo 403s until the gate is accepted on the account |
@@ -161,7 +177,8 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
2×**, precisely because the bkey plane is dtype-fixed. Secondary saving: a prefix-cache entry
tracks the attention plane (110.50 KiB at bf16 → 58.50 at fp8, per the `kv tier transfer`
lines), so the same 8 GiB host tier parks ~5.8 sessions of 200k at fp8 where it parked ~2.9 at bf16.
- **Pool sizing law.** `pool_mib ≥ tokens × 7.3125 KiB + 110 × seqs + 448` — the dtype-
- **Pool sizing law (the explicit-pair era — at 1.8.1 the cache is derived, so the resolver law below
owns it).** `pool_mib ≥ tokens × 7.3125 KiB + 110 × seqs + 448` — the dtype-
independent terms are also in the pool: `kv_gdn_state` 54 MiB × 2 copies per sequence (+conv/
tail/ple ≈ 110 MiB a sequence) and the prefix checkpoint region 8 × 55.99 MiB. Deployed: 3 ×
200,000 at fp8 = 4,285 + 330 + 448 = **5,063 MiB**, pool 5,120, which predicts (5120 − 330 −
@@ -174,13 +191,15 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
pool of 3,543 gave 386,904 (1.93 sessions), and the four-slot fp8 boot of 2026-10-13 19:21
backed 843,276. The ledger's `backs N sequences all at 200000 tokens at once` line is the
referee — a pair this tight is read off the boot, never assumed, and the concurrent probe is
its regression test. Fourth full-height conversation needs 6,601 MiB of pool: refusing it is
Frank's explicit call, and serving it is a `--vram-kv-mib` change, not a code change.
its regression test. A fourth full-height conversation needs 6,601 MiB of pool: refusing it is
Frank's explicit call, and under that pin serving it was a `--vram-kv-mib` change, not a code change.
At 1.8.1 the same decision is `MAX_SEQS` — the resolver buys the cache the demand signal implies.
- **The hard limit is per request.** The engine treats `kv group tokens >= max-model-len` as a
HARD limit — "one request longer than N tokens is accepted by the API and then refused as too
large to ever fit — this is a hard limit, not a backing ratio" — so the pool is sized for
`seqs × max-model-len`, not for one sequence. Radiance never truncates, at any size.
- **One VRAM budget per card:** `weights + kv ≤ 30,969 MiB`, read off the engine's own budget
- **One VRAM budget per card** (the arithmetic below is the pair era's; the ceiling survives, the knobs
do not): `weights + kv ≤ 30,969 MiB`, read off the engine's own budget
table (boot 2026-10-13 at pair 24000/5120): card 32,624 MiB, budgets claimed 29,120, and
"*3.42 GiB is neither budgeted nor accounted — the driver's context, the activation arena and
`--gpu-headroom-mib` come out of it; anything past those is throughput on the floor*", with
@@ -188,12 +207,34 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
static region). Reserve above the engine's floor = 2,058 − 209 = 1,849, ceiling 29,120 + 1,849
= 30,969; the deployed pair lands 458 MiB above that floor. The 2026-10-11 bound of 29,120
("31,856 − 1,710 outside both budgets − keep 1 GiB free") was a hand-picked reserve, not the
engine's wall. Boot with the explicit `--vram-weights-mib`/`--vram-kv-mib` pair, never
`--kv-cache-size-mib`; with the disk tier engaged the engine also forces ~9.6 GiB of weight
buffers + arena onto the weights side (~29.6 GiB/card effective).
engine's wall. Under that pin, boot with the explicit `--vram-weights-mib`/`--vram-kv-mib` pair,
never `--kv-cache-size-mib`; with the disk tier engaged the engine also forced ~9.6 GiB of weight
buffers + arena onto the weights side (~29.6 GiB/card effective). **Neither flag exists at 1.8.1:**
the same ceiling is enforced by the resolver, and passing the pair is a boot refusal.
- **At 1.8.1 the split belongs to the resolver, and it funds ~90 % of the worst case, never 100 %.**
The engine takes `--gpu-headroom-mib`, the cache bound (`--max-model-len` × `--max-num-seqs`) and
`--host-pool-mib`, prints `claimable → elastic → experts / kv cache` before allocating anything, and
sizes the expert plane from the model's own weight footprint plus slack — not from a number typed by
hand. Three boots of evidence:
| shape | demand | cache given | backing |
|---|---|---|---|
| 3 × 200,000, host pool 44,928 | 600,000 tok | 4.20 GiB | 93 % (2.8 sessions) |
| 3 × 200,000, host pool 45,568 | 600,000 tok | **4.20 GiB — byte-identical** | 93 % |
| 2 × 200,000, host pool 44,928 (live) | 400,000 tok | 2.76 GiB = 365,204 tok | 91 % (1.8 sessions) |
So `--max-num-seqs` is a **demand signal** for the resolver and an admission cap for the scheduler —
never a KV allocation — and no slot count fully funds the worst case. Growing host RAM by less than a
slab step (640 MiB) moved nothing: the expert plane's claim decides what is left for KV, so cache is
grown through slots × window or the dtype, and experts are grown by not leaving elastic pool idle.
`--host-pool-mib` is still load-bearing the other way — it is where the expert rows the cards cannot
hold get pinned, and under-sizing it has always been a boot failure (36,864 refused a boot in the fp8
era). The two-slot boot's own behavioural check: three concurrent conversations all answer `200` —
the third **queues** (`queue_cap: 16`), it is not refused; the 400 class is a single request over
`--max-model-len`, or three concurrent full-height conversations against a 1.8-session cache.
- **Idle KV VRAM is the expert cache's spillover — the price is paid twice.** `mover: 2275
slots can be lent out of the KV cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <=
256 tokens`; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 today)
256 tokens`; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 in the pair era)
so a bigger KV pool steals weight residency from the slab AND from the lendable cache. Priors
on this card/model: stealing 2.14 GiB of weights to buy `MAX_MODEL_LEN 262144` cost 2.5 %
decode; the other direction (`--expert-vs-cache-ratio` 0.82 → 0.84, +2.5 GiB of weights) cost
@@ -239,8 +280,8 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
- **MNBT 2048.** 2560 collapses decode to 2.6-7.7 t/s (MTP resonance); the w4 file tier streams
in place only at MNBT ≤ 2048 and the activation arena scales with the chunk (32768 → 33.3
GiB, refused).
- **Decode pays for the chunk it shares a step with.** Census of the live engine
(radiance 1.0.13, `/stats` + `/metrics`, cumulative over the 2026-10-15 boot):
- **Decode pays for the chunk it shares a step with.** Census of the live engine taken at the
3-slot shape on 1.0.13 (`/stats` + `/metrics`, cumulative over the 2026-10-15 boot):
1,435,585 decode-only steps at **18.7 ms** each against 127,991 prefill+mixed steps
at **317 ms** — 60 % of engine wall-clock spent inside 8 % of the steps, because the
chunk is `min(remaining, checkpoint interval 2048, MNBT 2048)` = 2048 rows and every
@@ -252,28 +293,33 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
busy-prefill window sits at the idle baseline (58-64 vs 77-85 t/s, the residual
confounded by co-scheduling), so no lingering expert-residency penalty at 2-minute
resolution, and `preemptions: 0` rules out preempt-and-re-prefill. Method:
[[LLM Performance Diagnosis]] §9.
- **The knobs for it are not in this pin.** `--busy-prefill-chunk` and
`--prefill-interleave` (radiance GUIDE §4.5 — a shared chunk's cap, and N decode-only
steps after each chunk) are absent from 1.0.13: proven on the running container
(`podman exec frank-archon radiance --help` prints no `prefill`/`chunk` flag), and the
2026-10-18 source tree carries both. The only crank this pin has is MNBT, and that is
an arm and not an edit: the vendor prices a 1024 chunk at −19 % prefill to halve the
stall, and this page's own MNBT law already knows 2560 collapses decode (MTP
resonance). An image bump is gated by the 1.2.0 numerics dead end below, so any try
re-scores draft acceptance before it is trusted.
- **Slots cost KV and rows, never speed.** `--max-num-seqs` carries no radiance constraint: it
adds 110 MiB of recurrent state per slot and declares the fused TP all-reduce over `seqs ×
(1 + spec)` rows, and `wht6` is bit-exact up to 25 rows (single-sequence decode and any step
of ≤ 25 rows is exact; only busier steps are lossy) — 3 × 3 = 9 today. Measured 2026-10-06:
seqs 2 and seqs 3 are statistically tied (30.9-33.1 t/s, inside the ±3 % floor). The "seqs 2
cannot boot" law belonged to the retired tcclaviger fork's `torch.compile` shape
specialization, and radiance booted seqs 2 on its first fp8-native start.
**Same era, separate law: a dequantize-to-compute path bills host RAM twice.** On the
EMULATION backend an offload request of **118 GiB** landed as **~238 GiB** of real host RAM —
the dequantized copy rides on top of the pinned weights, and the doubling is invisible in the
weight count. Read the engine log for `Using 'EMULATION' Mxfp4 MoE backend` before concluding
"doesn't fit"; the container-side procedure and the kernel gate are .
[[LLM Performance Diagnosis]] §9. The law is shape-independent; the fix lives at 1.8.1 (next law),
and nobody has re-run the census since the migration — so the fix is ordered, **not verified**.
- **The knobs for it arrived with 1.8.1, and only one of them is on.** `--busy-prefill-chunk` caps the
shared chunk so a step with waiting decoders takes a smaller piece (live: `auto`, the engine picks;
the explicit value is 512), and `--prefill-interleave N` inserts N decode-only steps after a chunk
step taken beside decoders. `interleave` is deliberately **0** — the env comment holds it there until
the decode-vs-prefill buckets ask for it, and those buckets have not been read since the migration.
The vendor prices a 1024 chunk at about a fifth off prefill throughput to halve the stall; that is a
knob for choosing tomorrow's chunk, not a measurement of this shape. MNBT remains an arm and not an
edit: 2560 collapses decode through MTP resonance. Any chunk change re-scores draft acceptance before
it is trusted.
- **Slots cost KV, rows, and window — never speed.** `--max-num-seqs` carries no radiance constraint: it
adds ≈110 MiB of recurrent state per slot (the live `kv_full_backing` arithmetic prices them exactly),
declares the fused TP all-reduce over `seqs × (1 + spec)` rows, and `wht6` is bit-exact up to 25 rows
(single-sequence decode and any step of ≤ 25 rows is exact; only busier steps are lossy) — 2 × 3 = 6
today. Measured 2026-10-06: seqs 2 and seqs 3 are statistically tied (30.9-33.1 t/s, inside the ±3 %
floor). Measured 2026-10-19 at 1.8.1: the third slot's real price is **KV, not rows** — dropping it
moved the resolver's cache from 4.20 GiB to 2.76 GiB and the backing from 2.8 to 1.8 sessions, while
the ≈4 GiB of host RAM it freed came back unbudgeted, landing in elastic pool that turned into a bigger
mover lend instead of cache (resolver law above). The old "seqs 2 cannot boot" law belonged to the
retired tcclaviger fork's `torch.compile` shape specialization, and radiance booted seqs 2 on its first
fp8-native start.
- **A dequantize-to-compute path bills host RAM twice.** On the EMULATION backend an offload request
of **118 GiB** landed as **~238 GiB** of real host RAM — the dequantized copy rides on top of the
pinned weights, and the doubling is invisible in the weight count. Read the engine log for
`Using 'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit"; a page that says a payload
"does not fit" without that line has not read the runtime.
- **One encoding per expert table** (`rad_arch_declare`): a whole expert kept bf16 is the only
thing legal beside the table, so a split tier (down=fp8, gate_up=w4) is unservable.
- **Descriptor-selection law.** The plugin is picked by the exact `(arch_id, quant descriptor)`