Ordered 2026-10-19: migrate frank-archon to the latest radiance so the endpoint gets
the step-sharing knobs. This is a shape-model cutover, not a digest swap.
Why: decode pays for the prefill chunk it shares a step with (decode-only step
18.7 ms vs 317 ms for a chunk step; decode median 13.3 t/s while prefill runs >=1000 t/s). --busy-prefill-chunk and --prefill-interleave are the engine's answers and neither
exists in the pinned 1.0.13.
The two breaks, both verified on the images themselves, not inferred from docs:
Version
Facts
1.0.13 (this pin)
has --vram-weights-mib / --vram-kv-mib and the prefix-cache host+disk tiers; has NO prefill knob
1.3.0
the VRAM pair is REMOVED — refusal, not warning: "the card is split between the weights and the cache from what the model needs"
1.4.0
--busy-prefill-chunk arrives
1.7.0
--prefill-interleave arrives
1.8.1 (target)
prefix cache is host RAM only; --prefix-cache-dir / -disk-mib are refusals
So there is NO version that carries the knobs and still accepts the old argv: the pair
died at 1.3.0, the knobs were born at 1.4.0. The migration therefore moves the endpoint
onto the engine's budget resolver, which derives the per-card split from --gpu-headroom-mib, the cache bound (--max-model-len, --max-num-seqs) and --host-pool-mib, and lets the cache lend idle blocks to the expert slab.
What changes on the host:
expert slab MiB and KV pool tokens become BOOT-LEDGER lines; the 3 x 200,000 fp8
capacity contract is verified live by the capacity leg of probe-frank-archon-kv.py,
not computed from an env number. The engine refuses to boot rather than shrink a pool,
so a shortfall is loud.
64 GiB of NVMe prefix cache becomes re-prefill. prefix_hit_rate is the watch item —
a drop is extra prefill, which is the very traffic that costs decode its cadence.
FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto is passed explicitly (auto vs off is the
difference between decode waiting one chunk and waiting a whole step); FRANK_ARCHON_PREFILL_INTERLEAVE=0 stays opt-in through a guarded argv array.
Pre-flighted before any downtime: the exact production argv was captured with a podman shim around the edited serve script and replayed against the 1.8.1 image without
device access — every flag accepted, startup report echoes --busy-prefill-chunk auto [command line] and stops only on "tp 2 needs 2 devices".
Tests: the two budget-arithmetic seams asserted knobs that no longer exist; they are
replaced by a seam asserting the retired knobs are ABSENT from env and argv (a stale line
is an ExecStart refusal, i.e. an outage) plus the tune-mount//data-absence contract. frank_archon_unit.bats 20/20; full suite in flight.
Acceptance gate: any pin past 1.2.0 must be re-scored (1.2.0 measured 0.7123 against
the 0.7549 control). The greedy probe leg runs against the booted service before this pin
is trusted; the numbers go into frank-archon with the boot ledger.
Ordered 2026-10-19: migrate `frank-archon` to the latest radiance so the endpoint gets
the step-sharing knobs. This is a shape-model cutover, not a digest swap.
**Why**: decode pays for the prefill chunk it shares a step with (decode-only step
18.7 ms vs 317 ms for a chunk step; decode median 13.3 t/s while prefill runs >=1000 t/s).
`--busy-prefill-chunk` and `--prefill-interleave` are the engine's answers and neither
exists in the pinned 1.0.13.
**The two breaks, both verified on the images themselves, not inferred from docs**:
|Version|Facts|
|---|---|
|1.0.13 (this pin)|has `--vram-weights-mib` / `--vram-kv-mib` and the prefix-cache host+disk tiers; has NO prefill knob|
|1.3.0|the VRAM pair is REMOVED — refusal, not warning: "the card is split between the weights and the cache from what the model needs"|
|1.4.0|`--busy-prefill-chunk` arrives|
|1.7.0|`--prefill-interleave` arrives|
|1.8.1 (target)|prefix cache is host RAM only; `--prefix-cache-dir` / `-disk-mib` are refusals|
So there is NO version that carries the knobs and still accepts the old argv: the pair
died at 1.3.0, the knobs were born at 1.4.0. The migration therefore moves the endpoint
onto the engine's budget resolver, which derives the per-card split from
`--gpu-headroom-mib`, the cache bound (`--max-model-len`, `--max-num-seqs`) and
`--host-pool-mib`, and lets the cache lend idle blocks to the expert slab.
**What changes on the host**:
- expert slab MiB and KV pool tokens become BOOT-LEDGER lines; the 3 x 200,000 fp8
capacity contract is verified live by the capacity leg of `probe-frank-archon-kv.py`,
not computed from an env number. The engine refuses to boot rather than shrink a pool,
so a shortfall is loud.
- 64 GiB of NVMe prefix cache becomes re-prefill. `prefix_hit_rate` is the watch item —
a drop is extra prefill, which is the very traffic that costs decode its cadence.
- `FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto` is passed explicitly (auto vs off is the
difference between decode waiting one chunk and waiting a whole step);
`FRANK_ARCHON_PREFILL_INTERLEAVE=0` stays opt-in through a guarded argv array.
**Pre-flighted before any downtime**: the exact production argv was captured with a
`podman` shim around the edited serve script and replayed against the 1.8.1 image without
device access — every flag accepted, startup report echoes `--busy-prefill-chunk auto
[command line]` and stops only on "tp 2 needs 2 devices".
**Tests**: the two budget-arithmetic seams asserted knobs that no longer exist; they are
replaced by a seam asserting the retired knobs are ABSENT from env and argv (a stale line
is an ExecStart refusal, i.e. an outage) plus the tune-mount/`/data`-absence contract.
`frank_archon_unit.bats` 20/20; full suite in flight.
**Acceptance gate**: any pin past 1.2.0 must be re-scored (1.2.0 measured 0.7123 against
the 0.7549 control). The greedy probe leg runs against the booted service before this pin
is trusted; the numbers go into [[frank-archon]] with the boot ledger.
---
cwd: /home/blank/.omp/wt/wt-archon-181 · session: 01a15366-7601-7175-bb69-c62186c3f8b5
The decode stall is step contention (measured: 18.7 ms decode-only steps
against 317 ms steps that carry a prefill chunk, decode at 13 t/s median
while prefill runs), and the two engine knobs that fix it arrived after this
endpoint's pin. Moving to 1.8.1 (2026-10-18) brings both, and brings two CLI
breaks with them:
- --vram-weights-mib / --vram-kv-mib were REMOVED at 1.3.0. The per-card
split between the expert slab and the KV pool is now derived by the
engine's budget resolver from --gpu-headroom-mib, the cache bound
(--max-model-len, --max-num-seqs) and --host-pool-mib, and the cache may
lend idle blocks to the expert slab. The slab size and the pool's token
count are boot-ledger lines from here on, so the capacity contract
(3 x 200,000 fp8) is verified live by the probe's capacity leg instead of
computed from an env number.
- The prefix cache has no disk tier any more: --prefix-cache-dir and
--prefix-cache-disk-mib are refusals and --prefix-cache-host-mib is
--prefix-cache-mib. 64 GiB of parked KV on NVMe becomes re-prefill, so
prefix_hit_rate is the thing to watch after this boot.
env: image re-pinned to the 1.8.1 digest; the pair and the disk tier dropped;
FRANK_ARCHON_PREFIX_CACHE_MIB replaces the host/disk pair; new
FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto (explicit, because auto vs off is the
difference between decode waiting one chunk and waiting a whole step) and
FRANK_ARCHON_PREFILL_INTERLEAVE=0 (opt-in through a guarded argv array).
serve: argv follows the new CLI, /data mount and the pre-created kvcache path
go with the disk tier, and the retired env keys are no longer required (they
would abort ExecStart). autoresearch-frank-archon.sh takes the same CLI, with
a cold prefill now coming from --prefix-cache-mib 0 instead of a per-run disk
directory. run-swift15-bf16.sh is untouched: it pins its own engines by design.
tests: the two budget-arithmetic seams asserted knobs that no longer exist and
are replaced by one seam that asserts the retired knobs are ABSENT from env and
argv - a stale line is a boot refusal, not a no-op. The container-contract seam
now asserts the tune mount and the absence of /data.
dank
merged commit 40ad1f4a60 into main2026-10-19 10:21:27 -07:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Ordered 2026-10-19: migrate
frank-archonto the latest radiance so the endpoint getsthe step-sharing knobs. This is a shape-model cutover, not a digest swap.
Why: decode pays for the prefill chunk it shares a step with (decode-only step
18.7 ms vs 317 ms for a chunk step; decode median 13.3 t/s while prefill runs >=1000 t/s).
--busy-prefill-chunkand--prefill-interleaveare the engine's answers and neitherexists in the pinned 1.0.13.
The two breaks, both verified on the images themselves, not inferred from docs:
--vram-weights-mib/--vram-kv-miband the prefix-cache host+disk tiers; has NO prefill knob--busy-prefill-chunkarrives--prefill-interleavearrives--prefix-cache-dir/-disk-mibare refusalsSo there is NO version that carries the knobs and still accepts the old argv: the pair
died at 1.3.0, the knobs were born at 1.4.0. The migration therefore moves the endpoint
onto the engine's budget resolver, which derives the per-card split from
--gpu-headroom-mib, the cache bound (--max-model-len,--max-num-seqs) and--host-pool-mib, and lets the cache lend idle blocks to the expert slab.What changes on the host:
capacity contract is verified live by the capacity leg of
probe-frank-archon-kv.py,not computed from an env number. The engine refuses to boot rather than shrink a pool,
so a shortfall is loud.
prefix_hit_rateis the watch item —a drop is extra prefill, which is the very traffic that costs decode its cadence.
FRANK_ARCHON_BUSY_PREFILL_CHUNK=autois passed explicitly (auto vs off is thedifference between decode waiting one chunk and waiting a whole step);
FRANK_ARCHON_PREFILL_INTERLEAVE=0stays opt-in through a guarded argv array.Pre-flighted before any downtime: the exact production argv was captured with a
podmanshim around the edited serve script and replayed against the 1.8.1 image withoutdevice access — every flag accepted, startup report echoes
--busy-prefill-chunk auto [command line]and stops only on "tp 2 needs 2 devices".Tests: the two budget-arithmetic seams asserted knobs that no longer exist; they are
replaced by a seam asserting the retired knobs are ABSENT from env and argv (a stale line
is an ExecStart refusal, i.e. an outage) plus the tune-mount/
/data-absence contract.frank_archon_unit.bats20/20; full suite in flight.Acceptance gate: any pin past 1.2.0 must be re-scored (1.2.0 measured 0.7123 against
the 0.7549 control). The greedy probe leg runs against the booted service before this pin
is trusted; the numbers go into frank-archon with the boot ledger.
cwd: /home/blank/.omp/wt/wt-archon-181 · session: 01a15366-7601-7175-bb69-c62186c3f8b5
The decode stall is step contention (measured: 18.7 ms decode-only steps against 317 ms steps that carry a prefill chunk, decode at 13 t/s median while prefill runs), and the two engine knobs that fix it arrived after this endpoint's pin. Moving to 1.8.1 (2026-10-18) brings both, and brings two CLI breaks with them: - --vram-weights-mib / --vram-kv-mib were REMOVED at 1.3.0. The per-card split between the expert slab and the KV pool is now derived by the engine's budget resolver from --gpu-headroom-mib, the cache bound (--max-model-len, --max-num-seqs) and --host-pool-mib, and the cache may lend idle blocks to the expert slab. The slab size and the pool's token count are boot-ledger lines from here on, so the capacity contract (3 x 200,000 fp8) is verified live by the probe's capacity leg instead of computed from an env number. - The prefix cache has no disk tier any more: --prefix-cache-dir and --prefix-cache-disk-mib are refusals and --prefix-cache-host-mib is --prefix-cache-mib. 64 GiB of parked KV on NVMe becomes re-prefill, so prefix_hit_rate is the thing to watch after this boot. env: image re-pinned to the 1.8.1 digest; the pair and the disk tier dropped; FRANK_ARCHON_PREFIX_CACHE_MIB replaces the host/disk pair; new FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto (explicit, because auto vs off is the difference between decode waiting one chunk and waiting a whole step) and FRANK_ARCHON_PREFILL_INTERLEAVE=0 (opt-in through a guarded argv array). serve: argv follows the new CLI, /data mount and the pre-created kvcache path go with the disk tier, and the retired env keys are no longer required (they would abort ExecStart). autoresearch-frank-archon.sh takes the same CLI, with a cold prefill now coming from --prefix-cache-mib 0 instead of a per-run disk directory. run-swift15-bf16.sh is untouched: it pins its own engines by design. tests: the two budget-arithmetic seams asserted knobs that no longer exist and are replaced by one seam that asserts the retired knobs are ABSENT from env and argv - a stale line is a boot refusal, not a no-op. The container-contract seam now asserts the tune mount and the absence of /data.