Commit Graph
2 Commits
Author SHA1 Message Date
Frank 96c5a0625c fix(qwen38): the rig's card split is inherited, not invented
run-qwen38-bf16.sh carried its own --host-pool-mib 12288 / --gpu-headroom-mib 768,
copied from the w4-era swift15 rig. The engine derives the whole per-card split from
those two numbers, and the KV cache's funding is `elastic - expert_floor` where the
expert floor is a function of the pinned host pool — so the pool buys cache as well
as residency. The first kldref boot got a 270 MiB / 7,436-position pool and the
recorder refused all 197 corpus documents ("document 0 is 10554 tokens and the KV
pool holds 7436"), exiting 1.

The rig now sources FRANK_ARCHON_HOST_POOL_MIB / FRANK_ARCHON_GPU_HEADROOM_MIB from
scripts/frank-archon.env, the file that owns them (49152 / 96), with `:?` guards, and
the header records the law instead of the claim that the pool never touches logits.
tests/qwen38_reference.bats pins the shape verbatim, so its SHAPE contract moves with
it; the suite is green (558 tests).
2026-10-27 21:54:19 -07:00
Frank a7329d190f feat(qwen38): the BF16 reference instrument - build it, drain once, record logits
Two tools the KLD gate depends on.

build-qwen38-bf16-rad.sh converts the checkpoint to a BF16 .rad with NO recipe
line, so every tensor stays bf16 - the reference must not carry the quantization
it is there to measure. It gates before writing 335 GiB: the checkpoint present
and fully verified by the fetcher's own verifier, the peak-disk bill (350,000
MiB: the checkpoint's 343,323 plus margin, because the .rad is built beside it),
an existing output retired to ~/.AGENTTRASH by --force instead of deleted, and
--plan-only before the real convert. --max-ctx 32768 is a declare-time check, not
a weight change: dropped, the resolver is handed the checkpoint's 262,144 window,
which at fp8 cache is 258.8 GiB on a 192 GiB machine.

run-qwen38-bf16.sh serves that container in three modes - kldref (record the
reference logits), score (record an arm's logits against that reference), calib
(open the Gram tap for the imatrix). It drains frank-archon and restores it on
every exit path, and the two logits modes ride ONE step size
(FRANK_QWEN38_MNBT, default 2048) and ONE window (FRANK_QWEN38_MAX_MODEL_LEN,
default 32768) so a reference and an arm cannot be measured at different shapes:
that is the whole validity of the KL number. Four slots over 2,048 matches
frank-archon.env's operating point (MAX_SEQS=4, MAX_BATCHED_TOKENS=2048); the
smaller host pool (12,288 vs 49,152) buys residency, which moves speed, not
logits. calib keeps 65,536 - decode-bound serving, never the gate.

kldref refuses before the drain when the corpus's longest record cannot fit the
declared window (limit = window x 3 chars, the pessimistic tokenizer bound; the
measured corpus is 38,263 chars, 2.6x inside 32,768). It runs on --dry-run too,
so the check is visible before a 20-minute drain is spent. 12 seam tests, each
red-proofed by neutralising the gate it pins.
2026-10-27 19:40:48 -07:00