fix(qwen38): the rig's card split is inherited, not invented
run-qwen38-bf16.sh carried its own --host-pool-mib 12288 / --gpu-headroom-mib 768,
copied from the w4-era swift15 rig. The engine derives the whole per-card split from
those two numbers, and the KV cache's funding is `elastic - expert_floor` where the
expert floor is a function of the pinned host pool — so the pool buys cache as well
as residency. The first kldref boot got a 270 MiB / 7,436-position pool and the
recorder refused all 197 corpus documents ("document 0 is 10554 tokens and the KV
pool holds 7436"), exiting 1.
The rig now sources FRANK_ARCHON_HOST_POOL_MIB / FRANK_ARCHON_GPU_HEADROOM_MIB from
scripts/frank-archon.env, the file that owns them (49152 / 96), with `:?` guards, and
the header records the law instead of the claim that the pool never touches logits.
tests/qwen38_reference.bats pins the shape verbatim, so its SHAPE contract moves with
it; the suite is green (558 tests).
This commit is contained in:
1 parent
a7329d190f
commit
96c5a0625c
2 files changed
+30
-17
No files matched your search
+29
-16
@@ -39,25 +39,33 @@
|
||||
# speculation, fp8 KV, a 32,768-token window, the PLE n-gram table pinned to RAM,
|
||||
# and the production chat template rather than the one baked into the container —
|
||||
# every scored container is asked the same tokenised questions the reference
|
||||
# answered. The window is 32,768 because this corpus is long: measured on
|
||||
# Dankserver 2026-10-27, kld.jsonl is 197 records with a longest of 38,263
|
||||
# characters (longest prompt FIELD 36,855, median record 27,228), which is ~9,600
|
||||
# tokens at the pessimistic chars/4 and ~7,700 at the ratio this repo actually
|
||||
# measured on the K3 corpus (380,143 chars = 76,568 tokens, chars/4.97). A 16,384
|
||||
# window cleared that by 1.7x; 32,768 clears every record of both corpora by at
|
||||
# least 2.6x, and the kldref gate below re-checks it against the file on disk.
|
||||
# answered. The window is 32,768: kld.jsonl is 197 records with a longest of
|
||||
# 38,263 characters (longest prompt FIELD 36,855, median record 27,228). Its
|
||||
# token length — 10,554 for that longest record, counted by the engine itself —
|
||||
# is in the funding paragraph below; the kldref gate re-checks the file against
|
||||
# the window before it boots anything.
|
||||
#
|
||||
# NO --vram-kv-mib: it was REMOVED at 1.3.0 and naming it is a boot refusal, not
|
||||
# a no-op — verified on the pinned image 2026-10-27: "E --vram-kv-mib was
|
||||
# removed: the card is split between the weights and the cache from what the
|
||||
# model needs." The budget resolver derives the per-card split from
|
||||
# --host-pool-mib and --gpu-headroom-mib, and those two are deliberately NOT
|
||||
# production's 49,152 / 1,536 (frank-archon.env:77): the pool decides WHICH
|
||||
# experts stay resident, so it buys speed and never logits — the shape the gate
|
||||
# has to hold identical is the kv dtype, the slot count, the step size, the spec
|
||||
# depth and the window. It is also the only figure that fits: a 335 GiB bf16
|
||||
# container cannot be booted against a 49 GiB resident set on a 192 GiB machine,
|
||||
# and a prefill-only pass does not care that the disk tier paces at ~3 GiB/s.
|
||||
# model needs." The budget resolver derives that split from --host-pool-mib and
|
||||
# --gpu-headroom-mib, and this rig inherits BOTH from frank-archon.env, the file
|
||||
# that owns them: the cache's funding is `elastic - expert_floor` and the expert
|
||||
# floor falls one MiB for every MiB of host pool (frank-archon.md, Laws), so the
|
||||
# pool buys KV as well as residency — a private pair is not a free knob. What
|
||||
# buying one cost, measured on the first kldref boot the same day with the w4
|
||||
# era's 12288 / 768: "I kv: the pool backs 6% of the worst case (4 sequences all
|
||||
# at 32768 tokens), 0.2 full-length sessions; a larger --host-pool-mib gives the
|
||||
# cache more" — a 270 MiB, 7,436-position cache, after which the recorder refused
|
||||
# every one of the 197 corpus documents ("E kld: document 0 is 10554 tokens and
|
||||
# the KV pool holds 7436, so it can never be scored") and exited 1. Production's
|
||||
# 49,152 per rank is affordable exactly when the rig runs: the rig stops
|
||||
# frank-archon first, and that stop frees the same 96 GiB the service was pinning.
|
||||
#
|
||||
# The 32,768-token window it pairs with is not up for grabs: the corpus's longest
|
||||
# prompt is 10,554 tokens by the engine's own count (the header's chars/4.97
|
||||
# estimate called that record ~7,700; the true ratio for this file is chars/3.6),
|
||||
# so the window clears it by 3x and the cache now funds it.
|
||||
#
|
||||
# The step size is ONE value for the recording and every scoring pass:
|
||||
# FRANK_QWEN38_MNBT, default 2,048. The gate's whole claim is that the reference,
|
||||
@@ -148,6 +156,11 @@ KLD_REF_DIR="$CAMPAIGN/kld-ref"
|
||||
CALIB_DIR="$CAMPAIGN/calib"
|
||||
C=qwen38-bf16
|
||||
SERVICE=frank-archon
|
||||
# The card split, taken from the file that owns it instead of a private pair:
|
||||
# --host-pool-mib also funds the KV cache (see the header), so hardcoding it
|
||||
# here once starved the recorder. The `:?` guards keep an empty env fatal.
|
||||
HOST_POOL_MIB="${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
|
||||
GPU_HEADROOM_MIB="${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
|
||||
|
||||
# ONE step size for both logits modes. The file tier streams only at <=2,048 and
|
||||
# the gate scores file-tier payloads (the served iq4r, the arms) against the
|
||||
@@ -201,7 +214,7 @@ fi
|
||||
# The measuring shape. Identical for kldref and score by construction — this
|
||||
# array is the only place the campaign's KL numbers are defined.
|
||||
ARGS=(--tp 2 --placement expert_tiered --weights-disk-tier
|
||||
--host-pool-mib 12288 --gpu-headroom-mib 768
|
||||
--host-pool-mib "$HOST_POOL_MIB" --gpu-headroom-mib "$GPU_HEADROOM_MIB"
|
||||
--deterministic --num-speculative-tokens 0
|
||||
--kv-cache-dtype fp8 --max-model-len "$MAX_MODEL_LEN"
|
||||
--ngram-placement ram
|
||||
|
||||
Reference in new issue
Block a user