fix(qwen38): calib rides the shared step; the wide one cost the container its cache
The calibration mode kept a private step (8 seats at MNBT 65,536) that it called "free throughput". The step's activation arena is billed to the card, so the wide step cost the container its own expert slab and KV: at 65,536 the arena is 10.65 GiB and the claim falls to 16.85 GiB (weights 14.24 GiB in VRAM, 56.24 GiB stranded on disk, KV 271 MiB for 8 seats), and the boot dies with "startup stopped: out of memory" (measured 2026-10-29, while the kldref boot at MNBT 1,024 — arena 504 MiB, claimable 30.87 GiB — had just recorded for 16 hours without fault). Every mode now takes its seat count from FRANK_ARCHON_MAX_SEQS, the file that owns it, exactly as PR #77 established for the card split, and its step from FRANK_QWEN38_MNBT. The per-mode literals in kldref and score follow, so one number defines the geometry for all three. The test that pinned the old exemption is re-pinned to the invariant that replaced it. tests/run.sh: 576/576 green.
This commit is contained in:
1 parent
fad87aa522
commit
a3ff55efd8
2 files changed
+40
-12
No files matched your search
@@ -94,9 +94,10 @@
|
||||
# bf16 container, kldref only), FRANK_QWEN38_CAMPAIGN (holds kld-ref/ and
|
||||
# calib/), FRANK_QWEN38_KLD_CORPUS, FRANK_QWEN38_TPL (the template copy
|
||||
# the container reads through /models), FRANK_QWEN38_LOG_DIR, and
|
||||
# FRANK_QWEN38_MNBT — the step size shared by kldref and score (default
|
||||
# FRANK_QWEN38_MNBT — the step size every mode boots at (default
|
||||
# 1,024: the file tier's ceiling is 2,048 and production runs there, but on
|
||||
# this payload the arena charged for a wider step is taken from the cache),
|
||||
# the slot count inherited as FRANK_ARCHON_MAX_SEQS — no per-mode literals —
|
||||
# and FRANK_QWEN38_MAX_MODEL_LEN — the window every mode boots at (32,768;
|
||||
# see the corpus census above, and raise it together with the corpus).
|
||||
# kldref refuses (exit 2) when the longest record of the corpus it is about to
|
||||
@@ -162,6 +163,15 @@ SERVICE=frank-archon
|
||||
# here once starved the recorder. The `:?` guards keep an empty env fatal.
|
||||
HOST_POOL_MIB="${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
|
||||
GPU_HEADROOM_MIB="${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
|
||||
# The slot count, from the same file that owns the card split: production runs
|
||||
# four slots and every mode measures at production's geometry. A per-mode
|
||||
# literal here is what let calib ask for eight slots and then starve its own
|
||||
# cache. The `:?` guard keeps an empty env fatal, as for the card split.
|
||||
MAX_SEQS="${FRANK_ARCHON_MAX_SEQS:?FRANK_ARCHON_MAX_SEQS must be set}"
|
||||
if ! [[ "$MAX_SEQS" =~ ^[0-9]+$ ]]; then
|
||||
echo "REFUSED: FRANK_ARCHON_MAX_SEQS must be a slot count (got: $MAX_SEQS)" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# ONE step size for both logits modes. The file tier streams only at <=2,048 and
|
||||
# the gate scores file-tier payloads (the served iq4r, the arms) against the
|
||||
@@ -232,7 +242,7 @@ case "$MODE" in
|
||||
# room the corpus needs. Corpus read-only, record dir rw.
|
||||
ARGS+=(--model "$MODEL_IN"
|
||||
--kld-record /data/kld-ref --kld-corpus /data/kld.jsonl
|
||||
--max-num-seqs 4 --max-num-batched-tokens "$MNBT")
|
||||
--max-num-seqs "$MAX_SEQS" --max-num-batched-tokens "$MNBT")
|
||||
MOUNTS+=(-v "$KLD_CORPUS:/data/kld.jsonl:ro"
|
||||
-v "$KLD_REF_DIR:/data/kld-ref")
|
||||
;;
|
||||
@@ -247,17 +257,26 @@ case "$MODE" in
|
||||
fi
|
||||
ARGS+=(--model "$MODEL_IN"
|
||||
--kld-ref /data/kld-ref --kld-out "/data/out/$(basename "$SCORE_OUT")"
|
||||
--max-num-seqs 4 --max-num-batched-tokens "$MNBT")
|
||||
--max-num-seqs "$MAX_SEQS" --max-num-batched-tokens "$MNBT")
|
||||
MOUNTS+=(-v "$KLD_REF_DIR:/data/kld-ref:ro"
|
||||
-v "$OUT_DIR:/data/out")
|
||||
;;
|
||||
calib)
|
||||
# Calibration is exempt from the shared step: the tap counts pooled
|
||||
# positions and nothing downstream is compared against a step size, so the
|
||||
# wide step is free throughput (a disk-tier step costs the same at 1 token
|
||||
# as at 64k).
|
||||
# Calibration rides the shared step like every other mode. The old
|
||||
# exemption — eight slots at MNBT 65,536 for "free throughput", because
|
||||
# "a disk-tier step costs the same at 1 token as at 64k" — is false: the
|
||||
# step's activation arena is billed on the card, not on the disk. The
|
||||
# engine's own ledger for this container: at MNBT 1,024 the arena is
|
||||
# 504 MiB and the card claims 30.87 GiB (weights 28.90 GiB of it); at
|
||||
# 65,536 the arena is 10.65 GiB (fixed region 1.25 GiB), the claim falls
|
||||
# to 16.85 GiB, the experts shrink to 10.98 GiB and the KV to 271 MiB —
|
||||
# one slot's worth of cache for eight slots, and the boot dies at
|
||||
# "startup stopped: out of memory" (measured 2026-10-29). A wide step
|
||||
# buys throughput with the expert slab, so price it like any other
|
||||
# VRAM knob. The slots follow frank-archon.env's FRANK_ARCHON_MAX_SEQS
|
||||
# through MAX_SEQS, exactly as kldref and score do.
|
||||
ARGS+=(--model "$MODEL_IN" --port 7770
|
||||
--max-num-seqs 8 --max-num-batched-tokens 65536)
|
||||
--max-num-seqs "$MAX_SEQS" --max-num-batched-tokens "$MNBT")
|
||||
MOUNTS+=(-v "$CALIB_DIR:/data/calib")
|
||||
TAPENV=(-e RADIANCE_CALIB_DIR=/data/calib
|
||||
-e RADIANCE_CALIB_TOKENS=12000000)
|
||||
|
||||
Reference in new issue
Block a user