fix(qwen38): calib rides the shared step; the wide one cost the container its cache
The calibration mode kept a private step (8 seats at MNBT 65,536) that it called "free throughput". The step's activation arena is billed to the card, so the wide step cost the container its own expert slab and KV: at 65,536 the arena is 10.65 GiB and the claim falls to 16.85 GiB (weights 14.24 GiB in VRAM, 56.24 GiB stranded on disk, KV 271 MiB for 8 seats), and the boot dies with "startup stopped: out of memory" (measured 2026-10-29, while the kldref boot at MNBT 1,024 — arena 504 MiB, claimable 30.87 GiB — had just recorded for 16 hours without fault). Every mode now takes its seat count from FRANK_ARCHON_MAX_SEQS, the file that owns it, exactly as PR #77 established for the card split, and its step from FRANK_QWEN38_MNBT. The per-mode literals in kldref and score follow, so one number defines the geometry for all three. The test that pinned the old exemption is re-pinned to the invariant that replaced it. tests/run.sh: 576/576 green.
This commit is contained in:
1 parent
fad87aa522
commit
a3ff55efd8
2 files changed
+40
-12
No files matched your search
@@ -247,16 +247,19 @@ flag_value() {
|
||||
[ ! -d "$L" ]
|
||||
}
|
||||
|
||||
@test "the reference and a score pass ride ONE step size, legal on the file tier" {
|
||||
@test "every mode rides ONE step size and ONE seat count, legal on the file tier" {
|
||||
# The gate's claim is one shape; that shape's ceiling is set by the tightest
|
||||
# payload it boots, and the served iq4r and every arm are file-tier
|
||||
# containers, which stream only at <=2048. A reference recorded above that is
|
||||
# a KL between two shapes, not two weights.
|
||||
# a KL between two shapes, not two weights. Calib used to exempt itself from
|
||||
# the shared step; the exemption cost the container its own cache, so the
|
||||
# test now holds every mode to the same two numbers.
|
||||
M="$BATS_TEST_TMPDIR/models"
|
||||
C="$BATS_TEST_TMPDIR/campaign"
|
||||
run env FRANK_QWEN38_MODELS="$M" FRANK_QWEN38_CAMPAIGN="$C" bash "$RUNNER" kldref --dry-run
|
||||
[ "$status" -eq 0 ]
|
||||
ref_step="$(flag_value --max-num-batched-tokens <<< "$output")"
|
||||
ref_seats="$(flag_value --max-num-seqs <<< "$output")"
|
||||
[ -n "$ref_step" ]
|
||||
[ "$ref_step" -le 2048 ]
|
||||
run env FRANK_QWEN38_MODELS="$M" FRANK_QWEN38_CAMPAIGN="$C" \
|
||||
@@ -271,10 +274,16 @@ flag_value() {
|
||||
run env FRANK_QWEN38_MODELS="$M" FRANK_QWEN38_CAMPAIGN="$C" FRANK_QWEN38_MNBT=1024 \
|
||||
bash "$RUNNER" score "$M/rad/arm-w5nl64.rad" "$C/scores/arm1.json" --dry-run
|
||||
[ "$(flag_value --max-num-batched-tokens <<< "$output")" = "1024" ]
|
||||
# calibration is exempt: grams are not logits, so the tap keeps the wide step
|
||||
# calibration rides the shared step too. Its old exemption (8 seats at
|
||||
# MNBT 65,536, "free throughput") is what made the container refuse to
|
||||
# boot: at that step the engine charges a 10.65 GiB activation arena to
|
||||
# the card, the experts fall to 10.98 GiB and the KV to 271 MiB —
|
||||
# "0.0 full-length sessions", "startup stopped: out of memory", measured
|
||||
# 2026-10-29 while kldref on the shared step ran 16 hours without fault.
|
||||
run env FRANK_QWEN38_MODELS="$M" FRANK_QWEN38_CAMPAIGN="$C" FRANK_QWEN38_MNBT=1024 \
|
||||
bash "$RUNNER" calib --dry-run
|
||||
[ "$(flag_value --max-num-batched-tokens <<< "$output")" = "65536" ]
|
||||
[ "$(flag_value --max-num-batched-tokens <<< "$output")" = "1024" ]
|
||||
[ "$(flag_value --max-num-seqs <<< "$output")" = "$ref_seats" ]
|
||||
# a step size that is not a token count is refused, never handed to the engine
|
||||
run env FRANK_QWEN38_MNBT=broad bash "$RUNNER" kldref --dry-run
|
||||
[ "$status" -eq 2 ]
|
||||
|
||||
Reference in new issue
Block a user