perf(quant): the calibration boot's step and slots are its own knobs (probe: 71 tok/s, 47 h tap)

Measured from the engine's own counters on the 2026-10-29 drain: prefill 71 tok/s steady state, a
180 s window showing 7 prefill steps of the capped 2,048 tokens and 3 preemptions. One 7,168-token
document is therefore ~100 s, round 1 (2,367 docs, 10.03M tokens) ~39 h, and the 12M-position
floor ~47 h of drained production — not the ~18 h the brief assumed.

The tap is the only phase whose cost is read volume per document: at the inherited 2,048-token step
a document is four steps, and each step re-streams every layer's routed expert union from the
container (43.4 GiB a rank live on disk). One wide step reads each layer's union once.

The 'one step, one slot count' audit law binds kldref and score because their KL numbers are only
comparable at an identical shape. Calib produces layer Hessians and no number any report compares,
so Q38_CALIB_MNBT / Q38_CALIB_SEQS are its own, defaulting to the inherited value — nothing moves
unless a probe asks. The KL modes ignore both.

Tests: 589/0 — the knob reaches calib's argv, does not reach kldref's, and a non-numeric value is
refused by name.
This commit is contained in:
Frank committed 2026-10-29 16:50:50 -07:00
1 parent a6fb0209e8
commit f6298e3501
2 files changed
+36 -2

No files matched your search

+15
View File
@@ -716,3 +716,18 @@ PY
[ "$status" -eq 0 ]
[ "$output" = "flat shrink grew done" ]
}
@test "the calibration boot's step is its own knob; the KL modes keep production's" {
# Calib is the only phase whose cost is read volume per document: four 2,048-token steps
# re-stream each layer's routed experts four times. Measured 2026-10-29: 71 tok/s, 47 h tap.
run env Q38_CALIB_MNBT=8192 bash "$RUNNER" calib --dry-run
[ "$status" -eq 0 ]
grep -A1 -x -- '--max-num-batched-tokens' <<<"$output" | grep -qx '8192'
# and the same knob must NOT reach the measuring shape those numbers are compared in
run env Q38_CALIB_MNBT=8192 bash "$RUNNER" kldref --dry-run
[ "$status" -eq 0 ]
! grep -qx '8192' <<<"$output"
run env Q38_CALIB_MNBT=wide bash "$RUNNER" calib --dry-run
[ "$status" -eq 2 ]
[[ "$output" == *"Q38_CALIB_MNBT must be a number"* ]]
}