frank-primary: env-driven spec/compilation/sampling knobs

Three engine knobs that were impossible to set without editing the run line:
FRANK_PRIMARY_SPEC (--speculative-config), FRANK_PRIMARY_COMPCONFIG
(--compilation-config) and FRANK_PRIMARY_GEN_CONFIG
(--override-generation-config). Each holds the whole JSON value; empty or
unset means the flag is absent, so the shipped shape E argv is unchanged.

The seam re-pins the law that justified the omission: "speculative decode is
ABSENT BY MEASUREMENT" compared the wrong pair — c1248d4 retired MTP by
weighing fp8+MTP4 (37.4-40.6 t/s, 54% acceptance) against the bf16 no-spec
boot (36.4-39.4), bf16 then proved unbootable, and fp8 came back WITHOUT MTP.
fp8+spec was never measured against fp8+no-spec on one shape, the no-spec
incumbent runs ~30 t/s under the fleet's 35 t/s target, and every vendor
flash-next recipe ships MTP3.

Also in the seam: a dead assertion is deleted (a failing `! grep` mid-test is
exempt from bats' errexit, so the pre-precise-memory `! grep
--expert-precise-memory` never ran), and the knob test asserts by RENDERING the
flag from the sourced env instead of grepping for an assignment.
This commit is contained in:
Frank committed 2026-10-02 16:37:52 -07:00
1 parent ea1132aa0a
commit d57174c8d3
3 files changed
+73 -14

No files matched your search

+29 -12
View File
@@ -140,7 +140,6 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
grep -q '^FRANK_PRIMARY_CACHE_DIR=/home/dank/.cache/frank-primary$' "$ENV_FILE"
grep -q '^FRANK_PRIMARY_GATHER_CHUNKS=32$' "$ENV_FILE"
grep -q '^FRANK_PRIMARY_GATHER_LANES=64$' "$ENV_FILE"
! grep -q '^FRANK_PRIMARY_SPEC=' "$ENV_FILE"
grep -q '^FRANK_PRIMARY_MODEL_SHARDS=131$' "$ENV_FILE"
}
@@ -152,7 +151,13 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
grep -q -- '--max-num-batched-tokens "$FRANK_PRIMARY_MNBT"' "$SERVE"
grep -q -- '-e R4D_LRU_GATHER_CHUNKS="$FRANK_PRIMARY_GATHER_CHUNKS"' "$SERVE"
grep -q -- '-e R4D_LRU_GATHER_LANES="$FRANK_PRIMARY_GATHER_LANES"' "$SERVE"
! grep -q -- '--speculative-config' "$SERVE"
# Speculative decode, the cudagraph capture ladder and the sampling
# defaults are all ENV-DRIVEN knobs (unset -> the flag is absent, which is
# the shipped shape today). JSON values carry no shell-special characters
# that this quoting cannot carry.
grep -q -- '${FRANK_PRIMARY_SPEC:+--speculative-config "$FRANK_PRIMARY_SPEC"}' "$SERVE"
grep -q -- '${FRANK_PRIMARY_COMPCONFIG:+--compilation-config "$FRANK_PRIMARY_COMPCONFIG"}' "$SERVE"
grep -q -- '${FRANK_PRIMARY_GEN_CONFIG:+--override-generation-config "$FRANK_PRIMARY_GEN_CONFIG"}' "$SERVE"
grep -q -- '--tensor-parallel-size 2' "$SERVE"
grep -q -- '--enable-prefix-caching' "$SERVE"
grep -q -- '--trust-remote-code' "$SERVE"
@@ -199,20 +204,32 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
# --enable-expert-offload flag in this arm; the legality of the pin is
# the unit's unbounded LimitMEMLOCK.
! grep -q -- '--enable-expert-offload' "$SERVE"
! grep -q -- '--expert-precise-memory' "$SERVE"
grep -q '^LimitMEMLOCK=infinity$' "$UNIT"
grep -q -- '--ipc=host --ulimit memlock=-1:-1' "$SERVE"
}
@test "speculative decode is ABSENT BY MEASUREMENT" {
# The 2026-10-01 overnight redo (prefetch OFF) re-measured the whole
# spec family against no-spec at matched geometry: no-spec measured
# 41.46/41.29/41.47 tok/s while the spec-era incumbent's 37-40 carried
# the prefetch tax, and MTP's 2.6 GiB/rank drafter reservation halves
# the pool (729,545 vs 975,342 tokens). The MTP3/4/6 untaxed re-measures
# never completed; until one shows a real win, no spec is the incumbent.
! grep -q '^FRANK_PRIMARY_SPEC=' "$ENV_FILE"
! grep -q -- '--speculative-config' "$SERVE"
@test "speculative decode is an ENV KNOB — the old retirement verdict was invalid" {
# The 2026-10-02 c1248d4 retirement of MTP compared the WRONG PAIR:
# fp8+MTP4 (37.4-40.6 tok/s, 54% acceptance) against the bf16 no-spec boot
# (36.4-39.4), then bf16 proved unbootable and fp8 came back WITHOUT MTP —
# so fp8+spec was never compared against fp8+no-spec on the same shape. The
# no-spec incumbent runs ~30 tok/s, under the fleet's 35 tok/s target, while
# EVERY vendor flash-next recipe ships MTP3, and our MTP4 run used the
# default GREEDY draft sampling (probabilistic is the other fork option and
# is what the recipes use under temp>0). So spec is a knob, not a law: the
# shipped value is whatever a matched-pair re-measurement wins.
grep -q 'THE RETIREMENT VERDICT WAS INVALID' "$ENV_FILE"
# The knobs are empty in the shipped env, so the `${VAR:+…}` wiring renders
# NO spec/compilation flag — asserted by rendering, not by grepping the
# assignment (shape E today has neither flag in its argv).
run bash -c 'set -a; . "$1" >/dev/null; set +a
printf %s "${FRANK_PRIMARY_SPEC:+--speculative-config $FRANK_PRIMARY_SPEC}${FRANK_PRIMARY_COMPCONFIG:+--compilation-config $FRANK_PRIMARY_COMPCONFIG}"' _ "$ENV_FILE"
[ -z "$output" ]
# The capture-ladder law: the piecewise graph ladder must contain
# num_speculative_tokens + 1, so an MTP3 shape captures [4, 8, 12] and the
# 2026-10-02 SEQS=2 failure (query_start_loc specialized to the constant 3)
# is why SEQS stays a multiple-compatible value.
grep -q 'num_speculative_tokens + 1' "$ENV_FILE"
}
@test "KV dtype is fp8 — the incumbent, and the only one that boots" {