frank-primary: prefix caching OFF — the fork's QFN align-mode GDN state fault
Measured 2026-10-04 18:46:07 PDT (live podman logs): 48 gdn_hip state faults in one 0.6 s burst — 'state out of ssm-cache range: max|S|=0 with non-finite values (slot=42 head=3|8)', one per TP rank, 23 GDN layers each — then the process-lifetime report cap. The guard's own text in this build names it: 'known Qwen3.8-Flash-Next (QFN) flaw under prefix caching … Mitigation: --no-enable-prefix-caching'. Mechanism: prefix caching puts QFN on mamba_cache_mode 'align', which checkpoints each sequence's GDN recurrent state into a mamba page per 3136-token block and re-seeds it from that page when a request resumes from a cached prefix. The NaN state landed at the 13th boundary (13 x 3136 = 40,768 — the reported ~40k mark) and persisted in the page: every later turn of that conversation re-seeded the same non-finite state for two heads, silently (fault-report cap already spent). RAM ruled out the same day: zero MCE/EDAC records since the boot under test, 8 GiB in-band pattern stable under live load, 4800 MT/s is the JEDEC floor on 5600-rated DIMMs, and the two-head single-slot NaN signature is a state-path fault, not scattered bit corruption. Fix: FRANK_PRIMARY_NO_PREFIX_CACHING=1 -> --no-enable-prefix-caching (flag renders in both directions from the knob). Cost: every turn re-prefills the whole history instead of at most 3135 tail tokens. Seam updated: the prefix-cache pin now asserts the conditional render and the shipped OFF value. Runbook carries the evidence + acceptance criteria; all 368 tests pass.
This commit is contained in:
@@ -142,6 +142,16 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
grep -q '^FRANK_PRIMARY_CACHE_DIR=/home/dank/.cache/frank-primary$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_GATHER_CHUNKS=32$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_GATHER_LANES=64$' "$ENV_FILE"
|
||||
# Prefix caching is OFF (2026-10-05): with the flag on, the fork runs
|
||||
# QFN on mamba_cache_mode 'align', and its GDN state path went
|
||||
# non-finite mid-conversation (48 gdn_hip state faults, one request,
|
||||
# slot 42 heads 3/8, at the 13th 3136-token boundary = 40,768 — the
|
||||
# "~40k corruption" report). The fork's own guard names the mitigation:
|
||||
# --no-enable-prefix-caching. RAM was ruled out the same day (zero MCE,
|
||||
# in-band pattern stable, 4800 MT/s is the JEDEC floor). The evidence
|
||||
# block is FRANK_PRIMARY_NO_PREFIX_CACHING in the env file; re-enable
|
||||
# only with a fixed align path and a re-verification past 40,768 tokens.
|
||||
grep -q '^FRANK_PRIMARY_NO_PREFIX_CACHING=1$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_MODEL_SHARDS=131$' "$ENV_FILE"
|
||||
}
|
||||
|
||||
@@ -161,7 +171,11 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
grep -q -- '${FRANK_PRIMARY_COMPCONFIG:+--compilation-config "$FRANK_PRIMARY_COMPCONFIG"}' "$SERVE"
|
||||
grep -q -- '${FRANK_PRIMARY_GEN_CONFIG:+--override-generation-config "$FRANK_PRIMARY_GEN_CONFIG"}' "$SERVE"
|
||||
grep -q -- '--tensor-parallel-size 2' "$SERVE"
|
||||
grep -q -- '--enable-prefix-caching' "$SERVE"
|
||||
# Prefix caching renders exactly ONE flag from the knob: set ->
|
||||
# --no-enable-prefix-caching (the fork-named mitigation), unset -> the
|
||||
# historical --enable-prefix-caching. Both directions are explicit so a
|
||||
# fork default change cannot silently flip the contract.
|
||||
grep -qF '${FRANK_PRIMARY_NO_PREFIX_CACHING:+--no-enable-prefix-caching}${FRANK_PRIMARY_NO_PREFIX_CACHING:---enable-prefix-caching}' "$SERVE"
|
||||
grep -q -- '--trust-remote-code' "$SERVE"
|
||||
# DCP is opt-in: the flag is passed only when FRANK_PRIMARY_DCP is set
|
||||
# (DCP reuses the TP ranks when PCP is off, so TP=2 + DCP=2 fits two GPUs).
|
||||
|
||||
Reference in New Issue
Block a user