frank-primary: prefix caching back ON — the OFF experiment did not fix the corruption
Frank: still corrupted at ~40k with --no-enable-prefix-caching deployed and mechanically verified (zero gdn_hip faults in the container serving that window, coherent 40,771-token boundary probe). The recurrence is FAULT-FREE, so the align-mode page re-seed is not the (only) mechanism and the mitigation bought whole-history re-prefill per turn for nothing. Revert: knob FRANK_PRIMARY_NO_PREFIX_CACHING commented out -> the serve branch renders --enable-prefix-caching (the shipped contract). The knob, the branch and the seam stay documented for a future fixed fork build. Open hypotheses recorded in the runbook: re-fed poisoned transcript (the same conversation cannot distinguish fixed from unfixed), or a second fault-free long-context path — fp8 KV at depth is the next suspect. Next diagnostic: fresh-conversation reproduction, then a KV-dtype A/B.
This commit is contained in:
@@ -142,16 +142,16 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
grep -q '^FRANK_PRIMARY_CACHE_DIR=/home/dank/.cache/frank-primary$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_GATHER_CHUNKS=32$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_GATHER_LANES=64$' "$ENV_FILE"
|
||||
# Prefix caching is OFF (2026-10-05): with the flag on, the fork runs
|
||||
# QFN on mamba_cache_mode 'align', and its GDN state path went
|
||||
# non-finite mid-conversation (48 gdn_hip state faults, one request,
|
||||
# slot 42 heads 3/8, at the 13th 3136-token boundary = 40,768 — the
|
||||
# "~40k corruption" report). The fork's own guard names the mitigation:
|
||||
# --no-enable-prefix-caching. RAM was ruled out the same day (zero MCE,
|
||||
# in-band pattern stable, 4800 MT/s is the JEDEC floor). The evidence
|
||||
# block is FRANK_PRIMARY_NO_PREFIX_CACHING in the env file; re-enable
|
||||
# only with a fixed align path and a re-verification past 40,768 tokens.
|
||||
grep -q '^FRANK_PRIMARY_NO_PREFIX_CACHING=1$' "$ENV_FILE"
|
||||
# Prefix caching is ON (reverted 2026-10-05 the same day it was
|
||||
# disabled): the fork's QFN align-mode GDN state fault (48 gdn_hip
|
||||
# state faults, slot 42 heads 3/8, at the 13th 3136-token boundary =
|
||||
# 40,768 — the "~40k corruption" report) was mitigated with
|
||||
# --no-enable-prefix-caching, but live long conversations STILL
|
||||
# degraded at ~40k with caching off and zero faults logged — cost
|
||||
# with no fix, so the historical flag returned. The knob stays
|
||||
# commented in the env file for a future fixed fork build; flip it to
|
||||
# 1 only together with a re-verification past 40,768 tokens.
|
||||
grep -qE '^# FRANK_PRIMARY_NO_PREFIX_CACHING=1$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_MODEL_SHARDS=131$' "$ENV_FILE"
|
||||
}
|
||||
|
||||
@@ -175,9 +175,10 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
# explicitly in the serve script (the ${VAR:--flag} double-expansion is
|
||||
# ambiguous bash — when set it parsed as a substring offset, measured
|
||||
# 2026-10-05): set -> --no-enable-prefix-caching (the fork-named
|
||||
# mitigation), unset -> the historical --enable-prefix-caching. Both
|
||||
# directions are spelled out so a fork default change cannot silently
|
||||
# flip the contract.
|
||||
# mitigation, tried and REVERTED the same day — live corruption
|
||||
# persisted at ~40k with zero faults), unset -> the shipped
|
||||
# --enable-prefix-caching. Both directions are spelled out so a fork
|
||||
# default change cannot silently flip the contract.
|
||||
grep -qF 'FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching' "$SERVE"
|
||||
grep -qF 'FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching' "$SERVE"
|
||||
grep -qF '"$FRANK_PRIMARY_PREFIX_CACHE_FLAG"' "$SERVE"
|
||||
@@ -192,7 +193,7 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching
|
||||
fi
|
||||
printf %s "$FRANK_PRIMARY_PREFIX_CACHE_FLAG"' _ "$ENV_FILE"
|
||||
[ "$output" = "--no-enable-prefix-caching" ]
|
||||
[ "$output" = "--enable-prefix-caching" ]
|
||||
grep -q -- '--trust-remote-code' "$SERVE"
|
||||
# DCP is opt-in: the flag is passed only when FRANK_PRIMARY_DCP is set
|
||||
# (DCP reuses the TP ranks when PCP is off, so TP=2 + DCP=2 fits two GPUs).
|
||||
|
||||
Reference in New Issue
Block a user