frank-primary: prefix caching back ON — the OFF experiment did not fix the corruption
Frank: still corrupted at ~40k with --no-enable-prefix-caching deployed and mechanically verified (zero gdn_hip faults in the container serving that window, coherent 40,771-token boundary probe). The recurrence is FAULT-FREE, so the align-mode page re-seed is not the (only) mechanism and the mitigation bought whole-history re-prefill per turn for nothing. Revert: knob FRANK_PRIMARY_NO_PREFIX_CACHING commented out -> the serve branch renders --enable-prefix-caching (the shipped contract). The knob, the branch and the seam stay documented for a future fixed fork build. Open hypotheses recorded in the runbook: re-fed poisoned transcript (the same conversation cannot distinguish fixed from unfixed), or a second fault-free long-context path — fp8 KV at depth is the next suspect. Next diagnostic: fresh-conversation reproduction, then a KV-dtype A/B.
This commit is contained in:
1 parent
1fee99c8ce
commit
35a45b8a66
4 files changed
+57
-26
No files matched your search
@@ -137,13 +137,14 @@ fi
|
||||
# available" while device_count() says 2). --ipc=host for the engine's shared
|
||||
# memory; memlock unbounded for the pinned host slabs.
|
||||
# The entrypoint already execs `vllm serve "$@"`, so no `serve` here.
|
||||
# Prefix caching: the fork's QFN mamba align path corrupts GDN SSM state
|
||||
# at ~40k tokens (2026-10-04, 48 gdn_hip state faults, one request — the
|
||||
# evidence block is in frank-primary.env). FRANK_PRIMARY_NO_PREFIX_CACHING
|
||||
# flips to the fork-named mitigation (--no-enable-prefix-caching); empty
|
||||
# keeps the historical flag. NO comment lines inside the backslash chain
|
||||
# below — a comment there terminates the continuation and the next line
|
||||
# runs as its own command (the 2026-10-05 exit-127 crash loop).
|
||||
# Prefix caching: ON — the shipped contract. FRANK_PRIMARY_NO_PREFIX_-
|
||||
# CACHING=1 flips to the fork-named mitigation (--no-enable-prefix-caching)
|
||||
# for the QFN align-mode GDN state fault; it was tried 2026-10-05 and
|
||||
# REVERTED the same day (live conversations still degraded at ~40k with
|
||||
# caching off and zero faults — see the evidence block in
|
||||
# frank-primary.env). NO comment lines inside the backslash chain below —
|
||||
# a comment there terminates the continuation and the next line runs as
|
||||
# its own command (the 2026-10-05 exit-127 crash loop).
|
||||
# The ${VAR:-word} / ${VAR:+word} double-expansion for the two directions is
|
||||
# AMBIGUOUS bash (${VAR:--flag} parses as a substring offset when set —
|
||||
# measured 2026-10-05: rendered "--no-enable-prefix-caching1"), so the flag
|
||||
|
||||
Reference in new issue
Block a user