frank-primary: fix the prefix-cache flag render (127 crash loop)
Two bash defects in the 9d8b185 prefix-cache wiring, both measured live
on the 2026-10-05 deploy attempt (unit crash-looped 190x, exit 127,
endpoint down for the whole 3600 s health window):
1. A # comment block inside the backslash-continued podman run
terminates the continuation: the comment swallows the joined line
and the flag line then runs as its own command
('--no-enable-prefix-caching: command not found'). The comment moves
above the run block, with a no-comments-in-chain note.
2. The ${VAR:+A}${VAR:--B} double-expansion is ambiguous: when the
variable is set, ${VAR:--B} parses as a SUBSTRING OFFSET, not the
:- default — measured render was '--no-enable-prefix-caching1'.
The flag is now branched explicitly into
FRANK_PRIMARY_PREFIX_CACHE_FLAG (exactly one of the two flags
reaches the engine; unset keeps the historical flag).
Seam: the render test sources the shipped env through the branch and
asserts the exact flag word the engine receives (render-assert, the
same pattern the spec knobs use). bash -n clean; all 22 frank-primary
seams pass.
This commit is contained in:
@@ -137,6 +137,22 @@ fi
|
||||
# available" while device_count() says 2). --ipc=host for the engine's shared
|
||||
# memory; memlock unbounded for the pinned host slabs.
|
||||
# The entrypoint already execs `vllm serve "$@"`, so no `serve` here.
|
||||
# Prefix caching: the fork's QFN mamba align path corrupts GDN SSM state
|
||||
# at ~40k tokens (2026-10-04, 48 gdn_hip state faults, one request — the
|
||||
# evidence block is in frank-primary.env). FRANK_PRIMARY_NO_PREFIX_CACHING
|
||||
# flips to the fork-named mitigation (--no-enable-prefix-caching); empty
|
||||
# keeps the historical flag. NO comment lines inside the backslash chain
|
||||
# below — a comment there terminates the continuation and the next line
|
||||
# runs as its own command (the 2026-10-05 exit-127 crash loop).
|
||||
# The ${VAR:-word} / ${VAR:+word} double-expansion for the two directions is
|
||||
# AMBIGUOUS bash (${VAR:--flag} parses as a substring offset when set —
|
||||
# measured 2026-10-05: rendered "--no-enable-prefix-caching1"), so the flag
|
||||
# is branched explicitly: exactly one of the two reaches the engine.
|
||||
if [ -n "${FRANK_PRIMARY_NO_PREFIX_CACHING:-}" ]; then
|
||||
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching
|
||||
else
|
||||
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching
|
||||
fi
|
||||
podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
|
||||
--publish "$FRANK_PRIMARY_PUBLISH" \
|
||||
--ipc=host --ulimit memlock=-1:-1 \
|
||||
@@ -162,12 +178,7 @@ podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
|
||||
--gpu-memory-utilization "$FRANK_PRIMARY_UTIL" \
|
||||
--kv-cache-memory-bytes "$FRANK_PRIMARY_KV_BYTES" \
|
||||
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
|
||||
# Prefix caching: the fork's QFN mamba align path corrupts GDN SSM state at
|
||||
# ~40k tokens (2026-10-04, 48 gdn_hip state faults, one request). FRANK_
|
||||
# PRIMARY_NO_PREFIX_CACHING flips to the fork-named mitigation
|
||||
# (--no-enable-prefix-caching); empty keeps the historical flag. The
|
||||
# evidence block is in frank-primary.env.
|
||||
${FRANK_PRIMARY_NO_PREFIX_CACHING:+--no-enable-prefix-caching}${FRANK_PRIMARY_NO_PREFIX_CACHING:---enable-prefix-caching} \
|
||||
"$FRANK_PRIMARY_PREFIX_CACHE_FLAG" \
|
||||
--enable-auto-tool-choice --tool-call-parser "$FRANK_PRIMARY_TOOL_PARSER" \
|
||||
--reasoning-parser "$FRANK_PRIMARY_REASONING_PARSER" \
|
||||
--default-chat-template-kwargs "$FRANK_PRIMARY_CHAT_TEMPLATE_KWARGS" \
|
||||
|
||||
@@ -171,11 +171,28 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
grep -q -- '${FRANK_PRIMARY_COMPCONFIG:+--compilation-config "$FRANK_PRIMARY_COMPCONFIG"}' "$SERVE"
|
||||
grep -q -- '${FRANK_PRIMARY_GEN_CONFIG:+--override-generation-config "$FRANK_PRIMARY_GEN_CONFIG"}' "$SERVE"
|
||||
grep -q -- '--tensor-parallel-size 2' "$SERVE"
|
||||
# Prefix caching renders exactly ONE flag from the knob: set ->
|
||||
# --no-enable-prefix-caching (the fork-named mitigation), unset -> the
|
||||
# historical --enable-prefix-caching. Both directions are explicit so a
|
||||
# fork default change cannot silently flip the contract.
|
||||
grep -qF '${FRANK_PRIMARY_NO_PREFIX_CACHING:+--no-enable-prefix-caching}${FRANK_PRIMARY_NO_PREFIX_CACHING:---enable-prefix-caching}' "$SERVE"
|
||||
# Prefix caching renders exactly ONE flag from the knob, branched
|
||||
# explicitly in the serve script (the ${VAR:--flag} double-expansion is
|
||||
# ambiguous bash — when set it parsed as a substring offset, measured
|
||||
# 2026-10-05): set -> --no-enable-prefix-caching (the fork-named
|
||||
# mitigation), unset -> the historical --enable-prefix-caching. Both
|
||||
# directions are spelled out so a fork default change cannot silently
|
||||
# flip the contract.
|
||||
grep -qF 'FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching' "$SERVE"
|
||||
grep -qF 'FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching' "$SERVE"
|
||||
grep -qF '"$FRANK_PRIMARY_PREFIX_CACHE_FLAG"' "$SERVE"
|
||||
# The observable contract: the exact flag word the engine receives,
|
||||
# rendered from the shipped env through the serve script's branch (the
|
||||
# same render-assert pattern the spec knobs use above).
|
||||
run bash -c '
|
||||
set -a; . "$1" >/dev/null; set +a
|
||||
if [ -n "${FRANK_PRIMARY_NO_PREFIX_CACHING:-}" ]; then
|
||||
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching
|
||||
else
|
||||
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching
|
||||
fi
|
||||
printf %s "$FRANK_PRIMARY_PREFIX_CACHE_FLAG"' _ "$ENV_FILE"
|
||||
[ "$output" = "--no-enable-prefix-caching" ]
|
||||
grep -q -- '--trust-remote-code' "$SERVE"
|
||||
# DCP is opt-in: the flag is passed only when FRANK_PRIMARY_DCP is set
|
||||
# (DCP reuses the TP ranks when PCP is off, so TP=2 + DCP=2 fits two GPUs).
|
||||
|
||||
Reference in New Issue
Block a user