frank-primary: fix the prefix-cache flag render (127 crash loop)
Two bash defects in the 9d8b185 prefix-cache wiring, both measured live
on the 2026-10-05 deploy attempt (unit crash-looped 190x, exit 127,
endpoint down for the whole 3600 s health window):
1. A # comment block inside the backslash-continued podman run
terminates the continuation: the comment swallows the joined line
and the flag line then runs as its own command
('--no-enable-prefix-caching: command not found'). The comment moves
above the run block, with a no-comments-in-chain note.
2. The ${VAR:+A}${VAR:--B} double-expansion is ambiguous: when the
variable is set, ${VAR:--B} parses as a SUBSTRING OFFSET, not the
:- default — measured render was '--no-enable-prefix-caching1'.
The flag is now branched explicitly into
FRANK_PRIMARY_PREFIX_CACHE_FLAG (exactly one of the two flags
reaches the engine; unset keeps the historical flag).
Seam: the render test sources the shipped env through the branch and
asserts the exact flag word the engine receives (render-assert, the
same pattern the spec knobs use). bash -n clean; all 22 frank-primary
seams pass.
This commit is contained in:
@@ -137,6 +137,22 @@ fi
|
|||||||
# available" while device_count() says 2). --ipc=host for the engine's shared
|
# available" while device_count() says 2). --ipc=host for the engine's shared
|
||||||
# memory; memlock unbounded for the pinned host slabs.
|
# memory; memlock unbounded for the pinned host slabs.
|
||||||
# The entrypoint already execs `vllm serve "$@"`, so no `serve` here.
|
# The entrypoint already execs `vllm serve "$@"`, so no `serve` here.
|
||||||
|
# Prefix caching: the fork's QFN mamba align path corrupts GDN SSM state
|
||||||
|
# at ~40k tokens (2026-10-04, 48 gdn_hip state faults, one request — the
|
||||||
|
# evidence block is in frank-primary.env). FRANK_PRIMARY_NO_PREFIX_CACHING
|
||||||
|
# flips to the fork-named mitigation (--no-enable-prefix-caching); empty
|
||||||
|
# keeps the historical flag. NO comment lines inside the backslash chain
|
||||||
|
# below — a comment there terminates the continuation and the next line
|
||||||
|
# runs as its own command (the 2026-10-05 exit-127 crash loop).
|
||||||
|
# The ${VAR:-word} / ${VAR:+word} double-expansion for the two directions is
|
||||||
|
# AMBIGUOUS bash (${VAR:--flag} parses as a substring offset when set —
|
||||||
|
# measured 2026-10-05: rendered "--no-enable-prefix-caching1"), so the flag
|
||||||
|
# is branched explicitly: exactly one of the two reaches the engine.
|
||||||
|
if [ -n "${FRANK_PRIMARY_NO_PREFIX_CACHING:-}" ]; then
|
||||||
|
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching
|
||||||
|
else
|
||||||
|
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching
|
||||||
|
fi
|
||||||
podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
|
podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
|
||||||
--publish "$FRANK_PRIMARY_PUBLISH" \
|
--publish "$FRANK_PRIMARY_PUBLISH" \
|
||||||
--ipc=host --ulimit memlock=-1:-1 \
|
--ipc=host --ulimit memlock=-1:-1 \
|
||||||
@@ -162,12 +178,7 @@ podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
|
|||||||
--gpu-memory-utilization "$FRANK_PRIMARY_UTIL" \
|
--gpu-memory-utilization "$FRANK_PRIMARY_UTIL" \
|
||||||
--kv-cache-memory-bytes "$FRANK_PRIMARY_KV_BYTES" \
|
--kv-cache-memory-bytes "$FRANK_PRIMARY_KV_BYTES" \
|
||||||
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
|
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
|
||||||
# Prefix caching: the fork's QFN mamba align path corrupts GDN SSM state at
|
"$FRANK_PRIMARY_PREFIX_CACHE_FLAG" \
|
||||||
# ~40k tokens (2026-10-04, 48 gdn_hip state faults, one request). FRANK_
|
|
||||||
# PRIMARY_NO_PREFIX_CACHING flips to the fork-named mitigation
|
|
||||||
# (--no-enable-prefix-caching); empty keeps the historical flag. The
|
|
||||||
# evidence block is in frank-primary.env.
|
|
||||||
${FRANK_PRIMARY_NO_PREFIX_CACHING:+--no-enable-prefix-caching}${FRANK_PRIMARY_NO_PREFIX_CACHING:---enable-prefix-caching} \
|
|
||||||
--enable-auto-tool-choice --tool-call-parser "$FRANK_PRIMARY_TOOL_PARSER" \
|
--enable-auto-tool-choice --tool-call-parser "$FRANK_PRIMARY_TOOL_PARSER" \
|
||||||
--reasoning-parser "$FRANK_PRIMARY_REASONING_PARSER" \
|
--reasoning-parser "$FRANK_PRIMARY_REASONING_PARSER" \
|
||||||
--default-chat-template-kwargs "$FRANK_PRIMARY_CHAT_TEMPLATE_KWARGS" \
|
--default-chat-template-kwargs "$FRANK_PRIMARY_CHAT_TEMPLATE_KWARGS" \
|
||||||
|
|||||||
@@ -171,11 +171,28 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
|||||||
grep -q -- '${FRANK_PRIMARY_COMPCONFIG:+--compilation-config "$FRANK_PRIMARY_COMPCONFIG"}' "$SERVE"
|
grep -q -- '${FRANK_PRIMARY_COMPCONFIG:+--compilation-config "$FRANK_PRIMARY_COMPCONFIG"}' "$SERVE"
|
||||||
grep -q -- '${FRANK_PRIMARY_GEN_CONFIG:+--override-generation-config "$FRANK_PRIMARY_GEN_CONFIG"}' "$SERVE"
|
grep -q -- '${FRANK_PRIMARY_GEN_CONFIG:+--override-generation-config "$FRANK_PRIMARY_GEN_CONFIG"}' "$SERVE"
|
||||||
grep -q -- '--tensor-parallel-size 2' "$SERVE"
|
grep -q -- '--tensor-parallel-size 2' "$SERVE"
|
||||||
# Prefix caching renders exactly ONE flag from the knob: set ->
|
# Prefix caching renders exactly ONE flag from the knob, branched
|
||||||
# --no-enable-prefix-caching (the fork-named mitigation), unset -> the
|
# explicitly in the serve script (the ${VAR:--flag} double-expansion is
|
||||||
# historical --enable-prefix-caching. Both directions are explicit so a
|
# ambiguous bash — when set it parsed as a substring offset, measured
|
||||||
# fork default change cannot silently flip the contract.
|
# 2026-10-05): set -> --no-enable-prefix-caching (the fork-named
|
||||||
grep -qF '${FRANK_PRIMARY_NO_PREFIX_CACHING:+--no-enable-prefix-caching}${FRANK_PRIMARY_NO_PREFIX_CACHING:---enable-prefix-caching}' "$SERVE"
|
# mitigation), unset -> the historical --enable-prefix-caching. Both
|
||||||
|
# directions are spelled out so a fork default change cannot silently
|
||||||
|
# flip the contract.
|
||||||
|
grep -qF 'FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching' "$SERVE"
|
||||||
|
grep -qF 'FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching' "$SERVE"
|
||||||
|
grep -qF '"$FRANK_PRIMARY_PREFIX_CACHE_FLAG"' "$SERVE"
|
||||||
|
# The observable contract: the exact flag word the engine receives,
|
||||||
|
# rendered from the shipped env through the serve script's branch (the
|
||||||
|
# same render-assert pattern the spec knobs use above).
|
||||||
|
run bash -c '
|
||||||
|
set -a; . "$1" >/dev/null; set +a
|
||||||
|
if [ -n "${FRANK_PRIMARY_NO_PREFIX_CACHING:-}" ]; then
|
||||||
|
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching
|
||||||
|
else
|
||||||
|
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching
|
||||||
|
fi
|
||||||
|
printf %s "$FRANK_PRIMARY_PREFIX_CACHE_FLAG"' _ "$ENV_FILE"
|
||||||
|
[ "$output" = "--no-enable-prefix-caching" ]
|
||||||
grep -q -- '--trust-remote-code' "$SERVE"
|
grep -q -- '--trust-remote-code' "$SERVE"
|
||||||
# DCP is opt-in: the flag is passed only when FRANK_PRIMARY_DCP is set
|
# DCP is opt-in: the flag is passed only when FRANK_PRIMARY_DCP is set
|
||||||
# (DCP reuses the TP ranks when PCP is off, so TP=2 + DCP=2 fits two GPUs).
|
# (DCP reuses the TP ranks when PCP is off, so TP=2 + DCP=2 fits two GPUs).
|
||||||
|
|||||||
Reference in New Issue
Block a user