frank-primary: fix the prefix-cache flag render (127 crash loop)

Two bash defects in the 9d8b185 prefix-cache wiring, both measured live
on the 2026-10-05 deploy attempt (unit crash-looped 190x, exit 127,
endpoint down for the whole 3600 s health window):

1. A # comment block inside the backslash-continued podman run
   terminates the continuation: the comment swallows the joined line
   and the flag line then runs as its own command
   ('--no-enable-prefix-caching: command not found'). The comment moves
   above the run block, with a no-comments-in-chain note.

2. The ${VAR:+A}${VAR:--B} double-expansion is ambiguous: when the
   variable is set, ${VAR:--B} parses as a SUBSTRING OFFSET, not the
   :- default — measured render was '--no-enable-prefix-caching1'.
   The flag is now branched explicitly into
   FRANK_PRIMARY_PREFIX_CACHE_FLAG (exactly one of the two flags
   reaches the engine; unset keeps the historical flag).

Seam: the render test sources the shipped env through the branch and
asserts the exact flag word the engine receives (render-assert, the
same pattern the spec knobs use). bash -n clean; all 22 frank-primary
seams pass.
This commit is contained in:
Frank
2026-10-05 05:18:53 -07:00
parent 9d8b18553d
commit b820d341f5
2 changed files with 39 additions and 11 deletions
+17 -6
View File
@@ -137,6 +137,22 @@ fi
# available" while device_count() says 2). --ipc=host for the engine's shared
# memory; memlock unbounded for the pinned host slabs.
# The entrypoint already execs `vllm serve "$@"`, so no `serve` here.
# Prefix caching: the fork's QFN mamba align path corrupts GDN SSM state
# at ~40k tokens (2026-10-04, 48 gdn_hip state faults, one request — the
# evidence block is in frank-primary.env). FRANK_PRIMARY_NO_PREFIX_CACHING
# flips to the fork-named mitigation (--no-enable-prefix-caching); empty
# keeps the historical flag. NO comment lines inside the backslash chain
# below — a comment there terminates the continuation and the next line
# runs as its own command (the 2026-10-05 exit-127 crash loop).
# The ${VAR:-word} / ${VAR:+word} double-expansion for the two directions is
# AMBIGUOUS bash (${VAR:--flag} parses as a substring offset when set —
# measured 2026-10-05: rendered "--no-enable-prefix-caching1"), so the flag
# is branched explicitly: exactly one of the two reaches the engine.
if [ -n "${FRANK_PRIMARY_NO_PREFIX_CACHING:-}" ]; then
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching
else
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching
fi
podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
--publish "$FRANK_PRIMARY_PUBLISH" \
--ipc=host --ulimit memlock=-1:-1 \
@@ -162,12 +178,7 @@ podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
--gpu-memory-utilization "$FRANK_PRIMARY_UTIL" \
--kv-cache-memory-bytes "$FRANK_PRIMARY_KV_BYTES" \
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
# Prefix caching: the fork's QFN mamba align path corrupts GDN SSM state at
# ~40k tokens (2026-10-04, 48 gdn_hip state faults, one request). FRANK_
# PRIMARY_NO_PREFIX_CACHING flips to the fork-named mitigation
# (--no-enable-prefix-caching); empty keeps the historical flag. The
# evidence block is in frank-primary.env.
${FRANK_PRIMARY_NO_PREFIX_CACHING:+--no-enable-prefix-caching}${FRANK_PRIMARY_NO_PREFIX_CACHING:---enable-prefix-caching} \
"$FRANK_PRIMARY_PREFIX_CACHE_FLAG" \
--enable-auto-tool-choice --tool-call-parser "$FRANK_PRIMARY_TOOL_PARSER" \
--reasoning-parser "$FRANK_PRIMARY_REASONING_PARSER" \
--default-chat-template-kwargs "$FRANK_PRIMARY_CHAT_TEMPLATE_KWARGS" \
+22 -5
View File
@@ -171,11 +171,28 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
grep -q -- '${FRANK_PRIMARY_COMPCONFIG:+--compilation-config "$FRANK_PRIMARY_COMPCONFIG"}' "$SERVE"
grep -q -- '${FRANK_PRIMARY_GEN_CONFIG:+--override-generation-config "$FRANK_PRIMARY_GEN_CONFIG"}' "$SERVE"
grep -q -- '--tensor-parallel-size 2' "$SERVE"
# Prefix caching renders exactly ONE flag from the knob: set ->
# --no-enable-prefix-caching (the fork-named mitigation), unset -> the
# historical --enable-prefix-caching. Both directions are explicit so a
# fork default change cannot silently flip the contract.
grep -qF '${FRANK_PRIMARY_NO_PREFIX_CACHING:+--no-enable-prefix-caching}${FRANK_PRIMARY_NO_PREFIX_CACHING:---enable-prefix-caching}' "$SERVE"
# Prefix caching renders exactly ONE flag from the knob, branched
# explicitly in the serve script (the ${VAR:--flag} double-expansion is
# ambiguous bash — when set it parsed as a substring offset, measured
# 2026-10-05): set -> --no-enable-prefix-caching (the fork-named
# mitigation), unset -> the historical --enable-prefix-caching. Both
# directions are spelled out so a fork default change cannot silently
# flip the contract.
grep -qF 'FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching' "$SERVE"
grep -qF 'FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching' "$SERVE"
grep -qF '"$FRANK_PRIMARY_PREFIX_CACHE_FLAG"' "$SERVE"
# The observable contract: the exact flag word the engine receives,
# rendered from the shipped env through the serve script's branch (the
# same render-assert pattern the spec knobs use above).
run bash -c '
set -a; . "$1" >/dev/null; set +a
if [ -n "${FRANK_PRIMARY_NO_PREFIX_CACHING:-}" ]; then
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching
else
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching
fi
printf %s "$FRANK_PRIMARY_PREFIX_CACHE_FLAG"' _ "$ENV_FILE"
[ "$output" = "--no-enable-prefix-caching" ]
grep -q -- '--trust-remote-code' "$SERVE"
# DCP is opt-in: the flag is passed only when FRANK_PRIMARY_DCP is set
# (DCP reuses the TP ranks when PCP is off, so TP=2 + DCP=2 fits two GPUs).