dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-primary: fix the prefix-cache flag render (127 crash loop)

Browse Source 
Two bash defects in the 9d8b185 prefix-cache wiring, both measured live
on the 2026-10-05 deploy attempt (unit crash-looped 190x, exit 127,
endpoint down for the whole 3600 s health window):

1. A # comment block inside the backslash-continued podman run
   terminates the continuation: the comment swallows the joined line
   and the flag line then runs as its own command
   ('--no-enable-prefix-caching: command not found'). The comment moves
   above the run block, with a no-comments-in-chain note.

2. The ${VAR:+A}${VAR:--B} double-expansion is ambiguous: when the
   variable is set, ${VAR:--B} parses as a SUBSTRING OFFSET, not the
   :- default — measured render was '--no-enable-prefix-caching1'.
   The flag is now branched explicitly into
   FRANK_PRIMARY_PREFIX_CACHE_FLAG (exactly one of the two flags
   reaches the engine; unset keeps the historical flag).

Seam: the render test sources the shipped env through the branch and
asserts the exact flag word the engine receives (render-assert, the
same pattern the spec knobs use). bash -n clean; all 22 frank-primary
seams pass.
This commit is contained in:
Frank
2026-10-05 05:18:53 -07:00
parent 9d8b18553d
commit b820d341f5
2 changed files with 39 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/serve-frank-primary.sh
+17 -6
View File
@@ -137,6 +137,22 @@ fi
# available" while device_count() says 2). --ipc=host for the engine's shared
# memory; memlock unbounded for the pinned host slabs.
# The entrypoint already execs `vllm serve "$@"`, so no `serve` here.
# Prefix caching: the fork's QFN mamba align path corrupts GDN SSM state
# at ~40k tokens (2026-10-04, 48 gdn_hip state faults, one request — the
# evidence block is in frank-primary.env). FRANK_PRIMARY_NO_PREFIX_CACHING
# flips to the fork-named mitigation (--no-enable-prefix-caching); empty
# keeps the historical flag. NO comment lines inside the backslash chain
# below — a comment there terminates the continuation and the next line
# runs as its own command (the 2026-10-05 exit-127 crash loop).
# The ${VAR:-word} / ${VAR:+word} double-expansion for the two directions is
# AMBIGUOUS bash (${VAR:--flag} parses as a substring offset when set —
# measured 2026-10-05: rendered "--no-enable-prefix-caching1"), so the flag
# is branched explicitly: exactly one of the two reaches the engine.
if [ -n "${FRANK_PRIMARY_NO_PREFIX_CACHING:-}" ]; then
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching
else
FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching
fi
podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
--publish "$FRANK_PRIMARY_PUBLISH" \
--ipc=host --ulimit memlock=-1:-1 \
@@ -162,12 +178,7 @@ podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
--gpu-memory-utilization "$FRANK_PRIMARY_UTIL" \
--kv-cache-memory-bytes "$FRANK_PRIMARY_KV_BYTES" \
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
# Prefix caching: the fork's QFN mamba align path corrupts GDN SSM state at
# ~40k tokens (2026-10-04, 48 gdn_hip state faults, one request). FRANK_
# PRIMARY_NO_PREFIX_CACHING flips to the fork-named mitigation
# (--no-enable-prefix-caching); empty keeps the historical flag. The
# evidence block is in frank-primary.env.
${FRANK_PRIMARY_NO_PREFIX_CACHING:+--no-enable-prefix-caching}${FRANK_PRIMARY_NO_PREFIX_CACHING:---enable-prefix-caching} \
"$FRANK_PRIMARY_PREFIX_CACHE_FLAG" \
--enable-auto-tool-choice --tool-call-parser "$FRANK_PRIMARY_TOOL_PARSER" \
--reasoning-parser "$FRANK_PRIMARY_REASONING_PARSER" \
--default-chat-template-kwargs "$FRANK_PRIMARY_CHAT_TEMPLATE_KWARGS" \