dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-primary: fix the SIGPIPE race in both image-digest gates

Browse Source 
The classic 'podman images | grep -q' pattern races under set -o pipefail:
grep exits on its first match, podman dies SIGPIPE (141) writing the
remaining RepoDigests lines, and pipefail promotes 141 to the pipeline
status — the gate reads 'no match' and refuses to deploy (2026-09-30,
dankserver). Same bug in the serve script (it would restart-loop the
same way). Fix: capture podman's output into a variable first, grep the
variable. Patched in the R9V September deploys by luck of timing.
This commit is contained in:
Frank
2026-09-30 01:05:05 -07:00
parent 01fedb0081
commit 2339faeaa4
2 changed files with 13 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/deploy-remote.sh
+7 -1
View File
@@ -1237,7 +1237,13 @@ deploy_llm_primary() {
echo "ERROR: frank-primary.env missing FRANK_PRIMARY_IMAGE/MODEL_DIR/MODEL_SHARDS" >&2 echo "ERROR: frank-primary.env missing FRANK_PRIMARY_IMAGE/MODEL_DIR/MODEL_SHARDS" >&2
return 1 return 1
fi fi
if ! podman images --format '{{.RepoDigests}}' | grep -qF "${image#*@}"; then # Capture podman's output BEFORE grepping: under set -o pipefail, the
# classic `podman images | grep -q` races (grep exits on match, podman
# dies SIGPIPE 141 writing the rest, pipefail calls it no-match — the
# 2026-09-30 deploy refusal).
local digests
digests="$(podman images --format '{{.RepoDigests}}')"
if ! grep -qF "${image#*@}" <<<"$digests"; then
echo "ERROR: pinned runtime image absent from rootless storage: $image" >&2 echo "ERROR: pinned runtime image absent from rootless storage: $image" >&2
echo "frank-primary: fetch it (podman pull $image)" >&2 echo "frank-primary: fetch it (podman pull $image)" >&2
return 1 return 1
scripts/serve-frank-primary.sh
+6 -5
View File
@@ -46,11 +46,12 @@ set +a
: "${FRANK_PRIMARY_SEQS:?FRANK_PRIMARY_SEQS must be set}" : "${FRANK_PRIMARY_SEQS:?FRANK_PRIMARY_SEQS must be set}"
: "${FRANK_PRIMARY_KV_BYTES:?FRANK_PRIMARY_KV_BYTES must be set (PER RANK)}" : "${FRANK_PRIMARY_KV_BYTES:?FRANK_PRIMARY_KV_BYTES must be set (PER RANK)}"
: "${FRANK_PRIMARY_KV_DTYPE:?FRANK_PRIMARY_KV_DTYPE must be set (fp8 = q8/q8 KV)}" : "${FRANK_PRIMARY_KV_DTYPE:?FRANK_PRIMARY_KV_DTYPE must be set (fp8 = q8/q8 KV)}"
: "${FRANK_PRIMARY_SPEC:?FRANK_PRIMARY_SPEC must be set (0 disables)}" # Capture podman's output BEFORE grepping: under `set -o pipefail`, the
: "${FRANK_PRIMARY_UTIL:?FRANK_PRIMARY_UTIL must be set}" # classic `podman images | grep -q` pattern races — grep exits on its first
# match, podman dies SIGPIPE (141) writing the remaining lines, and pipefail
# --- gates: never serve a half-finished payload or a missing image ----------- # turns the 141 into "no match" (the 2026-09-30 deploy refusal).
if ! podman images --format '{{.RepoDigests}}' | grep -qF "${FRANK_PRIMARY_IMAGE#*@}"; then DIGESTS="$(podman images --format '{{.RepoDigests}}')"
if ! grep -qF "${FRANK_PRIMARY_IMAGE#*@}" <<<"$DIGESTS"; then
echo "serve-frank-primary: pinned image absent from rootless storage: $FRANK_PRIMARY_IMAGE" >&2 echo "serve-frank-primary: pinned image absent from rootless storage: $FRANK_PRIMARY_IMAGE" >&2
echo " fetch it: podman pull $FRANK_PRIMARY_IMAGE" >&2 echo " fetch it: podman pull $FRANK_PRIMARY_IMAGE" >&2
exit 1 exit 1