frank-primary vllm: orphan-container self-heal + HOME + stop/rm order

Unit restart SIGKILLs conmon before --rm cleanup runs, leaving the container
name registered: every restart then dies on name-in-use while the orphan
holds the port and GPUs (observed live 2026-09-27, NRestarts=6 with the
container Up and the unit auto-restarting). The serve script now clears its
own name before exec podman run; the deployer stops the unit and removes
the container BEFORE waiting on the port.

HOME inside the container pointed at the root-owned WORKDIR /app, so
vLLM's registry cache write hit PermissionError; HOME now points at the
cache volume.

The vLLM cutover itself is blocked on a structural wall (AMD Qwen4Exp impl
creates the 51.2B PLE table as a bf16 on-GPU param: 47.69 GiB transient on a
31.86 GiB card; the pinned-host UVA path that fits this hardware is
CUDA-gated). llama.cpp is restored live at :7770; host HEAD unchanged.
This commit is contained in:
Frank committed 2026-09-27 01:08:21 -07:00
1 parent ecaad36654
commit baba84fc90
2 files changed
+13 -7

No files matched your search

+9 -3
View File
@@ -78,7 +78,7 @@ fi
mkdir -p /home/dank/.cache/frank-primary-vllm
SERVE_ARGS=(serve "$MODEL_DIR"
SERVE_ARGS=("$MODEL_DIR"
--served-model-name "$SERVED_NAME"
--host "$BIND_HOST"
--port "$PORT"
@@ -99,6 +99,10 @@ SERVE_ARGS=(serve "$MODEL_DIR"
if [ -n "${SPEC_CONFIG:-}" ]; then
SERVE_ARGS+=(--speculative-config "$SPEC_CONFIG")
fi
# A crashed/killed unit leaves the container name registered (systemd can
# SIGKILL conmon before the --rm cleanup runs); podman run would then die
# on name-in-use forever. Clear any orphan of OUR name first — idempotent.
podman rm -f "$CONTAINER_NAME" >/dev/null 2>&1 || true
exec podman run --rm \
--name "$CONTAINER_NAME" \
@@ -113,11 +117,13 @@ exec podman run --rm \
-e ROCR_VISIBLE_DEVICES="$ROCR_VISIBLE_DEVICES" \
-e HIP_VISIBLE_DEVICES="$HIP_VISIBLE_DEVICES" \
-e VLLM_ROCM_USE_AITER="$VLLM_ROCM_USE_AITER" \
-e NCCL_P2P_DISABLE="$NCCL_P2P_DISABLE" \
-e TRITON_CACHE_DIR="$CACHE_ROOT/triton" \
-e TORCHINDUCTOR_CACHE_DIR="$CACHE_ROOT/inductor" \
-e VLLM_CACHE_DIR="$CACHE_ROOT/vllm" \
-e HOME="$CACHE_ROOT" \
-e TRITON_ALLOW_NON_CONST_GLOBALS=1 \
--volume /home/dank/models:/models \
--volume /home/dank/.cache/frank-primary-vllm:"$CACHE_ROOT" \
"$IMAGE" \
vllm "${SERVE_ARGS[@]}"
"${SERVE_ARGS[@]}"