frank-primary vllm: orphan-container self-heal + HOME + stop/rm order

Unit restart SIGKILLs conmon before --rm cleanup runs, leaving the container
name registered: every restart then dies on name-in-use while the orphan
holds the port and GPUs (observed live 2026-09-27, NRestarts=6 with the
container Up and the unit auto-restarting). The serve script now clears its
own name before exec podman run; the deployer stops the unit and removes
the container BEFORE waiting on the port.

HOME inside the container pointed at the root-owned WORKDIR /app, so
vLLM's registry cache write hit PermissionError; HOME now points at the
cache volume.

The vLLM cutover itself is blocked on a structural wall (AMD Qwen4Exp impl
creates the 51.2B PLE table as a bf16 on-GPU param: 47.69 GiB transient on a
31.86 GiB card; the pinned-host UVA path that fits this hardware is
CUDA-gated). llama.cpp is restored live at :7770; host HEAD unchanged.
This commit is contained in:
Frank
2026-09-27 01:08:21 -07:00
parent ecaad36654
commit baba84fc90
2 changed files with 13 additions and 7 deletions
+4 -4
View File
@@ -1223,11 +1223,11 @@ PY
# A stale container with the same name makes `podman run --name` fail at
# start; remove any leftover from a failed previous attempt.
podman rm -f "$LLM_PRIMARY_CONTAINER" >/dev/null 2>&1 || true
# On a re-deploy the predecessor is THIS unit (the llama.cpp unit until
# the 2026-09-27 cutover, the vLLM unit after it): stop it before
# waiting on the port, or the wait deadlocks on the server it just
# stopped serving.
# A stopped unit can leave its container running: systemd's cgroup kill
# races conmon's --rm cleanup, and the orphan holds the port and GPUs.
# Stop the unit, then remove the container BEFORE waiting on the port.
run_privileged systemctl stop frank-primary.service 2>/dev/null || true
podman rm -f "$LLM_PRIMARY_CONTAINER" >/dev/null 2>&1 || true
# Nothing starts until the port is released: the lucebox cutover
# (2026-09-26) and the frank-large cutover (2026-09-16) both failed
# exactly here when the next engine bound 7770 a second too early.