frank-large: hand vLLM the CONTAINER model path; link the new runbook

The first crash-looping boot (51 restarts): serve received the host path
/home/dank/models/... but the models tree is mounted at /models inside the
container, so transformers failed 'Repo id must be in the form
namespace/repo_name' before the engine even built. Same conversion the spec
draft already used. Pinned by bats. Root docs index links the new runbook
(seam required it) and the work log records the crash-loop plus the 52 G
payload reclaim.
This commit is contained in:
Frank
2026-10-01 19:39:55 -07:00
parent de383fe53e
commit 780c307354
4 changed files with 39 additions and 11 deletions
+6 -3
View File
@@ -116,8 +116,11 @@ done
# --device /dev/mem exists for the fleet's other GPU containers; the GTT
# aperture (114 GiB) needs no size knob. Models mount read-only wholesale so
# spec drafts resolve from disk, and the HF cache mount keeps tokenizer
# metadata out of the container's ephemeral layer.
# spec drafts resolve from disk. THE PATH HANDED TO vllm IS THE CONTAINER
# PATH: the engine runs inside this container, and a host path that is not
# mounted at the same location dies as "Repo id must be in the form
# 'namespace/repo_name'" (2026-10-01 first boot, 51 crash-loop restarts).
MODEL_CONTAINER_DIR="/models/${FRANK_LARGE_MODEL_DIR##*/}"
podman run -d \
--name "$FRANK_LARGE_CONTAINER" \
--device /dev/kfd --device /dev/dri --device /dev/mem \
@@ -129,7 +132,7 @@ podman run -d \
-p "$FRANK_LARGE_PUBLISH" \
--entrypoint vllm \
"$FRANK_LARGE_IMAGE" \
serve "$FRANK_LARGE_MODEL_DIR" \
serve "$MODEL_CONTAINER_DIR" \
--served-model-name "$FRANK_LARGE_SERVED_NAME" \
--host 0.0.0.0 --port "${FRANK_LARGE_PUBLISH##*:}" \
--dtype bfloat16 \