frank-large: hand vLLM the CONTAINER model path; link the new runbook
The first crash-looping boot (51 restarts): serve received the host path /home/dank/models/... but the models tree is mounted at /models inside the container, so transformers failed 'Repo id must be in the form namespace/repo_name' before the engine even built. Same conversion the spec draft already used. Pinned by bats. Root docs index links the new runbook (seam required it) and the work log records the crash-loop plus the 52 G payload reclaim.
This commit is contained in:
@@ -116,8 +116,11 @@ done
|
||||
|
||||
# --device /dev/mem exists for the fleet's other GPU containers; the GTT
|
||||
# aperture (114 GiB) needs no size knob. Models mount read-only wholesale so
|
||||
# spec drafts resolve from disk, and the HF cache mount keeps tokenizer
|
||||
# metadata out of the container's ephemeral layer.
|
||||
# spec drafts resolve from disk. THE PATH HANDED TO vllm IS THE CONTAINER
|
||||
# PATH: the engine runs inside this container, and a host path that is not
|
||||
# mounted at the same location dies as "Repo id must be in the form
|
||||
# 'namespace/repo_name'" (2026-10-01 first boot, 51 crash-loop restarts).
|
||||
MODEL_CONTAINER_DIR="/models/${FRANK_LARGE_MODEL_DIR##*/}"
|
||||
podman run -d \
|
||||
--name "$FRANK_LARGE_CONTAINER" \
|
||||
--device /dev/kfd --device /dev/dri --device /dev/mem \
|
||||
@@ -129,7 +132,7 @@ podman run -d \
|
||||
-p "$FRANK_LARGE_PUBLISH" \
|
||||
--entrypoint vllm \
|
||||
"$FRANK_LARGE_IMAGE" \
|
||||
serve "$FRANK_LARGE_MODEL_DIR" \
|
||||
serve "$MODEL_CONTAINER_DIR" \
|
||||
--served-model-name "$FRANK_LARGE_SERVED_NAME" \
|
||||
--host 0.0.0.0 --port "${FRANK_LARGE_PUBLISH##*:}" \
|
||||
--dtype bfloat16 \
|
||||
|
||||
Reference in New Issue
Block a user