frank-large: hand vLLM the CONTAINER model path; link the new runbook

The first crash-looping boot (51 restarts): serve received the host path
/home/dank/models/... but the models tree is mounted at /models inside the
container, so transformers failed 'Repo id must be in the form
namespace/repo_name' before the engine even built. Same conversion the spec
draft already used. Pinned by bats. Root docs index links the new runbook
(seam required it) and the work log records the crash-loop plus the 52 G
payload reclaim.
This commit is contained in:
Frank
2026-10-01 19:39:55 -07:00
parent de383fe53e
commit 780c307354
4 changed files with 39 additions and 11 deletions
+14 -8
View File
@@ -213,14 +213,20 @@ The LLM inference fleet on the two hosts. Full index:
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
rollback record)
- [[frank-large-gufo-qwen38-27b]] — the ACTIVE frank-large runbook since
2026-09-28 (root systemd unit, port 7770): gufo (gufo-org/gufo) v0.1.1,
the Strix Halo inference engine, built from source on Dankclaw and
serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (a 27B dense model) with the
z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU — one 262,144-token
conversation, adaptive block policy, greedy, text-only, the two
from-source build workarounds, what changed in the client contract, the
measured prefill/decode trade against llama.cpp
- [[frank-large-vllm-mimo-v26-bf16]] — the ACTIVE frank-large runbook
since 2026-10-01 (root systemd unit, port 7770): the digest-pinned
kyuz0/vllm-therock-gfx1151 image (vLLM 0.30, TheRock ROCm 10) serving
XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B in FULL BF16 on the gfx1151 iGPU,
4 x 262,144 ctx, with CROSS-MODEL MTP spec decode from Qwen/Qwen3.5-9B —
the roof math (13.1 t/s base is ~94% of LPDDR5x BF16), the measured
MTP-vs-DFlash table (19.8 vs 17.4 t/s), the greedy-text numerics trap,
the `reasoning` field change, troubleshooting
- [[frank-large-gufo-qwen38-27b]] — RETIRED 2026-09-28 → 2026-10-01
(replaced by [[frank-large-vllm-mimo-v26-bf16]]; rollback via tag
frank-large-gufo-q4kxl-20261001): gufo v0.1.1 built from source serving
Qwen3.8-27B GGUF + the z-lab DFlash2 drafter on the gfx1151 iGPU — the
two from-source build workarounds, the client-contract delta, the
measured prefill/decode trade
- [[frank-large-unsloth-qwen38-27b-q8xl]] — RETIRED 2026-09-28 (replaced by
[[frank-large-gufo-qwen38-27b]]; rollback target): stock llama.cpp
(unsloth b11030, ROCm gfx1151) serving