frank-large: cutover to vLLM TheRock gfx1151, MiMo-V2.6 BF16 + cross-model MTP

Engine: digest-pinned docker.io/kyuz0/vllm-therock-gfx1151 (vLLM 0.30,
TheRock ROCm 10, Triton GDN kernels); the serve script gates the image by
REPO DIGEST, the payload (config + index + exact shard count), the spec
draft (BOTH weight layouts: single model.safetensors for dflash, sharded
index for the Qwen3.5-9B MTP head), and whole /dev/dri, sweeps the bench
identity vllm-mimo, then exec podman wait so a container death trips
Restart=always. Shape: 4 x 262144 at util 0.45 (KV grows only on the 8
full-attention layers), MTP 2 draft tokens.

Measured 2026-10-01 window (frank-large stopped): base 13.1 t/s decode
(~94% of the LPDDR5x BF16 roof), DFlash-8 17.4, MTP-2 19.8 with 71%/44%
per-position acceptance - the cross-model head works because vLLM reads the
draft from a separate path while MiMo ships zero mtp.* tensors. Spec arms
do not preserve greedy bf16 text (batched-verify reduction order).

The seam was rewritten around the new laws (18 tests): unit self-healing,
digest gating, engine wiring, the measured spec table, the retired gufo
gates, the deployer MainPID check. frank_small port test follows the
PORT -> FRANK_LARGE_PUBLISH rename. Rollback: tag frank-large-gufo-q4kxl-20261001.
This commit is contained in:
Frank committed 2026-10-01 18:48:32 -07:00
1 parent b3313340fa
commit dab483c79f
6 files changed
+487 -516

No files matched your search

+5 -4
View File
@@ -108,10 +108,11 @@ ENV_LARGE="$REPO_ROOT/scripts/frank-large.env"
grep -q '^ALIAS=frank-small$' "$ENV_FILE"
grep -q -- '--port "$LISTEN_PORT"' "$SERVE"
grep -q -- '--host "$BIND_ADDR"' "$SERVE"
# The two endpoints cannot share a port: frank-large holds 7770, this one
# 7771.
grep -q '^PORT=7770$' "$ENV_LARGE"
! grep -q '^PORT=7771$' "$ENV_LARGE"
# The two endpoints cannot share a port: frank-large publishes
# 7770:7770 through podman, this one binds 7771 natively. (The
# frank-large vLLM cutover renamed PORT to FRANK_LARGE_PUBLISH.)
grep -q '^FRANK_LARGE_PUBLISH=7770:7770$' "$ENV_LARGE"
! grep -q '^FRANK_LARGE_PUBLISH=7771' "$ENV_LARGE"
}
@test "the deploy handler installs the root unit and refuses a missing payload" {