frank-large: cutover to vLLM TheRock gfx1151, MiMo-V2.6 BF16 + cross-model MTP
Engine: digest-pinned docker.io/kyuz0/vllm-therock-gfx1151 (vLLM 0.30, TheRock ROCm 10, Triton GDN kernels); the serve script gates the image by REPO DIGEST, the payload (config + index + exact shard count), the spec draft (BOTH weight layouts: single model.safetensors for dflash, sharded index for the Qwen3.5-9B MTP head), and whole /dev/dri, sweeps the bench identity vllm-mimo, then exec podman wait so a container death trips Restart=always. Shape: 4 x 262144 at util 0.45 (KV grows only on the 8 full-attention layers), MTP 2 draft tokens. Measured 2026-10-01 window (frank-large stopped): base 13.1 t/s decode (~94% of the LPDDR5x BF16 roof), DFlash-8 17.4, MTP-2 19.8 with 71%/44% per-position acceptance - the cross-model head works because vLLM reads the draft from a separate path while MiMo ships zero mtp.* tensors. Spec arms do not preserve greedy bf16 text (batched-verify reduction order). The seam was rewritten around the new laws (18 tests): unit self-healing, digest gating, engine wiring, the measured spec table, the retired gufo gates, the deployer MainPID check. frank_small port test follows the PORT -> FRANK_LARGE_PUBLISH rename. Rollback: tag frank-large-gufo-q4kxl-20261001.
This commit is contained in:
1 parent
b3313340fa
commit
dab483c79f
6 files changed
+487
-516
No files matched your search
@@ -108,10 +108,11 @@ ENV_LARGE="$REPO_ROOT/scripts/frank-large.env"
|
||||
grep -q '^ALIAS=frank-small$' "$ENV_FILE"
|
||||
grep -q -- '--port "$LISTEN_PORT"' "$SERVE"
|
||||
grep -q -- '--host "$BIND_ADDR"' "$SERVE"
|
||||
# The two endpoints cannot share a port: frank-large holds 7770, this one
|
||||
# 7771.
|
||||
grep -q '^PORT=7770$' "$ENV_LARGE"
|
||||
! grep -q '^PORT=7771$' "$ENV_LARGE"
|
||||
# The two endpoints cannot share a port: frank-large publishes
|
||||
# 7770:7770 through podman, this one binds 7771 natively. (The
|
||||
# frank-large vLLM cutover renamed PORT to FRANK_LARGE_PUBLISH.)
|
||||
grep -q '^FRANK_LARGE_PUBLISH=7770:7770$' "$ENV_LARGE"
|
||||
! grep -q '^FRANK_LARGE_PUBLISH=7771' "$ENV_LARGE"
|
||||
}
|
||||
|
||||
@test "the deploy handler installs the root unit and refuses a missing payload" {
|
||||
|
||||
Reference in new issue
Block a user