docs: record the frank-small llama.cpp Q6_K cutover and the frank-large gufo rollback

This commit is contained in:
Frank committed 2026-10-01 23:50:52 -07:00
1 parent 8551bb006b
commit a7c68db9d0
6 files changed
+185 -59

No files matched your search

@@ -1,7 +1,7 @@
---
type: reference
title: "frank-large on Dankclaw — vLLM TheRock, MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP"
description: "The active frank-large runbook — the digest-pinned kyuz0 TheRock gfx1151 vLLM image serving XiaomiMiMo MiMo-V2.6-Distill-Qwen-9B in full BF16 on the Strix Halo iGPU: identity, the measured roof math, the cross-model Qwen3.5-9B MTP drafter and its acceptance curve, the greedy-text numerics trap, the client-contract field change, deployed flags, and what to check when it misbehaves."
title: "frank-large on Dankclaw — vLLM TheRock, MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (RETIRED)"
description: "The RETIRED frank-large runbook (2026-10-01, one day) — the digest-pinned kyuz0 TheRock gfx1151 vLLM image serving XiaomiMiMo MiMo-V2.6-Distill-Qwen-9B in full BF16 on the Strix Halo iGPU. Kept for the measured record: the roof math, the cross-model MTP drafter, and the autoresearch finding that spec decode is NET-NEGATIVE on real chat+reasoning traffic here (6.19 t/s with MTP-2 vs 9.01 t/s without). Rolled back to [[frank-large-gufo-qwen38-27b]] by Frank the same night."
tags:
- llm
- frank-large
@@ -17,12 +17,12 @@ timestamp: 2026-10-01
# frank-large — vLLM MiMo-V2.6-Distill-Qwen-9B BF16 (cross-model MTP)
> **ACTIVE since 2026-10-01.** Replaces [[frank-large-gufo-qwen38-27b]]
> (gufo + Qwen3.8-27B GGUF, now the rollback target, git tag
> `frank-large-gufo-q4kxl-20261001`). Same port (7770), same served alias
> (`frank-large`). This is the fleet's second vLLM endpoint and the first
> CROSS-MODEL speculative-decode deployment: the target is Xiaomi's distill,
> the draft head is Qwen's.
> **RETIRED 2026-10-01 night, one day active.** Frank's call after the
> autoresearch loop: battery decode 6.19 t/s (MTP-2) / 9.01 t/s (no draft)
> against gufo's ~15 t/s on the 27B — and every ≥6-bit quant path in vLLM
> 0.30 is dead on gfx1151 (quark int8 rejected; all FP8 kernels CDNA-only;
> torchao config unwired). Back to [[frank-large-gufo-qwen38-27b]]
> (Q4_K_XL, 1 x 262144). The measured content below stands as the record.
## Identity