docs: record the frank-small llama.cpp Q6_K cutover and the frank-large gufo rollback
This commit is contained in:
1 parent
8551bb006b
commit
a7c68db9d0
6 files changed
+185
-59
No files matched your search
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "frank-large on Dankclaw — vLLM TheRock, MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP"
|
||||
description: "The active frank-large runbook — the digest-pinned kyuz0 TheRock gfx1151 vLLM image serving XiaomiMiMo MiMo-V2.6-Distill-Qwen-9B in full BF16 on the Strix Halo iGPU: identity, the measured roof math, the cross-model Qwen3.5-9B MTP drafter and its acceptance curve, the greedy-text numerics trap, the client-contract field change, deployed flags, and what to check when it misbehaves."
|
||||
title: "frank-large on Dankclaw — vLLM TheRock, MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (RETIRED)"
|
||||
description: "The RETIRED frank-large runbook (2026-10-01, one day) — the digest-pinned kyuz0 TheRock gfx1151 vLLM image serving XiaomiMiMo MiMo-V2.6-Distill-Qwen-9B in full BF16 on the Strix Halo iGPU. Kept for the measured record: the roof math, the cross-model MTP drafter, and the autoresearch finding that spec decode is NET-NEGATIVE on real chat+reasoning traffic here (6.19 t/s with MTP-2 vs 9.01 t/s without). Rolled back to [[frank-large-gufo-qwen38-27b]] by Frank the same night."
|
||||
tags:
|
||||
- llm
|
||||
- frank-large
|
||||
@@ -17,12 +17,12 @@ timestamp: 2026-10-01
|
||||
|
||||
# frank-large — vLLM MiMo-V2.6-Distill-Qwen-9B BF16 (cross-model MTP)
|
||||
|
||||
> **ACTIVE since 2026-10-01.** Replaces [[frank-large-gufo-qwen38-27b]]
|
||||
> (gufo + Qwen3.8-27B GGUF, now the rollback target, git tag
|
||||
> `frank-large-gufo-q4kxl-20261001`). Same port (7770), same served alias
|
||||
> (`frank-large`). This is the fleet's second vLLM endpoint and the first
|
||||
> CROSS-MODEL speculative-decode deployment: the target is Xiaomi's distill,
|
||||
> the draft head is Qwen's.
|
||||
> **RETIRED 2026-10-01 night, one day active.** Frank's call after the
|
||||
> autoresearch loop: battery decode 6.19 t/s (MTP-2) / 9.01 t/s (no draft)
|
||||
> against gufo's ~15 t/s on the 27B — and every ≥6-bit quant path in vLLM
|
||||
> 0.30 is dead on gfx1151 (quark int8 rejected; all FP8 kernels CDNA-only;
|
||||
> torchao config unwired). Back to [[frank-large-gufo-qwen38-27b]]
|
||||
> (Q4_K_XL, 1 x 262144). The measured content below stands as the record.
|
||||
|
||||
## Identity
|
||||
|
||||
|
||||
Reference in new issue
Block a user