---
type: reference
title: "frank-dragoon on Dankclaw — vLLM TheRock, MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (RETIRED)"
description: "The RETIRED frank-dragoon runbook (2026-10-01, one day) — the digest-pinned kyuz0 TheRock gfx1151 vLLM image serving XiaomiMiMo MiMo-V2.6-Distill-Qwen-9B in full BF16 on the Strix Halo iGPU. Kept for the measured record: the roof math, the cross-model MTP drafter, and the autoresearch finding that spec decode is NET-NEGATIVE on real chat+reasoning traffic here (6.19 t/s with MTP-2 vs 9.01 t/s without). Rolled back to [[frank-dragoon-gufo-qwen38-27b]] by Frank the same night."
tags:
  - llm
  - frank-dragoon
  - dankclaw
  - vllm
  - strix-halo
  - mimo
  - mtp
  - spec-decode
  - bf16
timestamp: 2026-10-01
---

# frank-dragoon — vLLM MiMo-V2.6-Distill-Qwen-9B BF16 (cross-model MTP)

> **RETIRED 2026-10-01 night, one day active.** Frank's call after the
> autoresearch loop: battery decode 6.19 t/s (MTP-2) / 9.01 t/s (no draft)
> against gufo's ~15 t/s on the 27B — and every ≥6-bit quant path in vLLM
> 0.30 is dead on gfx1151 (quark int8 rejected; all FP8 kernels CDNA-only;
> torchao config unwired). Back to [[frank-dragoon-gufo-qwen38-27b]]
> (Q4_K_XL, 1 x 262144). The measured content below stands as the record.

## Identity

- **Endpoint:** `frank-dragoon` at `192.168.0.143:7770` (Dankclaw).
- **Runtime:** vLLM 0.30.0 in the **digest-pinned** `docker.io/kyuz0/vllm-therock-gfx1151`
  image (image ID `2d24a8c5a3fd`, digest `sha256:98c1f3ac9833…` — ROCm 10.0.0 /
  torch 2.11.0 / TheRock, Triton Gated DeltaNet kernels). The official vLLM
  images are CUDA-only; this fork is the fleet's gfx1151 path. Plain
  `podman run` driven by `scripts/serve-frank-dragoon.sh` under a root unit
  (`scripts/frank-dragoon.service`, `User=dank`, `Restart=always`,
  `ExecStop=podman stop -t 120`, `TimeoutStopSec=180` > the stop grace).
- **Model:** `XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B` in **full BF16**
  (17.53 GiB, 4 shards) — the same checkpoint frank-zealot serves as an INT8
  W8A16 AutoRound conversion on the 3090, here unquantized.
- **Draft:** `Qwen/Qwen3.5-9B` (18 GiB on disk; only the 15 `mtp.*` tensors
  are loaded into the draft pass).

## Why this shape (the roof math)

Decode speed for a dense model on Strix Halo is bandwidth-bound:
BF16 weights x one pass per token against the LPDDR5x aperture. Measured
2026-10-01 on this host: **13.1 t/s is ~94% of the roof** — the payload, not
a misconfiguration. The same INT8 payload reaches 83.9 t/s on the 3090
because it moves ~12 GiB through ~936 GB/s. The only way past a bandwidth
wall is generating more than one token per weight pass: **speculative decode
is the performance feature of this endpoint**, not a garnish.

The memory side is the opposite of a squeeze: KV grows ONLY on the 8 full
attention layers (32 KiB/token bf16: 8 x 2 x 4 heads x 256 dim x 2 B);
the 24 Gated DeltaNet layers hold a FIXED ~25 MiB state per running
sequence. `--max-model-len 262144` (the native window) with
`--max-num-seqs 4` at `--gpu-memory-utilization 0.45` boots clean — the
four-session contract the gufo shape ran, with the pool shared by the
scheduler instead of carved per session.

## Spec method: cross-model MTP — measured, not assumed

| arm | prefill | decode | acceptance |
|---|---|---|---|
| BF16 base | 17,876 t/s | 13.1 t/s | — |
| BF16 + DFlash 8 tok | 7,976 t/s | 17.4 t/s | pos0 59% pos1 26% (token rate 14.8%) |
| **BF16 + MTP 2 tok** | 7,380 t/s | **19.8 t/s** | pos0 71% pos1 44% (token rate 57.4%) |

One session, temperature 0, 28,647-token prefill / 512-token decode, same
client for all three arms (2026-10-01 bench window, frank-dragoon stopped per
Benchmarking Policy). **MTP wins (+51% vs +33%) and needs only 2 draft
tokens** — acceptance past position 1 is 44% and falling; DFlash's own curve
decays 59/26/14/7, so 8 proposals are mostly wasted verification.
`num_speculative_tokens` beyond 2 is UNMEASURED; do not raise it without a
fresh A/B. Both spec arms pay ~55–59% of prefill (the draft passes share
the compute queues) — this endpoint's thinking-heavy traffic spends its wall
clock in decode, which is the regime where that trade wins.

**Why a Qwen head accepts on a MiMo target:** MiMo-V2.6 is a distillation of
Qwen3.5-9B that preserves the hidden-state geometry — identical
4096-hidden/248320-vocab/32-layer/interval-4 structure, and the 71%/44%
per-position acceptance is the direct measurement that the distillation
preserved more than geometry. MiMo itself ships **ZERO `mtp.*` tensors**
(verified in both the BF16 base and the INT8 conversion) — which is exactly
why frank-zealot's SGLang NEXTN draft is a 0.00-acceptance ghost: SGLang
reads the head from the TARGET checkpoint, vLLM reads the draft from a
separate path. That difference is the whole trick:

```
--speculative-config '{"method":"mtp","model":"/models/Qwen3.5-9B","num_speculative_tokens":2}'
```

The alternative drafter (`z-lab/Qwen3.5-9B-DFlash`, 6 layers, target_layer_ids
[1,5,…,29]) stays fetched on disk; flip `FRANK_DRAGOON_SPEC=dflash` and repoint
`FRANK_DRAGOON_SPEC_DIR` to use it. Its card's "vLLM PR #40898 unmerged" claim
is FALSE — `vllm/v1/spec_decode/dflash.py` ships in 0.30.

## The numerics trap (greedy text is NOT preserved by spec decode)

Measured: temperature-0 completions DIFFER across arms (sha `1318aeb6` base,
`0208b45d` dflash, `20c41c38` mtp). Batched verification changes the bf16
reduction order, argmax near-ties diverge, and the divergence cascades.
vLLM spec decode is distribution-preserving, not bitwise-greedy-preserving.
The gufo-era "greedy == autoregressive text" law does not carry over; do not
chase text drift between spec and no-spec arms as a regression.

## Payload

- Target: 4 safetensors shards (17.53 GiB) + config/tokenizer under
  `/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-BF16/`, each shard
  byte-count + LFS sha256-verified by
  `old/download-frank-dragoon-mimo.sh` (`--verify-only` re-checks; no token on
  the curl command line).
- MTP source: `old/download-frank-dragoon-qwen35.sh` →
  `/home/dank/models/Qwen3.5-9B/` (4 shards behind
  `model.safetensors.index.json`; the shards carry the odd HF asset names
  `model.safetensors-0000N-of-00004.safetensors` — the index resolves them,
  the engine boots from exactly this tree).
- DFlash drafter: `old/download-frank-dragoon-dflash.sh` →
  `/home/dank/models/Qwen3.5-9B-DFlash/`.
- **Payload status 2026-10-02:** all three trees moved to
  `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` (59 GiB staged with
  the other retired frank payloads) and the fetch trio moved to `old/` — the
  endpoint is two runtimes past this one, and re-fetch is cheaper than disk.
- The serve script gates the image by REPO DIGEST (rootless podman resolves
  tags from the network; the pin exists to boot exactly these bytes
  offline), then config + index + exact shard count, then the spec draft
  (either weight layout: single `model.safetensors` or a sharded index),
  then `/dev/kfd` + whole `/dev/dri` (never individual renderD nodes —
  Dankserver 2026-09-29 lesson).

## Client contract

- **Alias:** `frank-dragoon` (the `id` at `/v1/models`).
- **Reasoning:** `--reasoning-parser qwen3`; vLLM emits the thinking text in
  **`reasoning`**, NOT llama.cpp/gufo's `reasoning_content` — the harness
  `models.yml` compat block for this provider sets
  `reasoningContentField: "reasoning"`. Miss that and thinking silently
  vanishes from transcripts.
- **Tools:** `--enable-auto-tool-choice --tool-call-parser qwen3_coder`.
- **Vision:** the checkpoint carries the `qwen3_5_vision` tower but the
  serve script passes `--language-model-only` — the endpoint stays
  text-only by decision; dropping the flag brings vision back.
- **Sampling:** request-side; the harness sends the Qwen3.5-family recipe
  (temp 0.7, top_p 0.8) via `compat.extraBody`, and
  `chat_template_kwargs.{enable_thinking,preserve_thinking}` — the
  `chat_template_kwargs` form is what vLLM reads; a top-level
  `preserve_thinking` is ignored there.
- **Env hygiene:** every tunable is `FRANK_DRAGOON_`-prefixed because HIP
  reads bare `TEMP`/`HOST`/`GPU_MAX_HEAP_SIZE` and the serve script exports
  the env file with `set -a`.

## Ops

- Boot ~200–240 s warm; health 503 while loading (do not kill it).
- The serve script sweeps stale container identities before starting:
  `frank-dragoon-mimo` itself and the bench-window container `vllm-mimo`.
- Container death → `podman wait` returns non-zero from ExecStart →
  `Restart=always` self-heals (a clean exit-0 drain also restarts, the
  frank-archon lesson).
- Bench windows: `systemctl stop frank-dragoon` is enough (mask fails on the
  relinked unit; the deploy restores).
- First-deploy facts (2026-10-01, deploy #3): healthy after **230 s**;
  boot log `GPU KV cache size: 819,332 tokens, Maximum concurrency for
  262,144 tokens per request: 3.13x` — four sessions share the pool
  (the scheduler, not a carve). Live battery green: alias `frank-dragoon`,
  reasoning split into `reasoning`, tool_calls with correct args, and
  `/metrics` acceptance pos0 69% pos1 44% (54/86 draft tokens accepted),
  matching the bench window.

## Rollback

Git tag **`frank-dragoon-gufo-q4kxl-20261001`** — gufo v0.1.1 (built from
source, `scripts/build-gufo.sh` stays on disk) + the Q4_K_XL Qwen3.8-27B
payload + DFlash2 drafter, all still on the host. Restoring the tag's
unit/env/serve trio and redeploying returns the 27B endpoint.

# Citations

- Model: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
- Draft source: https://huggingface.co/Qwen/Qwen3.5-9B
- DFlash drafter: https://huggingface.co/z-lab/Qwen3.5-9B-DFlash
- Engine image: https://hub.docker.com/r/kyuz0/vllm-therock-gfx1151
- Serve script: `scripts/serve-frank-dragoon.sh`; tunables: `scripts/frank-dragoon.env`
- Predecessor: [[frank-dragoon-gufo-qwen38-27b]]; sibling on the same arch:
  [[frank-zealot-sglang-mimo-v26]]