docs: record the 2026-10-02 swaps — frank-large on ROCmFPX, frank-small on Q5_K_M

frank-large: new runbook frank-large-rocmfpx-q8agent-mtp.md — the ftype-115
tensor-type lock-in that forced the fork, the pinned build (set_rows history),
the measured MTP A/B (17.0 vs 7.6 t/s decode, acceptance 0.462), the live
client contract (reasoning_content, effort fold, vision restored) and rollback.
frank-small: new runbook frank-small-llama-mimo-q5km.md; q6k marked retired
(rollback target). gufo runbook marked retired (rollback target). Both indexes,
the fleet topology table, the vision/models.yml notes, the benchmarking law and
three new Key Lessons updated.
This commit is contained in:
Frank committed 2026-10-02 01:51:11 -07:00
1 parent 125d04967f
commit 72bd4c967e
7 files changed
+522 -147

No files matched your search

@@ -1,7 +1,7 @@
---
type: reference
title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF"
description: "The active frank-small runbook — a CUDA (sm_86) build of llama.cpp master serving bartowski's MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF on the RTX 3090: two native-window sessions, the asymmetric KV contract (f16 K + q8_0 V), the flash-attention dependency that comes with quantized V, the VRAM fit, and what to check when it misbehaves."
title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF (retired)"
description: "The retired 2026-10-01 frank-small runbook — the first llama.cpp + GGUF shape on the RTX 3090: two native-window sessions, the asymmetric KV contract (f16 K + q8_0 V) that motivated the move off SGLang, the VRAM fit at Q6_K, and what to check when it misbehaves. Superseded the same night by the Q5_K_M payload."
tags:
- llm
- frank-small
@@ -16,11 +16,14 @@ timestamp: 2026-10-01
# frank-small — llama.cpp MiMo-V2.6 Q6_K (RTX 3090)
> **ACTIVE since 2026-10-01 night.** Replaces
> [[frank-small-sglang-mimo-v26]] (SGLang + INT8-W8A16-AutoRound safetensors,
> same architecture). Rolled in by Frank's night order: move this endpoint to
> the GGUF path for **asymmetric KV** — unquantized K + q8_0 V — which only
> llama.cpp exposes per-tensor.
> **RETIRED 2026-10-01 → 10-02** (one night): superseded by
> [[frank-small-llama-mimo-q5km]], the same runtime and shape one quant rung
> down. The Q6_K file stays at `/home/dank/models/MiMo-Q6K/` as the rollback
> target.
>
> Original order (Frank, 2026-10-01 night): move this endpoint to the GGUF
> path for **asymmetric KV** — unquantized K + q8_0 V — which only llama.cpp
> exposes per-tensor. That contract is what the current deployment kept.
## Identity