docs: record the 2026-10-02 swaps — frank-large on ROCmFPX, frank-small on Q5_K_M
frank-large: new runbook frank-large-rocmfpx-q8agent-mtp.md — the ftype-115 tensor-type lock-in that forced the fork, the pinned build (set_rows history), the measured MTP A/B (17.0 vs 7.6 t/s decode, acceptance 0.462), the live client contract (reasoning_content, effort fold, vision restored) and rollback. frank-small: new runbook frank-small-llama-mimo-q5km.md; q6k marked retired (rollback target). gufo runbook marked retired (rollback target). Both indexes, the fleet topology table, the vision/models.yml notes, the benchmarking law and three new Key Lessons updated.
This commit is contained in:
1 parent
125d04967f
commit
72bd4c967e
7 files changed
+522
-147
No files matched your search
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF"
|
||||
description: "The active frank-small runbook — a CUDA (sm_86) build of llama.cpp master serving bartowski's MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF on the RTX 3090: two native-window sessions, the asymmetric KV contract (f16 K + q8_0 V), the flash-attention dependency that comes with quantized V, the VRAM fit, and what to check when it misbehaves."
|
||||
title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF (retired)"
|
||||
description: "The retired 2026-10-01 frank-small runbook — the first llama.cpp + GGUF shape on the RTX 3090: two native-window sessions, the asymmetric KV contract (f16 K + q8_0 V) that motivated the move off SGLang, the VRAM fit at Q6_K, and what to check when it misbehaves. Superseded the same night by the Q5_K_M payload."
|
||||
tags:
|
||||
- llm
|
||||
- frank-small
|
||||
@@ -16,11 +16,14 @@ timestamp: 2026-10-01
|
||||
|
||||
# frank-small — llama.cpp MiMo-V2.6 Q6_K (RTX 3090)
|
||||
|
||||
> **ACTIVE since 2026-10-01 night.** Replaces
|
||||
> [[frank-small-sglang-mimo-v26]] (SGLang + INT8-W8A16-AutoRound safetensors,
|
||||
> same architecture). Rolled in by Frank's night order: move this endpoint to
|
||||
> the GGUF path for **asymmetric KV** — unquantized K + q8_0 V — which only
|
||||
> llama.cpp exposes per-tensor.
|
||||
> **RETIRED 2026-10-01 → 10-02** (one night): superseded by
|
||||
> [[frank-small-llama-mimo-q5km]], the same runtime and shape one quant rung
|
||||
> down. The Q6_K file stays at `/home/dank/models/MiMo-Q6K/` as the rollback
|
||||
> target.
|
||||
>
|
||||
> Original order (Frank, 2026-10-01 night): move this endpoint to the GGUF
|
||||
> path for **asymmetric KV** — unquantized K + q8_0 V — which only llama.cpp
|
||||
> exposes per-tensor. That contract is what the current deployment kept.
|
||||
|
||||
## Identity
|
||||
|
||||
|
||||
Reference in new issue
Block a user