docs: frank-large Q6 uncensored cutover + frank-small 2x200704 f16-KV reslice

- new active runbook frank-large-rocmfpx-q6agent-uncensored-mtp.md: the
  Q6_0_ROCMFPX_AGENT file (ftype 114, 339 type-102 + 159 type-103 tensors),
  27.6-28.1 t/s at 86.8% draft acceptance, the measured ngram-mod chain
  rejection (-15% prose: recurrent-state rollback disabled), the
  bench-window hardening laws
- frank-large-rocmfpx-q8agent-mtp.md retired; rollback section corrected
  (gufo payloads removed from disk 2026-10-03 - that rollback is now a
  re-download; the aligned Q8 file is the direct one, still on disk)
- frank-small-llama-mimo-q5km.md reshaped to 2 x 200704 with fully
  unquantized f16 KV: boot-pair table gains the measured row (20,530 used /
  4,046 free), 104.84 tok/s, slot-window contract at 200704, rollback to
  the 3x180224 + q8_0-V shape documented
- AGENTS.md: fleet table rows, both endpoint sections, the chain law
  extended with the recurrent-model measurement, the --parallel law with
  the reslice, runbook index (active + retired)
- LLM/index.md + root index entries updated
This commit is contained in:
Frank
2026-10-03 08:01:44 -07:00
parent 47a2c1287b
commit b80886e318
6 changed files with 438 additions and 216 deletions
+20 -14
View File
@@ -213,15 +213,21 @@ The LLM inference fleet on the two hosts. Full index:
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
rollback record)
- [[frank-large-rocmfpx-q8agent-mtp]] — the ACTIVE frank-large runbook
since 2026-10-02 (root systemd unit, port 7770): the pinned kingjones30/
ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF **Q8_0_ROCMFPX_AGENT** — ftype 115,
311 tensors of GGML_TYPE_Q8_0_ROCMFPX, a format no other loader on the
host reads — with the repo's Q4_0 MTP head drafting 4 tokens and its BF16
projector, one session at the native 262,144 window
- [[frank-large-gufo-qwen38-27b]] — RETIRED 2026-10-01 → 10-02 (rollback
target, payload and `scripts/build-gufo.sh` both on disk): gufo v0.1.1
- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — the ACTIVE frank-large
runbook since 2026-10-03 (root systemd unit, port 7770): the pinned
kingjones30/ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF
**Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT** — ftype 114, 339 tensors of
GGML_TYPE_Q6_0_ROCMFPX + 159 type-103, a format no other loader on the
host reads — with the repo's Q4_0 MTP head drafting 4 tokens (27.6-28.1
t/s measured, 86.8 % acceptance; the ngram-mod chain measured and
rejected) and its BF16 projector, one session at the native 262,144 window
- [[frank-large-rocmfpx-q8agent-mtp]] — RETIRED 2026-10-02 → 10-03 (rollback
target, aligned Q8 payload still on disk): the same fork serving the
ALIGNED Q8_0_ROCMFPX_AGENT at 17.0 t/s — the +123 % MTP A/B, the traps
- [[frank-large-gufo-qwen38-27b]] — RETIRED 2026-10-01 → 10-02 (history;
the payload dirs were removed from disk 2026-10-03, `scripts/build-gufo.sh`
remains): gufo v0.1.1
built from source serving Qwen3.8-27B UD-Q4_K_XL + the z-lab DFlash2
drafter on the gfx1151 iGPU — the two from-source build workarounds, the
template-hash allowlist, the measured prefill/decode trade
@@ -283,11 +289,11 @@ The LLM inference fleet on the two hosts. Full index:
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
- [[frank-small-llama-mimo-q5km]] — the ACTIVE frank-small runbook since
2026-10-02: a CUDA (sm_86) build of llama.cpp master serving
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** on Dankclaw's RTX
3090, port 7771 — THREE conversations at 180,224 tokens each, the
asymmetric KV contract (f16 K + q8_0 V), vision through the repo's mmproj,
and the two VRAM measurements that set the pool size; ~100 tok/s decode
2026-10-02, resliced 2026-10-03: a CUDA (sm_86) build of llama.cpp master
serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** on Dankclaw's
RTX 3090, port 7771 — TWO conversations at 200,704 tokens each with fully
unquantized f16 KV, vision through the repo's mmproj, and the boot-pair
arithmetic that sizes the pool; 104.8 tok/s decode, 4,046 MiB free
- [[frank-small-llama-mimo-q6k]] — RETIRED 2026-10-01 (same day; replaced
by [[frank-small-llama-mimo-q5km]], payload still on disk): the first
llama.cpp + GGUF shape for this endpoint, Q6_K at the same