docs: record the 2026-10-02 swaps — frank-large on ROCmFPX, frank-small on Q5_K_M
frank-large: new runbook frank-large-rocmfpx-q8agent-mtp.md — the ftype-115 tensor-type lock-in that forced the fork, the pinned build (set_rows history), the measured MTP A/B (17.0 vs 7.6 t/s decode, acceptance 0.462), the live client contract (reasoning_content, effort fold, vision restored) and rollback. frank-small: new runbook frank-small-llama-mimo-q5km.md; q6k marked retired (rollback target). gufo runbook marked retired (rollback target). Both indexes, the fleet topology table, the vision/models.yml notes, the benchmarking law and three new Key Lessons updated.
This commit is contained in:
@@ -213,20 +213,27 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
|
||||
Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
|
||||
rollback record)
|
||||
- [[frank-large-vllm-mimo-v26-bf16]] — the ACTIVE frank-large runbook
|
||||
since 2026-10-01 (root systemd unit, port 7770): the digest-pinned
|
||||
- [[frank-large-rocmfpx-q8agent-mtp]] — the ACTIVE frank-large runbook
|
||||
since 2026-10-02 (root systemd unit, port 7770): the pinned kingjones30/
|
||||
ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
|
||||
Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF **Q8_0_ROCMFPX_AGENT** — ftype 115,
|
||||
311 tensors of GGML_TYPE_Q8_0_ROCMFPX, a format no other loader on the
|
||||
host reads — with the repo's Q4_0 MTP head drafting 4 tokens and its BF16
|
||||
projector, one session at the native 262,144 window
|
||||
- [[frank-large-gufo-qwen38-27b]] — RETIRED 2026-10-01 → 10-02 (rollback
|
||||
target, payload and `scripts/build-gufo.sh` both on disk): gufo v0.1.1
|
||||
built from source serving Qwen3.8-27B UD-Q4_K_XL + the z-lab DFlash2
|
||||
drafter on the gfx1151 iGPU — the two from-source build workarounds, the
|
||||
template-hash allowlist, the measured prefill/decode trade
|
||||
- [[frank-large-vllm-mimo-v26-bf16]] — RETIRED 2026-10-01 (same day;
|
||||
replaced by the gufo rollback, then by
|
||||
[[frank-large-rocmfpx-q8agent-mtp]]): the digest-pinned
|
||||
kyuz0/vllm-therock-gfx1151 image (vLLM 0.30, TheRock ROCm 10) serving
|
||||
XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B in FULL BF16 on the gfx1151 iGPU,
|
||||
4 x 262,144 ctx, with CROSS-MODEL MTP spec decode from Qwen/Qwen3.5-9B —
|
||||
the roof math (13.1 t/s base is ~94% of LPDDR5x BF16), the measured
|
||||
MTP-vs-DFlash table (19.8 vs 17.4 t/s), the greedy-text numerics trap,
|
||||
the `reasoning` field change, troubleshooting
|
||||
- [[frank-large-gufo-qwen38-27b]] — RETIRED 2026-09-28 → 2026-10-01
|
||||
(replaced by [[frank-large-vllm-mimo-v26-bf16]]; rollback via tag
|
||||
frank-large-gufo-q4kxl-20261001): gufo v0.1.1 built from source serving
|
||||
Qwen3.8-27B GGUF + the z-lab DFlash2 drafter on the gfx1151 iGPU — the
|
||||
two from-source build workarounds, the client-contract delta, the
|
||||
measured prefill/decode trade
|
||||
- [[frank-large-unsloth-qwen38-27b-q8xl]] — RETIRED 2026-09-28 (replaced by
|
||||
[[frank-large-gufo-qwen38-27b]]; rollback target): stock llama.cpp
|
||||
(unsloth b11030, ROCm gfx1151) serving
|
||||
@@ -275,12 +282,21 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
||||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||||
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
|
||||
- [[frank-small-sglang-mimo-v26]] — the ACTIVE frank-small runbook since
|
||||
2026-10-01: SGLang in a self-contained python3.12 venv serving
|
||||
- [[frank-small-llama-mimo-q5km]] — the ACTIVE frank-small runbook since
|
||||
2026-10-02: a CUDA (sm_86) build of llama.cpp master serving
|
||||
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** on Dankclaw's RTX
|
||||
3090, port 7771 — 2 x 262,144 native windows, the asymmetric KV contract
|
||||
(f16 K + q8_0 V) and the VRAM arithmetic behind it, ~100 tok/s decode
|
||||
- [[frank-small-llama-mimo-q6k]] — RETIRED 2026-10-01 (same day; replaced
|
||||
by [[frank-small-llama-mimo-q5km]], payload still on disk): the first
|
||||
llama.cpp + GGUF shape for this endpoint, Q6_K at the same
|
||||
2 x 262,144 / f16-K + q8_0-V contract, 20.9 GiB measured
|
||||
- [[frank-small-sglang-mimo-v26]] — RETIRED 2026-10-01 (rollback target):
|
||||
SGLang in a self-contained python3.12 venv serving
|
||||
letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound (qwen3_5
|
||||
hybrid, INT8 W8A16, in-checkpoint vision, NO MTP weights shipped) on
|
||||
Dankclaw's RTX 3090, port 7771 — fp8 KV, pinned 606,208-token pool,
|
||||
4 x 151,552 slots, bf16 mamba states, eager prefill
|
||||
2 x 262,144 slots, bf16 mamba states, eager prefill
|
||||
- [[frank-small-gemma4-12b]] — RETIRED 2026-10-01 (rollback target):
|
||||
mainline llama.cpp serving
|
||||
HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced at Q4_K_M
|
||||
|
||||
Reference in New Issue
Block a user