docs: frank-large Q6 uncensored cutover + frank-small 2x200704 f16-KV reslice
- new active runbook frank-large-rocmfpx-q6agent-uncensored-mtp.md: the Q6_0_ROCMFPX_AGENT file (ftype 114, 339 type-102 + 159 type-103 tensors), 27.6-28.1 t/s at 86.8% draft acceptance, the measured ngram-mod chain rejection (-15% prose: recurrent-state rollback disabled), the bench-window hardening laws - frank-large-rocmfpx-q8agent-mtp.md retired; rollback section corrected (gufo payloads removed from disk 2026-10-03 - that rollback is now a re-download; the aligned Q8 file is the direct one, still on disk) - frank-small-llama-mimo-q5km.md reshaped to 2 x 200704 with fully unquantized f16 KV: boot-pair table gains the measured row (20,530 used / 4,046 free), 104.84 tok/s, slot-window contract at 200704, rollback to the 3x180224 + q8_0-V shape documented - AGENTS.md: fleet table rows, both endpoint sections, the chain law extended with the recurrent-model measurement, the --parallel law with the reslice, runbook index (active + retired) - LLM/index.md + root index entries updated
This commit is contained in:
@@ -213,15 +213,21 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
|
||||
Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
|
||||
rollback record)
|
||||
- [[frank-large-rocmfpx-q8agent-mtp]] — the ACTIVE frank-large runbook
|
||||
since 2026-10-02 (root systemd unit, port 7770): the pinned kingjones30/
|
||||
ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
|
||||
Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF **Q8_0_ROCMFPX_AGENT** — ftype 115,
|
||||
311 tensors of GGML_TYPE_Q8_0_ROCMFPX, a format no other loader on the
|
||||
host reads — with the repo's Q4_0 MTP head drafting 4 tokens and its BF16
|
||||
projector, one session at the native 262,144 window
|
||||
- [[frank-large-gufo-qwen38-27b]] — RETIRED 2026-10-01 → 10-02 (rollback
|
||||
target, payload and `scripts/build-gufo.sh` both on disk): gufo v0.1.1
|
||||
- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — the ACTIVE frank-large
|
||||
runbook since 2026-10-03 (root systemd unit, port 7770): the pinned
|
||||
kingjones30/ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
|
||||
Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF
|
||||
**Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT** — ftype 114, 339 tensors of
|
||||
GGML_TYPE_Q6_0_ROCMFPX + 159 type-103, a format no other loader on the
|
||||
host reads — with the repo's Q4_0 MTP head drafting 4 tokens (27.6-28.1
|
||||
t/s measured, 86.8 % acceptance; the ngram-mod chain measured and
|
||||
rejected) and its BF16 projector, one session at the native 262,144 window
|
||||
- [[frank-large-rocmfpx-q8agent-mtp]] — RETIRED 2026-10-02 → 10-03 (rollback
|
||||
target, aligned Q8 payload still on disk): the same fork serving the
|
||||
ALIGNED Q8_0_ROCMFPX_AGENT at 17.0 t/s — the +123 % MTP A/B, the traps
|
||||
- [[frank-large-gufo-qwen38-27b]] — RETIRED 2026-10-01 → 10-02 (history;
|
||||
the payload dirs were removed from disk 2026-10-03, `scripts/build-gufo.sh`
|
||||
remains): gufo v0.1.1
|
||||
built from source serving Qwen3.8-27B UD-Q4_K_XL + the z-lab DFlash2
|
||||
drafter on the gfx1151 iGPU — the two from-source build workarounds, the
|
||||
template-hash allowlist, the measured prefill/decode trade
|
||||
@@ -283,11 +289,11 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||||
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
|
||||
- [[frank-small-llama-mimo-q5km]] — the ACTIVE frank-small runbook since
|
||||
2026-10-02: a CUDA (sm_86) build of llama.cpp master serving
|
||||
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** on Dankclaw's RTX
|
||||
3090, port 7771 — THREE conversations at 180,224 tokens each, the
|
||||
asymmetric KV contract (f16 K + q8_0 V), vision through the repo's mmproj,
|
||||
and the two VRAM measurements that set the pool size; ~100 tok/s decode
|
||||
2026-10-02, resliced 2026-10-03: a CUDA (sm_86) build of llama.cpp master
|
||||
serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** on Dankclaw's
|
||||
RTX 3090, port 7771 — TWO conversations at 200,704 tokens each with fully
|
||||
unquantized f16 KV, vision through the repo's mmproj, and the boot-pair
|
||||
arithmetic that sizes the pool; 104.8 tok/s decode, 4,046 MiB free
|
||||
- [[frank-small-llama-mimo-q6k]] — RETIRED 2026-10-01 (same day; replaced
|
||||
by [[frank-small-llama-mimo-q5km]], payload still on disk): the first
|
||||
llama.cpp + GGUF shape for this endpoint, Q6_K at the same
|
||||
|
||||
Reference in New Issue
Block a user