docs: frank-large Q6 uncensored cutover + frank-small 2x200704 f16-KV reslice

- new active runbook frank-large-rocmfpx-q6agent-uncensored-mtp.md: the
  Q6_0_ROCMFPX_AGENT file (ftype 114, 339 type-102 + 159 type-103 tensors),
  27.6-28.1 t/s at 86.8% draft acceptance, the measured ngram-mod chain
  rejection (-15% prose: recurrent-state rollback disabled), the
  bench-window hardening laws
- frank-large-rocmfpx-q8agent-mtp.md retired; rollback section corrected
  (gufo payloads removed from disk 2026-10-03 - that rollback is now a
  re-download; the aligned Q8 file is the direct one, still on disk)
- frank-small-llama-mimo-q5km.md reshaped to 2 x 200704 with fully
  unquantized f16 KV: boot-pair table gains the measured row (20,530 used /
  4,046 free), 104.84 tok/s, slot-window contract at 200704, rollback to
  the 3x180224 + q8_0-V shape documented
- AGENTS.md: fleet table rows, both endpoint sections, the chain law
  extended with the recurrent-model measurement, the --parallel law with
  the reslice, runbook index (active + retired)
- LLM/index.md + root index entries updated
This commit is contained in:
Frank
2026-10-03 08:01:44 -07:00
parent 47a2c1287b
commit b80886e318
6 changed files with 438 additions and 216 deletions
+31 -22
View File
@@ -1,6 +1,6 @@
# LLM Fleet — Knowledge Base
Documentation for the LLM inference fleet on the two hosts (2026-10-02):
Documentation for the LLM inference fleet on the two hosts (2026-10-03):
**Dankserver runs the tcclaviger RDNA4 vLLM fork on frank-primary** (the
digest-pinned `tcclaviger/vllm` image driving `podman run` directly, serving
orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 across both Radeon AI PRO R9700s
@@ -15,11 +15,12 @@ mounted cache dir** (the image ships no `/root/.cache` at all; unmounted, every
SEQS 2 cannot compile this model. Rollback frank-primary-fp8-mtp-20261001),
and Dankclaw
runs two endpoints — **ROCmFPX llama.cpp (frank-large, port 7770:
kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF Q8_0_ROCMFPX_AGENT, a ftype-115
file only that fork can read, with the repo's MTP head drafting 4 tokens, ONE
session at 262,144, f16 KV, vision via the bundled projector)** and
kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF
Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT, a ftype-114 file only that fork can
read, with the repo's MTP head drafting 4 tokens at 27.6-28.1 t/s decode, ONE
session at 262,144, f16 KV, vision via the BF16 projector)** and
**llama.cpp CUDA (frank-small, port 7771: MiMo-V2.6-Distill-Qwen-9B Q5_K_M
GGUF, 3 x 180,224, asymmetric KV f16 K + q8_0 V, on the RTX 3090)**.
GGUF, 2 x 200,704, fully unquantized f16 KV, on the RTX 3090)**.
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -46,12 +47,12 @@ expert-precise-memory arena 7,522 slots, /root/.cache mounted]
gfx1151 iGPU 114 GiB GTT]
E[ROCmFPX llama.cpp gfx1151, pinned fork build
frank-large, port 7770]
F[Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP-Q4_0 head
F[Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP-Q4_0 head
1 x 262144 ctx, f16 KV, draft n-max 4, mmproj vision]
G[llama.cpp master CUDA sm_86
frank-small, port 7771]
H[MiMo-V2.6 Q5_K_M GGUF
3 x 180224 ctx, f16 K + q8_0 V, flash-attn, mmproj vision]
2 x 200704 ctx, f16 K + f16 V, flash-attn, mmproj vision]
E2 --> E --> F
E2 --> G --> H
end
@@ -70,20 +71,23 @@ frank-small, port 7771]
## Active Deployment
- [[frank-large-rocmfpx-q8agent-mtp]] — **ACTIVE since 2026-10-02**
- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — **ACTIVE since 2026-10-03**
(root systemd unit, port 7770): the pinned kingjones30/ROCmFPX fork serving
the repo's highest-quality rung `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`
(26.28 GiB, ftype 115 = 187 plain-Q8_0 + 311 GGML_TYPE_Q8_0_ROCMFPX
tensors) with the bundled `mtp-Qwen3.8-27B-Q4_0.gguf` head drafting 4
tokens, one 262,144 window, f16 KV, the BF16 projector for vision. Why the
runtime is not interchangeable, the pinned-build set_rows history, the
measured envelope, rollback to gufo
- [[frank-small-llama-mimo-q5km]] — **ACTIVE since 2026-10-02** (root
systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF Q5_K_M on the RTX 3090,
THREE slots of 180,224 tokens (pool 540,672) with K f16 + V q8_0 and the
mmproj for vision, ~100 tok/s decode, ~2.7 GiB of the card left free.
The KV arithmetic, the flash-attn dependency, rollback to Q6_K
the uncensored protected-heads rung `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`
(23.15 GiB, ftype 114 = 159 type-103 + 339 GGML_TYPE_Q6_0_ROCMFPX tensors,
64 protected output heads) with the repo's `mtp-Qwen3.8-27B-Q4_0.gguf` head
drafting 4 tokens, one 262,144 window, f16 KV, the BF16 projector for
vision. Measured 27.6-28.1 t/s prose decode at 86.8 % draft acceptance; the
ngram-mod chain onto MTP was measured and REJECTED (-15 %: it disables the
hybrid's recurrent state rollback). Rollback: the aligned Q8 file, still on
disk
- [[frank-small-llama-mimo-q5km]] — **ACTIVE since 2026-10-02, resliced
2026-10-03** (root systemd unit, port 7771): llama.cpp master (CUDA sm_86)
serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF Q5_K_M on the RTX 3090,
TWO slots of 200,704 tokens (pool 401,408) with FULLY UNQUANTIZED KV (f16 K
+ f16 V) and the mmproj for vision, 104.8 tok/s decode, 4,046 MiB of the
card left free. The boot-pair pool arithmetic, the slot-window contract,
rollback to 3 x 180,224 or Q6_K
- [[dankserver-qwen38-fp8-expert-offload-tuning]] — **ACTIVE since
2026-10-02 (shape E)** (root systemd unit, port 7770): the tcclaviger RDNA4
vLLM fork serving orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 across both
@@ -192,8 +196,13 @@ frank-small, port 7771]
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[frank-large-gufo-qwen38-27b]] — **RETIRED 2026-10-02 (the rollback
target)** (root systemd unit, port 7770): gufo
- [[frank-large-rocmfpx-q8agent-mtp]] — **RETIRED 2026-10-02 → 10-03 (the
rollback target; payload still on disk)**: the same pinned ROCmFPX fork
serving the ALIGNED `Q8_0_ROCMFPX_AGENT` (ftype 115, 17.0 t/s, MTP
acceptance 0.462) — the MTP-vs-none +123 % A/B, the trap catalogue
- [[frank-large-gufo-qwen38-27b]] — **RETIRED 2026-10-02 (history; the
payload dirs were removed from disk 2026-10-03, so this rollback is a
re-download)** (root systemd unit, port 7770): gufo
(gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from
source on Dankclaw, serving unsloth/Qwen3.8-27B-GGUF **UD-Q4_K_XL**
with the z-lab DFlash2 Q4_K_M drafter — ONE session at the full native