frank-primary: R9V runbook + docs cutover

New ACTIVE runbook (setup/doctor PASS=34/FAIL=0, UVA transport verified
in serving workers, wire-verified alias/reasoning/metrics), index +
vault entries flipped: llama.cpp b11030 endpoint RETIRED 2026-09-27 to
rollback target. Topology diagram updated.
This commit is contained in:
Frank
2026-09-27 04:35:05 -07:00
parent 7562e41ba4
commit ca6ba6d0e9
3 changed files with 161 additions and 16 deletions
+26 -14
View File
@@ -1,14 +1,14 @@
# LLM Fleet — Knowledge Base
Documentation for the LLM inference fleet on the two hosts (2026-09-26):
**llama.cpp on all three endpoints** — Dankserver runs unsloth llama.cpp
(the b11030-mix prebuilt, ROCm gfx120X) serving frank-primary
(unsloth Qwen3.8-Flash-Next Q8_0 + the shared Q8_0 MTP head + ngram-mod +
vision) across both Radeon AI PRO R9700s on :7770, and Dankclaw runs two
endpoints — stock llama.cpp (unsloth b11030, ROCm gfx1151; frank-large,
unsloth Qwen3.8-27B UD-Q8_K_XL + the z-lab DFlash2 Q4_K_M drafter on the
gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small, HauhauCS
Gemma4-12B QAT Q4_K_M with MTP and vision on the RTX 3090, port 7771).
Documentation for the LLM inference fleet on the two hosts (2026-09-27):
**Dankserver runs R9V radiance-vLLM on frank-primary** (the drwolfen fork
serving the abliterated Dyluhn Qwen3.8-Flash-Next-Uncensored IQ4_XS + MTP4
across both Radeon AI PRO R9700s on :7770, docker-pinned runtime driven
through a docker→podman shim), and Dankclaw runs two llama.cpp endpoints —
stock llama.cpp (unsloth b11030, ROCm gfx1151; frank-large, unsloth
Qwen3.8-27B UD-Q8_K_XL + the z-lab DFlash2 Q4_K_M drafter on the gfx1151
iGPU, port 7770) and mainline llama.cpp (frank-small, HauhauCS Gemma4-12B
QAT Q4_K_M with MTP and vision on the RTX 3090, port 7771).
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -22,10 +22,11 @@ graph LR
subgraph Dankserver[AMD Dankserver - Ubuntu 26.04]
A[2x Radeon AI PRO R9700
gfx1201, 32 GiB each]
B2[unsloth llama.cpp b11030-mix, ROCm gfx120X
B2[R9V radiance-vLLM fork, docker-pinned
via docker-podman shim
frank-primary, port 7770]
C[Qwen3.8-Flash-Next Q8_0 + shared-Q8_0 MTP head
layer split on both cards, 262144 ctx / 1 slot, ngram-mod, vision]
C[Qwen3.8-Flash-Next-Uncensored IQ4_XS
TP=2 tiered experts, MTP4, CED, 196608 ctx, vision]
end
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
@@ -57,8 +58,19 @@ frank-small, port 7771]
## Active Deployment
- [[frank-primary-llamacpp-qwen38-flash-next-q8]] — the ACTIVE
frank-primary runbook (2026-09-26, root systemd unit, port 7770):
- [[frank-primary-r9v-radiance-qwen38-uncensored]] — the ACTIVE
frank-primary runbook (2026-09-27, root systemd unit, port 7770): the
R9V radiance-vLLM fork (drwolfen/radiance-vllm-qwen4exp, docker-pinned
image bundle `2dac17a2`, driven through a docker→podman shim) serving
Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS (92.39 GiB, 3 shards) +
blockfp8 MTP4 (depth 4) + F16 vision on BOTH R9700s — TP=2 tiered
experts (UVA pinned host copy, 62 hot + 160-slot LRU on rank 0, 428
static on rank 1), PLE n-gram on ssd-mmap, CED on, 196608 ctx, reasoning
channel `reasoning`, parser `qwen3_coder`. Root unit + LimitMEMLOCK;
the tuned 104-key shape lives in the on-host setup state. Rollback:
[[frank-primary-llamacpp-qwen38-flash-next-q8]]
- [[frank-primary-llamacpp-qwen38-flash-next-q8]] — RETIRED 2026-09-27
(rollback target; the llama.cpp b11030 endpoint of 2026-09-26):
unsloth llama.cpp (b11030-mix prebuilt, ROCm gfx120X, bundled runtime)
serving unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 across both R9700s —
routed experts and the 51B-param n-gram table computed in host RAM