frank-primary: R9V runbook + docs cutover
New ACTIVE runbook (setup/doctor PASS=34/FAIL=0, UVA transport verified in serving workers, wire-verified alias/reasoning/metrics), index + vault entries flipped: llama.cpp b11030 endpoint RETIRED 2026-09-27 to rollback target. Topology diagram updated.
This commit is contained in:
@@ -1,14 +1,14 @@
|
||||
# LLM Fleet — Knowledge Base
|
||||
|
||||
Documentation for the LLM inference fleet on the two hosts (2026-09-26):
|
||||
**llama.cpp on all three endpoints** — Dankserver runs unsloth llama.cpp
|
||||
(the b11030-mix prebuilt, ROCm gfx120X) serving frank-primary
|
||||
(unsloth Qwen3.8-Flash-Next Q8_0 + the shared Q8_0 MTP head + ngram-mod +
|
||||
vision) across both Radeon AI PRO R9700s on :7770, and Dankclaw runs two
|
||||
endpoints — stock llama.cpp (unsloth b11030, ROCm gfx1151; frank-large,
|
||||
unsloth Qwen3.8-27B UD-Q8_K_XL + the z-lab DFlash2 Q4_K_M drafter on the
|
||||
gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small, HauhauCS
|
||||
Gemma4-12B QAT Q4_K_M with MTP and vision on the RTX 3090, port 7771).
|
||||
Documentation for the LLM inference fleet on the two hosts (2026-09-27):
|
||||
**Dankserver runs R9V radiance-vLLM on frank-primary** (the drwolfen fork
|
||||
serving the abliterated Dyluhn Qwen3.8-Flash-Next-Uncensored IQ4_XS + MTP4
|
||||
across both Radeon AI PRO R9700s on :7770, docker-pinned runtime driven
|
||||
through a docker→podman shim), and Dankclaw runs two llama.cpp endpoints —
|
||||
stock llama.cpp (unsloth b11030, ROCm gfx1151; frank-large, unsloth
|
||||
Qwen3.8-27B UD-Q8_K_XL + the z-lab DFlash2 Q4_K_M drafter on the gfx1151
|
||||
iGPU, port 7770) and mainline llama.cpp (frank-small, HauhauCS Gemma4-12B
|
||||
QAT Q4_K_M with MTP and vision on the RTX 3090, port 7771).
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||
@@ -22,10 +22,11 @@ graph LR
|
||||
subgraph Dankserver[AMD Dankserver - Ubuntu 26.04]
|
||||
A[2x Radeon AI PRO R9700
|
||||
gfx1201, 32 GiB each]
|
||||
B2[unsloth llama.cpp b11030-mix, ROCm gfx120X
|
||||
B2[R9V radiance-vLLM fork, docker-pinned
|
||||
via docker-podman shim
|
||||
frank-primary, port 7770]
|
||||
C[Qwen3.8-Flash-Next Q8_0 + shared-Q8_0 MTP head
|
||||
layer split on both cards, 262144 ctx / 1 slot, ngram-mod, vision]
|
||||
C[Qwen3.8-Flash-Next-Uncensored IQ4_XS
|
||||
TP=2 tiered experts, MTP4, CED, 196608 ctx, vision]
|
||||
end
|
||||
|
||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||
@@ -57,8 +58,19 @@ frank-small, port 7771]
|
||||
|
||||
## Active Deployment
|
||||
|
||||
- [[frank-primary-llamacpp-qwen38-flash-next-q8]] — the ACTIVE
|
||||
frank-primary runbook (2026-09-26, root systemd unit, port 7770):
|
||||
- [[frank-primary-r9v-radiance-qwen38-uncensored]] — the ACTIVE
|
||||
frank-primary runbook (2026-09-27, root systemd unit, port 7770): the
|
||||
R9V radiance-vLLM fork (drwolfen/radiance-vllm-qwen4exp, docker-pinned
|
||||
image bundle `2dac17a2`, driven through a docker→podman shim) serving
|
||||
Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS (92.39 GiB, 3 shards) +
|
||||
blockfp8 MTP4 (depth 4) + F16 vision on BOTH R9700s — TP=2 tiered
|
||||
experts (UVA pinned host copy, 62 hot + 160-slot LRU on rank 0, 428
|
||||
static on rank 1), PLE n-gram on ssd-mmap, CED on, 196608 ctx, reasoning
|
||||
channel `reasoning`, parser `qwen3_coder`. Root unit + LimitMEMLOCK;
|
||||
the tuned 104-key shape lives in the on-host setup state. Rollback:
|
||||
[[frank-primary-llamacpp-qwen38-flash-next-q8]]
|
||||
- [[frank-primary-llamacpp-qwen38-flash-next-q8]] — RETIRED 2026-09-27
|
||||
(rollback target; the llama.cpp b11030 endpoint of 2026-09-26):
|
||||
unsloth llama.cpp (b11030-mix prebuilt, ROCm gfx120X, bundled runtime)
|
||||
serving unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 across both R9700s —
|
||||
routed experts and the 51B-param n-gram table computed in host RAM
|
||||
|
||||
Reference in New Issue
Block a user