docs(frank-large): unsloth Qwen3.8-Flash-Next runbook; retire the CIRU Orca runbook

New runbook frank-large-unsloth-qwen38-flash-next.md (runtime, payload,
deployed config, the explicit -md, the 114 GiB GTT memory model, client
contract, rollback). frank-large-ciru-orca.md marked RETIRED (rollback
target, payload and runtime intact on disk). AGENTS.md topology row,
frank-large section, harness note, runbook index, and the GPU-budget
lesson (GTT aperture ~114 GiB, not the stale 64 GiB rocm-smi figure —
the CIRU Orca held 92.6 GiB on the same device). LLM index header,
mermaid, active list, and the vault root index updated. Work-log entry.
This commit is contained in:
Frank
2026-09-21 12:30:35 -07:00
parent b149d02187
commit 7010f5b754
6 changed files with 307 additions and 63 deletions
+23 -16
View File
@@ -1,11 +1,11 @@
# LLM Fleet — Knowledge Base
Documentation for the LLM inference fleet on the two hosts (2026-09-20):
Documentation for the LLM inference fleet on the two hosts (2026-09-21):
**vllm-radiance on Dankserver** serving frank-medium as ONE tensor-parallel
process across both Radeon AI PRO R9700 cards (just1moremodel
Qwen3.8-27B-Uncensored MXFP4-awq + the z-lab DFlash2-FP8 drafter, TP=2 over the
R4D one-shot allreduce), and TWO endpoints on Dankclaw — the CIRU v4.4.1
runtime (frank-large, jcbtc Qwen3.8-Flash-CIRU-STRIX-Orca w4b on the gfx1151
process across both Radeon AI PRO R9700 cards (orcarouter
Qwen3.8-27B-Uncensored-FP8 + the z-lab DFlash2 drafter, TP=2 over the
R4D one-shot allreduce), and TWO endpoints on Dankclaw — the unsloth
llama.cpp prebuilt (frank-large, Qwen3.8-Flash-Next UD-IQ4_XS on the gfx1151
iGPU, port 7770) and mainline llama.cpp (frank-small, HauhauCS Gemma4-12B QAT
Q4_K_M with MTP and vision on the RTX 3090, port 7771).
@@ -23,18 +23,18 @@ graph LR
gfx1201, 32 GiB each]
B2[vllm-radiance 0.9.3, TP=2
frank-medium, port 7770]
C[Qwen3.8-27B MXFP4-awq + DFlash2-FP8
C[Qwen3.8-27B FP8 + DFlash2 drafter
ONE process on both cards, 262144 ctx, up to 8 seqs]
A --> B2 --> C
end
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
E2[RTX 3090 (PCIe x4 dock) +
gfx1151 iGPU 64 GiB unified memory]
E[CIRU v4.4.1 runtime
gfx1151 iGPU 114 GiB GTT]
E[unsloth llama.cpp b11030 (ROCm gfx1151)
frank-large, port 7770]
F[jcbtc Qwen3.8-Flash-CIRU-STRIX-Orca w4b
262144 ctx / 1 slot, MTP depth 4 + PLE]
F[Qwen3.8-Flash-Next UD-IQ4_XS
200000 ctx / 1 slot, MTP depth 2]
G[mainline llama.cpp 37b53fd
frank-small, port 7771]
H[Gemma4-12B QAT Q4_K_M
@@ -96,11 +96,18 @@ frank-small, port 7771]
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[frank-large-ciru-orca]] — ACTIVE frank-large runbook since 2026-09-19
(root systemd unit, port 7770): the CIRU v4.4.1 runtime (private host build,
QSA hotfix) serving jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca w4b on the gfx1151
iGPU — one 262,144-token conversation, MTP depth 4, 4096 MiB PLE cache,
runtime-root gate, client contract, troubleshooting
- [[frank-large-unsloth-qwen38-flash-next]] — ACTIVE frank-large runbook
since 2026-09-21 (root systemd unit, port 7770): the unsloth llama.cpp
prebuilt (b11030-mix-5ff778e, ROCm gfx1151) serving
unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS on the gfx1151 iGPU — one
200,000-token conversation, MTP depth 2 via the explicit -md shared-Q8_0
head, the 114 GiB GTT memory model, client contract, troubleshooting
- [[frank-large-ciru-orca]] — RETIRED 2026-09-21 (replaced by
[[frank-large-unsloth-qwen38-flash-next]]; rollback target): the CIRU
v4.4.1 runtime (private host build, QSA hotfix) serving
jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca w4b on the gfx1151 iGPU — one
262,144-token conversation, MTP depth 4, 4096 MiB PLE cache, runtime-root
gate, client contract, troubleshooting (2026-09-19 → 09-21)
- [[halogen-flash-server]] — RETIRED 2026-09-19 (replaced by
[[frank-large-ciru-orca]]; rollback target): peonist-ai halogen-flash-server
0.5.8 serving peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b
@@ -161,7 +168,7 @@ frank-small, port 7771]
(deployed and reverted 2026-08-22)
- [[Dankserver System Specs]] — the 2x R9700 box (frank-medium)
- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on the
CIRU v4.4.1 runtime since 2026-09-19, frank-small on the RTX 3090)
unsloth llama.cpp prebuilt since 2026-09-21, frank-small on the RTX 3090)
- [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
on Dankclaw (Gemma4 E4B vision+audio pack, XDNA2, port 7771), kept as
the retirement record — the IOMMU/SVA err=-19 root cause and where