docs: frank-large Kairic Edge runbook + frank-medium clean-KV updates

- New runbook frank-large-kairic-edge-qwen38-27b.md (ACTIVE frank-large,
  Kairic Edge runtime, jcbtc Qwen3.8-27B-IU4, 262144 ctx, MTP depth 4 +
  n-gram drafting, compatibility mode)
- AGENTS.md: frank-large section + topology row + fleet contract + runbook
  index (Kairic Edge ACTIVE, unsloth RETIRED same-day); frank-medium
  section body (MXFP4 + unquantized bf16 KV, 486,464-token pool)
- LLM index + top-level index: Kairic Edge ACTIVE, unsloth RETIRED,
  system specs entry
This commit is contained in:
Frank committed 2026-09-21 19:16:51 -07:00
1 parent b2a74004df
commit e04f510883
4 files changed
+282 -69

No files matched your search

+21 -13
View File
@@ -4,10 +4,11 @@ Documentation for the LLM inference fleet on the two hosts (2026-09-21):
**vllm-radiance on Dankserver** serving frank-medium as ONE tensor-parallel
process across both Radeon AI PRO R9700 cards (just1moremodel
Qwen3.8-27B-Uncensored-MXFP4 + the z-lab DFlash2 drafter, bf16 KV, TP=2 over the
R4D one-shot allreduce), and TWO endpoints on Dankclaw — the unsloth
llama.cpp prebuilt (frank-large, Qwen3.8-Flash-Next UD-IQ4_XS on the gfx1151
iGPU, port 7770) and mainline llama.cpp (frank-small, HauhauCS Gemma4-12B QAT
Q4_K_M with MTP and vision on the RTX 3090, port 7771).
R4D one-shot allreduce), and TWO endpoints on Dankclaw — the Kairic Edge
runtime (frank-large, ciru-ai/ROCmFPX v1.2, jcbtc Qwen3.8-27B-IU4-Kairic-Edge
IU4 W4A4 on the gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small,
HauhauCS Gemma4-12B QAT Q4_K_M with MTP and vision on the RTX 3090, port
7771).
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -31,10 +32,10 @@ ONE process on both cards, 262144 ctx, up to 8 seqs]
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
E2[RTX 3090 (PCIe x4 dock) +
gfx1151 iGPU 114 GiB GTT]
E[unsloth llama.cpp b11030 (ROCm gfx1151)
E[Kairic Edge (ciru-ai/ROCmFPX v1.2, gfx1151)
frank-large, port 7770]
F[Qwen3.8-Flash-Next UD-IQ4_XS
200000 ctx / 1 slot, MTP depth 2]
F[Qwen3.8-27B-IU4-Kairic-Edge (IU4 W4A4)
262144 ctx / 1 slot, MTP depth 4 + n-gram]
G[mainline llama.cpp 37b53fd
frank-small, port 7771]
H[Gemma4-12B QAT Q4_K_M
@@ -95,11 +96,18 @@ frank-small, port 7771]
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[frank-large-unsloth-qwen38-flash-next]] — ACTIVE frank-large runbook
since 2026-09-21 (root systemd unit, port 7770): the unsloth llama.cpp
prebuilt (b11030-mix-5ff778e, ROCm gfx1151) serving
unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS on the gfx1151 iGPU — one
200,000-token conversation, MTP depth 2 via the explicit -md shared-Q8_0
- [[frank-large-kairic-edge-qwen38-27b]] — ACTIVE frank-large runbook
since 2026-09-21 (root systemd unit, port 7770): the Kairic Edge runtime
(ciru-ai/ROCmFPX v1.2, built from source) serving
jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (IU4 W4A4, 27B dense) on the gfx1151
iGPU — one full 262,144-token conversation, MTP depth 4 + n-gram drafting,
the three PromptForge sidecars, compatibility mode, the 114 GiB GTT memory
model, client contract, troubleshooting
- [[frank-large-unsloth-qwen38-flash-next]] — RETIRED 2026-09-21 (same-day
swap, replaced by [[frank-large-kairic-edge-qwen38-27b]]; rollback target):
the unsloth llama.cpp prebuilt (b11030-mix-5ff778e, ROCm gfx1151) serving
unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (125B MoE) on the gfx1151 iGPU —
one 200,000-token conversation, MTP depth 2 via the explicit -md shared-Q8_0
head, the 114 GiB GTT memory model, client contract, troubleshooting
- [[frank-large-ciru-orca]] — RETIRED 2026-09-21 (replaced by
[[frank-large-unsloth-qwen38-flash-next]]; rollback target): the CIRU
@@ -167,7 +175,7 @@ frank-small, port 7771]
(deployed and reverted 2026-08-22)
- [[Dankserver System Specs]] — the 2x R9700 box (frank-medium)
- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on the
unsloth llama.cpp prebuilt since 2026-09-21, frank-small on the RTX 3090)
Kairic Edge runtime since 2026-09-21, frank-small on the RTX 3090)
- [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
on Dankclaw (Gemma4 E4B vision+audio pack, XDNA2, port 7771), kept as
the retirement record — the IOMMU/SVA err=-19 root cause and where