frank-large: document the ik_llama.cpp deployment and the decode performance campaign

- Runbook with the measured envelope (prefill ~192 t/s at the VNNI ceiling;
  decode ~3.8-4.3 t/s gated by 100 backend splits/token, GPU 95% idle), the
  root-cause evidence chain, tuning levers, and co-tenancy traps
- AGENTS.md: fleet topology row + Active Services section for the ik
  deployment; halogen demoted to rollback; bench policy unit type fixed
- LLM + root indexes relinked (frank-large-ik-uncensored ACTIVE,
  halogen-flash-server retired)
This commit is contained in:
Frank committed 2026-09-16 02:42:41 -07:00
1 parent 0c1e10a5ef
commit 795b0b3a5e
4 files changed
+164 -43

No files matched your search

+15 -9
View File
@@ -1,10 +1,10 @@
# LLM Fleet — Knowledge Base
Documentation for the LLM inference fleet on the two hosts
(2026-09-15): charlie12345/ROCmFPX llama.cpp on Dankserver (frank-medium,
(2026-09-16): charlie12345/ROCmFPX llama.cpp on Dankserver (frank-medium,
kingjones777 Qwen3.8-27B-ROCmFPX-Q8_0-AGENT on 2x Radeon AI PRO R9700) and
halogen-flash-server 0.5.8 on Dankclaw (frank-large, peonist-ai
Qwen3.8-Flash-Next w4b).
ik_llama.cpp on Dankclaw (frank-large, mradermacher Qwen3.8-Flash-Next-
Uncensored i1-Q4_K_M on the RTX 3090 + Strix Halo hybrid).
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -26,11 +26,12 @@ layer split, 131k ctx / 1 slot, MTP]
end
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
E2[gfx1151 iGPU 64 GiB unified memory]
E[halogen-flash-server 0.5.8
E2[RTX 3090 (PCIe x4 dock) +
gfx1151 iGPU 64 GiB unified memory]
E[ik_llama.cpp master d85a1d7
frank-large, port 7770]
F[peonist-ai Qwen3.8-Flash-Next
w4b .hgn + overlay, 262k ctx / 4 slots]
F[mradermacher Qwen3.8-Flash-Next-Uncensored
i1-Q4_K_M GGUF, 262k ctx / 1 slot, vision]
E2 --> E --> F
end
@@ -58,8 +59,13 @@ w4b .hgn + overlay, 262k ctx / 4 slots]
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[halogen-flash-server]] — ACTIVE frank-large runbook since 2026-09-11:
peonist-ai/halogen-flash-server 0.5.8 serving
- [[frank-large-ik-uncensored]] — ACTIVE frank-large runbook since
2026-09-15: ik_llama.cpp (root systemd unit) serving mradermacher/
Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M on the RTX 3090 (dense/attention/
KV) + LPDDR (experts/PLE) hybrid; measured performance envelope, the
decode root-cause chain, tuning levers, and the co-tenancy traps
- [[halogen-flash-server]] — RETIRED 2026-09-15 (rollback target): the
peonist-ai halogen-flash-server 0.5.8 runbook serving
peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn +
quality overlay), rootless Quadlet container on Dankclaw port 7770 —
deployed flags, memory model, client contract, troubleshooting