frank-large: document the ik_llama.cpp deployment and the decode performance campaign
- Runbook with the measured envelope (prefill ~192 t/s at the VNNI ceiling; decode ~3.8-4.3 t/s gated by 100 backend splits/token, GPU 95% idle), the root-cause evidence chain, tuning levers, and co-tenancy traps - AGENTS.md: fleet topology row + Active Services section for the ik deployment; halogen demoted to rollback; bench policy unit type fixed - LLM + root indexes relinked (frank-large-ik-uncensored ACTIVE, halogen-flash-server retired)
This commit is contained in:
1 parent
0c1e10a5ef
commit
795b0b3a5e
4 files changed
+164
-43
No files matched your search
@@ -125,8 +125,12 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
|
||||
Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
|
||||
rollback record)
|
||||
- [[halogen-flash-server]] — the ACTIVE frank-large runbook since
|
||||
2026-09-11: peonist-ai/halogen-flash-server 0.5.8 serving
|
||||
- [[frank-large-ik-uncensored]] — the ACTIVE frank-large runbook since
|
||||
2026-09-15: ik_llama.cpp serving mradermacher/Qwen3.8-Flash-Next-
|
||||
Uncensored-i1-Q4_K_M on the RTX 3090 + Strix Halo hybrid (root systemd
|
||||
unit, port 7770) — measured performance envelope and traps
|
||||
- [[halogen-flash-server]] — the RETIRED 2026-09-15 halogen runbook and
|
||||
rollback target: peonist-ai/halogen-flash-server 0.5.8 serving
|
||||
peonist-ai/halogen-qwen3.8-flash-next (w4b) as a rootless Quadlet
|
||||
container on Dankclaw, port 7770 — flags, memory model, client contract
|
||||
- [[dankclaw-iommu-translation-off]] — why Dankclaw boots with
|
||||
|
||||
Reference in new issue
Block a user