docs: the vault and AGENTS.md describe two Dankclaw endpoints
halogen-flash-server is ACTIVE again with its 2026-09-16 measured ledger (budget 33.5 GiB against a need of 27.2 — no clamp — and 14.7 GiB left for everything else), ik_llama.cpp is retired with its measurements kept as the record of why, and the new frank-small runbook covers the -mtp file, the 64 KiB/token KV arithmetic, the MTP contract, and the reasoning_effort trap that cost me every subagent on the endpoint. AGENTS.md: topology, Active Services, runbook index, and three new llama.cpp lessons (MTP is a property of the file; --jinja is what `tools` needs; a model's template can reject the harness's thinking level). Three pre-existing defects found while editing it: a truncated sentence mid-file, a harness path that does not exist on either server, and the benchmarking stop/mask recipe still calling frank-large a root unit. Truthfulness notes worth keeping: the co-tenant stall NOTE prints again, so the claim that it "no longer prints" is gone and the three observed block counts are recorded instead of a guessed threshold; and the Observability page says out loud why frank-large has no scrape job rather than leaving a silent absence.
This commit is contained in:
Frank committed
1 parent
40850bbc8f
commit
49e8dfff66
8 files changed
+565
-153
No files matched your search
@@ -1,10 +1,11 @@
|
||||
# LLM Fleet — Knowledge Base
|
||||
|
||||
Documentation for the LLM inference fleet on the two hosts
|
||||
(2026-09-16): charlie12345/ROCmFPX llama.cpp on Dankserver (frank-medium,
|
||||
kingjones777 Qwen3.8-27B-ROCmFPX-Q8_0-AGENT on 2x Radeon AI PRO R9700) and
|
||||
ik_llama.cpp on Dankclaw (frank-large, mradermacher Qwen3.8-Flash-Next-
|
||||
Uncensored i1-Q4_K_M on the RTX 3090 + Strix Halo hybrid).
|
||||
Documentation for the LLM inference fleet on the two hosts (2026-09-16):
|
||||
vllm-radiance on Dankserver (frank-medium, just1moremodel
|
||||
Qwen3.8-27B-Uncensored-MXFP4-awq on 2x Radeon AI PRO R9700) and TWO endpoints on
|
||||
Dankclaw — halogen-flash-server (frank-large, Qwen3.8-Flash-Next w4b on the
|
||||
gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small, ISTA-DASLab
|
||||
Qwen3.8-27B-GSQ-RCO IQ3_S with MTP and vision on the RTX 3090, port 7771).
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||
@@ -18,26 +19,31 @@ graph LR
|
||||
subgraph Dankserver[AMD Dankserver - Ubuntu 26.04]
|
||||
A[2x Radeon AI PRO R9700
|
||||
gfx1201, 32 GiB each]
|
||||
B2[ROCmFPX llama.cpp
|
||||
B2[vllm-radiance MXFP4
|
||||
frank-medium, port 7770]
|
||||
C[Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
|
||||
layer split, 131k ctx / 1 slot, MTP]
|
||||
C[Qwen3.8-27B-Uncensored-MXFP4-awq
|
||||
2 single-GPU replicas, 114688 ctx / 4 seqs, DFlash2]
|
||||
A --> B2 --> C
|
||||
end
|
||||
|
||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||
E2[RTX 3090 (PCIe x4 dock) +
|
||||
gfx1151 iGPU 64 GiB unified memory]
|
||||
E[ik_llama.cpp master d85a1d7
|
||||
E[halogen-flash-server 0.5.8
|
||||
frank-large, port 7770]
|
||||
F[mradermacher Qwen3.8-Flash-Next-Uncensored
|
||||
i1-Q4_K_M GGUF, 262k ctx / 1 slot, vision]
|
||||
F[peonist-ai Qwen3.8-Flash-Next w4b
|
||||
2 x 200192 ctx over one 400384-position pool]
|
||||
G[mainline llama.cpp 37b53fd
|
||||
frank-small, port 7771]
|
||||
H[Qwen3.8-27B GSQ-RCO IQ3_S
|
||||
131072 ctx / 1 slot, MTP + vision]
|
||||
E2 --> E --> F
|
||||
E2 --> G --> H
|
||||
end
|
||||
|
||||
CLI[LLM Clients / API] -->|HTTP :7770| B2
|
||||
CLI -->|HTTP :7770| E
|
||||
```
|
||||
CLI -->|HTTP :7771| G
|
||||
|
||||
## Servers
|
||||
|
||||
@@ -59,23 +65,31 @@ i1-Q4_K_M GGUF, 262k ctx / 1 slot, vision]
|
||||
2026-09-15): charlie12345/ROCmFPX serving
|
||||
kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF; kept as the record of the
|
||||
runtime it replaced
|
||||
- [[freetoken-frank-small]] — RETIRED 2026-09-15 with the RTX 3090: the
|
||||
FreeToken v0.1.2 frank-small runbook (batsclamp Huihui-Qwen3.6-35B-A3B-
|
||||
abliterated-FP8), kept as the retirement record
|
||||
- [[freetoken-frank-small]] — RETIRED 2026-09-15, when the RTX 3090 moved to
|
||||
frank-large: the FreeToken v0.1.2 frank-small runbook (batsclamp
|
||||
Huihui-Qwen3.6-35B-A3B-abliterated-FP8), kept as the retirement record. The
|
||||
NAME was reused on 2026-09-16 by a different service on Dankclaw — see
|
||||
[[frank-small-qwen38-gsq-rco]]
|
||||
- [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep
|
||||
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
|
||||
- [[deployment]] — deploy and update process for both servers
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[frank-large-ik-uncensored]] — ACTIVE frank-large runbook since
|
||||
2026-09-15: ik_llama.cpp (root systemd unit) serving mradermacher/
|
||||
Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M on the RTX 3090 (dense/attention/
|
||||
KV) + LPDDR (experts/PLE) hybrid; measured performance envelope, the
|
||||
decode root-cause chain, tuning levers, and the co-tenancy traps
|
||||
- [[halogen-flash-server]] — RETIRED 2026-09-15 (rollback target): the
|
||||
peonist-ai halogen-flash-server 0.5.8 runbook serving
|
||||
peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn +
|
||||
quality overlay), rootless Quadlet container on Dankclaw port 7770 —
|
||||
deployed flags, memory model, client contract, troubleshooting
|
||||
- [[halogen-flash-server]] — ACTIVE frank-large runbook again since 2026-09-16
|
||||
(rootless Quadlet, port 7770): peonist-ai halogen-flash-server 0.5.8 serving
|
||||
peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn + quality
|
||||
overlay) on the gfx1151 iGPU — two 200k conversations over one
|
||||
400,384-position pool, deployed flags, memory model, client contract,
|
||||
troubleshooting
|
||||
- [[frank-small-qwen38-gsq-rco]] — ACTIVE frank-small runbook since 2026-09-16
|
||||
(root systemd unit, port 7771): mainline llama.cpp serving
|
||||
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO IQ3_S on the RTX 3090, with the model's own
|
||||
Multi-Token Prediction head for speculative decoding and the BF16 projector
|
||||
for the fleet's image path
|
||||
- [[frank-large-ik-uncensored]] — RETIRED 2026-09-16 (rollback target): the
|
||||
ik_llama.cpp runbook (root systemd unit, 2026-09-15 → 09-16) serving
|
||||
mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M on the RTX 3090
|
||||
(dense/attention/KV) + LPDDR (experts/PLE) hybrid; its measured envelope and
|
||||
decode root-cause chain are the record of why the fleet went back to halogen
|
||||
- [[dankclaw-iommu-translation-off]] — why Dankclaw boots with
|
||||
`amd_iommu=off` since 2026-09-11 (13-16 % prefill, measured upstream),
|
||||
what it cost (the NPU endpoint), what was deliberately not copied from
|
||||
@@ -100,8 +114,8 @@ i1-Q4_K_M GGUF, 262k ctx / 1 slot, vision]
|
||||
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment
|
||||
(deployed and reverted 2026-08-22)
|
||||
- [[Dankserver System Specs]] — the 2x R9700 box (frank-medium)
|
||||
- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large,
|
||||
on halogen-flash-server since 2026-09-11)
|
||||
- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on
|
||||
halogen-flash-server since 2026-09-16, frank-small on the RTX 3090)
|
||||
- [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
|
||||
on Dankclaw (Gemma4 E4B vision+audio pack, XDNA2, port 7771), kept as
|
||||
the retirement record — the IOMMU/SVA err=-19 root cause and where
|
||||
|
||||
Reference in new issue
Block a user