docs: wrap up the 2026-09-19 fleet cutover
frank-large -> CIRU v4.4.1 Orca (new runbook), frank-medium -> the deployed
orcarouter Q8_0 / DFlash2 Q4_K_M / balanced 32:33 / 327680 state. The d374ab9
frank-medium deploy updated only scripts/tests and left AGENTS.md, the runbook
and both indexes on the predecessor (ajgazin Swift UD-Q8_K_XL, 35:30, 393216).
- AGENTS.md: frank-large section (CIRU Orca, root unit, runtime gate, PLE, MTP)
+ frank-medium section and topology row to the deployed shape
- new runbook frank-large-ciru-orca.md
- frank-medium runbook header/intro: orcarouter Q8_0 swap, 32:33, 327680;
predecessor kept as calibration history
- LLM index.md: intro + mermaid (halogen->CIRU, GSQ-RCO->Gemma4, Swift->orcarouter),
add ciru-orca, retire halogen
- top-level index.md: add ciru-orca link (docs-index seam), retire halogen,
fix frank-medium entry
This commit is contained in:
Frank committed
1 parent
f68bda56c7
commit
764cf41222
5 files changed
+340
-106
No files matched your search
@@ -1,13 +1,13 @@
|
||||
# LLM Fleet — Knowledge Base
|
||||
|
||||
Documentation for the LLM inference fleet on the two hosts (2026-09-16):
|
||||
Documentation for the LLM inference fleet on the two hosts (2026-09-19):
|
||||
**mainline llama.cpp on Dankserver** serving frank-medium as one layer-split
|
||||
process across both Radeon AI PRO R9700 cards (ajgazin Swift-Qwen3.8-27B-
|
||||
Uncensored-Dynamic-MTP UD-Q8_K_XL + the z-lab DFlash2 drafter), and TWO
|
||||
endpoints on Dankclaw — halogen-flash-server (frank-large, Qwen3.8-Flash-Next
|
||||
w4b on the gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small,
|
||||
ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S with MTP and vision on the RTX 3090, port
|
||||
7771).
|
||||
process across both Radeon AI PRO R9700 cards (orcarouter Qwen3.8-27B-
|
||||
Uncensored Q8_0 + the z-lab DFlash2 Q4_K_M drafter, balanced 32:33), and TWO
|
||||
endpoints on Dankclaw — the CIRU v4.4.1 runtime (frank-large, jcbtc
|
||||
Qwen3.8-Flash-CIRU-STRIX-Orca w4b on the gfx1151 iGPU, port 7770) and mainline
|
||||
llama.cpp (frank-small, HauhauCS Gemma4-12B QAT Q4_K_M with MTP and vision on
|
||||
the RTX 3090, port 7771).
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||
@@ -23,22 +23,22 @@ graph LR
|
||||
gfx1201, 32 GiB each]
|
||||
B2[llama.cpp, layer split
|
||||
frank-medium, port 7770]
|
||||
C[Swift-Qwen3.8-27B-Uncensored UD-Q8_K_XL
|
||||
ONE process on both cards, 262144 ctx / 1 slot, DFlash2]
|
||||
C[orcarouter Qwen3.8-27B Q8_0 + DFlash2 Q4_K_M
|
||||
ONE process on both cards, 2x163840 ctx, balanced 32:33]
|
||||
A --> B2 --> C
|
||||
end
|
||||
|
||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||
E2[RTX 3090 (PCIe x4 dock) +
|
||||
gfx1151 iGPU 64 GiB unified memory]
|
||||
E[halogen-flash-server 0.5.8
|
||||
E[CIRU v4.4.1 runtime
|
||||
frank-large, port 7770]
|
||||
F[peonist-ai Qwen3.8-Flash-Next w4b
|
||||
2 x 200192 ctx over one 400384-position pool]
|
||||
F[jcbtc Qwen3.8-Flash-CIRU-STRIX-Orca w4b
|
||||
262144 ctx / 1 slot, MTP depth 4 + PLE]
|
||||
G[mainline llama.cpp 37b53fd
|
||||
frank-small, port 7771]
|
||||
H[Qwen3.8-27B GSQ-RCO IQ3_S
|
||||
131072 ctx / 1 slot, MTP + vision]
|
||||
H[Gemma4-12B QAT Q4_K_M
|
||||
4 x 262144 ctx, MTP + vision + audio]
|
||||
E2 --> E --> F
|
||||
E2 --> G --> H
|
||||
end
|
||||
@@ -57,11 +57,12 @@ frank-small, port 7771]
|
||||
## Active Deployment
|
||||
|
||||
- [[frank-medium-llamacpp-dflash2]] — the ACTIVE frank-medium runbook since
|
||||
2026-09-16: mainline llama.cpp serving
|
||||
ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF at UD-Q8_K_XL with the
|
||||
z-lab DFlash2 Q8_0 drafter on Dankserver's 2x Radeon AI PRO R9700, as ONE
|
||||
layer-split process behind :7770 (tensor parallelism is impossible on this
|
||||
host — the second card cannot dispatch an RCCL collective)
|
||||
2026-09-16 (model swapped 2026-09-19): mainline llama.cpp serving
|
||||
orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M
|
||||
drafter on Dankserver's 2x Radeon AI PRO R9700, as ONE layer-split process
|
||||
behind :7770 (tensor parallelism is impossible on this host — the second card
|
||||
cannot dispatch an RCCL collective), balanced 32:33 over two 163,840-token
|
||||
slots
|
||||
- [[frank-medium-radiance-mxfp4]] — RETIRED 2026-09-16, one day after it was
|
||||
deployed: the vllm-radiance MXFP4 stack (stilldeadcode/vllm-radiance:0.9.3 +
|
||||
ggz14/radiance-vllm-mxfp4 patches) serving
|
||||
@@ -81,12 +82,17 @@ frank-small, port 7771]
|
||||
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
|
||||
- [[deployment]] — deploy and update process for both servers
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[halogen-flash-server]] — ACTIVE frank-large runbook again since 2026-09-16
|
||||
(rootless Quadlet, port 7770): peonist-ai halogen-flash-server 0.5.8 serving
|
||||
peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn + quality
|
||||
overlay) on the gfx1151 iGPU — two 200k conversations over one
|
||||
- [[frank-large-ciru-orca]] — ACTIVE frank-large runbook since 2026-09-19
|
||||
(root systemd unit, port 7770): the CIRU v4.4.1 runtime (private host build,
|
||||
QSA hotfix) serving jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca w4b on the gfx1151
|
||||
iGPU — one 262,144-token conversation, MTP depth 4, 4096 MiB PLE cache,
|
||||
runtime-root gate, client contract, troubleshooting
|
||||
- [[halogen-flash-server]] — RETIRED 2026-09-19 (replaced by
|
||||
[[frank-large-ciru-orca]]; rollback target): peonist-ai halogen-flash-server
|
||||
0.5.8 serving peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b
|
||||
.hgn + quality overlay) on the gfx1151 iGPU — two 200k conversations over one
|
||||
400,384-position pool, deployed flags, memory model, client contract,
|
||||
troubleshooting
|
||||
troubleshooting (2026-09-16 → 09-19)
|
||||
- [[frank-small-gemma4-12b]] — ACTIVE frank-small runbook since 2026-09-17
|
||||
(root systemd unit, port 7771): mainline llama.cpp serving
|
||||
HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced Q4_K_M on the RTX
|
||||
@@ -140,8 +146,8 @@ frank-small, port 7771]
|
||||
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment
|
||||
(deployed and reverted 2026-08-22)
|
||||
- [[Dankserver System Specs]] — the 2x R9700 box (frank-medium)
|
||||
- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on
|
||||
halogen-flash-server since 2026-09-16, frank-small on the RTX 3090)
|
||||
- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on the
|
||||
CIRU v4.4.1 runtime since 2026-09-19, frank-small on the RTX 3090)
|
||||
- [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
|
||||
on Dankclaw (Gemma4 E4B vision+audio pack, XDNA2, port 7771), kept as
|
||||
the retirement record — the IOMMU/SVA err=-19 root cause and where
|
||||
|
||||
Reference in new issue
Block a user