docs: the LLM index and the cooperation page catch up with layer split

The LLM index still opened with vllm-radiance as frank-medium's runtime and
drew the replica pair in its topology diagram; the cooperation page's thesis
(collectives are impossible, so the answer is hardware) now has to name the
path the fleet actually took: llama.cpp's layer split, which launches no RCCL
kernel on either card. That reframes the M.2 riser as a means to TENSOR
parallelism rather than a means to using both cards at all.
This commit is contained in:
Frank committed 2026-09-16 15:39:47 -07:00
1 parent 87492c7b10
commit c97e359bc2
2 files changed
+49 -19

No files matched your search

+21 -14
View File
@@ -1,11 +1,13 @@
# LLM Fleet — Knowledge Base
Documentation for the LLM inference fleet on the two hosts (2026-09-16):
vllm-radiance on Dankserver (frank-medium, just1moremodel
Qwen3.8-27B-Uncensored-MXFP4-awq on 2x Radeon AI PRO R9700) and TWO endpoints on
Dankclaw — halogen-flash-server (frank-large, Qwen3.8-Flash-Next w4b on the
gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small, ISTA-DASLab
Qwen3.8-27B-GSQ-RCO IQ3_S with MTP and vision on the RTX 3090, port 7771).
**mainline llama.cpp on Dankserver** serving frank-medium as one layer-split
process across both Radeon AI PRO R9700 cards (ajgazin Swift-Qwen3.8-27B-
Uncensored-Dynamic-MTP UD-Q8_K_XL + the z-lab DFlash2 drafter), and TWO
endpoints on Dankclaw — halogen-flash-server (frank-large, Qwen3.8-Flash-Next
w4b on the gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small,
ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S with MTP and vision on the RTX 3090, port
7771).
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -19,10 +21,10 @@ graph LR
subgraph Dankserver[AMD Dankserver - Ubuntu 26.04]
A[2x Radeon AI PRO R9700
gfx1201, 32 GiB each]
B2[vllm-radiance MXFP4
B2[llama.cpp, layer split
frank-medium, port 7770]
C[Qwen3.8-27B-Uncensored-MXFP4-awq
2 single-GPU replicas, 114688 ctx / 4 seqs, DFlash2]
C[Swift-Qwen3.8-27B-Uncensored UD-Q8_K_XL
ONE process on both cards, 262144 ctx / 1 slot, DFlash2]
A --> B2 --> C
end
@@ -54,13 +56,18 @@ frank-small, port 7771]
## Active Deployment
- [[frank-medium-radiance-mxfp4]] — the ACTIVE frank-medium runbook: the
vllm-radiance MXFP4 stack (stilldeadcode/vllm-radiance:0.9.3 +
- [[frank-medium-llamacpp-dflash2]] — the ACTIVE frank-medium runbook since
2026-09-16: mainline llama.cpp serving
ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF at UD-Q8_K_XL with the
z-lab DFlash2 Q8_0 drafter on Dankserver's 2x Radeon AI PRO R9700, as ONE
layer-split process behind :7770 (tensor parallelism is impossible on this
host — the second card cannot dispatch an RCCL collective)
- [[frank-medium-radiance-mxfp4]] — RETIRED 2026-09-16, one day after it was
deployed: the vllm-radiance MXFP4 stack (stilldeadcode/vllm-radiance:0.9.3 +
ggz14/radiance-vllm-mxfp4 patches) serving
just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq on Dankserver's 2x Radeon
AI PRO R9700 as two single-GPU replicas behind a round-robin proxy, with
the PCIe-atomics evidence that rules out TP=2 on this host (since
2026-09-15/16)
just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq as two single-GPU replicas
behind a round-robin proxy; kept as the record of the deployment it
succeeded, plus the PCIe-atomics evidence that rules out TP=2 on this host
- [[frank-medium-rocmfpx]] — PRIOR frank-medium deployment (llama.cpp era,
2026-09-15): charlie12345/ROCmFPX serving
kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF; kept as the record of the