docs: the LLM index and the cooperation page catch up with layer split
The LLM index still opened with vllm-radiance as frank-medium's runtime and drew the replica pair in its topology diagram; the cooperation page's thesis (collectives are impossible, so the answer is hardware) now has to name the path the fleet actually took: llama.cpp's layer split, which launches no RCCL kernel on either card. That reframes the M.2 riser as a means to TENSOR parallelism rather than a means to using both cards at all.
This commit is contained in:
1 parent
87492c7b10
commit
c97e359bc2
2 files changed
+49
-19
No files matched your search
@@ -1,11 +1,13 @@
|
||||
# LLM Fleet — Knowledge Base
|
||||
|
||||
Documentation for the LLM inference fleet on the two hosts (2026-09-16):
|
||||
vllm-radiance on Dankserver (frank-medium, just1moremodel
|
||||
Qwen3.8-27B-Uncensored-MXFP4-awq on 2x Radeon AI PRO R9700) and TWO endpoints on
|
||||
Dankclaw — halogen-flash-server (frank-large, Qwen3.8-Flash-Next w4b on the
|
||||
gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small, ISTA-DASLab
|
||||
Qwen3.8-27B-GSQ-RCO IQ3_S with MTP and vision on the RTX 3090, port 7771).
|
||||
**mainline llama.cpp on Dankserver** serving frank-medium as one layer-split
|
||||
process across both Radeon AI PRO R9700 cards (ajgazin Swift-Qwen3.8-27B-
|
||||
Uncensored-Dynamic-MTP UD-Q8_K_XL + the z-lab DFlash2 drafter), and TWO
|
||||
endpoints on Dankclaw — halogen-flash-server (frank-large, Qwen3.8-Flash-Next
|
||||
w4b on the gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small,
|
||||
ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S with MTP and vision on the RTX 3090, port
|
||||
7771).
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||
@@ -19,10 +21,10 @@ graph LR
|
||||
subgraph Dankserver[AMD Dankserver - Ubuntu 26.04]
|
||||
A[2x Radeon AI PRO R9700
|
||||
gfx1201, 32 GiB each]
|
||||
B2[vllm-radiance MXFP4
|
||||
B2[llama.cpp, layer split
|
||||
frank-medium, port 7770]
|
||||
C[Qwen3.8-27B-Uncensored-MXFP4-awq
|
||||
2 single-GPU replicas, 114688 ctx / 4 seqs, DFlash2]
|
||||
C[Swift-Qwen3.8-27B-Uncensored UD-Q8_K_XL
|
||||
ONE process on both cards, 262144 ctx / 1 slot, DFlash2]
|
||||
A --> B2 --> C
|
||||
end
|
||||
|
||||
@@ -54,13 +56,18 @@ frank-small, port 7771]
|
||||
|
||||
## Active Deployment
|
||||
|
||||
- [[frank-medium-radiance-mxfp4]] — the ACTIVE frank-medium runbook: the
|
||||
vllm-radiance MXFP4 stack (stilldeadcode/vllm-radiance:0.9.3 +
|
||||
- [[frank-medium-llamacpp-dflash2]] — the ACTIVE frank-medium runbook since
|
||||
2026-09-16: mainline llama.cpp serving
|
||||
ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF at UD-Q8_K_XL with the
|
||||
z-lab DFlash2 Q8_0 drafter on Dankserver's 2x Radeon AI PRO R9700, as ONE
|
||||
layer-split process behind :7770 (tensor parallelism is impossible on this
|
||||
host — the second card cannot dispatch an RCCL collective)
|
||||
- [[frank-medium-radiance-mxfp4]] — RETIRED 2026-09-16, one day after it was
|
||||
deployed: the vllm-radiance MXFP4 stack (stilldeadcode/vllm-radiance:0.9.3 +
|
||||
ggz14/radiance-vllm-mxfp4 patches) serving
|
||||
just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq on Dankserver's 2x Radeon
|
||||
AI PRO R9700 as two single-GPU replicas behind a round-robin proxy, with
|
||||
the PCIe-atomics evidence that rules out TP=2 on this host (since
|
||||
2026-09-15/16)
|
||||
just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq as two single-GPU replicas
|
||||
behind a round-robin proxy; kept as the record of the deployment it
|
||||
succeeded, plus the PCIe-atomics evidence that rules out TP=2 on this host
|
||||
- [[frank-medium-rocmfpx]] — PRIOR frank-medium deployment (llama.cpp era,
|
||||
2026-09-15): charlie12345/ROCmFPX serving
|
||||
kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF; kept as the record of the
|
||||
|
||||
Reference in new issue
Block a user