docs: frank-medium is llama.cpp with DFlash2 on both cards
The runbook for the new deployment, plus the sweep of everything that still described the vllm-radiance replica pair: the vault index and its ACTIVE marker, the superseded banner on the radiance page, the pointer from the ROCmFPX page, the homepage card, the Grafana dashboard description and the observability page's scrape table. The new page carries the two facts a future agent needs and the flags do not say: llama.cpp's layer split is what lets BOTH cards serve ONE model without any collective (vLLM's PP handoff is an RCCL send/recv, which is the primitive this host cannot dispatch), and the rocBLAS Tensile-path trap that makes a healthy-looking build fail on the first request.
This commit is contained in:
1 parent
c9c42e7489
commit
ab51d61198
7 files changed
+266
-21
No files matched your search
@@ -114,12 +114,19 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
|
||||
- [[deployment]] — unified deploy and update process for both servers
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[frank-medium-radiance-mxfp4]] — the ACTIVE frank-medium runbook: the
|
||||
vllm-radiance MXFP4 stack serving
|
||||
just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq on Dankserver's 2x Radeon
|
||||
AI PRO R9700 as two single-GPU replicas behind a round-robin proxy, plus
|
||||
the PCIe-atomics evidence that rules out TP=2 on this host (since
|
||||
2026-09-15/16)
|
||||
- [[frank-medium-llamacpp-dflash2]] — the ACTIVE frank-medium runbook since
|
||||
2026-09-16: mainline llama.cpp (gfx1201 HIP build) serving
|
||||
ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF at UD-Q8_K_XL with
|
||||
the DFlash2 drafter (z-lab/Qwen3.8-27B-DFlash2-GGUF at Q8_0) on
|
||||
Dankserver's 2x Radeon AI PRO R9700 as one layer-split process behind the
|
||||
:7770 endpoint — no tensor parallelism, because the second card cannot
|
||||
dispatch an RCCL collective
|
||||
- [[frank-medium-radiance-mxfp4]] — RETIRED 2026-09-16: the vllm-radiance
|
||||
MXFP4 stack serving just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq on
|
||||
Dankserver's 2x Radeon AI PRO R9700 as two single-GPU replicas behind a
|
||||
round-robin proxy; kept as the record of the deployment
|
||||
[[frank-medium-llamacpp-dflash2]] replaced, plus the PCIe-atomics evidence
|
||||
that rules out TP=2 on this host (since 2026-09-15/16)
|
||||
- [[Dual-R9700-Cooperation]] — why the two R9700s cannot run tensor-parallel
|
||||
together: the PCIe-atomics gate quoted from ROCclr, the measured RCCL matrix
|
||||
(collectives *and* P2P `send`/`recv` both fail on card #2, which rules out
|
||||
@@ -157,6 +164,11 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
||||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||||
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
|
||||
- [[frank-small-qwen38-gsq-rco]] — the ACTIVE frank-small runbook since
|
||||
2026-09-16: mainline llama.cpp serving ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-
|
||||
GGUF at IQ3_S (own MTP head + BF16 vision projector) on Dankclaw's
|
||||
RTX 3090, port 7771 — the 3090 freed when halogen took the iGPU back for
|
||||
frank-large
|
||||
- [[ikllama-ornith-dankserver]] — the retired ik_llama.cpp runtime for the old frank-small (q8_0 + Hadamard KV cache, split-mode none; 2026-08-26)
|
||||
- [[freetoken-frank-small]] — RETIRED 2026-09-15: the FreeToken/CUDA frank-small runbook, retired with the RTX 3090
|
||||
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
|
||||
|
||||
Reference in new issue
Block a user