The radiance MXFP4 stack cannot run TP=2 on this host. RCCL's collective
kernels carry a hidden hostcall-buffer parameter, and ROCr refuses to
dispatch them on a device without PCIe atomics: the AQL dispatch fails with
hipErrorIllegalState ("the operation cannot be performed in the present
state"), which is what every two-rank collective hit. The second R9700 sits
behind the chipset bridges 05:00.0/06:09.0, which advertise
`AtomicOpsCap: Routing-` — a read-only capability, so no software can enable
atomic routing on that path and the driver's
pci_enable_atomic_ops_to_root() fails for that card. The first card, on CPU
root port 00:01.1, has atomics enabled.
Ruled out: kernel version (7.0.0-28/-30/-31 identical, after a full upgrade),
iommu=pt, P2P hardware (45.3 GB/s between the cards), the shipped librccl (a
from-source rebuild reproduces the failure at the same call path), vLLM and
pynccl (a bare two-rank torch.distributed allreduce fails the same way), all
NCCL transport knobs, and single-rank tests (NCCL short-circuits at
world_size=1, so one-card success proves nothing).
Deployment: three rootless Quadlets — frank-medium-a (:7771, card 1),
frank-medium-b (:7772, card 2), both TP=1 at 114688 context (262144 and
131072 do not fit one card's KV) with vLLM-profiled KV — and
frank-medium-proxy (:7770), a round-robin that preserves the fleet contract
of one endpoint and one served model name. frank-medium.container stays as
the TP=2 target for a host whose second card is atomic-capable; the deploy
handler does not install it and trashes any stale copy.
Measured through the proxy (vllm bench serve, random dataset, --ignore-eos):
conc 1 42.8 tok/s, conc 4 147.6 tok/s, conc 8 230.8 tok/s, 32k prefill at
~2.8k tok/s per card, TPOT p50 23.6-28.8 ms.
Also: healthcheck reads PORT so one probe serves all three units, the hermes
example and the harness models.yml drop to the real 114688 context, and the
runbook records the full evidence chain.
description:vllm-radiance MXFP4 serving just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq as frank-medium on Dankserver (2x AMD Radeon AI PRO R9700, gfx1201), as two single-GPU replicas behind a round-robin proxy — and the PCIe-atomics reason TP=2 cannot run on this host.
tags:
- frank-medium
- dankserver
- vllm
- vllm-radiance
- mxfp4
- gfx1201
- r9700
- rccl
- pcie-atomics
- quadlet
timestamp:2026-09-16
---
# frank-medium vllm-radiance MXFP4 Runbook
The ACTIVE frank-medium deployment (since 2026-09-15/16): the
`stilldeadcode/vllm-radiance:0.9.3` image plus the `ggz14/radiance-vllm-mxfp4`
patches, serving `just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq` on the two
Radeon AI PRO R9700 (gfx1201) cards. It replaced the charlie12345/ROCmFPX
llama.cpp deployment ([[frank-medium-rocmfpx]]).
**Deployed shape: two single-GPU replicas behind a round-robin proxy.** The
intended shape — one tensor-parallel process across both cards (TP=2) — cannot
launch a collective on this host. The reason is a hardware property of the
motherboard's PCIe topology, not the image, the RCCL build, or the kernel
version. The evidence is in [Why TP=2 cannot run here](#why-tp2-cannot-run-here).
Card #1 is on a CPU root port; card #2 is behind the chipset bridges
`05:00.0`/`06:09.0`, which advertise `AtomicOpsCap: Routing-`. That capability
bit is **read-only** — no software can enable atomic routing through those
bridges — so the driver's atomic enablement call fails for card #2 and the KFD
marks the CPU↔GPU2 link `NO_ATOMICS`.
Writing the AtomicOp Requester bit into the endpoint's Device Control 2 by hand
(`setpci -s 0b:00.0 CAP_EXP+28.w`) does not help: the driver reads the chain at
probe time, and ROCr consumes the driver's cached answer. Only a re-probe (a
boot with the write already in place) would change it — and even then the two
`Routing-` bridges would still make `pci_enable_atomic_ops_to_root()` fail.
### What was ruled out
| Suspect | Result |
|---|---|
| Kernel version | 7.0.0-28, -30 and -31 fail identically (full `apt full-upgrade` to the newest available kernel changed nothing) |
| IOMMU | identical before and after `iommu=pt` |
| P2P hardware | fine: 45.3 GB/s bidirectional copy between the two cards; both cards pass `rocm-bandwidth-test` |
| The shipped librccl | a from-source rebuild of `ROCm/rccl` develop (with local fixes for that branch's dangling references) reproduces the same failure at the same call path — the library is not at fault |
| vLLM/pynccl | a bare two-rank `torch.distributed` allreduce with no vLLM involved fails the same way; single-rank collectives succeed on both cards (NCCL short-circuits at `world_size=1`, which is why a one-card test proves nothing) |
| Transport knobs | `NCCL_PROTO=Simple`, `NCCL_P2P_DISABLE`, `NCCL_SHM_DISABLE`, `NCCL_CUMEM_ENABLE=0`, MSCCL off, channel counts, launch modes, `HSA_ENABLE_SDMA=0` — all identical |
| Image/support matrix | the same image on a different host with the same kernel (Proxmox VM, Codeberg issue #22) fails byte-identically |
### What would fix it
- **Move card #2 to a slot whose path is atomic-capable** (a CPU root port, as
card #1 has). This is the only software-free fix and the one that restores
true TP=2. On this AM5 platform the CPU provides the x16 slot plus x4 lanes,
so a second x16-class card necessarily lands on the chipset path — a
platform with more CPU lanes (Threadripper/EPYC) is the structural answer.
- Until then, **two replicas** are how both cards serve.
## Deployed configuration
Both replicas, uniform except GPU index and port:
| Knob | Value | Why |
|---|---|---|
| `TP` | 1 | one card per replica |
| `MAXLEN` | 114688 | the largest shape that boots on one card: 262144 needs 9.61 GiB of KV against 5.14 available, and 131072 needs 5.58 against 5.22 |
| `KV_MEM` | 0 | vLLM profiles this shape; the upstream pin (2x7551-32624) is a TP=2/262144 measurement and is invalid here |
| `MAXSEQS` | 4 | matches the per-replica share of the fleet's concurrency |
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Frank