Answers Frank's question: given the 2x R9700 box, which of DeepSeek-V4.1-Flash,
MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash runs best, on what runtime, and can
Affinity / vllm-radiance / FreeToken-AMD be modified to run them fast.
Sizing, measured from the HF API rather than the model cards:
- DS4.1 payload 475.27 GiB, of which 189.13 GiB is Engram (two 94.56 GiB
shards, the layer 1 and 14 lookup tables). Lucebox never maps them, so
they do not evict the expert page cache.
- 240 routed experts x ~11 MB per token at the 37-of-384-per-layer residency
the hot stack can hold = ~2.4 GB across PCIe per token. At 55 GB/s real on
Gen5 x8 that is a ~23 tok/s ceiling, under the 35 tok/s goal before any
attention or dense work.
- The backend refuses layer-major prefill for V4.1, so the only legal
prefill is token-by-token; 1000 t/s is ~20x away.
Affinity (Codeberg StillDeadcode) builds clean here (hipcc 23, gfx1201, zero
errors) and claims 110 tok/s for the 284B V4-Flash, because the 101 GiB
container leaves the cold expert set in a pinned host pool in RAM. V4.1 is
2.5x too big to repeat that. Port scope, from both checkpoints' real
safetensors indices: expert machinery ports unchanged (identical tensor
naming, 15,360 tensors at 40x384), 26/32 per-layer tensors and 34/35 config
keys present. Five gaps - Engram, the disjoint {0,1,2} vs {0,4,128} ratio
vocabulary, CED, the DSpark draft geometry, vision. num_hash_layers is the
only key V4.1 lacks, so model.h:36's default of 3 silently survives and
would route layers 0-2 through a table that does not exist.
Also corrects: Dankserver RAM 60 -> 246 GiB (stale since the 2026-09-20 board
swap) and storage; the top-level index credited V4.1 with V4's geometry
(43 layers / 256 experts vs the actual 40 / 384).
Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI PRO R9700 gfx1201 on CPU lanes x8/x8) serving frank-medium.
dankserver
hardware
r9700
gfx1201
rocm
2026-09-20
Dankserver System Specs
Server Overview
Dankserver is a media/library server and the frank-medium LLM host. The
host runs Ubuntu 26.04 (Resolute Raccoon). The RTX 3090 that served the
FreeToken/CUDA frank-small era was removed on 2026-09-15 and replaced with
two AMD Radeon AI PRO R9700 cards.
Hardware Specifications
Motherboard
Property
Value
Board
ASUS ProArt B850-Creator WiFi NEO (Rev 1.xx) — swapped in 2026-09-20
Previous
ASUS TUF GAMING B650-E WIFI (card #2 was behind the chipset there; see Dual-R9700-Cooperation)
GPU lanes
CPU GPP ports bifurcated x8/x8 at Gen5: PCIEX16(G5)_1 → 00:01.1, PCIEX16(G5)_2 → 00:01.3; ~31.5 GB/s per card
Atomic routing
AtomicOpsCap: Routing+ on every bridge in both GPU chains, AtomicOpsCtl: ReqEn+ on both endpoints — the condition that ruled out TP=2 for the previous board
2x Realtek RTL8126 5GbE, RTL8922AE WiFi 7, PROM21 SATA controller
CPU
Property
Value
Architecture
x86_64
Vendor
AuthenticAMD
Model
AMD Ryzen 7 7700X
Cores
8 cores / 16 threads (1 socket) — SMT is on; the BIOS defaults reset with the board swap. llama.cpp still runs --threads 8 --threads-batch 8, so this is headroom, not a change in the serving shape
NUMA Nodes
1 (node 0)
GPUs
Property
Value
Model
2x AMD Radeon AI PRO R9700 (Navi 48, DID 0x7551, rev c0)
Architecture
gfx1201 (RDNA4)
VRAM
32,624 MiB each (rocm-smi reports 34,208,743,424 B per card)
Link
PCIe 5.0 x8 each (CPU GPP 00:01.1 / 00:01.3), no XGMI; separate IOMMU groups (15 and 20), ACS ReqRedir+ on both root ports, so P2P hairpins at the root complex
Atomic ops
Routing+ on all four bridges per chain; ReqEn+ on both endpoints (CAP_EXP+28.w = 0x1440)
PCI
03:00.0 and 07:00.0 (were 03:00.0 / 0b:00.0 on the B650-E)
iGPU
Raphael (gfx1036) at 11:00.0 — enumerates as ROCm device 2, hidden from HIP
The RTX 3090 is gone; stale nvidia-smi binaries and the disabled NVIDIA
apt lists remain on disk but no NVIDIA PCI device exists.
Memory
Property
Value
System RAM
246 GiB (upgraded for the Lucebox DeepSeek deployment of 2026-09-26; this page said 60 GiB until 2026-09-28 — a stale figure that survived the board swap)
Swap
63 GiB (/swapfile)
/dev/shm
32 GiB
Storage
Filesystem
Size
Used
Avail
Use%
/ (nvme0n1p2)
1.8T
927G
~822G
~54%
/mnt/entropy (dm-0)
91T
87T
244G
100% — full; do not plan model payload here
Models live under /home/dank/models; the serving build tree under
/home/dank/GitPackages/llama.cpp-dflash2 (the older charlie-ROCmFPX tree is
still on disk), both sharing the self-contained ROCm 7.14 nightly toolchain
under ~/.local/share/rocm-7.14.0a20260624 — which also carries librccl.so.1
and hipcc, so an RCCL-backed rebuild needs no new host packages.
Network
Interface
Type
Speed
IP
Notes
eno1
onboard RTL8126 5GbE
2500 Mb/s (negotiated)
192.168.0.235, static
LLM endpoint (port 7770). The address models.yml, Prometheus and Pi-hole all point at — preserved across the swap.
eno2
onboard RTL8126 5GbE, second port
2500 Mb/s
192.168.0.247, DHCP
New since the swap: cabled, and it took the lower-metric default route (101, against eno1's 102). Two live addresses and two default routes for one host — either declare it or bring the link down.
wlp14s0
WiFi 7 (RTL8922AE)
-
-
Renamed with the board (was wlp8s0); unused.
OS and Software
Property
Value
OS
Ubuntu 26.04 LTS (Resolute Raccoon)
Kernel
7.0.0-31-generic (iommu=pt)
Python
3.14.4
ROCm userspace
Ubuntu 7.1 packages in /usr (hipcc 7.1.1, rocminfo, rocm-smi 7.1.1)
Serving stack
mainline llama.cpp fb27a525 at ~/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714, RPATH-bundled with the ROCm 7.14 nightly runtime
The stale AMD-repo ROCm 6.3.3 packages (167, ~24.04-suffixed) that used to
hijack libhsa-runtime64 were purged on 2026-09-15 — see
frank-medium-rocmfpx.
Active Services
The active inference service is frank-medium.service: mainline llama-server on
port 7770, layer-split 32:33 across both R9700s with the DFlash2 drafter pinned to
ROCm1. See frank-medium-llamacpp-dflash2. The former
frank-small.service (FreeToken/CUDA) was retired with the 2026-09-15 GPU swap;
its unit is archived in ~/.AGENTTRASH/20260915-frank-small-retire/host-units/,
and the vllm-radiance replica trio in ~/.AGENTTRASH/20260916-vllm-radiance-retire/.
See deployment.
Hardware Relevance
2x 32 GB VRAM: the 27.0 GiB Q8_0 checkpoint plus the 1.06 GiB DFlash2 drafter
and the f16 KV pool for 327,680 tokens fit with the split at 32:33; live
headroom is ~6.3 GiB on ROCm0 and ~5.2 GiB on ROCm1, which is why a batch or
context change needs a re-fit and not a guess.
60 GB system RAM is mostly headroom; nothing in the serving path is
CPU-resident.
The iGPU is deliberately unused: HIP_VISIBLE_DEVICES=0,1 keeps work off
the shared-memory Raphael device.
Frank
d29d104f01