Frank
d29d104f01
Answers Frank's question: given the 2x R9700 box, which of DeepSeek-V4.1-Flash,
MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash runs best, on what runtime, and can
Affinity / vllm-radiance / FreeToken-AMD be modified to run them fast.
Sizing, measured from the HF API rather than the model cards:
- DS4.1 payload 475.27 GiB, of which 189.13 GiB is Engram (two 94.56 GiB
shards, the layer 1 and 14 lookup tables). Lucebox never maps them, so
they do not evict the expert page cache.
- 240 routed experts x ~11 MB per token at the 37-of-384-per-layer residency
the hot stack can hold = ~2.4 GB across PCIe per token. At 55 GB/s real on
Gen5 x8 that is a ~23 tok/s ceiling, under the 35 tok/s goal before any
attention or dense work.
- The backend refuses layer-major prefill for V4.1, so the only legal
prefill is token-by-token; 1000 t/s is ~20x away.
Affinity (Codeberg StillDeadcode) builds clean here (hipcc 23, gfx1201, zero
errors) and claims 110 tok/s for the 284B V4-Flash, because the 101 GiB
container leaves the cold expert set in a pinned host pool in RAM. V4.1 is
2.5x too big to repeat that. Port scope, from both checkpoints' real
safetensors indices: expert machinery ports unchanged (identical tensor
naming, 15,360 tensors at 40x384), 26/32 per-layer tensors and 34/35 config
keys present. Five gaps - Engram, the disjoint {0,1,2} vs {0,4,128} ratio
vocabulary, CED, the DSpark draft geometry, vision. num_hash_layers is the
only key V4.1 lacks, so model.h:36's default of 3 silently survives and
would route layers 0-2 through a table that does not exist.
Also corrects: Dankserver RAM 60 -> 246 GiB (stale since the 2026-09-20 board
swap) and storage; the top-level index credited V4.1 with V4's geometry
(43 layers / 256 experts vs the actual 40 / 384).
132 lines
6.0 KiB
Markdown
132 lines
6.0 KiB
Markdown
---
|
|
type: configuration
|
|
title: Dankserver System Specs
|
|
description: Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI PRO R9700 gfx1201 on CPU lanes x8/x8) serving frank-medium.
|
|
tags:
|
|
- dankserver
|
|
- hardware
|
|
- r9700
|
|
- gfx1201
|
|
- rocm
|
|
timestamp: 2026-09-20
|
|
---
|
|
|
|
# Dankserver System Specs
|
|
|
|
## Server Overview
|
|
|
|
Dankserver is a media/library server and the `frank-medium` LLM host. The
|
|
host runs Ubuntu 26.04 (Resolute Raccoon). The RTX 3090 that served the
|
|
FreeToken/CUDA `frank-small` era was removed on 2026-09-15 and replaced with
|
|
two AMD Radeon AI PRO R9700 cards.
|
|
|
|
## Hardware Specifications
|
|
|
|
### Motherboard
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| Board | **ASUS ProArt B850-Creator WiFi NEO** (Rev 1.xx) — swapped in 2026-09-20 |
|
|
| Previous | ASUS TUF GAMING B650-E WIFI (card #2 was behind the chipset there; see [[Dual-R9700-Cooperation]]) |
|
|
| GPU lanes | CPU GPP ports bifurcated **x8/x8** at Gen5: `PCIEX16(G5)_1` → `00:01.1`, `PCIEX16(G5)_2` → `00:01.3`; ~31.5 GB/s per card |
|
|
| Atomic routing | `AtomicOpsCap: Routing+` on every bridge in both GPU chains, `AtomicOpsCtl: ReqEn+` on both endpoints — the condition that ruled out TP=2 for the previous board |
|
|
| Free | `PCIEX16(G4)` (chipset, x4), M.2_2, M.2_3 |
|
|
| Occupied M.2 | M.2_1 — boot NVMe (Samsung PM9C1a, DRAM-less), trained **x2** at Gen5 (63 Gb/s ≈ 7.9 GB/s) |
|
|
| Onboard | 2x Realtek RTL8126 5GbE, RTL8922AE WiFi 7, PROM21 SATA controller |
|
|
|
|
### CPU
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| Architecture | x86_64 |
|
|
| Vendor | AuthenticAMD |
|
|
| Model | AMD Ryzen 7 7700X |
|
|
| Cores | 8 cores / 16 threads (1 socket) — SMT is **on**; the BIOS defaults reset with the board swap. `llama.cpp` still runs `--threads 8 --threads-batch 8`, so this is headroom, not a change in the serving shape |
|
|
| NUMA Nodes | 1 (node 0) |
|
|
|
|
### GPUs
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| Model | 2x AMD Radeon AI PRO R9700 (Navi 48, DID 0x7551, rev c0) |
|
|
| Architecture | gfx1201 (RDNA4) |
|
|
| VRAM | 32,624 MiB each (`rocm-smi` reports 34,208,743,424 B per card) |
|
|
| Link | PCIe 5.0 **x8** each (CPU GPP `00:01.1` / `00:01.3`), no XGMI; separate IOMMU groups (15 and 20), ACS `ReqRedir+` on both root ports, so P2P hairpins at the root complex |
|
|
| Atomic ops | `Routing+` on all four bridges per chain; `ReqEn+` on both endpoints (`CAP_EXP+28.w = 0x1440`) |
|
|
| PCI | 03:00.0 and **07:00.0** (were `03:00.0` / `0b:00.0` on the B650-E) |
|
|
| iGPU | Raphael (gfx1036) at **11:00.0** — enumerates as ROCm device 2, hidden from HIP |
|
|
|
|
The RTX 3090 is gone; stale `nvidia-smi` binaries and the disabled NVIDIA
|
|
apt lists remain on disk but no NVIDIA PCI device exists.
|
|
|
|
### Memory
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| System RAM | **246 GiB** (upgraded for the Lucebox DeepSeek deployment of 2026-09-26; this page said 60 GiB until 2026-09-28 — a stale figure that survived the board swap) |
|
|
| Swap | 63 GiB (`/swapfile`) |
|
|
| /dev/shm | 32 GiB |
|
|
|
|
### Storage
|
|
|
|
| Filesystem | Size | Used | Avail | Use% |
|
|
|------------|------|------|-------|------|
|
|
| / (nvme0n1p2) | 1.8T | 927G | ~822G | ~54% |
|
|
| /mnt/entropy (dm-0) | 91T | 87T | **244G** | **100%** — full; do not plan model payload here |
|
|
|
|
Models live under `/home/dank/models`; the serving build tree under
|
|
`/home/dank/GitPackages/llama.cpp-dflash2` (the older `charlie-ROCmFPX` tree is
|
|
still on disk), both sharing the self-contained ROCm 7.14 nightly toolchain
|
|
under `~/.local/share/rocm-7.14.0a20260624` — which also carries `librccl.so.1`
|
|
and `hipcc`, so an RCCL-backed rebuild needs no new host packages.
|
|
|
|
### Network
|
|
|
|
| Interface | Type | Speed | IP | Notes |
|
|
|-----------|------|-------|----|-------|
|
|
| eno1 | onboard RTL8126 5GbE | 2500 Mb/s (negotiated) | 192.168.0.235, **static** | LLM endpoint (port 7770). The address `models.yml`, Prometheus and Pi-hole all point at — preserved across the swap. |
|
|
| eno2 | onboard RTL8126 5GbE, second port | 2500 Mb/s | 192.168.0.247, DHCP | **New since the swap**: cabled, and it took the lower-metric default route (101, against eno1's 102). Two live addresses and two default routes for one host — either declare it or bring the link down. |
|
|
| wlp14s0 | WiFi 7 (RTL8922AE) | - | - | Renamed with the board (was `wlp8s0`); unused. |
|
|
|
|
### OS and Software
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
|
|
| Kernel | 7.0.0-31-generic (`iommu=pt`) |
|
|
| Python | 3.14.4 |
|
|
| ROCm userspace | Ubuntu 7.1 packages in /usr (hipcc 7.1.1, rocminfo, rocm-smi 7.1.1) |
|
|
| Serving stack | mainline llama.cpp `fb27a525` at `~/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714`, RPATH-bundled with the ROCm 7.14 nightly runtime |
|
|
|
|
The stale AMD-repo ROCm 6.3.3 packages (167, `~24.04`-suffixed) that used to
|
|
hijack `libhsa-runtime64` were purged on 2026-09-15 — see
|
|
[[frank-medium-rocmfpx]].
|
|
|
|
### Active Services
|
|
|
|
The active inference service is `frank-medium.service`: mainline llama-server on
|
|
port 7770, layer-split 32:33 across both R9700s with the DFlash2 drafter pinned to
|
|
ROCm1. See [[frank-medium-llamacpp-dflash2]]. The former
|
|
`frank-small.service` (FreeToken/CUDA) was retired with the 2026-09-15 GPU swap;
|
|
its unit is archived in `~/.AGENTTRASH/20260915-frank-small-retire/host-units/`,
|
|
and the vllm-radiance replica trio in `~/.AGENTTRASH/20260916-vllm-radiance-retire/`.
|
|
See [[deployment]].
|
|
|
|
## Hardware Relevance
|
|
|
|
- 2x 32 GB VRAM: the 27.0 GiB Q8_0 checkpoint plus the 1.06 GiB DFlash2 drafter
|
|
and the f16 KV pool for 327,680 tokens fit with the split at 32:33; live
|
|
headroom is ~6.3 GiB on ROCm0 and ~5.2 GiB on ROCm1, which is why a batch or
|
|
context change needs a re-fit and not a guess.
|
|
- 60 GB system RAM is mostly headroom; nothing in the serving path is
|
|
CPU-resident.
|
|
- The iGPU is deliberately unused: `HIP_VISIBLE_DEVICES=0,1` keeps work off
|
|
the shared-memory Raphael device.
|
|
|
|
## See Also
|
|
|
|
- [[frank-medium-llamacpp-dflash2]] — the active serving runbook
|
|
- [[Dual-R9700-Cooperation]] — why the PCIe path matters, and what the swap changed
|
|
- [[deployment]] — the deploy flow
|
|
- [[Shared Memory Architecture Dankclaw]] — for contrast, Dankclaw's unified memory
|