serverAppConfigs/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md

---
type: research
title: Frontier MoE on Dankserver — what fits, on which runtime
description: VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded.
tags:
  - dankserver
  - r9700
  - gfx1201
  - moe
  - deepseek-v4.1
  - deepseek-v4
  - mimo
  - glm
  - affinity
  - lucebox
  - feasibility
timestamp: 2026-09-28
---

# Frontier MoE on Dankserver — what fits, on which runtime

> **Question this page answers.** Given Dankserver's hardware, which of
> DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash runs best, on
> which runtime, and can Affinity / vllm-radiance / FreeToken-AMD be modified
> to run them fast?
>
> **Answer in one line.** V4.1-Flash does not fit this box at any
> configuration; the engine that does run it (Lucebox) measures
> 0.8–9.1 tok/s on R9700 because decode is bound by PCIe expert streaming
> rather than by the GPUs. The 284B **V4-Flash** on **Affinity** is the only
> shape that clears 35 tok/s, at 110.

Everything below was measured or read from source on 2026-09-28. Claims
carried over from the engine authors are marked as theirs.

## Hardware (measured live, 2026-09-28)

| Property | Value |
|---|---|
| GPUs | 2x AMD Radeon AI PRO R9700 (Navi 48, **gfx1201**), 32,624 MiB each = **63.7 GiB** total |
| Interconnect | PCIe 5.0 **x8** each, CPU root ports `00:01.1` / `00:01.3`, ~31.5 GB/s per card |
| PCIe atomics | `CAP_EXP+28.w = 0x1440` (ReqEn+) on both endpoints — RCCL collectives are legal |
| CPU | Ryzen 7 7700X — **8 cores / 16 threads** (small; CPU expert compute is expensive) |
| RAM | **246 GiB** + 63 GiB swap |
| Disk | 1.8 TB NVMe. `/mnt/entropy` is a separate 91 TB ext4 at **100%** — do not plan payload there |

The knowledge base was wrong about RAM: [[Dankserver System Specs]] said
60 GiB until 2026-09-28 and has been corrected. The Lucebox runbook's 246 GiB
was right.

## The three candidates

| Model | Total / active | Routed experts | Attention & KV | Working gfx1201 runtime |
|---|---|---|---|---|
| **DeepSeek-V4.1-Flash** | 552B / 8B prefill, 16B decode | 384 top-6, 40 layers, `moe_intermediate_size` 2304, hidden 5120 | CED 20 enc + 20 dec; CSA2 Full/Reindex/Reuse; **FP4 main KV, 890 B/token**; Engram 196B at layers 1 & 14 | Lucebox `feat/ds41-engine` only (unmerged) |
| **MiMo-V2.6-Flash-MOPD** | 309B / 15B | 256 top-8, no shared expert, 48 layers (39 SWA / 9 GA) | hybrid SWA (window 128) + GA; 5-layer MTP | vLLM / SGLang, NVIDIA-first. No AMD path |
| **GLM-5.3-Flash** | 320B / 18B | MoE | KDA (linear) + DSA (sparse) hybrid; mHC | **tcclaviger vLLM fork — plausible native path (2026-09-29 source reading: `glm5next` registered with an `amd/` backend dir + DSA indexer + mHC path; r4d.so carries native mxfp4/dsfp4 MoE entries). Gate: a compressed-tensors MXFP4 payload (~165 GiB, none exists publicly) + a HIP boot probe.** llama.cpp GGUF (community R9700 guide) remains the verified-but-slower path |

## Sizing — the arithmetic that decides it

| Quantity | Value | Note |
|---|---:|---|
| DS4.1 official checkpoint | **475.27 GiB** | 48 shards, summed from the HF API |
|   of which Engram | **189.13 GiB** | exactly two 94.56 GiB shards — the layer 1 and 14 lookup tables |
| DS4.1 routed-expert params | 543.6 B | 40 x 384 x 3 x 5120 x 2304 |
|   shipped as | 269 GiB (fp4) | DeepSeek's own `expert_dtype: fp4` |
| Lucebox `ROCMFP23.gguf` | **377.51 GiB** | 179 GiB experts (2.5/3.5 bpw) + 189 GiB Engram + ~9.5 GiB dense |
| V4-Flash official checkpoint | 148.67 GiB | **3.2x smaller than V4.1** |
| Affinity container for V4-Flash | 101.1 GiB | 92.9 GiB experts @ 2.875 bpw + 6.2 dense + 2.0 embed |

The Engram tables are **never mapped** by Lucebox, so they do not evict the
expert page cache — that is the one piece of good news in the 377 GiB file.

### The decode wall

A V4.1 token selects `6 x 40 = 240` routed experts at ~11 MB each. At the
residency the hot stack can hold (37 of 384 per layer, 9.6%), **~2.4 GB of
expert weights cross PCIe per token**. PCIe 5.0 x8 does ~55 GB/s real:

```
2.4 GB / 55 GB/s = 44 ms  ->  ~23 tok/s absolute ceiling
```

That is **below the 35 tok/s target before any attention, Engram or dense work
is counted.** Raising residency is the only lever, and VRAM is what bounds it:
63.7 GiB / 11 MB = 5,800 of 15,360 experts = 37.8% maximum, and that assumes
VRAM holds nothing else.

The engine author measured the same wall on an R9700 with 61 GB of RAM
(DS41.md, "Measured"): 0.8 tok/s cold / 1.8 tok/s warm at a 6 GB hot budget,
0.9 / 1.9 at 16 GB, and 9.1 tok/s for the best tuned three-tier case. They
name the cause: *"90 to 96% of the 240 routed experts a token selects are still
cold, and decode moves roughly 2.4 GB of expert weights per token through the
synchronous stream path. That, not the GPU, bounds decode today."*

## Affinity — a DeepSeek-V4-Flash engine, and what a V4.1 port costs

[Affinity](https://codeberg.org/StillDeadcode/affinity) is *"an inference
engine for DeepSeek-V4-Flash on one or two RDNA4 GPUs"*, C++/HIP, no framework.
**It builds clean on this host**: CMake 3.24+, `hipcc` 23, `HIP enabled:
gfx1201`, zero errors, producing `affinity`, `aff-quantize`, `aff-info` and
`aff-gpucheck`. Its README claims **~110 tok/s on two R9700s** against 8 on
one. That is 3x Frank's 35 tok/s goal, for the 284B model.

The 110 works because the 101 GiB container leaves the cold expert set in a
**pinned host pool in RAM** — the SSD tier that Affinity calls "a cliff" is
never reached. V4.1 would need 182 GiB of experts at the same 2.875 bpw, plus
189 GiB of Engram, against the same 246 GiB of RAM.

### What a V4.1 port would reuse

The config parser is **fully data-driven** — every dimension is read from
`config.json`, so nothing is compiled in at the config layer. And the expert
tensor naming is *identical* between the two checkpoints:

```
layers.N.ffn.experts.E.w1.weight   (11,008 in V4  = 43 x 256)
layers.N.ffn.experts.E.w1.weight   (15,360 in V4.1 = 40 x 384)
```

So the quantiser, the `.aff` container, the expert slab, the host pool and the
heat/placement engine — **the part that produces the 110 tok/s** — all carry
over. Hyper-connections are the same family (`hc_mult 4`, Sinkhorn 20).

### What a V4.1 port would have to write

Verified against both checkpoints' real `model.safetensors.index.json` on
2026-09-28 — not inferred from the config prose.

**Tensor presence.** `aff-quantize` requires 32 per-layer tensors. V4.1
supplies 26 plus every global (`embed`, `head`, `norm`, `hc_head_*`) and the
whole expert set. Six are missing:

| Missing in V4.1 | Replaced in V4.1 by |
|---|---|
| `ffn.gate.tid2eid` | the Engram tables |
| `attn.compressor.ape` | nothing — ratio-2 is *pooled, no APE* |
| `attn.indexer.compressor.{wkv,wgate,ape,norm}.weight` | `attn.indexer.wk` + `attn.indexer.k_norm` |

**Config keys.** Of the 35 Affinity reads, V4.1 supplies **34**. The one it
lacks is the dangerous one:

| Gap | Detail |
|---|---|
| **Engram — fails silently** | `num_hash_layers` (V4: 3) is absent, so `cfg_int` returns its default and `model.h:36`'s `n_hash_layer = 3` survives. It would route layers 0–2 through a table that does not exist: fluent output, wrong numbers, no error. This is the only silent failure; the other four gaps fail loudly. |
| **Ratio vocabulary is disjoint** | V4 uses `{0, 4, 128}`; V4.1 uses `{0, 1, 2}` — **no overlap in the non-zero case**. Affinity computes `coff = (ratio == 4) ? 2 : 1`, so V4.1's 18 ratio-2 layers get the ratio-1 width; `comp_ape` is requested 38 times for a tensor that does not exist; and the `ratio == 4` indexer branch fires on **0** of the 8 layers that need one (`index_source_layer_ids` = 2, 8, 14, 20, 24, 28, 32, 36) — those 8 would run dense. |
| **CED** | A 20-layer encoder feeding a 20-layer decoder whose global KV is projected from the final encoder hidden state, through an `aligner`. Affinity's `model.cpp` is a flat layer stack. |
| **DSpark** | V4.1's draft carries **its own** 128-expert MoE at top-3 (`dspark_n_routed_experts`, `dspark_num_experts_per_tok`), 3 stages rather than 1. Affinity's draft shares the target's slab and routing. |
| Vision | V4.1 is multimodal; Affinity has no vision path. Text-only is an acceptable loss. |

V4.1 also adds 31 config keys Affinity never reads, including the whole
`engram_*` block, `kv_source_layer_ids`, `index_source_layer_ids` and the
candidate block pre-selection.

**Verdict.** The reusable half is the *right* half — the expert quantiser,
the `.aff` container, the slab, the host pool and the heat/placement engine
are what produce the 110 tok/s. The rest is 6 tensor bindings, one new ratio
vocabulary, the CED aligner, a new draft geometry and Engram: weeks of work
in a third-party codebase, with no cheap way to prove correctness. And it
meets the same capacity wall regardless, because 182 GiB of experts plus
189 GiB of Engram against 246 GiB of RAM is exactly the problem Lucebox has.
Not advisable.

## The other runtimes

| Runtime | gfx1201 | Architectures | Note |
|---|---|---|---|
| **Affinity** | yes, built here | **V4-Flash only** — "the geometry is compiled in; there is no architecture abstraction" | 110 tok/s for 284B. Best engine for this box by an order of magnitude, wrong model |
| **tcclaviger vLLM fork** | **yes — proven 2026-09-29** | qwen4exp measured 37.4-40.6 tok/s decode (see [[dankserver-qwen38-fp8-expert-offload-tuning]]); **`glm5next` registered** (`Glm5NextForCausalLM`/`MTP`, backend-split `amd/`/`common/`/`nvidia/`, `sparse_indexer.py` for DSA); r4d.so has **native mxfp4 + dsfp4 MoE entries** (`has_dsfp4() == True`); W4A16 MoE class (`R4dW4a16MoEExperts`) wired into the same tiered offload; TunableOp silently skipped under offload (needs `CLAV_TUNABLEOP_WITH_OFFLOAD=1`, and then corrupts numerics — do not use) | the ad hoc qwen-fp8 container's runtime; **the old "MXFP4 = emulation/missing kernels" verdict predates this fork and is dead** — the GLM-5.3-Flash gate is now the payload format + a boot probe, not kernels |
| **Lucebox** | yes, image on host | Qwen 3.8, DS4; **V4.1 on the unmerged `feat/ds41-engine` branch** | The only V4.1 path. **Refuses layer split and layer-major prefill for V4.1** |
| **radiance-vLLM** (drwolfen) | yes — served frank-archon 2026-09-27 → 09-30 | welded to `qwen4_exp`; PLE / QSA / packed-HC live inside that model class | 12–20 person-days per architecture |
| **FreeToken-ROCm** | partial | block-FP8 safetensors only (modelopt); known `CpuMoeExecutor` issue under ROCm | cannot load a GGUF |
| **llama.cpp** | yes | GLM-5.3 confirmed by a community R9700 guide; DS-V4.1 CED unverified | no expert LRU; **8 CPU cores** make CPU expert placement the wrong trade |
| **TokenSpeed** | claimed | GLM-5.x incl. a 5.3-Flash recipe | NVIDIA-first, unverified on gfx1201 |
| **KTransformers** | no | GLM-5.3 tutorial exists | built for Intel AMX + NVIDIA |
| **radiance** (StillDeadcode — the standalone C++/HIP engine, NOT the vLLM fork of the same name) | image `stilldeadcode/radiance` prebuilt for gfx1201 | qwen35 dense/MoE at fp8, qwen4exp quantised-at-convert, llama-shaped | see the probe below: our FP8 payload is REFUSED; tiers, MTP and vision native |

### radiance (standalone) — 2026-10-05 probe

`codeberg.org/StillDeadcode/radiance` — Deadcode's from-scratch engine
(v1.0.13): same author as the `vllm-radiance` fork we ran in September
([[frank-medium-radiance-mxfp4]]), an unrelated codebase — C++/HIP, no
Python, kernels/architectures/quantisers as `.so` plugins, flags written
for two 32 GB R9700s, native Qwen3.8-Flash-Next with `--placement
expert_tiered`, MTP and vision.

Probe with the DEPLOYED payload (`rad-convert --plan-only`, which stops
at declare — no GPU touched, no bytes written):

```
E no architecture plugin claims 'qwen4exp' at quantisation 'fp8'.
  'qwen4exp' is served at: (unquantised).
```

So the 173 G orcarouter FP8 file cannot enter this engine at all: the
qwen4exp plugin takes bf16/f16/f32/GGUF in and every quantisation happens
at `rad-convert` time by recipe. Not a lock-in to their published quant —
recipes are per-weight-class rules (`rtn`/`gptq`/`awq`/`autoround`/
`paroquant`/`ggml`, grids from fp8 128×128 to i4/w4nl/w5nl codebooks,
custom `--quant` rules go ahead of the file's), and libr4d lists routed
experts-as-fp8 among the encodings it reads — so today's quantization is
reproducible **from a bf16 source**, not from our file. Uncensored-family
bf16 source candidates: `windowsxp811203/Qwen3.8-Flash-Next-Abliterated-GGUF`
(ships the bf16 GGUF) and `mradermacher/Qwen3.8-Flash-Next-Uncensored-GGUF`
(i1 quants only). 622 G free fits source + container. A trial is:
download → `rad-convert` (CPU-side) → stop+mask frank-archon window to
serve. Caveats on record: `--tp-wire wht6` is a lossy int8 all-reduce
wire; the recurrent GDN state is f32 by default (144 MiB/sequence) with
an opt-in f16 that changes generated text; every pool is budgeted
explicitly — no profiling pass, no utilisation fraction.

The two-R9700 three-owner split (primary + secondary + stream) is **not
available on this host**: the retired quadlet recorded that
`LUCE_DS4_MOE_TP_INPROC` is qualified for gfx1151+gfx1201 pairs, *not*
gfx1201+gfx1201. hip:0 takes dense weights and the hot expert stack;
`LUCE_EXPERT_STREAM_DEVICE=1` gives hip:1 the streamed-expert slot cache.

## Measured on Dankserver

<!-- MEASUREMENT -->

## See Also

- [[frank-archon-lucebox-ds4]] — the retired Lucebox frank-archon runbook
- [[Dual-R9700-Cooperation]] — the PCIe/atomics facts behind the board swap
- [[Dankserver System Specs]] — the host (RAM corrected 2026-09-28)
- [[frank-archon-r9v-radiance-qwen38-q8]] — the ACTIVE frank-archon