serverAppConfigs/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md

---
type: research
title: Frontier MoE on Dankserver — what fits, on which runtime
description: VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded.
tags:
  - dankserver
  - r9700
  - gfx1201
  - moe
  - deepseek-v4.1
  - deepseek-v4
  - mimo
  - glm
  - affinity
  - lucebox
  - feasibility
timestamp: 2026-09-28
---

# Frontier MoE on Dankserver — what fits, on which runtime

> **Question this page answers.** Given Dankserver's hardware, which of
> DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash runs best, on
> which runtime, and can Affinity / vllm-radiance / FreeToken-AMD be modified
> to run them fast?
>
> **Answer in one line.** V4.1-Flash does not fit this box at any
> configuration; the engine that does run it (Lucebox) measures
> 0.8–9.1 tok/s on R9700 because decode is bound by PCIe expert streaming
> rather than by the GPUs. The 284B **V4-Flash** on **Affinity** is the only
> shape that clears 35 tok/s, at 110.

Everything below was measured or read from source on 2026-09-28. Claims
carried over from the engine authors are marked as theirs.

## Hardware (measured live, 2026-09-28)

| Property | Value |
|---|---|
| GPUs | 2x AMD Radeon AI PRO R9700 (Navi 48, **gfx1201**), 32,624 MiB each = **63.7 GiB** total |
| Interconnect | PCIe 5.0 **x8** each, CPU root ports `00:01.1` / `00:01.3`, ~31.5 GB/s per card |
| PCIe atomics | `CAP_EXP+28.w = 0x1440` (ReqEn+) on both endpoints — RCCL collectives are legal |
| CPU | Ryzen 7 7700X — **8 cores / 16 threads** (small; CPU expert compute is expensive) |
| RAM | **246 GiB** + 63 GiB swap |
| Disk | 1.8 TB NVMe. `/mnt/entropy` is a separate 91 TB ext4 at **100%** — do not plan payload there |

The knowledge base was wrong about RAM: [[Dankserver System Specs]] said
60 GiB until 2026-09-28 and has been corrected. The Lucebox runbook's 246 GiB
was right.

## The three candidates

| Model | Total / active | Routed experts | Attention & KV | Working gfx1201 runtime |
|---|---|---|---|---|
| **DeepSeek-V4.1-Flash** | 552B / 8B prefill, 16B decode | 384 top-6, 40 layers, `moe_intermediate_size` 2304, hidden 5120 | CED 20 enc + 20 dec; CSA2 Full/Reindex/Reuse; **FP4 main KV, 890 B/token**; Engram 196B at layers 1 & 14 | Lucebox `feat/ds41-engine` only (unmerged) |
| **MiMo-V2.6-Flash-MOPD** | 309B / 15B | 256 top-8, no shared expert, 48 layers (39 SWA / 9 GA) | hybrid SWA (window 128) + GA; 5-layer MTP | vLLM / SGLang, NVIDIA-first. No AMD path |
| **GLM-5.3-Flash** | 320B / 18B | MoE | KDA (linear) + DSA (sparse) hybrid; mHC | llama.cpp (community R9700 guide); TokenSpeed recipe |

## Sizing — the arithmetic that decides it

| Quantity | Value | Note |
|---|---:|---|
| DS4.1 official checkpoint | **475.27 GiB** | 48 shards, summed from the HF API |
|   of which Engram | **189.13 GiB** | exactly two 94.56 GiB shards — the layer 1 and 14 lookup tables |
| DS4.1 routed-expert params | 543.6 B | 40 x 384 x 3 x 5120 x 2304 |
|   shipped as | 269 GiB (fp4) | DeepSeek's own `expert_dtype: fp4` |
| Lucebox `ROCMFP23.gguf` | **377.51 GiB** | 179 GiB experts (2.5/3.5 bpw) + 189 GiB Engram + ~9.5 GiB dense |
| V4-Flash official checkpoint | 148.67 GiB | **3.2x smaller than V4.1** |
| Affinity container for V4-Flash | 101.1 GiB | 92.9 GiB experts @ 2.875 bpw + 6.2 dense + 2.0 embed |

The Engram tables are **never mapped** by Lucebox, so they do not evict the
expert page cache — that is the one piece of good news in the 377 GiB file.

### The decode wall

A V4.1 token selects `6 x 40 = 240` routed experts at ~11 MB each. At the
residency the hot stack can hold (37 of 384 per layer, 9.6%), **~2.4 GB of
expert weights cross PCIe per token**. PCIe 5.0 x8 does ~55 GB/s real:

```
2.4 GB / 55 GB/s = 44 ms  ->  ~23 tok/s absolute ceiling
```

That is **below the 35 tok/s target before any attention, Engram or dense work
is counted.** Raising residency is the only lever, and VRAM is what bounds it:
63.7 GiB / 11 MB = 5,800 of 15,360 experts = 37.8% maximum, and that assumes
VRAM holds nothing else.

The engine author measured the same wall on an R9700 with 61 GB of RAM
(DS41.md, "Measured"): 0.8 tok/s cold / 1.8 tok/s warm at a 6 GB hot budget,
0.9 / 1.9 at 16 GB, and 9.1 tok/s for the best tuned three-tier case. They
name the cause: *"90 to 96% of the 240 routed experts a token selects are still
cold, and decode moves roughly 2.4 GB of expert weights per token through the
synchronous stream path. That, not the GPU, bounds decode today."*

## Affinity — a DeepSeek-V4-Flash engine, and what a V4.1 port costs

[Affinity](https://codeberg.org/StillDeadcode/affinity) is *"an inference
engine for DeepSeek-V4-Flash on one or two RDNA4 GPUs"*, C++/HIP, no framework.
**It builds clean on this host**: CMake 3.24+, `hipcc` 23, `HIP enabled:
gfx1201`, zero errors, producing `affinity`, `aff-quantize`, `aff-info` and
`aff-gpucheck`. Its README claims **~110 tok/s on two R9700s** against 8 on
one. That is 3x Frank's 35 tok/s goal, for the 284B model.

The 110 works because the 101 GiB container leaves the cold expert set in a
**pinned host pool in RAM** — the SSD tier that Affinity calls "a cliff" is
never reached. V4.1 would need 182 GiB of experts at the same 2.875 bpw, plus
189 GiB of Engram, against the same 246 GiB of RAM.

### What a V4.1 port would reuse

The config parser is **fully data-driven** — every dimension is read from
`config.json`, so nothing is compiled in at the config layer. And the expert
tensor naming is *identical* between the two checkpoints:

```
layers.N.ffn.experts.E.w1.weight   (11,008 in V4  = 43 x 256)
layers.N.ffn.experts.E.w1.weight   (15,360 in V4.1 = 40 x 384)
```

So the quantiser, the `.aff` container, the expert slab, the host pool and the
heat/placement engine — **the part that produces the 110 tok/s** — all carry
over. Hyper-connections are the same family (`hc_mult 4`, Sinkhorn 20).

### What a V4.1 port would have to write

Verified against both checkpoints' real `model.safetensors.index.json` on
2026-09-28 — not inferred from the config prose.

**Tensor presence.** `aff-quantize` requires 32 per-layer tensors. V4.1
supplies 26 plus every global (`embed`, `head`, `norm`, `hc_head_*`) and the
whole expert set. Six are missing:

| Missing in V4.1 | Replaced in V4.1 by |
|---|---|
| `ffn.gate.tid2eid` | the Engram tables |
| `attn.compressor.ape` | nothing — ratio-2 is *pooled, no APE* |
| `attn.indexer.compressor.{wkv,wgate,ape,norm}.weight` | `attn.indexer.wk` + `attn.indexer.k_norm` |

**Config keys.** Of the 35 Affinity reads, V4.1 supplies **34**. The one it
lacks is the dangerous one:

| Gap | Detail |
|---|---|
| **Engram — fails silently** | `num_hash_layers` (V4: 3) is absent, so `cfg_int` returns its default and `model.h:36`'s `n_hash_layer = 3` survives. It would route layers 0–2 through a table that does not exist: fluent output, wrong numbers, no error. This is the only silent failure; the other four gaps fail loudly. |
| **Ratio vocabulary is disjoint** | V4 uses `{0, 4, 128}`; V4.1 uses `{0, 1, 2}` — **no overlap in the non-zero case**. Affinity computes `coff = (ratio == 4) ? 2 : 1`, so V4.1's 18 ratio-2 layers get the ratio-1 width; `comp_ape` is requested 38 times for a tensor that does not exist; and the `ratio == 4` indexer branch fires on **0** of the 8 layers that need one (`index_source_layer_ids` = 2, 8, 14, 20, 24, 28, 32, 36) — those 8 would run dense. |
| **CED** | A 20-layer encoder feeding a 20-layer decoder whose global KV is projected from the final encoder hidden state, through an `aligner`. Affinity's `model.cpp` is a flat layer stack. |
| **DSpark** | V4.1's draft carries **its own** 128-expert MoE at top-3 (`dspark_n_routed_experts`, `dspark_num_experts_per_tok`), 3 stages rather than 1. Affinity's draft shares the target's slab and routing. |
| Vision | V4.1 is multimodal; Affinity has no vision path. Text-only is an acceptable loss. |

V4.1 also adds 31 config keys Affinity never reads, including the whole
`engram_*` block, `kv_source_layer_ids`, `index_source_layer_ids` and the
candidate block pre-selection.

**Verdict.** The reusable half is the *right* half — the expert quantiser,
the `.aff` container, the slab, the host pool and the heat/placement engine
are what produce the 110 tok/s. The rest is 6 tensor bindings, one new ratio
vocabulary, the CED aligner, a new draft geometry and Engram: weeks of work
in a third-party codebase, with no cheap way to prove correctness. And it
meets the same capacity wall regardless, because 182 GiB of experts plus
189 GiB of Engram against 246 GiB of RAM is exactly the problem Lucebox has.
Not advisable.

## The other runtimes

| Runtime | gfx1201 | Architectures | Note |
|---|---|---|---|
| **Affinity** | yes, built here | **V4-Flash only** — "the geometry is compiled in; there is no architecture abstraction" | 110 tok/s for 284B. Best engine for this box by an order of magnitude, wrong model |
| **Lucebox** | yes, image on host | Qwen 3.8, DS4; **V4.1 on the unmerged `feat/ds41-engine` branch** | The only V4.1 path. **Refuses layer split and layer-major prefill for V4.1** |
| **radiance-vLLM** (drwolfen) | yes — the active frank-primary | welded to `qwen4_exp`; PLE / QSA / packed-HC live inside that model class | 12–20 person-days per architecture |
| **FreeToken-ROCm** | partial | block-FP8 safetensors only (modelopt); known `CpuMoeExecutor` issue under ROCm | cannot load a GGUF |
| **llama.cpp** | yes | GLM-5.3 confirmed by a community R9700 guide; DS-V4.1 CED unverified | no expert LRU; **8 CPU cores** make CPU expert placement the wrong trade |
| **TokenSpeed** | claimed | GLM-5.x incl. a 5.3-Flash recipe | NVIDIA-first, unverified on gfx1201 |
| **KTransformers** | no | GLM-5.3 tutorial exists | built for Intel AMX + NVIDIA |

The two-R9700 three-owner split (primary + secondary + stream) is **not
available on this host**: the retired quadlet recorded that
`LUCE_DS4_MOE_TP_INPROC` is qualified for gfx1151+gfx1201 pairs, *not*
gfx1201+gfx1201. hip:0 takes dense weights and the hot expert stack;
`LUCE_EXPERT_STREAM_DEVICE=1` gives hip:1 the streamed-expert slot cache.

## Measured on Dankserver

<!-- MEASUREMENT -->

## See Also

- [[frank-primary-lucebox-ds4]] — the retired Lucebox frank-primary runbook
- [[Dual-R9700-Cooperation]] — the PCIe/atomics facts behind the board swap
- [[Dankserver System Specs]] — the host (RAM corrected 2026-09-28)
- [[frank-primary-r9v-radiance-qwen38-q8]] — the ACTIVE frank-primary