serverAppConfigs/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
research	Frontier MoE on Dankserver — what fits, on which runtime	VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded.	dankserver r9700 gfx1201 moe deepseek-v4.1 deepseek-v4 mimo glm affinity lucebox feasibility	2026-09-28

Frontier MoE on Dankserver — what fits, on which runtime

Question this page answers. Given Dankserver's hardware, which of DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash runs best, on which runtime, and can Affinity / vllm-radiance / FreeToken-AMD be modified to run them fast?

Answer in one line. V4.1-Flash does not fit this box at any configuration; the engine that does run it (Lucebox) measures 0.8–9.1 tok/s on R9700 because decode is bound by PCIe expert streaming rather than by the GPUs. The 284B V4-Flash on Affinity is the only shape that clears 35 tok/s, at 110.

Everything below was measured or read from source on 2026-09-28. Claims carried over from the engine authors are marked as theirs.

Hardware (measured live, 2026-09-28)

Property	Value
GPUs	2x AMD Radeon AI PRO R9700 (Navi 48, gfx1201), 32,624 MiB each = 63.7 GiB total
Interconnect	PCIe 5.0 x8 each, CPU root ports 00:01.1 / 00:01.3, ~31.5 GB/s per card
PCIe atomics	CAP_EXP+28.w = 0x1440 (ReqEn+) on both endpoints — RCCL collectives are legal
CPU	Ryzen 7 7700X — 8 cores / 16 threads (small; CPU expert compute is expensive)
RAM	246 GiB + 63 GiB swap
Disk	1.8 TB NVMe. /mnt/entropy is a separate 91 TB ext4 at 100% — do not plan payload there

The knowledge base was wrong about RAM: Dankserver System Specs said 60 GiB until 2026-09-28 and has been corrected. The Lucebox runbook's 246 GiB was right.

The three candidates

Model	Total / active	Routed experts	Attention & KV	Working gfx1201 runtime
DeepSeek-V4.1-Flash	552B / 8B prefill, 16B decode	384 top-6, 40 layers, moe_intermediate_size 2304, hidden 5120	CED 20 enc + 20 dec; CSA2 Full/Reindex/Reuse; FP4 main KV, 890 B/token; Engram 196B at layers 1 & 14	Lucebox feat/ds41-engine only (unmerged)
MiMo-V2.6-Flash-MOPD	309B / 15B	256 top-8, no shared expert, 48 layers (39 SWA / 9 GA)	hybrid SWA (window 128) + GA; 5-layer MTP	vLLM / SGLang, NVIDIA-first. No AMD path
GLM-5.3-Flash	320B / 18B	MoE	KDA (linear) + DSA (sparse) hybrid; mHC	tcclaviger vLLM fork — plausible native path (2026-09-29 source reading: glm5next registered with an amd/ backend dir + DSA indexer + mHC path; r4d.so carries native mxfp4/dsfp4 MoE entries). Gate: a compressed-tensors MXFP4 payload (~165 GiB, none exists publicly) + a HIP boot probe. llama.cpp GGUF (community R9700 guide) remains the verified-but-slower path

Sizing — the arithmetic that decides it

Quantity	Value	Note
DS4.1 official checkpoint	475.27 GiB	48 shards, summed from the HF API
of which Engram	189.13 GiB	exactly two 94.56 GiB shards — the layer 1 and 14 lookup tables
DS4.1 routed-expert params	543.6 B	40 x 384 x 3 x 5120 x 2304
shipped as	269 GiB (fp4)	DeepSeek's own expert_dtype: fp4
Lucebox ROCMFP23.gguf	377.51 GiB	179 GiB experts (2.5/3.5 bpw) + 189 GiB Engram + ~9.5 GiB dense
V4-Flash official checkpoint	148.67 GiB	3.2x smaller than V4.1
Affinity container for V4-Flash	101.1 GiB	92.9 GiB experts @ 2.875 bpw + 6.2 dense + 2.0 embed

The Engram tables are never mapped by Lucebox, so they do not evict the expert page cache — that is the one piece of good news in the 377 GiB file.

The decode wall

A V4.1 token selects 6 x 40 = 240 routed experts at ~11 MB each. At the residency the hot stack can hold (37 of 384 per layer, 9.6%), ~2.4 GB of expert weights cross PCIe per token. PCIe 5.0 x8 does ~55 GB/s real:

2.4 GB / 55 GB/s = 44 ms  ->  ~23 tok/s absolute ceiling

That is below the 35 tok/s target before any attention, Engram or dense work is counted. Raising residency is the only lever, and VRAM is what bounds it: 63.7 GiB / 11 MB = 5,800 of 15,360 experts = 37.8% maximum, and that assumes VRAM holds nothing else.

The engine author measured the same wall on an R9700 with 61 GB of RAM (DS41.md, "Measured"): 0.8 tok/s cold / 1.8 tok/s warm at a 6 GB hot budget, 0.9 / 1.9 at 16 GB, and 9.1 tok/s for the best tuned three-tier case. They name the cause: "90 to 96% of the 240 routed experts a token selects are still cold, and decode moves roughly 2.4 GB of expert weights per token through the synchronous stream path. That, not the GPU, bounds decode today."

Affinity — a DeepSeek-V4-Flash engine, and what a V4.1 port costs

Affinity is "an inference engine for DeepSeek-V4-Flash on one or two RDNA4 GPUs", C++/HIP, no framework. It builds clean on this host: CMake 3.24+, hipcc 23, HIP enabled: gfx1201, zero errors, producing affinity, aff-quantize, aff-info and aff-gpucheck. Its README claims ~110 tok/s on two R9700s against 8 on one. That is 3x Frank's 35 tok/s goal, for the 284B model.

The 110 works because the 101 GiB container leaves the cold expert set in a pinned host pool in RAM — the SSD tier that Affinity calls "a cliff" is never reached. V4.1 would need 182 GiB of experts at the same 2.875 bpw, plus 189 GiB of Engram, against the same 246 GiB of RAM.

What a V4.1 port would reuse

The config parser is fully data-driven — every dimension is read from config.json, so nothing is compiled in at the config layer. And the expert tensor naming is identical between the two checkpoints:

layers.N.ffn.experts.E.w1.weight   (11,008 in V4  = 43 x 256)
layers.N.ffn.experts.E.w1.weight   (15,360 in V4.1 = 40 x 384)

So the quantiser, the .aff container, the expert slab, the host pool and the heat/placement engine — the part that produces the 110 tok/s — all carry over. Hyper-connections are the same family (hc_mult 4, Sinkhorn 20).

What a V4.1 port would have to write

Verified against both checkpoints' real model.safetensors.index.json on 2026-09-28 — not inferred from the config prose.

Tensor presence. aff-quantize requires 32 per-layer tensors. V4.1 supplies 26 plus every global (embed, head, norm, hc_head_*) and the whole expert set. Six are missing:

Missing in V4.1	Replaced in V4.1 by
ffn.gate.tid2eid	the Engram tables
attn.compressor.ape	nothing — ratio-2 is pooled, no APE
attn.indexer.compressor.{wkv,wgate,ape,norm}.weight	attn.indexer.wk + attn.indexer.k_norm

Config keys. Of the 35 Affinity reads, V4.1 supplies 34. The one it lacks is the dangerous one:

Gap	Detail
Engram — fails silently	num_hash_layers (V4: 3) is absent, so cfg_int returns its default and model.h:36's n_hash_layer = 3 survives. It would route layers 0–2 through a table that does not exist: fluent output, wrong numbers, no error. This is the only silent failure; the other four gaps fail loudly.
Ratio vocabulary is disjoint	V4 uses {0, 4, 128}; V4.1 uses {0, 1, 2} — no overlap in the non-zero case. Affinity computes coff = (ratio == 4) ? 2 : 1, so V4.1's 18 ratio-2 layers get the ratio-1 width; comp_ape is requested 38 times for a tensor that does not exist; and the ratio == 4 indexer branch fires on 0 of the 8 layers that need one (index_source_layer_ids = 2, 8, 14, 20, 24, 28, 32, 36) — those 8 would run dense.
CED	A 20-layer encoder feeding a 20-layer decoder whose global KV is projected from the final encoder hidden state, through an aligner. Affinity's model.cpp is a flat layer stack.
DSpark	V4.1's draft carries its own 128-expert MoE at top-3 (dspark_n_routed_experts, dspark_num_experts_per_tok), 3 stages rather than 1. Affinity's draft shares the target's slab and routing.
Vision	V4.1 is multimodal; Affinity has no vision path. Text-only is an acceptable loss.

V4.1 also adds 31 config keys Affinity never reads, including the whole engram_* block, kv_source_layer_ids, index_source_layer_ids and the candidate block pre-selection.

Verdict. The reusable half is the right half — the expert quantiser, the .aff container, the slab, the host pool and the heat/placement engine are what produce the 110 tok/s. The rest is 6 tensor bindings, one new ratio vocabulary, the CED aligner, a new draft geometry and Engram: weeks of work in a third-party codebase, with no cheap way to prove correctness. And it meets the same capacity wall regardless, because 182 GiB of experts plus 189 GiB of Engram against 246 GiB of RAM is exactly the problem Lucebox has. Not advisable.

The other runtimes

Runtime	gfx1201	Architectures	Note
Affinity	yes, built here	V4-Flash only — "the geometry is compiled in; there is no architecture abstraction"	110 tok/s for 284B. Best engine for this box by an order of magnitude, wrong model
tcclaviger vLLM fork	yes — proven 2026-09-29	qwen4exp measured 37.4-40.6 tok/s decode (see dankserver-qwen38-fp8-expert-offload-tuning); glm5next registered (Glm5NextForCausalLM/MTP, backend-split amd//common//nvidia/, sparse_indexer.py for DSA); r4d.so has native mxfp4 + dsfp4 MoE entries (has_dsfp4() == True); W4A16 MoE class (R4dW4a16MoEExperts) wired into the same tiered offload; TunableOp silently skipped under offload (needs CLAV_TUNABLEOP_WITH_OFFLOAD=1, and then corrupts numerics — do not use)	the ad hoc qwen-fp8 container's runtime; the old "MXFP4 = emulation/missing kernels" verdict predates this fork and is dead — the GLM-5.3-Flash gate is now the payload format + a boot probe, not kernels
Lucebox	yes, image on host	Qwen 3.8, DS4; V4.1 on the unmerged feat/ds41-engine branch	The only V4.1 path. Refuses layer split and layer-major prefill for V4.1
radiance-vLLM (drwolfen)	yes — the active frank-primary	welded to qwen4_exp; PLE / QSA / packed-HC live inside that model class	12–20 person-days per architecture
FreeToken-ROCm	partial	block-FP8 safetensors only (modelopt); known CpuMoeExecutor issue under ROCm	cannot load a GGUF
llama.cpp	yes	GLM-5.3 confirmed by a community R9700 guide; DS-V4.1 CED unverified	no expert LRU; 8 CPU cores make CPU expert placement the wrong trade
TokenSpeed	claimed	GLM-5.x incl. a 5.3-Flash recipe	NVIDIA-first, unverified on gfx1201
KTransformers	no	GLM-5.3 tutorial exists	built for Intel AMX + NVIDIA

The two-R9700 three-owner split (primary + secondary + stream) is not available on this host: the retired quadlet recorded that LUCE_DS4_MOE_TP_INPROC is qualified for gfx1151+gfx1201 pairs, not gfx1201+gfx1201. hip:0 takes dense weights and the hot expert stack; LUCE_EXPERT_STREAM_DEVICE=1 gives hip:1 the streamed-expert slot cache.

Measured on Dankserver

See Also

• frank-primary-lucebox-ds4 — the retired Lucebox frank-primary runbook
• Dual-R9700-Cooperation — the PCIe/atomics facts behind the board swap
• Dankserver System Specs — the host (RAM corrected 2026-09-28)
• frank-primary-r9v-radiance-qwen38-q8 — the ACTIVE frank-primary