dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: frontier-MoE feasibility on Dankserver (V4.1 / MiMo / GLM-5.3)

Browse Source 
Answers Frank's question: given the 2x R9700 box, which of DeepSeek-V4.1-Flash,
MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash runs best, on what runtime, and can
Affinity / vllm-radiance / FreeToken-AMD be modified to run them fast.

Sizing, measured from the HF API rather than the model cards:
- DS4.1 payload 475.27 GiB, of which 189.13 GiB is Engram (two 94.56 GiB
  shards, the layer 1 and 14 lookup tables). Lucebox never maps them, so
  they do not evict the expert page cache.
- 240 routed experts x ~11 MB per token at the 37-of-384-per-layer residency
  the hot stack can hold = ~2.4 GB across PCIe per token. At 55 GB/s real on
  Gen5 x8 that is a ~23 tok/s ceiling, under the 35 tok/s goal before any
  attention or dense work.
- The backend refuses layer-major prefill for V4.1, so the only legal
  prefill is token-by-token; 1000 t/s is ~20x away.

Affinity (Codeberg StillDeadcode) builds clean here (hipcc 23, gfx1201, zero
errors) and claims 110 tok/s for the 284B V4-Flash, because the 101 GiB
container leaves the cold expert set in a pinned host pool in RAM. V4.1 is
2.5x too big to repeat that. Port scope, from both checkpoints' real
safetensors indices: expert machinery ports unchanged (identical tensor
naming, 15,360 tensors at 40x384), 26/32 per-layer tensors and 34/35 config
keys present. Five gaps - Engram, the disjoint {0,1,2} vs {0,4,128} ratio
vocabulary, CED, the DSpark draft geometry, vision. num_hash_layers is the
only key V4.1 lacks, so model.h:36's default of 3 silently survives and
would route layers 0-2 through a table that does not exist.

Also corrects: Dankserver RAM 60 -> 246 GiB (stale since the 2026-09-20 board
swap) and storage; the top-level index credited V4.1 with V4's geometry
(43 layers / 256 experts vs the actual 40 / 384).
This commit is contained in:
Frank
2026-09-28 14:58:59 -07:00
parent 564a781539
commit d29d104f01
5 changed files with 251 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/LLM/Dankserver System Specs.md
+4 -4
View File
@@ -63,16 +63,16 @@ apt lists remain on disk but no NVIDIA PCI device exists.
| Property | Value |
|----------|-------|
| System RAM | 60 GiB |
| Swap | 63 GiB |
| System RAM | **246 GiB** (upgraded for the Lucebox DeepSeek deployment of 2026-09-26; this page said 60 GiB until 2026-09-28 — a stale figure that survived the board swap) |
| Swap | 63 GiB (`/swapfile`) |
| /dev/shm | 32 GiB |
### Storage
| Filesystem | Size | Used | Avail | Use% |
|------------|------|------|-------|------|
| / (nvme0n1p2) | 915G | ~640G | ~280G | ~70% |
| /mnt/entropy (dm-0) | 91T | 86T | 910G | 99% |
| / (nvme0n1p2) | 1.8T | 927G | ~822G | ~54% |
| /mnt/entropy (dm-0) | 91T | 87T | **244G** | **100%** — full; do not plan model payload here |
Models live under `/home/dank/models`; the serving build tree under
`/home/dank/GitPackages/llama.cpp-dflash2` (the older `charlie-ROCmFPX` tree is