frank-primary: V4.1 launch shape (feat/ds41-engine source)

The deepseek41 architecture exists only on Luce-Org/lucebox's
feat/ds41-engine branch; main rejects the checkpoint with
"unsupported model architecture 'deepseek41'". Re-aim the image build at
that branch and fix the launch shape, which had been copied from the V4
DS4.md recipe:

- drop --ds4-fused-decode and --ds4-fused-verify-f16-kv, and use
  --ds4-prefill dense: V4.1's validate_model_features refuses the V4-only
  prefill/fused paths (layer split and --mmproj too).
- --chunk 4096 (the V4.1 dense-prefill measurement point), --max-ctx
  131072 (the V4.1 default window), --prefix/--prefill-cache-slots 0 as
  the DS41.md bring-up recipe.
- drop --cache-type-k/v: the DeepSeek4 backend ignores them; the MLA
  compressed caches are F16 by design.
- expose BOTH R9700s and set LUCE_EXPERT_STREAM_DEVICE=1, so hip:0 owns
  the dense weights plus the hot expert stack while hip:1 holds the
  streamed-expert slot cache instead of competing with it.
- quadlet: 384 routed experts per layer, Engram tables, 40 layers.
This commit is contained in:
Frank committed 2026-09-26 17:00:46 -07:00
1 parent fc472e89d4
commit 846140c2be
2 files changed
+61 -37

No files matched your search

@@ -7,13 +7,14 @@
# runtime (github.com/Luce-Org/lucebox, local image localhost/lucebox-rocm:
# gfx1201 built from Dockerfile.rocm with LUCE_HIP_ARCHES=gfx1201 and
# ROCM_VERSION=7.14.1-full to match the host driver's ROCm 7.x line)
# serving DeepSeek-V4.1-Flash-ROCMFP23 (377.2 GiB GGUF, 43-layer MoE,
# 256 routed experts + 1 shared per layer, top-6 routing) on a single
# Radeon AI PRO R9700 (gfx1201). The model does not fit VRAM: dense work
# and the MLA KV cache sit on the R9700, the routed experts are computed
# on CPU from the mmapped page cache (246 GiB RAM holds the working set;
# the rest streams from NVMe). Lazy mmap page-in is the engine default —
# no flag.
# serving DeepSeek-V4.1-Flash-ROCMFP23 (377.2 GiB GGUF, 40-layer MoE,
# 384 routed experts + 1 shared per layer, top-6 routing, Engram tables at
# layers 1 and 14) on the two Radeon AI PRO R9700s (gfx1201). The model
# cannot be resident: dense weights, the compressed MLA caches and a hot
# expert stack sit on hip:0, the streamed-expert slot cache on hip:1, and
# every other routed expert is pulled out of the mmapped page cache (246 GiB
# RAM holds most of the 179 GiB expert set; the rest reads from NVMe). Lazy
# mmap page-in is the engine default — no flag.
#
# The image's entrypoint only forwards a fixed Qwen/DFlash flag set and
# cannot express the DS4 flags, so the unit bypasses it (--entrypoint) and
@@ -25,11 +26,14 @@
# Open WebUI metadata) — the fleet contract is unchanged from
# frank-medium.
#
# DEVICES. /dev/kfd + /dev/dri through the amdgpu/KFD stack. HIP_VISIBLE_
# DEVICES=0 pins the first R9700 (the Raphael iGPU enumerates as index 2
# and stays out). The second R9700 is not used: the in-process
# heterogeneous expert-parallel path (LUCE_DS4_MOE_TP_INPROC) is qualified
# for gfx1151+gfx1201 pairs, not gfx1201+gfx1201.
# DEVICES. /dev/kfd + /dev/dri through the amdgpu/KFD stack. Both R9700s are
# visible (indices 0 and 1; the Raphael iGPU enumerates as index 2 and stays
# out). They are NOT driven as a symmetric pair: the in-process expert-
# parallel path (LUCE_DS4_MOE_TP_INPROC) is qualified for gfx1151+gfx1201
# pairs, not gfx1201+gfx1201. Instead hip:0 owns the dense weights and the
# hot expert stack while LUCE_EXPERT_STREAM_DEVICE=1 (entrypoint) gives hip:1
# the streamed-expert slot cache, so each card carries one tier and neither
# waits on the other.
#
# MOUNTS. /models (ro — the GGUF is mmapped, nothing writes), and the
# entrypoint + healthcheck scripts straight out of this checkout
@@ -58,9 +62,10 @@ AddDevice=/dev/kfd
AddDevice=/dev/dri
GroupAdd=keep-groups
PodmanArgs=--privileged --ipc=host --security-opt seccomp=unconfined --cap-add SYS_PTRACE --network=host --entrypoint /usr/local/bin/frank-primary-entrypoint.sh
# First R9700 only (index pin excludes the Raphael iGPU at index 2).
Environment=HIP_VISIBLE_DEVICES=0
Environment=ROCR_VISIBLE_DEVICES=0
# Both R9700s: hip:0 target, hip:1 the streamed-expert slot cache. Index 2,
# the Raphael iGPU, stays out.
Environment=HIP_VISIBLE_DEVICES=0,1
Environment=ROCR_VISIBLE_DEVICES=0,1
# The 377 GiB GGUF, mmapped; pages pull from disk/page cache on demand.
Volume=/home/dank/models:/models:ro
# Entrypoint + healthcheck from this checkout, mounted read-only.