The runtime's auto budget is gpu_free - kv - warm - safety (22.35 GiB), which
leaves only the 0.50 GiB safety margin free while the fit check demands the
1.50 GiB discrete headroom, so auto-sizing refuses on any 32 GiB target:
"device 0 memory: 30.97 of 31.86 GiB used, 0.89 GiB free (headroom 1.50
GiB): does not fit". 18000 MB is the value DS41.md measured its R9700 rows
at: ~1500 experts resident, ~5 GiB free on device 0.
--ds4-prefill dense made requires_monolithic_model() true, and a monolithic
load asks ggml for the entire 187 GiB expert set as one device-0 buffer:
"allocating 191932.25 MiB on device 0: cudaMalloc failed: out of memory",
followed by "monolithic HIP load required for dense prefill". Approximate
prefill (dense, sparse) is expressible only where the experts are resident
(in-process owners). exact feeds the prompt token by token, which is the
path the hybrid tier streams experts for. Chunk follows DS41.md's
cold-expert recipe (1024).
The deepseek41 architecture exists only on Luce-Org/lucebox's
feat/ds41-engine branch; main rejects the checkpoint with
"unsupported model architecture 'deepseek41'". Re-aim the image build at
that branch and fix the launch shape, which had been copied from the V4
DS4.md recipe:
- drop --ds4-fused-decode and --ds4-fused-verify-f16-kv, and use
--ds4-prefill dense: V4.1's validate_model_features refuses the V4-only
prefill/fused paths (layer split and --mmproj too).
- --chunk 4096 (the V4.1 dense-prefill measurement point), --max-ctx
131072 (the V4.1 default window), --prefix/--prefill-cache-slots 0 as
the DS41.md bring-up recipe.
- drop --cache-type-k/v: the DeepSeek4 backend ignores them; the MLA
compressed caches are F16 by design.
- expose BOTH R9700s and set LUCE_EXPERT_STREAM_DEVICE=1, so hip:0 owns
the dense weights plus the hot expert stack while hip:1 holds the
streamed-expert slot cache instead of competing with it.
- quadlet: 384 routed experts per layer, Engram tables, 40 layers.
Replaces frank-medium (vllm-radiance TP=2) on :7770. The 377.2 GiB
43-layer MoE (256 routed experts, top-6) is mmapped: dense work and the
MLA KV cache on the R9700, routed experts computed on CPU from the
page cache (246 GiB RAM). The unit bypasses the image entrypoint
(Qwen/DFlash-only flag set) and execs luce_server with the DS4 launch
shape; --ds4-fused-decode's monolithic-load fallback IS the serving
mode here. deploy_llm_primary gates on image + model byte count,
stops/masks the radiance unit, waits for :7770, installs and starts.
Runbook: frank-primary-lucebox-ds4.md.
Frank