frank-primary: V4.1 launch shape (feat/ds41-engine source)

The deepseek41 architecture exists only on Luce-Org/lucebox's
feat/ds41-engine branch; main rejects the checkpoint with
"unsupported model architecture 'deepseek41'". Re-aim the image build at
that branch and fix the launch shape, which had been copied from the V4
DS4.md recipe:

- drop --ds4-fused-decode and --ds4-fused-verify-f16-kv, and use
  --ds4-prefill dense: V4.1's validate_model_features refuses the V4-only
  prefill/fused paths (layer split and --mmproj too).
- --chunk 4096 (the V4.1 dense-prefill measurement point), --max-ctx
  131072 (the V4.1 default window), --prefix/--prefill-cache-slots 0 as
  the DS41.md bring-up recipe.
- drop --cache-type-k/v: the DeepSeek4 backend ignores them; the MLA
  compressed caches are F16 by design.
- expose BOTH R9700s and set LUCE_EXPERT_STREAM_DEVICE=1, so hip:0 owns
  the dense weights plus the hot expert stack while hip:1 holds the
  streamed-expert slot cache instead of competing with it.
- quadlet: 384 routed experts per layer, Engram tables, 40 layers.
This commit is contained in:
Frank committed 2026-09-26 17:00:46 -07:00
1 parent fc472e89d4
commit 846140c2be
2 files changed
+61 -37

No files matched your search

@@ -8,22 +8,37 @@
# C++ server reads LUCE_* environment variables straight from the process
# environment, so no env plumbing is needed here.
#
# LAUNCH SHAPE (server/docs/DS4.md, server/docs/RECOMMENDED_SETUPS.md):
# --ds4-fused-decode requests a monolithic GPU load; on a 377 GiB model
# that allocation fails and the backend falls back
# to hybrid expert placement + layered decode (dense
# work on the R9700, routed experts computed on CPU
# from the mmapped page cache). That fallback IS the
# intended serving mode on this host.
# --ds4-prefill sparse the batched prefill; use for prompts beyond a few
# hundred tokens (exact is token-wise decode cost).
# --ds4-expert-top-k 6 the model default. Do NOT reduce on adaptive
# (qtype-105/106) artifacts: measured degenerate
# output at top-4 (DS4.md, 2026-08-05).
# --chunk 8192 the qualified Strix Halo profile's chunk.
# --max-ctx 131072 the qualified Strix Halo window; DS4's MLA KV is
# compressed, so the pool is small.
# --cache-type-k/v q4_0 the qualified launch's KV dtype.
# SOURCE BRANCH. The `deepseek41` architecture exists only on the
# `feat/ds41-engine` branch of Luce-Org/lucebox: main carries `deepseek4`
# alone and rejects this checkpoint with "unsupported model architecture
# 'deepseek41'" (server/src/common/backend_plan.cpp). The image is built from
# that branch — see serverAppConfigs-docs/LLM/frank-primary-lucebox-ds41.md.
#
# LAUNCH SHAPE (server/docs/DS41.md — the V4.1 page, not the V4 DS4.md one):
# --ds4-prefill dense the V4.1 batched prefill. The default `exact` walks
# the prompt token by token, and the V4-only modes are
# REFUSED for V4.1: `sparse`, --ds4-fused-decode,
# --ds4-fused-verify-f16-kv, layer split and --mmproj
# all fail validate_model_features at load.
# --chunk 4096 the chunk the V4.1 dense-prefill numbers were taken
# at; the runtime caps it to what the device holds.
# --max-ctx 131072 the V4.1 default window. The MLA compressed caches
# are F16 by design and stay small; --cache-type-k/v
# are ignored by the DeepSeek4 backend, so they are
# not passed.
# --prefix-cache-slots 0 --prefill-cache-slots 0
# the DS41.md bring-up recipe (the server default is
# 32 prefix slots). Lift them once capacity under load
# has been measured, not before.
#
# EXPERT PLACEMENT. 377 GiB of weights against 64 GiB of VRAM: only the dense
# weights, the compressed caches and a hot expert stack fit on hip:0, and the
# remaining routed experts are streamed out of the mmapped page cache/NVMe by
# the hybrid expert tier. LUCE_EXPERT_STREAM_DEVICE=1 moves the streamed-
# expert slot cache onto the second R9700 — it defaults to the target device,
# where it would compete with the hot stack. Both the owner stack and the
# slot cache then auto-size to their device's free memory minus the 1.5 GiB
# discrete headroom.
#
# No DSpark draft is published for DeepSeek-V4.1-Flash, so this is plain
# autoregressive decode (no --draft, no LUCE_DS4_SPEC).
@@ -32,6 +47,12 @@ set -euo pipefail
MODEL="${FRANK_PRIMARY_MODEL:-/models/DeepSeek-V4.1-Flash/DeepSeek-V4.1-Flash-ROCMFP23.gguf}"
BIN="${FRANK_PRIMARY_BIN:-/opt/lucebox-hub/server/build/luce_server}"
# The second R9700 holds the streamed-expert slot cache and computes them.
export LUCE_EXPERT_STREAM_DEVICE="${LUCE_EXPERT_STREAM_DEVICE:-1}"
# rocBLAS/BLAS dispatch and queue depth, as the fleet's other ROCm servers.
export ROCBLAS_USE_HIPBLASLT="${ROCBLAS_USE_HIPBLASLT:-0}"
export GPU_MAX_HW_QUEUES="${GPU_MAX_HW_QUEUES:-8}"
if [[ ! -f "$MODEL" ]]; then
echo "frank-primary: model missing at $MODEL" >&2
exit 1
@@ -47,9 +68,7 @@ exec "$BIN" "$MODEL" \
--model-name frank-primary \
--target-device hip:0 \
--max-ctx 131072 \
--chunk 8192 \
--cache-type-k q4_0 \
--cache-type-v q4_0 \
--ds4-fused-decode \
--ds4-prefill sparse \
--ds4-expert-top-k 6
--chunk 4096 \
--ds4-prefill dense \
--prefix-cache-slots 0 \
--prefill-cache-slots 0