--- type: configuration title: frank-archon Lucebox DeepSeek V4 Runbook description: RETIRED 2026-09-26 frank-archon runbook (active for hours on 2026-09-26) — the Lucebox runtime (github.com/Luce-Org/lucebox, local image localhost/lucebox-rocm:gfx1201) serving DeepSeek-V4.1-Flash-ROCMFP23 (377.2 GiB GGUF, 43-layer MoE, 256 routed experts, top-6) on Dankserver's R9700 with the routed experts computed on CPU from the mmapped page cache. Replaced the same day by [[frank-archon-llamacpp-qwen38-flash-next-q8]] (llama.cpp Qwen3.8-Flash-Next Q8_0); the image is still in rootless storage but the payload is gone from the host, and frank-archon runs [[frank-archon-radiance-qwen38-iq4r]] today (2026-10-18). tags: - frank-archon - dankserver - lucebox - deepseek-v4 - ds4 - rocmfp23 - gfx1201 - r9700 - quadlet - moe timestamp: 2026-09-26 --- # frank-archon Lucebox DeepSeek V4 Runbook > [!warning] RETIRED 2026-09-26 (same day it went live) > Replaced by [[frank-archon-llamacpp-qwen38-flash-next-q8]] — unsloth > llama.cpp serving Qwen3.8-Flash-Next Q8_0 on both R9700s. This runtime > served for hours; the image `localhost/lucebox-rocm:gfx1201` is still in > dank's rootless storage, but the 378 GiB payload was deleted from the host > (`ls /home/dank/models/` on Dankserver, 2026-10-18: `StillDeadcode`, `rad`, > `tpl.jinja` only), and the quadlet-era repo files were deleted by `8500600` > (2026-09-26) — they survive only in git history, not in a `.AGENTTRASH` > directory (`dankserver/.AGENTTRASH/` holds only `20260916-vllm-radiance`). The dankserver LLM endpoint (:7770, alias **frank-archon**) as it stood on 2026-09-26: the [Lucebox](https://github.com/Luce-Org/lucebox) runtime serving `Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF` (377.2 GiB, 405,349,859,328 bytes, LFS oid `2daea6ca1715bc987aae201e63445d3f7abdc2337c05f5f569f5e50237ce8be2`) on a single Radeon AI PRO R9700 (gfx1201). It replaced [[frank-medium-radiance-mxfp4]] (vllm-radiance TP=2), retired earlier the same day. ## Why this shape DeepSeek-V4.1-Flash is a 43-layer MoE: 256 routed experts + 1 shared expert per layer, top-6 routing. The ROCMFP23 quantization is fp2 gate/up with fp3 down experts (Q2_0_ROCMFP2 type 107 + Q3_0_ROCMFPX type 104). At 377.2 GiB the model fits neither the 63.7 GiB of combined VRAM nor the 246 GiB of RAM, so it is served the way llama.cpp serves big models by default: the GGUF is mmapped, pages pull from disk/page cache on demand, dense work (attention, shared expert, embeddings, output projection) and the MLA KV cache run on the R9700, and the routed experts are computed on CPU. The 246 GiB RAM (upgraded for this deployment) holds the working set of active experts; the rest streams from the 1.8 T NVMe. Lucebox's `--ds4-fused-decode` requests a monolithic GPU load; on a model this size that allocation fails and the backend falls back to **hybrid expert placement + layered decode** — that fallback is the intended serving mode here, not an error. ## Image `localhost/lucebox-rocm:gfx1201`, built on the host from `github.com/Luce-Org/lucebox` `Dockerfile.rocm`: ``` git clone --recurse-submodules https://github.com/Luce-Org/lucebox.git cd lucebox # Three local modifications (tracked at the time as # dankserver/frank-primary/lucebox-gfx1201.patch, deleted by 8500600): # 1. Dockerfile.rocm drops the ROCm *-dev apt packages (the 7.14.1-full # base already ships them in /opt/rocm; the apt repo does not serve # them) and the three runtime math libs. # 2. Dockerfile.rocm installs the official cmake 3.31.6 binary (jammy's # 3.22 predates the `block` command ROCm 7.14 hipblaslt-config.cmake # requires, CMake >= 3.25). # 3. ggml-cuda.cu: `ggml_cuda_node_launches_nothing` moved above the # `#ifdef USE_CUDA_GRAPH` guard — `ggml_cuda_node_is_noop` calls it # unconditionally, but the macro is never defined by the build # system, so the in-guard definition was dead and the use failed. git apply ../lucebox-gfx1201.patch podman build --build-arg LUCE_HIP_ARCHES=gfx1201 \ --build-arg ROCM_VERSION=7.14.1-full \ -t localhost/lucebox-rocm:gfx1201 -f Dockerfile.rocm . ``` - The prebuilt `ghcr.io/luce-org/lucebox-hub:rocm` is **not usable as-is**: it is built for gfx1151 + ROCm 6.4.1, and gfx1200/gfx1201 are not code-object compatible. - `ROCM_VERSION=7.14.1-full` matches the host driver's ROCm 7.x line (the radiance image's core-7.14 userspace runs on this host). A 6.4.x userspace on a 7.x driver can segfault at model load. Only `-full` tags exist for 7.14.x. - The binary lands at `/opt/lucebox-hub/server/build/luce_server`. ## Model `/home/dank/models/DeepSeek-V4.1-Flash/DeepSeek-V4.1-Flash-ROCMFP23.gguf`, fetched with `huggingface_hub` (the xet-native path — plain `curl` full GETs 400 against HF's xethub CDN for this repo; see the fetch note in `scripts/hf-fetch.sh`). Byte count and sha256 are deploy-gated in `deploy_llm_archon`. ## Deployed flags (2026-09-26) From `dankserver/frank-primary/frank-primary-entrypoint.sh` (deleted by `8500600`; recover it with `git show 8500600^:dankserver/frank-primary/frank-primary-entrypoint.sh`) — the unit bypasses the image's entrypoint, which only forwards a fixed Qwen/DFlash flag set and cannot express the DS4 flags): | Setting | Value | |---|---| | Model | `DeepSeek-V4.1-Flash-ROCMFP23.gguf` (377.2 GiB) | | Device | `--target-device hip:0` (`HIP_VISIBLE_DEVICES=0`; the Raphael iGPU at index 2 stays out) | | Window | `--max-ctx 131072` (the qualified Strix Halo window; DS4's MLA KV is compressed, so the pool is small) | | Chunk | `--chunk 8192` (the qualified profile's chunk) | | KV dtype | `--cache-type-k q4_0 --cache-type-v q4_0` (the qualified launch) | | Decode | `--ds4-fused-decode` (falls back to hybrid expert placement on this host) | | Prefill | `--ds4-prefill sparse` (the batched prefill; `exact` runs at token-wise decode cost) | | Routing | `--ds4-expert-top-k 6` (the model default; do NOT reduce on adaptive qtype-105/106 artifacts — measured degenerate output at top-4, DS4.md 2026-08-05) | | Draft | none — no DSpark draft is published for V4.1-Flash; plain autoregressive decode | | Identity | `--model-name frank-archon`, `--host 0.0.0.0 --port 7770` | ## API surface OpenAI Chat Completions (streaming + tool metadata), `/v1/models`, `/health`, OpenAI Responses, Anthropic Messages, and Open WebUI model metadata — the fleet contract is unchanged from frank-medium. ## Cutover procedure 1. Image built and model fetched (gates in `deploy_llm_archon`). 2. `deploy.sh` → `deploy_llm_archon`: stops and **masks** the radiance unit (`frank-medium`), waits for :7770 to free, installs `dankserver/frank-primary/quadlet/frank-primary.container` (also deleted by `8500600`; recover it from `git show 8500600^`), starts `frank-archon.service`, waits for health (1800 s start period — first start page-ins the dense weights and builds the graphs). 3. Verify: `/health` 200 is not enough — a real chat completion, a tool call, and an Open WebUI completion. 4. Retire the radiance unit to the home trash (never `rm`); the MXFP6 checkpoint payload did not stay — it is gone from the host (see Rollback). ## Rollback Unmask and start the radiance unit (`systemctl --user unmask frank-medium && systemctl --user start frank-medium`) and stop/mask frank-archon. The `localhost/radiance-vllm:0.13.0-e060895` image is still in dank's rootless storage; the `/models/Qwen3.8-27B-PARO-MXFP6` checkpoint is not — Dankserver's `/home/dank/models/` holds only `StillDeadcode`, `rad` and `tpl.jinja` as of 2026-10-18, so this rollback needs a re-fetch of the payload. ## Verification (2026-09-26) _Filled in after the cutover: health, real completion, tool call, Open WebUI completion, measured prefill/decode, `systemctl --user is-enabled/is-active` state._ ## Dual-R9700 follow-up (2026-09-28 probe era) After this single-card runbook was retired, the DS41 engine got a dual-R9700 attempt on the `feat/ds41-engine` branch (`main` rejects the checkpoint with `unsupported model architecture 'deepseek41'`). Findings that outlive the probe: **Budget both cards explicitly, or you waste half the box.** The engine sizes the streamed-expert cache LAST, after the model, so uncapped it absorbs every spare byte on the stream device — symptom: card1 at 33.0/34.2 GB looking busy while card0 sits at 14.2/34.2 GB; the busy card is holding a cache of things still streaming from NVMe. Pin all four: | env | card | meaning | |---|---|---| | `LUCE_EXPERT_BUDGET_MB` | hip:0 | primary hot experts (~11.96 MiB each) | | `LUCE_EXPERT_SECONDARY_BUDGET_MB` | hip:1 | secondary experts (needs `LUCE_DS4_MOE_TP*`) | | `LUCE_EXPERT_STREAM_CACHE_MB` | hip:1 | cap the auto-sized stream cache | | `LUCE_EXPERT_STREAM_DEVICE=1` | hip:1 | which card holds the slots and computes them | Working full-card shape: 21000 + 20000 + 8000 → card0 ~28.6/31.86 GiB, card1 ~27.3/31.86 GiB, resident experts 3,360 of 15,360. **Expert TP DOES work on gfx1201+gfx1201.** A retired quadlet recorded `LUCE_DS4_MOE_TP_INPROC` as "qualified for gfx1151+gfx1201, not gfx1201+gfx1201" — that is WRONG; peer access negotiates and the secondary tier comes up clean: `[deepseek4-moe-tp] enabled mode=in-process local=hip:0 secondary=hip:1`. It is the only lever that materially raises resident expert count, and resident count is the entire bottleneck. Confirm every run with `expert ownership:` and `hybrid experts ready:` lines. **The placement file must live under the mounted models dir.** JSON `{"owners": [...]}`, 15,360 entries indexed `layer*384+expert`, values `primary`/`secondary`/`stream`. It is a ranking, not a promise: each device keeps what fits its budget and demotes the rest. Anywhere outside `/models/...` gives `cannot open expert placement` and a SILENT single-tier fallback — a silent-wrongness failure, not a crash. **V4.1 refuses layer split and layer-major prefill** — only `--ds4-prefill exact` (token-by-token) is legal on a 32 GiB card and `--ds4-fused-decode` is refused there. With ~22% of experts resident each token pulls ~187 cold experts × ~12 MiB; if `/proc//io` `read_bytes` climbs into terabytes the model is larger than the box's RAM and you are in page-cache thrash at the NVMe ceiling — a capacity verdict, not a tuning problem. Low GPU util with a pegged link is the EXPECTED signature of PCIe-expert-stream-bound decode, not a broken run. **Harness traps, each of which produced a wrong number:** never `env -i` the podman call (rootless podman runs as `dank`; `env -i HOME=/root` makes PODMAN try `/root/.config` and die `stat /root/.config: permission denied` before the container exists — pass container env with `-e`); never put a large prompt on curl's argv (`Argument list too long` at ~128 KB — use `--data-binary @file`); size prompts by growing the file until `wc -c` reaches the byte target AND assert depth against the engine's own `usage.prompt_tokens` (nominal 4 chars/token was really 5.73; a chunk loop that appends a fixed ~152-byte line per nominal 200-byte chunk undershot 30%); the server runs as the container's PID 1, so tear down with `podman rm -f`, never host `pkill`; drop page cache between arms (`sync; echo 3 > /proc/sys/vm/drop_caches`) or the first arm reads warm and every later one is a lie. Read the placement log — it is the whole diagnosis (`arch=deepseek41 layers=40 embd=5120 experts=384 used=6 ff=2304 hash_layers=0` — hash_layers=0 is correct for V4.1; `engram: layers=[1,14] … 2 table(s) embedded (not mapped)` — the 189.13 GiB Engram tables are never mapped, so they do not evict the expert page cache).