frank-primary: Lucebox DeepSeek-V4.1-Flash-ROCMFP23 quadlet + deploy handler
Replaces frank-medium (vllm-radiance TP=2) on :7770. The 377.2 GiB 43-layer MoE (256 routed experts, top-6) is mmapped: dense work and the MLA KV cache on the R9700, routed experts computed on CPU from the page cache (246 GiB RAM). The unit bypasses the image entrypoint (Qwen/DFlash-only flag set) and execs luce_server with the DS4 launch shape; --ds4-fused-decode's monolithic-load fallback IS the serving mode here. deploy_llm_primary gates on image + model byte count, stops/masks the radiance unit, waits for :7770, installs and starts. Runbook: frank-primary-lucebox-ds4.md.
This commit is contained in:
1 parent
441c4f1c99
commit
acd14ab7df
8 files changed
+352
-1
No files matched your search
@@ -124,6 +124,11 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
|
||||
- [[deployment]] — unified deploy and update process for both servers
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[frank-primary-lucebox-ds4]] — the ACTIVE frank-primary runbook:
|
||||
the Lucebox runtime serving DeepSeek-V4.1-Flash-ROCMFP23 (377.2 GiB
|
||||
GGUF, 43-layer MoE, 256 routed experts top-6) on Dankserver's R9700
|
||||
with the routed experts computed on CPU from the mmapped page cache —
|
||||
replaces frank-medium on :7770 (cutover 2026-09-26)
|
||||
- [[frank-medium-radiance-mxfp4]] — the ACTIVE frank-medium runbook:
|
||||
vllm-radiance as ONE tensor-parallel process (TP=2 over the R4D one-shot
|
||||
allreduce) serving just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq (MXFP4
|
||||
|
||||
Reference in new issue
Block a user