frank-primary: swap Dankserver to llama.cpp Qwen3.8-Flash-Next Q8_0
The 2026-09-26 same-day cutover that ended the Lucebox DS4.1 detour:
Dankserver's :7770 endpoint is now unsloth llama.cpp (the b11030-mix
prebuilt, ROCm gfx120X, bundled runtime) serving
unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 — 175.30 GiB across six sha256-
verified split parts — on both Radeon AI PRO R9700s, as a ROOT unit (the
frank-large shape: no container, no generated unit).
The deployed contract:
- Placement law: layer split -ts 1,1 with -ngl 999; the routed experts
and the 51B-param n-gram (PLE) table are COMPUTED in host RAM via
-ot exps=CPU,ngram_embeddings=CPU (the x4-pair measurement: streaming
over PCIe loses at 3.7 t/s, computing in RAM does not care).
per_layer_token_embd.weight=CPU is required by the shared MTP head's
borrow path (a GPU-resident borrow crashes the draft load).
mmap stays on; --cache-ram 16384.
- Spec chain draft-mtp,ngram-mod: the shared Q8_0 MTP head
(mtp-...-shared-Q8_0.gguf, 2,786,568,256 B) at depth 2, with the
draftless ngram-mod matcher chained behind it — the only legal way to
layer two spec implementations in one process.
- Vision explicit: --mmproj names the BF16 projector (this build never
autoloads; a server without it comes up healthy and text-only).
- Reasoning: --reasoning-format deepseek → reasoning_content; the
Qwen3.8 template's {xhigh, medium, low} reasoning_effort gate folded
by reasoningEffortMap in models.yml.
- One stream, the model's full 262,144 context; greedy server default
with Qwen-recommended sampling riding compat.extraBody.
The deployer's deploy_llm_primary is rewritten for the root unit: it
gates every artifact at its declared byte count (stat -c %s, never wc -c
— which read the whole 377 GiB lucebox file for minutes on 2026-09-26),
retires the lucebox user quadlet to dated home trash, waits out port
7770, relinks/enables/restarts the root unit and waits up to 1800 s for
/health. The quadlet-era tree under dankserver/frank-primary/ is
archived (dankserver/.AGENTTRASH/20260926-llamacpp-cutover/ on this
box; git history everywhere else) and its planner mapping replaced by
the scripts/ trio.
Observability flips back to the llamacpp namespace: a frank-primary
Prometheus job at 192.168.0.235:7770 (cross-host LAN — the netavark
published-port quirk that forces host.containers.internal for the two
Dankclaw endpoints is same-host only), the LlmEndpointDown runbook
text, and the dashboard description for three llama.cpp endpoints.
Docs: the new runbook (frank-primary-llamacpp-qwen38-flash-next-q8.md),
RETIRED banners on the lucebox and radiance runbooks and their index
entries, the AGENTS.md fleet section rewritten for the root-unit shape,
and the homepage frank-medium card renamed to frank-primary with the
Q8_0 Flash-Next spec (docs.html #models and the modality note updated:
images on frank-primary and frank-small, audio on frank-small only).
models.yml: the dankserver provider is "llamacpp (dankserver)" serving
frank-primary, contextWindow 262144, maxTokens 65536 (compaction-gap
law: 183,501 + 65,536 <= 262,144), input [text, image].
Suite: 346 green (frank_primary_unit.bats rewritten for the root unit —
16 tests pinning the placement law, the shared-head requirement, the
explicit projector, HIP env hygiene (BIND_HOST, never bare HOST), and
the deployer's byte-count gates; the quadlet-era planner and
changed-tests mappings rewritten for the trio; the observability scrape
test asserts three llamacpp jobs again).
Rollback: the lucebox image (localhost/lucebox-rocm:gfx1201) and the
378 GiB payload stay on the host; the Qwen payload stays fetched, so a
re-swap in either direction is a redeploy, not a refetch.
This commit is contained in:
1 parent
d89813363e
commit
8500600d09
25 files changed
+856
-544
No files matched your search
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
title: frank-primary Lucebox DeepSeek V4 Runbook
|
||||
description: ACTIVE frank-primary runbook — the Lucebox runtime (github.com/Luce-Org/lucebox, local image localhost/lucebox-rocm:gfx1201) serving DeepSeek-V4.1-Flash-ROCMFP23 (377.2 GiB GGUF, 43-layer MoE, 256 routed experts, top-6) on Dankserver's R9700 with the routed experts computed on CPU from the mmapped page cache. Replaces frank-medium (vllm-radiance TP=2) on :7770, cutover 2026-09-26.
|
||||
description: RETIRED 2026-09-26 frank-primary runbook (active for hours on 2026-09-26) — the Lucebox runtime (github.com/Luce-Org/lucebox, local image localhost/lucebox-rocm:gfx1201) serving DeepSeek-V4.1-Flash-ROCMFP23 (377.2 GiB GGUF, 43-layer MoE, 256 routed experts, top-6) on Dankserver's R9700 with the routed experts computed on CPU from the mmapped page cache. Replaced the same day by [[frank-primary-llamacpp-qwen38-flash-next-q8]] (llama.cpp Qwen3.8-Flash-Next Q8_0); payload and image retained on the host as the rollback target.
|
||||
tags:
|
||||
- frank-primary
|
||||
- dankserver
|
||||
@@ -18,14 +18,22 @@ timestamp: 2026-09-26
|
||||
|
||||
# frank-primary Lucebox DeepSeek V4 Runbook
|
||||
|
||||
The ACTIVE dankserver LLM endpoint (:7770, alias **frank-primary**): the
|
||||
> [!warning] RETIRED 2026-09-26 (same day it went live)
|
||||
> Replaced by [[frank-primary-llamacpp-qwen38-flash-next-q8]] — unsloth
|
||||
> llama.cpp serving Qwen3.8-Flash-Next Q8_0 on both R9700s. This runtime
|
||||
> served for hours; the rollback material (image `localhost/lucebox-rocm:
|
||||
> gfx1201` and the 378 GiB payload) is retained on the host, and the
|
||||
> quadlet-era files live in `dankserver/.AGENTTRASH/20260926-llamacpp-cutover/`.
|
||||
|
||||
The dankserver LLM endpoint (:7770, alias **frank-primary**) as it stood on
|
||||
2026-09-26: the
|
||||
[Lucebox](https://github.com/Luce-Org/lucebox) runtime serving
|
||||
`Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF` (377.2 GiB, 405,349,859,328
|
||||
bytes, LFS oid
|
||||
`2daea6ca1715bc987aae201e63445d3f7abdc2337c05f5f569f5e50237ce8be2`) on a
|
||||
single Radeon AI PRO R9700 (gfx1201). It replaces
|
||||
[[frank-medium-radiance-mxfp4]] (vllm-radiance TP=2), retired on
|
||||
2026-09-26.
|
||||
single Radeon AI PRO R9700 (gfx1201). It replaced
|
||||
[[frank-medium-radiance-mxfp4]] (vllm-radiance TP=2), retired earlier the
|
||||
same day.
|
||||
|
||||
## Why this shape
|
||||
|
||||
|
||||
Reference in new issue
Block a user