feat(llm): swap frank-large to halogen-flash-server, retire frank-npu

frank-large moves from the kingjones30/ROCmFPX llama.cpp build to
peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b
.hgn 115.55 GiB + quality overlay). Upstream measures this engine at
roughly 4x the prefill throughput of the llama.cpp family on the same
gfx1151 silicon, and prefill dominates any prompt with real context.

Shape: a rootless Quadlet container (dankclaw/frank-large/quadlet/),
engine on container loopback :8730 (never published), OpenAI front-end
:8731 published as host 7770, HALOGEN_MODEL_ID=frank-large so no client
renamed anything. Context 262144, KV pool 262144 positions (~7.2 GiB,
conservative for a shared 121 GiB host), 4 slots, host reserve 24,
vision tower on, weights pinned, all other flags at vendor defaults.
Weights are fetched by HALOGEN_DOWNLOAD on first start into a writable
/home/dank/models/halogen.

The llama.cpp era (unit, env, serve script) is staged in .AGENTTRASH/
rather than deleted; the STRIX payload and the ROCmFPX build stay on
disk as the rollback target. deploy_llm_large becomes a user-quadlet
handler: it retires the old ROOT unit (moving its checkout symlink into
the home trash, never rm) and gates the first start on the download
rather than reporting a failure for a fetch that is working exactly as
designed.

frank-npu is retired in the same change. The kernel cmdline moves from
iommu=pt to amd_iommu=off for the 13-16% of prefill upstream measured,
and that takes the XDNA2 NPU with it: without PASID/SVA the driver
cannot open /dev/accel/accel0 (err=-19). Its five dependent routes are
repointed (Hermes vision -> frank-large, Hermes title generation ->
frank-small, harness vision -> frank-large, tiny -> frank-small, task ->
frank-large:medium) and its provider blocks are removed from both the
harness config and Hermes's live config. Audio input leaves the fleet
with no replacement; image input survives on halogen's vision tower.

Docs: new halogen-flash-server runbook, new IOMMU decision record, and
frank-npu rewritten as a retirement record. AGENTS.md topology, runtime
sections, benchmarking policy (frank-large is a user unit now) and the
harness model-config section are corrected.
This commit is contained in:
Frank committed 2026-09-11 14:48:28 -07:00
1 parent 184df9aa0d
commit 301914b1c0
21 files changed
+929 -279

No files matched your search

@@ -1,7 +1,7 @@
---
type: reference
title: kingjones Qwen3.8-Flash-Next ROCmFP4 STRIX on Dankclaw
description: ACTIVE frank-large runbook. kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF served by the card's pinned kingjones30/ROCmFPX fork. 177B params (56B MoE + 51.2B PLE streamed via mmap), single 113.47 GiB GGUF, 401408 ctx x 2 slots.
title: kingjones Qwen3.8-Flash-Next ROCmFP4 STRIX on Dankclaw (prior, rollback target)
description: PRIOR frank-large runbook and rollback target (2026-08-30 → 2026-09-11). kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF served by the card's pinned kingjones30/ROCmFPX fork. 177B params (56B MoE + 51.2B PLE streamed via mmap), single 113.47 GiB GGUF, 401408 ctx x 2 slots.
tags:
- llama.cpp
- rocm
@@ -16,12 +16,22 @@ timestamp: 2026-08-30
# kingjones Qwen3.8-Flash-Next ROCmFP4 STRIX on Dankclaw
> **Active again since 2026-08-30** on the model card's **pinned runtime**:
> the `kingjones30/ROCmFPX` fork (replaced the agentionai FAST deployment
> which ran on charlie's `build-qwen38`). Text-only tier; the agentionai
> f16 vision tower is paired for mmproj and loads clean. Speculative
> decoding is OFF (the card ships no spec flags; the Flash-Next MTP heads
> do not load on this loader family — see MTP Draft Head).
> **PRIOR / ROLLBACK TARGET since 2026-09-11** — frank-large moved to
> [[halogen-flash-server]] (peonist-ai halogen-flash-server 0.5.8 serving
> Qwen3.8-Flash-Next w4b), which upstream measures at roughly 4x this
> runtime's prefill. Everything below documents the llama.cpp era and stays
> accurate for a rollback: the payload is still at
> `/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX`, the binary
> at `/home/dank/GitPackages/kingjones-ROCmFPX/build`, and the unit, env and
> serve script are staged in `serverAppConfigs/.AGENTTRASH/` (`.llama.cpp`).
> Rolling back also requires removing the halogen Quadlet from
> `~/.config/containers/systemd/` so it stops claiming port 7770.
>
> Ran 2026-08-30 → 2026-09-11 on the model card's **pinned runtime**: the
> `kingjones30/ROCmFPX` fork. Text-only tier; the agentionai f16 vision
> tower was paired for mmproj and loads clean. Speculative decoding is OFF
> (the card ships no spec flags; the Flash-Next MTP heads do not load on
> this loader family — see MTP Draft Head).
`frank-large` (port 7770) serves
`Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf` (single file, 113.47 GiB)