feat(llm): swap frank-large to halogen-flash-server, retire frank-npu

frank-large moves from the kingjones30/ROCmFPX llama.cpp build to
peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b
.hgn 115.55 GiB + quality overlay). Upstream measures this engine at
roughly 4x the prefill throughput of the llama.cpp family on the same
gfx1151 silicon, and prefill dominates any prompt with real context.

Shape: a rootless Quadlet container (dankclaw/frank-large/quadlet/),
engine on container loopback :8730 (never published), OpenAI front-end
:8731 published as host 7770, HALOGEN_MODEL_ID=frank-large so no client
renamed anything. Context 262144, KV pool 262144 positions (~7.2 GiB,
conservative for a shared 121 GiB host), 4 slots, host reserve 24,
vision tower on, weights pinned, all other flags at vendor defaults.
Weights are fetched by HALOGEN_DOWNLOAD on first start into a writable
/home/dank/models/halogen.

The llama.cpp era (unit, env, serve script) is staged in .AGENTTRASH/
rather than deleted; the STRIX payload and the ROCmFPX build stay on
disk as the rollback target. deploy_llm_large becomes a user-quadlet
handler: it retires the old ROOT unit (moving its checkout symlink into
the home trash, never rm) and gates the first start on the download
rather than reporting a failure for a fetch that is working exactly as
designed.

frank-npu is retired in the same change. The kernel cmdline moves from
iommu=pt to amd_iommu=off for the 13-16% of prefill upstream measured,
and that takes the XDNA2 NPU with it: without PASID/SVA the driver
cannot open /dev/accel/accel0 (err=-19). Its five dependent routes are
repointed (Hermes vision -> frank-large, Hermes title generation ->
frank-small, harness vision -> frank-large, tiny -> frank-small, task ->
frank-large:medium) and its provider blocks are removed from both the
harness config and Hermes's live config. Audio input leaves the fleet
with no replacement; image input survives on halogen's vision tower.

Docs: new halogen-flash-server runbook, new IOMMU decision record, and
frank-npu rewritten as a retirement record. AGENTS.md topology, runtime
sections, benchmarking policy (frank-large is a user unit now) and the
harness model-config section are corrected.
This commit is contained in:
Frank
2026-09-11 14:48:28 -07:00
parent 184df9aa0d
commit 301914b1c0
21 changed files with 929 additions and 279 deletions
+9 -9
View File
@@ -59,7 +59,6 @@ SERVICE_ORDER = (
"lidarr",
"readarr",
"frank-large",
"frank-npu",
"frank-small",
)
@@ -89,7 +88,6 @@ DANKCLAW_SERVICES = {
"caddy",
"homepage",
"frank-large",
"frank-npu",
}
DANKSERVER_SERVICES = {
@@ -297,6 +295,11 @@ def services_for_path(path: str, target: str) -> set[str]:
if len(parts) >= 2 and parts[1] == "open-webui":
_add(services, "open-webui")
return services
if len(parts) >= 2 and parts[1] == "frank-large":
# The service is a rootless Quadlet container since 2026-09-11;
# its unit and fleet settings live in one .container file.
_add(services, "frank-large")
return services
if len(parts) >= 2 and parts[1] == "gitea":
if len(parts) >= 4 and parts[2] == "quadlet":
name = parts[3]
@@ -346,15 +349,12 @@ def services_for_path(path: str, target: str) -> set[str]:
if parts[0] == "scripts":
name = p.name
if name in {"frank-large.env", "frank-large.service", "serve-frank-large.sh"}:
_add(services, "frank-large")
elif name in {"frank-npu.env", "frank-npu.service", "serve-frank-npu.sh"}:
_add(services, "frank-npu")
elif name in {"frank-small.env", "frank-small.service", "serve-frank-small.sh"}:
if name in {"frank-small.env", "frank-small.service", "serve-frank-small.sh"}:
_add(services, "frank-small")
elif name == "chat-template.jinja":
# Only the selected target's LLM consumes this shared template.
_add(services, "frank-large" if target == "dankclaw" else "frank-small")
# Only frank-small consumes this template now: frank-large runs
# halogen-flash-server, which owns its own chat handling.
_add(services, "frank-small")
elif name in {"homepage-status.py", "homepage-status.service", "homepage-status.timer"}:
_add(services, "homepage")
# Deployment/test/benchmark scripts and other repo tooling do not