Replace the CIRU v4.4.1 Orca runtime with the unsloth llama.cpp prebuilt
(b11030-mix-5ff778e, ROCm gfx1151, bundles its own ROCm) serving
unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (87.3 GiB, three HF split parts
concatenated) on the Strix Halo iGPU. One conversation, 200,000 ctx,
PARALLEL_SLOTS=1, MTP depth 2 via the explicit -md shared-Q8_0 head
(sidecar auto-discovery does not search the mtp/ subfolder), text-only.
Fleet contract unchanged: port 7770, alias frank-large.
- serve-frank-large.sh: unsloth binary, -m/-md, --spec-type draft-mtp
--spec-draft-n-max 2, --jinja, preserve_thinking; gates on binary,
GGUF, MTP head; conditional --mmproj on ENABLE_VISION=1
- download-frank-large-unsloth.sh: resumable fetch of 3 parts + MTP head
+ mmproj, each verified against its LFS sha256 fetched live from the
Hub tree API; concatenates parts, records/verifies SHA256SUMS
- deploy-remote.sh: LLM_LARGE_MODEL/LLM_LARGE_DRAFT paths, pkill pattern
on the model path (not the binary name), wait_port_free gate
- tests: seams for 200k/1-slot, explicit -md, text-only, handler gates,
sweep safety
Operator call 2026-09-19: switch the iGPU endpoint from
halogen-flash-server to jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca on the CIRU
v4.4.1 runtime with the QSA window hotfix (host build against the private
ROCm 10 SDK), one stream of the full 262144 trained window. The service
leaves the quadlet shape for a root unit (the same shape as frank-small and
frank-medium: a private-SDK host binary has no image to install and no
generated unit to dodge); the halogen quadlet is archived, the deploy
handler retires it on the host, gates on 7770, and its stray-run sweep
matches the model path so it cannot take frank-small down.
Dankclaw's endpoints are re-split. frank-large is the halogen-flash-server
quadlet again -- the 2026-09-13 unit file was never touched by the two-day
ik_llama.cpp interlude, so the runtime was still in the repo and its image is
still in local storage -- retuned to the shape asked for: HALOGEN_CTX 200192
with a 400,384-position pool and two slots, exactly two 200k conversations.
frank-small is new: mainline llama.cpp on the RTX 3090 serving
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF at IQ3_S, using the -mtp build (it carries
blk.64.nextn.*, which is what lets --spec-type draft-mtp run on the main
model's own weights instead of a second 12 GiB copy) plus the BF16 vision
projector, as one 131,072-token slot with f16 KV on port 7771.
The host is shared now, and that is what the halogen retune pays for:
HALOGEN_MAX_TOK 24576 (arena 20.6 -> 16.4 GiB, ~4-5% prefill) and the vision
tower dropped (1.97 GiB) hand ~6 GiB back to the host, and the reserve rises to
20 to keep the fit check honest. Vision moved wholesale to frank-small.
Retired to scripts/.AGENTTRASH/: the ik trio (env, service, serve script), the
Qwen-Sharp template only that runtime read, and four benchmark harnesses bound
to retired runtimes. Because retiring a file makes any test that greps it pass
by absence, the seams assert the archive exists AND the live path does not.
frank-large moves from the kingjones30/ROCmFPX llama.cpp build to
peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b
.hgn 115.55 GiB + quality overlay). Upstream measures this engine at
roughly 4x the prefill throughput of the llama.cpp family on the same
gfx1151 silicon, and prefill dominates any prompt with real context.
Shape: a rootless Quadlet container (dankclaw/frank-large/quadlet/),
engine on container loopback :8730 (never published), OpenAI front-end
:8731 published as host 7770, HALOGEN_MODEL_ID=frank-large so no client
renamed anything. Context 262144, KV pool 262144 positions (~7.2 GiB,
conservative for a shared 121 GiB host), 4 slots, host reserve 24,
vision tower on, weights pinned, all other flags at vendor defaults.
Weights are fetched by HALOGEN_DOWNLOAD on first start into a writable
/home/dank/models/halogen.
The llama.cpp era (unit, env, serve script) is staged in .AGENTTRASH/
rather than deleted; the STRIX payload and the ROCmFPX build stay on
disk as the rollback target. deploy_llm_large becomes a user-quadlet
handler: it retires the old ROOT unit (moving its checkout symlink into
the home trash, never rm) and gates the first start on the download
rather than reporting a failure for a fetch that is working exactly as
designed.
frank-npu is retired in the same change. The kernel cmdline moves from
iommu=pt to amd_iommu=off for the 13-16% of prefill upstream measured,
and that takes the XDNA2 NPU with it: without PASID/SVA the driver
cannot open /dev/accel/accel0 (err=-19). Its five dependent routes are
repointed (Hermes vision -> frank-large, Hermes title generation ->
frank-small, harness vision -> frank-large, tiny -> frank-small, task ->
frank-large:medium) and its provider blocks are removed from both the
harness config and Hermes's live config. Audio input leaves the fleet
with no replacement; image input survives on halogen's vision tower.
Docs: new halogen-flash-server runbook, new IOMMU decision record, and
frank-npu rewritten as a retirement record. AGENTS.md topology, runtime
sections, benchmarking policy (frank-large is a user unit now) and the
harness model-config section are corrected.
frank-default-large -> frank-large (Dankclaw :7770), frank-default-small ->
frank-small (Dankserver :7770), frank-default-sightsound -> frank-npu
(Dankclaw :7771). Units, env files, serve scripts, deploy-plan mappings,
deploy-remote dispatch (stop/disable old units + relink + enable new), and the
test seams move together. Retired frank-default-npu.* artifacts moved to
scripts/.AGENTTRASH/ (tracked-in-history; ignored on disk).
Served aliases become frank-large / frank-small via --alias; the NPU endpoint
serves the whole FLM registry, so its rename changes no served tags.
FrankandClaude