feat(llm): swap frank-large to halogen-flash-server, retire frank-npu
frank-large moves from the kingjones30/ROCmFPX llama.cpp build to peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b .hgn 115.55 GiB + quality overlay). Upstream measures this engine at roughly 4x the prefill throughput of the llama.cpp family on the same gfx1151 silicon, and prefill dominates any prompt with real context. Shape: a rootless Quadlet container (dankclaw/frank-large/quadlet/), engine on container loopback :8730 (never published), OpenAI front-end :8731 published as host 7770, HALOGEN_MODEL_ID=frank-large so no client renamed anything. Context 262144, KV pool 262144 positions (~7.2 GiB, conservative for a shared 121 GiB host), 4 slots, host reserve 24, vision tower on, weights pinned, all other flags at vendor defaults. Weights are fetched by HALOGEN_DOWNLOAD on first start into a writable /home/dank/models/halogen. The llama.cpp era (unit, env, serve script) is staged in .AGENTTRASH/ rather than deleted; the STRIX payload and the ROCmFPX build stay on disk as the rollback target. deploy_llm_large becomes a user-quadlet handler: it retires the old ROOT unit (moving its checkout symlink into the home trash, never rm) and gates the first start on the download rather than reporting a failure for a fetch that is working exactly as designed. frank-npu is retired in the same change. The kernel cmdline moves from iommu=pt to amd_iommu=off for the 13-16% of prefill upstream measured, and that takes the XDNA2 NPU with it: without PASID/SVA the driver cannot open /dev/accel/accel0 (err=-19). Its five dependent routes are repointed (Hermes vision -> frank-large, Hermes title generation -> frank-small, harness vision -> frank-large, tiny -> frank-small, task -> frank-large:medium) and its provider blocks are removed from both the harness config and Hermes's live config. Audio input leaves the fleet with no replacement; image input survives on halogen's vision tower. Docs: new halogen-flash-server runbook, new IOMMU decision record, and frank-npu rewritten as a retirement record. AGENTS.md topology, runtime sections, benchmarking policy (frank-large is a user unit now) and the harness model-config section are corrected.
This commit is contained in:
@@ -18,15 +18,23 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
||||
|
||||
@test "deployment planner maps large-model changes to the large model only" {
|
||||
run python3 "$PLANNER" --target dankclaw --paths \
|
||||
scripts/frank-large.env scripts/frank-large.service
|
||||
dankclaw/frank-large/quadlet/frank-large.container
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$output" = "frank-large" ]
|
||||
}
|
||||
|
||||
@test "shared chat-template changes map to the selected host's inference service" {
|
||||
@test "the retired NPU endpoint is absent from the deployment inventory" {
|
||||
! grep -q '"frank-npu"' "$PLANNER"
|
||||
! grep -q 'frank_npu\|frank-npu' "$REMOTE_DEPLOY"
|
||||
}
|
||||
|
||||
@test "shared chat-template changes map only to the service that reads it" {
|
||||
# frank-large runs halogen-flash-server, which owns its own chat handling;
|
||||
# frank-small is the only consumer of the tracked template now, so a
|
||||
# template edit is a no-op on Dankclaw.
|
||||
run python3 "$PLANNER" --target dankclaw --paths scripts/chat-template.jinja
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$output" = "frank-large" ]
|
||||
[ -z "$output" ]
|
||||
|
||||
run python3 "$PLANNER" --target dankserver --paths scripts/chat-template.jinja
|
||||
[ "$status" -eq 0 ]
|
||||
@@ -68,10 +76,6 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
||||
run python3 "$PLANNER" --target dankclaw --paths dankclaw/hermes/config.yaml.example
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$output" = "hermes" ]
|
||||
|
||||
run python3 "$PLANNER" --target dankclaw --paths scripts/serve-frank-npu.sh scripts/frank-npu.env
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$output" = "frank-npu" ]
|
||||
}
|
||||
|
||||
@test "homepage is owned by Dankclaw's consolidated proxy" {
|
||||
|
||||
Reference in New Issue
Block a user