Files
serverAppConfigs/tests/frank_large_quadlet.bats
T
Frank 301914b1c0 feat(llm): swap frank-large to halogen-flash-server, retire frank-npu
frank-large moves from the kingjones30/ROCmFPX llama.cpp build to
peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b
.hgn 115.55 GiB + quality overlay). Upstream measures this engine at
roughly 4x the prefill throughput of the llama.cpp family on the same
gfx1151 silicon, and prefill dominates any prompt with real context.

Shape: a rootless Quadlet container (dankclaw/frank-large/quadlet/),
engine on container loopback :8730 (never published), OpenAI front-end
:8731 published as host 7770, HALOGEN_MODEL_ID=frank-large so no client
renamed anything. Context 262144, KV pool 262144 positions (~7.2 GiB,
conservative for a shared 121 GiB host), 4 slots, host reserve 24,
vision tower on, weights pinned, all other flags at vendor defaults.
Weights are fetched by HALOGEN_DOWNLOAD on first start into a writable
/home/dank/models/halogen.

The llama.cpp era (unit, env, serve script) is staged in .AGENTTRASH/
rather than deleted; the STRIX payload and the ROCmFPX build stay on
disk as the rollback target. deploy_llm_large becomes a user-quadlet
handler: it retires the old ROOT unit (moving its checkout symlink into
the home trash, never rm) and gates the first start on the download
rather than reporting a failure for a fetch that is working exactly as
designed.

frank-npu is retired in the same change. The kernel cmdline moves from
iommu=pt to amd_iommu=off for the 13-16% of prefill upstream measured,
and that takes the XDNA2 NPU with it: without PASID/SVA the driver
cannot open /dev/accel/accel0 (err=-19). Its five dependent routes are
repointed (Hermes vision -> frank-large, Hermes title generation ->
frank-small, harness vision -> frank-large, tiny -> frank-small, task ->
frank-large:medium) and its provider blocks are removed from both the
harness config and Hermes's live config. Audio input leaves the fleet
with no replacement; image input survives on halogen's vision tower.

Docs: new halogen-flash-server runbook, new IOMMU decision record, and
frank-npu rewritten as a retirement record. AGENTS.md topology, runtime
sections, benchmarking policy (frank-large is a user unit now) and the
harness model-config section are corrected.
2026-09-11 14:48:28 -07:00

96 lines
4.1 KiB
Bash
Executable File

#!/usr/bin/env bats
# Seam: the frank-large quadlet on dankclaw.
# frank-large became a rootless Quadlet container on 2026-09-11 when
# halogen-flash-server replaced the llama.cpp/kingjones runtime. This seam pins
# the properties that make it work and that no reviewer can see by reading the
# flags alone: GPU device access and the image's required podman posture (the
# engine maps the checkpoint and registers it with the GPU in place), the
# single published port that keeps every client pointed where it has always
# pointed, the read-write weights mount the first-start download needs, boot
# persistence, and a deploy handler that retires the old root unit without
# deleting anything.
load helpers
UNIT="$REPO_ROOT/dankclaw/frank-large/quadlet/frank-large.container"
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
@test "frank-large quadlet exists under dankclaw/frank-large/quadlet/" {
[ -f "$UNIT" ]
}
@test "frank-large unit is boot-persistent (WantedBy=default.target)" {
# Generated units cannot be enabled on this host's podman 5.x v1 quadlet
# generator; [Install] plus linger is the entire boot-persistence story.
grep -q '^WantedBy=default.target$' "$UNIT"
}
@test "frank-large unit is self-healing" {
grep -q '^Restart=on-failure$' "$UNIT"
grep -q '^RestartSec=' "$UNIT"
}
@test "frank-large image is pinned to an exact version tag" {
grep -Eq '^Image=localhost/peonist-ai/halogen-flash-server:[0-9]+\.[0-9]+\.[0-9]+$' "$UNIT"
}
@test "frank-large can reach the GPU (kfd + DRM render node)" {
# Without both devices the engine cannot map the checkpoint at all; the
# failure is a load error, not a degraded mode.
grep -q '^AddDevice=/dev/kfd$' "$UNIT"
grep -q '^AddDevice=/dev/dri$' "$UNIT"
}
@test "frank-large runs with the podman posture the image requires" {
# The image documents --ipc=host, an unconfined seccomp profile and an
# unlimited memlock as hard requirements.
grep -q -- '--ipc=host' "$UNIT"
grep -q -- '--security-opt seccomp=unconfined' "$UNIT"
grep -q '^Ulimit=memlock=-1:-1$' "$UNIT"
}
@test "frank-large publishes exactly one port and it is the client port" {
# 8731 is the OpenAI front-end; the engine on 8730 is unauthenticated and
# must never be published.
[ "$(grep -c '^PublishPort=' "$UNIT")" -eq 1 ]
grep -q '^PublishPort=7770:8731$' "$UNIT"
}
@test "frank-large keeps the fleet identity every client already points at" {
# The id at /v1/models, in /health and in every response body.
grep -q '^Environment=HALOGEN_MODEL_ID=frank-large$' "$UNIT"
}
@test "the weights mount is writable, because the first start downloads into it" {
grep -q '^Volume=/home/dank/models/halogen:/models$' "$UNIT"
! grep -q '^Volume=.*:/models:ro$' "$UNIT"
grep -q '^Environment=HALOGEN_DOWNLOAD=' "$UNIT"
}
@test "frank-large unit carries no secret material" {
! grep -Eq '^(Environment|EnvironmentFile)=.*(TOKEN|PASSWORD|SECRET|KEY)' "$UNIT"
}
@test "the deploy handler retires the old root unit without deleting it" {
grep -q 'retire_root_llm_large' "$REMOTE_DEPLOY"
grep -q 'systemctl stop frank-large.service' "$REMOTE_DEPLOY"
# Never `rm`: the retired unit moves into the home trash so the llama.cpp
# era stays recoverable.
grep -q 'mv -- /etc/systemd/system/frank-large.service' "$REMOTE_DEPLOY"
! grep -qE '(^|[^[:alnum:]_])rm (-[a-zA-Z]+ )?/etc/systemd/system/frank-large' "$REMOTE_DEPLOY"
}
@test "the deploy handler never tries to enable a generated unit" {
# `systemctl --user enable frank-large.service` fails on this host
# ("transient or generated"); the deploy must install and restart instead.
! grep -q 'systemctl --user enable frank-large.service' "$REMOTE_DEPLOY"
grep -q 'install_quadlet frank-large dankclaw/frank-large/quadlet frank-large' "$REMOTE_DEPLOY"
grep -q 'systemctl --user restart frank-large.service' "$REMOTE_DEPLOY"
}
@test "the deploy handler gates the first start on the download, not on a failure" {
grep -q 'wait_for_llm_large_ready' "$REMOTE_DEPLOY"
grep -q 'qwen38-flash-next-w4b.hgn' "$REMOTE_DEPLOY"
grep -q '127.0.0.1:7770/health' "$REMOTE_DEPLOY"
}