dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(frank-dragoon): run the XDNA2 NPU lane beside the flash model

Browse Source 
halogen-flash-server already ships an NPU path on the pin we run: for each
model named in HALOGEN_NPU_MODELS it loads the vendor's precompiled AIE
programs (XDNA2 executes only AIE-compiled graphs) and answers them on
/v1/embeddings inside the SAME container behind port 7770. That gives the
fleet an in-house embedding endpoint without touching the iGPU trunk.

The lane is opt-in and one line wide: HALOGEN_NPU_MODELS blank is the
pre-lane endpoint byte for byte (no device, no mounts, no engine env).

The three host preconditions the vendor names are gated in the serve script
and mirrored in the deployer (the 2026-10-01 twin law), each refused by name:
  - the staged model files. /models is read-only and HALOGEN_DOWNLOAD stays
    unset so the container opens no outbound connection at boot, so
    scripts/download-frank-dragoon-npu.sh stages them from the pinned
    revision with the catalog's own byte counts and sha256s (the engine
    re-checks both at boot);
  - the distro XRT's libxrt_coreutil/libxrt_core/libxrt_driver_xdna, each
    mounted twice — where the image looks for XRT and at its own path —
    because those libraries find each other where they were installed and
    an /opt/xilinx/xrt of symlinks dangles inside the container;
  - the GPU fabric clock HELD. On Strix Halo the iGPU and the NPU co-running
    hang the machine (and corrupt NPU results) while FCLK changes speed, and
    halogen refuses to start in that state; it cannot read host sysfs from
    inside, so the hold is proven host-side. dankclaw/host/ carries the
    vendor's halogen-fabric-clock{,.service} verbatim at tag v0.16.2 (sha256
    a3a4f7f4e0142e71… / 5f9f1cc4c85a1c02…); the deployer installs, enables
    and gates it only while the lane is named, and releases the clock plus
    drops the boot hook when it is not.

After health, the deployer's lane gate is a real embedding through
/v1/embeddings — a 200 on /health proves nothing about the NPU programs.

Planner: dankclaw/host/halogen-fabric-clock* re-dispatches frank-dragoon
(the deployer installs it); dankclaw/host/grub and the fetch script plan
nothing.

Seam: 11 new tests — the off arm really renders nothing lane-specific, the
on arm renders the device/mounts/env, every missing precondition refuses by
name, the deployer twins and the host unit shape are pinned.

Co-Authored-By: omp <omp@dankclaw>
This commit is contained in:
Frank
2026-10-19 21:22:50 -07:00
co-authored by omp
parent 6215f1a6c8
commit 53ef121180
9 changed files with 605 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/frank_dragoon_unit.bats
+200
View File
@@ -27,6 +27,9 @@ setup() {
ENV_FILE="$ROOT/scripts/frank-dragoon.env"
UNIT="$ROOT/scripts/frank-dragoon.service"
DEPLOYER="$ROOT/scripts/deploy-remote.sh"
HOST_UNIT="$ROOT/dankclaw/host/halogen-fabric-clock"
HOST_SERVICE="$ROOT/dankclaw/host/halogen-fabric-clock.service"
NPU_FETCH="$ROOT/scripts/download-frank-dragoon-npu.sh"
BUILD="$ROOT/scripts/build-rocmfpx.sh"
TEST_TMP="$(mktemp -d)"
touch "$TEST_TMP/empty.env"
@@ -167,6 +170,14 @@ good_env() {
kv_gib=$(( pool * 28 * 1024 / 1073741824 ))
arena_tenths=$(( 38 + 84 * mt / 16384 ))
total_gib=$(( 62 + kv_gib + arena_tenths / 10 + 2 + reserve + 15 ))
# The NPU lane bills extra pinned host RAM per named model (the vendor's
# budget note: ~2 GiB for an embedding/decider/rerank-sized program, more
# for FLUX-class ones), and the lane is named in the shipped env, so the
# ledger must still close on a 121 GiB unified-memory host.
npu_models=0
npu="$(sed -nE 's/^HALOGEN_NPU_MODELS=(.*)$/\1/p' "$ENV_FILE")"
[ -n "$npu" ] && npu_models="$(echo "${npu//,/ }" | wc -w)"
total_gib=$(( total_gib + 2 * npu_models ))
[ "$total_gib" -le 121 ]
}
@@ -339,6 +350,13 @@ $(cat "$TEST_TMP/serve.log")"
grep -qF "LLM_DRAGOON_TOK_FILES=\"\${LLM_DRAGOON_TOK_FILES:-$tokf}\"" "$DEPLOYER"
bytes="$(sed -nE 's/^FRANK_DRAGOON_CHECKPOINT_BYTES=([0-9]+)$/\1/p' "$ENV_FILE")"
grep -qF "LLM_DRAGOON_CKPT_BYTES=\"\${LLM_DRAGOON_CKPT_BYTES:-$bytes}\"" "$DEPLOYER"
# The NPU twins: the lane switches on the same value on both sides. A blank
# env line with a named deployer default (or the reverse) is the 2026-10-01
# disagreement again — one side gates a lane the other does not build.
npu_dir="$(sed -nE 's/^FRANK_DRAGOON_NPU_DIR=(.*)$/\1/p' "$ENV_FILE")"
npu_models="$(sed -nE 's/^HALOGEN_NPU_MODELS=(.*)$/\1/p' "$ENV_FILE")"
grep -qF "LLM_DRAGOON_NPU_DIR=\"\${LLM_DRAGOON_NPU_DIR:-$npu_dir}\"" "$DEPLOYER"
grep -qF "LLM_DRAGOON_NPU_MODELS=\"\${LLM_DRAGOON_NPU_MODELS:-$npu_models}\"" "$DEPLOYER"
}
@test "the deployer points a missing artifact at the v2abl fetch script" {
@@ -383,3 +401,185 @@ $(cat "$TEST_TMP/serve.log")"
grep -qE 'GPU_TARGET=.*gfx1151' "$BUILD"
grep -qE 'DLLAMA_BUILD_WEBUI=OFF' "$BUILD"
}
# --- the NPU lane (XDNA2) ----------------------------------------------------
# Opt-in lane added 2026-10-19: halogen runs the vendor's precompiled AIE
# programs for the models HALOGEN_NPU_MODELS names on the NPU inside the SAME
# container (they answer /v1/embeddings on 7770). Two things make this seam
# worth pinning: the lane must be genuinely OFF when the variable is blank (the
# rollback is one line, so the render must show nothing lane-specific), and
# every host precondition the engine needs — the accel node, the distro XRT's
# three libraries, the staged model files, the held GPU fabric clock — must be
# refused BY NAME instead of surfacing as an engine crash-loop.
npu_fixture() {
touch "$TEST_TMP/accel0"
mkdir -p "$TEST_TMP/xrt"
for l in libxrt_coreutil.so.2 libxrt_core.so.2 libxrt_driver_xdna.so.2; do
touch "$TEST_TMP/xrt/$l"
done
mkdir -p "$TEST_TMP/npu/qwen3-embedding-0.6b/devices"
mkdir -p "$TEST_TMP/sys/class/drm/card1/device"
printf '0: 400Mhz \n7: 2000Mhz *\n' > "$TEST_TMP/sys/class/drm/card1/device/pp_dpm_fclk"
printf 'manual\n' > "$TEST_TMP/sys/class/drm/card1/device/power_dpm_force_performance_level"
NPU_ENV=(
"HALOGEN_NPU_MODELS=qwen3-embedding-0.6b"
"FRANK_DRAGOON_NPU_DIR=$TEST_TMP/npu"
"FRANK_DRAGOON_DEV_ACCEL=$TEST_TMP/accel0"
"FRANK_DRAGOON_XRT_DIR=$TEST_TMP/xrt"
"FRANK_DRAGOON_SYSFS=$TEST_TMP/sys"
)
}
# run_with_log <args...> — run the serve script and fold its own stderr plus
# the podman stub's argv (which the script redirects into the log) into
# $output, so both refusal text and render assertions read one variable.
run_with_log() {
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" "${NPU_ENV[@]}" "$@" "$SERVE"
if [ -f "$TEST_TMP/serve.log" ]; then
output="$output
$(cat "$TEST_TMP/serve.log")"
fi
}
@test "the lane is off when HALOGEN_NPU_MODELS is blank: no device, no mounts, no engine env" {
# The rollback is blanking one env line, so "off" has to be observable in
# the command podman would run — not just implied by an absent log line.
podman_stub
good_env
run_with_log HALOGEN_NPU_MODELS=
[ "$status" -eq 0 ]
[[ "$output" == *"--publish 7770:8731"* ]]
! grep -q '/models/npu' "$TEST_TMP/serve.log"
! grep -q 'HALOGEN_NPU_MODELS' "$TEST_TMP/serve.log"
! grep -q 'xilinx' "$TEST_TMP/serve.log"
! grep -q 'accel0' "$TEST_TMP/serve.log"
}
@test "the lane arms the accel node, the staged model dir and each XRT library twice" {
# Each library mounts at the image's XRT path AND at its own path: the
# distro libraries find each other where they were installed, and an
# /opt/xilinx/xrt of symlinks into the system dir dangles inside the
# container (the vendor calls that out by name).
podman_stub
good_env
npu_fixture
run_with_log
[ "$status" -eq 0 ]
[[ "$output" == *"--device $TEST_TMP/accel0"* ]]
[[ "$output" == *"-v $TEST_TMP/npu:/models/npu:ro"* ]]
[[ "$output" == *"-e HALOGEN_NPU_MODELS=qwen3-embedding-0.6b"* ]]
for l in libxrt_coreutil.so.2 libxrt_core.so.2 libxrt_driver_xdna.so.2; do
[[ "$output" == *"-v $TEST_TMP/xrt/$l:/opt/xilinx/xrt/lib/$l:ro"* ]]
[[ "$output" == *"-v $TEST_TMP/xrt/$l:$TEST_TMP/xrt/$l:ro"* ]]
done
# The Flash payload still rides the same container: the lane is additive.
[[ "$output" == *"-e HALOGEN_CHECKPOINT=/models/qwen.hgn"* ]]
}
@test "the lane refuses a model that is not staged and names the fetch script" {
# The engine cannot fetch these at boot here (/models is read-only and
# HALOGEN_DOWNLOAD stays unset), so an un-staged name must stop the unit
# before podman runs, with the command that fixes it.
podman_stub
good_env
npu_fixture
rm -rf "$TEST_TMP/npu/qwen3-embedding-0.6b"
run_with_log
[ "$status" -ne 0 ]
[[ "$output" == *"qwen3-embedding-0.6b"*"not staged"* ]]
[[ "$output" == *"download-frank-dragoon-npu.sh"* ]]
}
@test "the lane refuses without the accel node and says what to check" {
podman_stub
good_env
npu_fixture
rm "$TEST_TMP/accel0"
run_with_log
[ "$status" -ne 0 ]
[[ "$output" == *"$TEST_TMP/accel0 missing"* ]]
[[ "$output" == *"iommu=pt"* ]]
}
@test "the lane refuses when an XRT NPU library is missing" {
podman_stub
good_env
npu_fixture
rm "$TEST_TMP/xrt/libxrt_driver_xdna.so.2"
run_with_log
[ "$status" -ne 0 ]
[[ "$output" == *"libxrt_driver_xdna.so.2 missing"* ]]
}
@test "the lane refuses while the fabric clock is moving" {
# The engine itself refuses to start in this state (GPU+NPU co-run hangs
# Strix Halo while FCLK changes speed) but cannot see host sysfs from
# inside the container — the proof has to happen on the host side.
podman_stub
good_env
npu_fixture
printf 'auto\n' > "$TEST_TMP/sys/class/drm/card1/device/power_dpm_force_performance_level"
run_with_log
[ "$status" -ne 0 ]
[[ "$output" == *"not held"* ]]
[[ "$output" == *"sudo halogen-fabric-clock hold"* ]]
}
@test "the lane refuses when no fabric-clock control can be read" {
# A host whose sysfs exposes nothing cannot PROVE the hold, and a silent
# pass here is a boot that dies in the engine instead.
podman_stub
good_env
npu_fixture
rm -rf "$TEST_TMP/sys/class/drm"
run_with_log
[ "$status" -ne 0 ]
[[ "$output" == *"cannot prove"* ]]
}
@test "the lane also passes when the clock is held at the high level" {
# The vendor's status accepts `high` as well as `manual` + the top fclk
# level; refusing a correctly-held host would block a deploy for nothing.
podman_stub
good_env
npu_fixture
printf 'high\n' > "$TEST_TMP/sys/class/drm/card1/device/power_dpm_force_performance_level"
run_with_log
[ "$status" -eq 0 ]
[[ "$output" == *"-e HALOGEN_NPU_MODELS=qwen3-embedding-0.6b"* ]]
}
@test "the deployer gates the same lane artifacts and installs the fabric-clock unit" {
grep -qE 'LLM_DRAGOON_NPU_MODELS' "$DEPLOYER"
grep -qE 'install -m 0755 dankclaw/host/halogen-fabric-clock' "$DEPLOYER"
grep -qE 'install -m 0644 dankclaw/host/halogen-fabric-clock\.service' "$DEPLOYER"
grep -qE 'halogen-fabric-clock status' "$DEPLOYER"
grep -qE 'download-frank-dragoon-npu\.sh' "$DEPLOYER"
# The lane's own health gate is a real embedding, not a 200 on /health.
grep -qE '/v1/embeddings' "$DEPLOYER"
}
@test "the fabric-clock host unit is the vendor shape and boots on its own" {
[ -x "$HOST_UNIT" ]
grep -qE '^ (hold\)|release\)|status\))' "$HOST_UNIT"
grep -qE 'pp_dpm_fclk' "$HOST_UNIT"
grep -qE '^Type=oneshot$' "$HOST_SERVICE"
grep -qE '^WantedBy=.*multi-user\.target' "$HOST_SERVICE"
# A resume hands the clock back to the driver, so the unit must re-run then.
grep -qE '^WantedBy=.*suspend\.target' "$HOST_SERVICE"
grep -qE '^ExecStart=.*/usr/local/sbin/halogen-fabric-clock hold$' "$HOST_SERVICE"
}
@test "the NPU fetch script pins the vendor catalog by size and sha256" {
# XDNA2 runs only AIE-compiled programs, so the vendor's per-model repo is
# the only source; the engine re-checks size+sha256 against its own catalog
# at boot, so this manifest has to agree with it or the boot refuses.
[ -x "$NPU_FETCH" ]
grep -qE 'curl -fL' "$NPU_FETCH"
grep -qE 'qwen3-embedding-0\.6b\.hnpw 813032248 [0-9a-f]{64}' "$NPU_FETCH"
grep -qE 'sha256sum' "$NPU_FETCH"
# curl -f alone is not enough on Hugging Face: the resolve URL redirects
# and curl drops Range across it (the sibling 62 GiB fetch, 2026-10-18).
grep -qE 'url_effective' "$NPU_FETCH"
}