Frankandomp
53ef121180
halogen-flash-server already ships an NPU path on the pin we run: for each
model named in HALOGEN_NPU_MODELS it loads the vendor's precompiled AIE
programs (XDNA2 executes only AIE-compiled graphs) and answers them on
/v1/embeddings inside the SAME container behind port 7770. That gives the
fleet an in-house embedding endpoint without touching the iGPU trunk.
The lane is opt-in and one line wide: HALOGEN_NPU_MODELS blank is the
pre-lane endpoint byte for byte (no device, no mounts, no engine env).
The three host preconditions the vendor names are gated in the serve script
and mirrored in the deployer (the 2026-10-01 twin law), each refused by name:
- the staged model files. /models is read-only and HALOGEN_DOWNLOAD stays
unset so the container opens no outbound connection at boot, so
scripts/download-frank-dragoon-npu.sh stages them from the pinned
revision with the catalog's own byte counts and sha256s (the engine
re-checks both at boot);
- the distro XRT's libxrt_coreutil/libxrt_core/libxrt_driver_xdna, each
mounted twice — where the image looks for XRT and at its own path —
because those libraries find each other where they were installed and
an /opt/xilinx/xrt of symlinks dangles inside the container;
- the GPU fabric clock HELD. On Strix Halo the iGPU and the NPU co-running
hang the machine (and corrupt NPU results) while FCLK changes speed, and
halogen refuses to start in that state; it cannot read host sysfs from
inside, so the hold is proven host-side. dankclaw/host/ carries the
vendor's halogen-fabric-clock{,.service} verbatim at tag v0.16.2 (sha256
a3a4f7f4e0142e71… / 5f9f1cc4c85a1c02…); the deployer installs, enables
and gates it only while the lane is named, and releases the clock plus
drops the boot hook when it is not.
After health, the deployer's lane gate is a real embedding through
/v1/embeddings — a 200 on /health proves nothing about the NPU programs.
Planner: dankclaw/host/halogen-fabric-clock* re-dispatches frank-dragoon
(the deployer installs it); dankclaw/host/grub and the fetch script plan
nothing.
Seam: 11 new tests — the off arm really renders nothing lane-specific, the
on arm renders the device/mounts/env, every missing precondition refuses by
name, the deployer twins and the host unit shape are pinned.
Co-Authored-By: omp <omp@dankclaw>
113 lines
6.8 KiB
Bash
113 lines
6.8 KiB
Bash
# frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
|
|
# Strix Halo iGPU (gfx1151), host port 7770.
|
|
#
|
|
# ONE OWNER: payload identity, shape, eras, laws and every measured number live
|
|
# in serverAppConfigs-docs/LLM/frank-dragoon.md. This file carries the knobs
|
|
# plus only the reason a reader editing THIS line cannot recover from the page.
|
|
# Where the two disagree, this file wins on values; the boot ledger arbitrates
|
|
# the memory figures.
|
|
#
|
|
# The .hgn format is the lock: it loads ONLY in the flash-server line, so the
|
|
# engine moves with the payload, never ahead of it.
|
|
# Image digest-pinned (a tag pull silently re-points; the digest is what was
|
|
# qualified — 0.16.2 is the version the model card names as its quickstart pin).
|
|
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
|
# Container identity kept from the 27B era: the unit's ExecStop names it.
|
|
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
|
|
# Predecessor identity the serve script clears if it is still holding the GPUs.
|
|
FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
|
|
# host:container — the frontend listens on 8731 inside; the engine's own port
|
|
# (8730) has no auth and is NEVER published.
|
|
FRANK_DRAGOON_PUBLISH=7770:8731
|
|
|
|
FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-v2-abliterated
|
|
# Same filename family, DIFFERENT artifact: this is the native v2 conversion,
|
|
# 32 bytes smaller than the writer transplant that used to carry the name.
|
|
# Never infer the payload from the filename — this size and sha256 are the gate.
|
|
FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-abliterated.hgn
|
|
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678432
|
|
FRANK_DRAGOON_CHECKPOINT_SHA256=b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
|
|
# The n-gram table is REQUIRED (the engine reads it beside the checkpoint) and
|
|
# disk-backed: mmap'd page cache, never pinned. It is a stock peonist companion
|
|
# hardlinked from the previous payload dir — a hardlink is a regular file, and
|
|
# the serve gate refuses symlinks — with the engine's own NIAH/PPL verdicts on
|
|
# the page as the referee.
|
|
FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
|
|
FRANK_DRAGOON_NGRAM_BYTES=51200246144
|
|
FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
|
# Vision tower: optional on the engine, ENABLED here (image input).
|
|
FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
|
|
FRANK_DRAGOON_VISION_BYTES=897916416
|
|
FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
|
# The tokenizer is a FLAT DIRECTORY ($FRANK_DRAGOON_MODEL_DIR/tokenizer) mounted
|
|
# whole. The serve gate and the deployer refuse a missing file or ANY symlink:
|
|
# an HF cache snapshot symlinks each file into blobs/ and those links dangle
|
|
# inside the container mount.
|
|
FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
|
|
|
|
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
|
# stays at the image default, which is what was qualified) ------------------
|
|
# Served id. OpenWebUI api_configs model_ids for the :7770 backend MUST name it.
|
|
HALOGEN_MODEL_ID=frank-dragoon
|
|
# PER-REQUEST ceiling: a longer prompt is a 400 naming the limit, never a
|
|
# truncation. The harness window must sit ~45k UNDER this (the 400-zone law on
|
|
# the page) — a window equal to the ceiling makes every long request a 400.
|
|
HALOGEN_CTX=200704
|
|
# ONE pool behind both slots, sized so BOTH can carry a maximum-length request
|
|
# at once. Pool positions buy concurrency; the per-request window never moves
|
|
# with them. Legal band: ctx <= pool <= slots * ctx (the seam enforces it), and
|
|
# RAM the pool takes is RAM the n-gram table's page cache loses.
|
|
HALOGEN_KV_POOL_POSITIONS=401408
|
|
# Two concurrent conversations: the MTP drafter is not slot-locked on this
|
|
# engine, so the second slot is capacity, not a speculation tax.
|
|
HALOGEN_KV_SLOTS=2
|
|
HALOGEN_HOST_RESERVE_GIB=20
|
|
# The MTP head is INSIDE the checkpoint — do NOT set HALOGEN_MTP_HEAD.
|
|
# 0 is mandatory: the adaptive default disarms the head for whole requests after
|
|
# low acceptance and halves decode. Depth 2 is the card's best-all-round default.
|
|
HALOGEN_SPEC_ADAPT=0
|
|
# Coupled trio: MAX_TOK is only the default for requests that omit max_tokens;
|
|
# the cap bounds how long ONE request holds the GPU and the queue timeout bounds
|
|
# how long the NEXT client waits. Move all three together or none.
|
|
HALOGEN_MAX_TOK=8192
|
|
HALOGEN_MAX_TOKENS_CAP=65536
|
|
HALOGEN_QUEUE_TIMEOUT=7200
|
|
# The engine has a server-side sampler (an absent temperature is NOT greedy
|
|
# here); models.yml extraBody carries the same recipe.
|
|
HALOGEN_TEMPERATURE=1.0
|
|
HALOGEN_TOP_P=0.95
|
|
HALOGEN_TOP_K=20
|
|
# Weights stay pinned: the engine pins by default and only the retired w4b build
|
|
# honored HALOGEN_FLASH_PIN_TRUNK=0, which this payload rejects.
|
|
HALOGEN_WEIGHTS_LOCK=1
|
|
|
|
# --- NPU lane (XDNA2) -------------------------------------------------------
|
|
# HALOGEN_NPU_MODELS is the whole switch: EMPTY or unset = the endpoint exactly
|
|
# as it ran before this lane existed (no accel device, no XRT mount, no
|
|
# fabric-clock dependency, no /v1/embeddings). Rolling the lane back is blanking
|
|
# this one line and re-deploying — nothing else moves.
|
|
# Names are the vendor catalog's folder names (`/opt/halogen/npu/models.txt`
|
|
# inside the image). The engine cannot fetch them here: /models is read-only and
|
|
# HALOGEN_DOWNLOAD stays unset so the container opens no outbound connection at
|
|
# boot, so scripts/download-frank-dragoon-npu.sh must stage them into
|
|
# $FRANK_DRAGOON_NPU_DIR first (the serve gate and the deployer both refuse by
|
|
# name when a named model is not staged).
|
|
# Cost: ~1.5 GiB extra pinned host RAM per named model, and a hard dependency on
|
|
# the host holding the GPU fabric clock — on Strix Halo the iGPU and the NPU
|
|
# together hang the machine (and corrupt NPU results) while FCLK changes speed,
|
|
# so halogen refuses to start with this variable set unless the clock is held
|
|
# (dankclaw/host/halogen-fabric-clock; the deployer installs and gates it).
|
|
HALOGEN_NPU_MODELS=qwen3-embedding-0.6b
|
|
FRANK_DRAGOON_NPU_DIR=/home/dank/models/halogen-npu
|
|
# Node paths and the XRT directory are variables so the bats seam can drive
|
|
# these gates on a box with no NPU. Ubuntu's distro XRT (libxrt2 + libxrt-npu2)
|
|
# is mounted FILE BY FILE and TWICE EACH — once where the image looks for XRT,
|
|
# once at its own path, because those libraries find each other at their install
|
|
# path; the vendor forbids mounting an /opt/xilinx/xrt of symlinks into the
|
|
# system dir, which dangle inside the container.
|
|
FRANK_DRAGOON_DEV_ACCEL=/dev/accel/accel0
|
|
FRANK_DRAGOON_XRT_DIR=/usr/lib/x86_64-linux-gnu
|
|
# sysfs root the fabric-clock preflight reads (same knob the vendor's
|
|
# halogen-fabric-clock status reads; the seam points it at a fixture).
|
|
FRANK_DRAGOON_SYSFS=/sys
|