Frank
b19f1c49d6
The lane's first boot died at container init, not at the engine:
runc create failed: ... create mountpoint for /models/npu mount:
mkdirat .../merged/models/npu: read-only file system
The image ships no /models, and the serve script binds the payload dir
there read-only, so a nested `-v $NPU_DIR:/models/npu:ro` had to create
its own mountpoint underneath a read-only parent — impossible. The
engine's own entrypoint confirms the container path is fixed: it rejects
HALOGEN_NPU_DIR outright ("our NPU models live under /models/npu/<id>/").
Fix is the mount shape, not the engine: FRANK_DRAGOON_NPU_DIR now lives
inside FRANK_DRAGOON_MODEL_DIR, so the payload's single read-only mount
puts the files at /models/npu/<id>/ by itself and the nested bind is
gone. Verified on the running container: after the move, `podman exec
frank-dragoon-halogen ls /models/npu/qwen3-embedding-0.6b` answers with
the staged tree while the lane is still armed off.
Both gates refuse the old shape by name (serve + deployer parity) so a
future payload swap that moves the model dir cannot reintroduce a crash
that lands after the deployer has already stopped the running unit.
128 lines
7.8 KiB
Bash
128 lines
7.8 KiB
Bash
# frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
|
|
# Strix Halo iGPU (gfx1151), host port 7770.
|
|
#
|
|
# ONE OWNER: payload identity, shape, eras, laws and every measured number live
|
|
# in serverAppConfigs-docs/LLM/frank-dragoon.md. This file carries the knobs
|
|
# plus only the reason a reader editing THIS line cannot recover from the page.
|
|
# Where the two disagree, this file wins on values; the boot ledger arbitrates
|
|
# the memory figures.
|
|
#
|
|
# The .hgn format is the lock: it loads ONLY in the flash-server line, so the
|
|
# engine moves with the payload, never ahead of it.
|
|
# Image digest-pinned (a tag pull silently re-points; the digest is what was
|
|
# qualified — 0.16.2 is the version the model card names as its quickstart pin).
|
|
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
|
# Container identity kept from the 27B era: the unit's ExecStop names it.
|
|
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
|
|
# Predecessor identity the serve script clears if it is still holding the GPUs.
|
|
FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
|
|
# host:container — the frontend listens on 8731 inside; the engine's own port
|
|
# (8730) has no auth and is NEVER published.
|
|
FRANK_DRAGOON_PUBLISH=7770:8731
|
|
|
|
FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-v2-abliterated
|
|
# Same filename family, DIFFERENT artifact: this is the native v2 conversion,
|
|
# 32 bytes smaller than the writer transplant that used to carry the name.
|
|
# Never infer the payload from the filename — this size and sha256 are the gate.
|
|
FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-abliterated.hgn
|
|
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678432
|
|
FRANK_DRAGOON_CHECKPOINT_SHA256=b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
|
|
# The n-gram table is REQUIRED (the engine reads it beside the checkpoint) and
|
|
# disk-backed: mmap'd page cache, never pinned. It is a stock peonist companion
|
|
# hardlinked from the previous payload dir — a hardlink is a regular file, and
|
|
# the serve gate refuses symlinks — with the engine's own NIAH/PPL verdicts on
|
|
# the page as the referee.
|
|
FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
|
|
FRANK_DRAGOON_NGRAM_BYTES=51200246144
|
|
FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
|
# Vision tower: optional on the engine, ENABLED here (image input).
|
|
FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
|
|
FRANK_DRAGOON_VISION_BYTES=897916416
|
|
FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
|
# The tokenizer is a FLAT DIRECTORY ($FRANK_DRAGOON_MODEL_DIR/tokenizer) mounted
|
|
# whole. The serve gate and the deployer refuse a missing file or ANY symlink:
|
|
# an HF cache snapshot symlinks each file into blobs/ and those links dangle
|
|
# inside the container mount.
|
|
FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
|
|
|
|
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
|
# stays at the image default, which is what was qualified) ------------------
|
|
# Served id. OpenWebUI api_configs model_ids for the :7770 backend MUST name it.
|
|
HALOGEN_MODEL_ID=frank-dragoon
|
|
# PER-REQUEST ceiling: a longer prompt is a 400 naming the limit, never a
|
|
# truncation. The harness window must sit ~45k UNDER this (the 400-zone law on
|
|
# the page) — a window equal to the ceiling makes every long request a 400.
|
|
HALOGEN_CTX=200704
|
|
# ONE pool behind both slots, sized so BOTH can carry a maximum-length request
|
|
# at once. Pool positions buy concurrency; the per-request window never moves
|
|
# with them. Legal band: ctx <= pool <= slots * ctx (the seam enforces it), and
|
|
# RAM the pool takes is RAM the n-gram table's page cache loses.
|
|
HALOGEN_KV_POOL_POSITIONS=401408
|
|
# Two concurrent conversations: the MTP drafter is not slot-locked on this
|
|
# engine, so the second slot is capacity, not a speculation tax.
|
|
HALOGEN_KV_SLOTS=2
|
|
HALOGEN_HOST_RESERVE_GIB=20
|
|
# The MTP head is INSIDE the checkpoint — do NOT set HALOGEN_MTP_HEAD.
|
|
# 0 is mandatory: the adaptive default disarms the head for whole requests after
|
|
# low acceptance and halves decode. Depth 2 is the card's best-all-round default.
|
|
HALOGEN_SPEC_ADAPT=0
|
|
# Coupled trio: MAX_TOK is only the default for requests that omit max_tokens;
|
|
# the cap bounds how long ONE request holds the GPU and the queue timeout bounds
|
|
# how long the NEXT client waits. Move all three together or none.
|
|
HALOGEN_MAX_TOK=8192
|
|
HALOGEN_MAX_TOKENS_CAP=65536
|
|
HALOGEN_QUEUE_TIMEOUT=7200
|
|
# The engine has a server-side sampler (an absent temperature is NOT greedy
|
|
# here); models.yml extraBody carries the same recipe.
|
|
HALOGEN_TEMPERATURE=1.0
|
|
HALOGEN_TOP_P=0.95
|
|
HALOGEN_TOP_K=20
|
|
# Weights stay pinned: the engine pins by default and only the retired w4b build
|
|
# honored HALOGEN_FLASH_PIN_TRUNK=0, which this payload rejects.
|
|
HALOGEN_WEIGHTS_LOCK=1
|
|
|
|
# --- NPU lane (XDNA2) -------------------------------------------------------
|
|
# HALOGEN_NPU_MODELS is the whole switch: EMPTY or unset = the endpoint exactly
|
|
# as it ran before this lane existed (no accel device, no XRT mount, no
|
|
# fabric-clock dependency, no /v1/embeddings). Rolling the lane back is blanking
|
|
# this one line and re-deploying — nothing else moves.
|
|
# Names are the vendor catalog's folder names (`/opt/halogen/npu/models.txt`
|
|
# inside the image) and the engine reads a model ONLY at /models/npu/<name>/ —
|
|
# HALOGEN_NPU_DIR is gone from the image and setting it is a boot refusal.
|
|
# The engine cannot fetch anything here: /models is read-only and
|
|
# HALOGEN_DOWNLOAD stays unset so the container opens no outbound connection at
|
|
# boot, so scripts/download-frank-dragoon-npu.sh must stage the files first
|
|
# (the serve gate and the deployer both refuse by name when a named model is
|
|
# missing). FRANK_DRAGOON_NPU_DIR therefore lives INSIDE the payload dir: the
|
|
# payload mount is the container's only handle on /models, and runc cannot
|
|
# mkdir a nested /models/npu mountpoint through a read-only bind — the lane's
|
|
# first boot died on precisely that (2026-10-19). Swap the payload dir and this
|
|
# path moves with it.
|
|
# Cost: ~1.5 GiB extra pinned host RAM per named model, and a hard dependency on
|
|
# the host holding the GPU fabric clock — on Strix Halo the iGPU and the NPU
|
|
# together hang the machine (and corrupt NPU results) while FCLK changes speed,
|
|
# so halogen refuses to start with this variable set unless the clock is held
|
|
# (dankclaw/host/halogen-fabric-clock; the deployer installs and gates it).
|
|
HALOGEN_NPU_MODELS=qwen3-embedding-0.6b
|
|
FRANK_DRAGOON_NPU_DIR=/home/dank/models/halogen-v2-abliterated/npu
|
|
# Node paths and the XRT directory are variables so the bats seam can drive
|
|
# these gates on a box with no NPU. Ubuntu's distro XRT (libxrt2 + libxrt-npu2)
|
|
# is mounted FILE BY FILE and TWICE EACH — once where the image looks for XRT,
|
|
# once at its own path, because those libraries find each other at their install
|
|
# path; the vendor forbids mounting an /opt/xilinx/xrt of symlinks into the
|
|
# system dir, which dangle inside the container.
|
|
FRANK_DRAGOON_DEV_ACCEL=/dev/accel/accel0
|
|
FRANK_DRAGOON_XRT_DIR=/usr/lib/x86_64-linux-gnu
|
|
# sysfs root the fabric-clock preflight reads (same knob the vendor's
|
|
# halogen-fabric-clock status reads; the seam points it at a fixture).
|
|
FRANK_DRAGOON_SYSFS=/sys
|
|
# Container network. Empty = podman's rootless default (`pasta`): a private
|
|
# netns that nothing else can join — podman refuses a second network while the
|
|
# primary is pasta, the published port is refused from dankclaw_default
|
|
# containers, and host.containers.internal resolves link-local, which
|
|
# open-notebook's URL guard refuses to store. Named = the app bridge, so
|
|
# open-notebook can dial http://frank-dragoon-halogen:8731 for the NPU
|
|
# endpoints. The engine's unauthenticated 8730 stays private either way (it
|
|
# binds the container's own loopback); only the API port becomes name-resolvable.
|
|
FRANK_DRAGOON_NETWORK=dankclaw_default
|