Frank
c6e2887f94
The lane's child dlopens libxrt_core.so.2 from /opt/xilinx/lib/<multiarch>/
as well; our mounts filled only /opt/xilinx/xrt/lib and the install path.
Since the host's libxrt2 was reconfigured 2026-10-20 the child dies naming
the missing prefix ('No such library
/opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2') and the engine
supervisor takes the WHOLE container down on a dead lane — measured
2026-10-21: frank-dragoon boot-looped 88 times with chat dead while the
GPU lines looked healthy.
Each XRT library now mounts THREE times from the same real file (never a
symlink); the multiarch name derives from FRANK_DRAGOON_XRT_DIR's basename
so the bats fixture proves the derivation. Proven end-to-end before commit:
production-shaped scratch boot + the three extra mounts answers the full
boot ledger, npu_api buckets u0..u9, and /health ok at the 262144 ceiling.
A dead lane is not survivable on this engine — arm it only with all three
prefixes (page law updated).
138 lines
8.5 KiB
Bash
138 lines
8.5 KiB
Bash
# frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
|
|
# Strix Halo iGPU (gfx1151), host port 7770.
|
|
#
|
|
# ONE OWNER: payload identity, shape, eras, laws and every measured number live
|
|
# in serverAppConfigs-docs/LLM/frank-dragoon.md. This file carries the knobs
|
|
# plus only the reason a reader editing THIS line cannot recover from the page.
|
|
# Where the two disagree, this file wins on values; the boot ledger arbitrates
|
|
# the memory figures.
|
|
#
|
|
# The .hgn format is the lock: it loads ONLY in the flash-server line, so the
|
|
# engine moves with the payload, never ahead of it.
|
|
# Image digest-pinned (a tag pull silently re-points; the digest is what was
|
|
# qualified — 0.16.2 is the version the model card names as its quickstart pin).
|
|
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
|
# Container identity kept from the 27B era: the unit's ExecStop names it.
|
|
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
|
|
# Predecessor identity the serve script clears if it is still holding the GPUs.
|
|
FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
|
|
# host:container — the frontend listens on 8731 inside; the engine's own port
|
|
# (8730) has no auth and is NEVER published.
|
|
FRANK_DRAGOON_PUBLISH=7770:8731
|
|
|
|
FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-v2-abliterated
|
|
# Same filename family, DIFFERENT artifact: this is the native v2 conversion,
|
|
# 32 bytes smaller than the writer transplant that used to carry the name.
|
|
# Never infer the payload from the filename — this size and sha256 are the gate.
|
|
FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-abliterated.hgn
|
|
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678432
|
|
FRANK_DRAGOON_CHECKPOINT_SHA256=b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
|
|
# The n-gram table is REQUIRED (the engine reads it beside the checkpoint) and
|
|
# disk-backed: mmap'd page cache, never pinned. It is a stock peonist companion
|
|
# hardlinked from the previous payload dir — a hardlink is a regular file, and
|
|
# the serve gate refuses symlinks — with the engine's own NIAH/PPL verdicts on
|
|
# the page as the referee.
|
|
FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
|
|
FRANK_DRAGOON_NGRAM_BYTES=51200246144
|
|
FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
|
# Vision tower: optional on the engine, ENABLED here (image input).
|
|
FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
|
|
FRANK_DRAGOON_VISION_BYTES=897916416
|
|
FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
|
# The tokenizer is a FLAT DIRECTORY ($FRANK_DRAGOON_MODEL_DIR/tokenizer) mounted
|
|
# whole. The serve gate and the deployer refuse a missing file or ANY symlink:
|
|
# an HF cache snapshot symlinks each file into blobs/ and those links dangle
|
|
# inside the container mount.
|
|
FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
|
|
|
|
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
|
# stays at the image default, which is what was qualified) ------------------
|
|
# Served id. OpenWebUI api_configs model_ids for the :7770 backend MUST name it.
|
|
HALOGEN_MODEL_ID=frank-dragoon
|
|
# PER-REQUEST ceiling, set to the checkpoint's NATIVE window (262,144 — a
|
|
# longer prompt is a 400 naming the limit, never a truncation; stretching
|
|
# past native would need HALOGEN_ROPE_YARN and is not on plan). The harness
|
|
# `contextWindow` must sit ~45k under THIS number (the 400-zone law on the
|
|
# page): with the window at 200,000 the gap is 62,144, which absorbs the
|
|
# chars/4 under-count. A window at or near the ceiling makes every long
|
|
# request a guaranteed 400.
|
|
HALOGEN_CTX=262144
|
|
# ONE pool behind both slots. The harness window is 200,000, so 401,408
|
|
# still carries TWO maximum harness-length conversations at once
|
|
# (2 x 200,000 + one 45,056 budget <= pool + cap headroom); two literal
|
|
# ceiling-length requests would queue instead — buying those means raising
|
|
# the pool, and RAM the pool takes is RAM the n-gram table's page cache
|
|
# loses. Pool positions buy concurrency; the per-request window never moves
|
|
# with them. Legal band: ctx <= pool <= slots * ctx (the seam enforces it).
|
|
HALOGEN_KV_POOL_POSITIONS=401408
|
|
# Two concurrent conversations: the MTP drafter is not slot-locked on this
|
|
# engine, so the second slot is capacity, not a speculation tax.
|
|
HALOGEN_KV_SLOTS=2
|
|
HALOGEN_HOST_RESERVE_GIB=20
|
|
# The MTP head is INSIDE the checkpoint — do NOT set HALOGEN_MTP_HEAD.
|
|
# 0 is mandatory: the adaptive default disarms the head for whole requests after
|
|
# low acceptance and halves decode. Depth 2 is the card's best-all-round default.
|
|
HALOGEN_SPEC_ADAPT=0
|
|
# Coupled trio: MAX_TOK is only the default for requests that omit max_tokens;
|
|
# the cap bounds how long ONE request holds the GPU and the queue timeout bounds
|
|
# how long the NEXT client waits. Move all three together or none.
|
|
HALOGEN_MAX_TOK=8192
|
|
HALOGEN_MAX_TOKENS_CAP=65536
|
|
HALOGEN_QUEUE_TIMEOUT=7200
|
|
# The engine has a server-side sampler (an absent temperature is NOT greedy
|
|
# here); models.yml extraBody carries the same recipe.
|
|
HALOGEN_TEMPERATURE=1.0
|
|
HALOGEN_TOP_P=0.95
|
|
HALOGEN_TOP_K=20
|
|
# Weights stay pinned: the engine pins by default and only the retired w4b build
|
|
# honored HALOGEN_FLASH_PIN_TRUNK=0, which this payload rejects.
|
|
HALOGEN_WEIGHTS_LOCK=1
|
|
|
|
# --- NPU lane (XDNA2) -------------------------------------------------------
|
|
# HALOGEN_NPU_MODELS is the whole switch: EMPTY or unset = the endpoint exactly
|
|
# as it ran before this lane existed (no accel device, no XRT mount, no
|
|
# fabric-clock dependency, no /v1/embeddings). Rolling the lane back is blanking
|
|
# this one line and re-deploying — nothing else moves.
|
|
# Names are the vendor catalog's folder names (`/opt/halogen/npu/models.txt`
|
|
# inside the image) and the engine reads a model ONLY at /models/npu/<name>/ —
|
|
# HALOGEN_NPU_DIR is gone from the image and setting it is a boot refusal.
|
|
# The engine cannot fetch anything here: /models is read-only and
|
|
# HALOGEN_DOWNLOAD stays unset so the container opens no outbound connection at
|
|
# boot, so scripts/download-frank-dragoon-npu.sh must stage the files first
|
|
# (the serve gate and the deployer both refuse by name when a named model is
|
|
# missing). FRANK_DRAGOON_NPU_DIR therefore lives INSIDE the payload dir: the
|
|
# payload mount is the container's only handle on /models, and runc cannot
|
|
# mkdir a nested /models/npu mountpoint through a read-only bind — the lane's
|
|
# first boot died on precisely that (2026-10-19). Swap the payload dir and this
|
|
# path moves with it.
|
|
# Cost: ~1.5 GiB extra pinned host RAM per named model, and a hard dependency on
|
|
# the host holding the GPU fabric clock — on Strix Halo the iGPU and the NPU
|
|
# together hang the machine (and corrupt NPU results) while FCLK changes speed,
|
|
# so halogen refuses to start with this variable set unless the clock is held
|
|
# (dankclaw/host/halogen-fabric-clock; the deployer installs and gates it).
|
|
HALOGEN_NPU_MODELS=qwen3-embedding-0.6b
|
|
FRANK_DRAGOON_NPU_DIR=/home/dank/models/halogen-v2-abliterated/npu
|
|
# Node paths and the XRT directory are variables so the bats seam can drive
|
|
# these gates on a box with no NPU. Ubuntu's distro XRT (libxrt2 + libxrt-npu2)
|
|
# is mounted FILE BY FILE and THREE TIMES EACH — where the image looks for XRT
|
|
# (/opt/xilinx/xrt/lib), at its own path (those libraries find each other at
|
|
# their install path), and at /opt/xilinx/lib/<multiarch>/ (the NPU child
|
|
# searches that third prefix; without it the lane dies naming it and the
|
|
# supervisor takes the container down — 2026-10-21 boot loop). The basename of
|
|
# this directory IS the multiarch name. The vendor forbids mounting an
|
|
# /opt/xilinx/xrt of symlinks into the system dir, which dangle in the container.
|
|
FRANK_DRAGOON_DEV_ACCEL=/dev/accel/accel0
|
|
FRANK_DRAGOON_XRT_DIR=/usr/lib/x86_64-linux-gnu
|
|
# sysfs root the fabric-clock preflight reads (same knob the vendor's
|
|
# halogen-fabric-clock status reads; the seam points it at a fixture).
|
|
FRANK_DRAGOON_SYSFS=/sys
|
|
# Container network. Empty = podman's rootless default (`pasta`): a private
|
|
# netns that nothing else can join — podman refuses a second network while the
|
|
# primary is pasta, the published port is refused from dankclaw_default
|
|
# containers, and host.containers.internal resolves link-local, which
|
|
# open-notebook's URL guard refuses to store. Named = the app bridge, so
|
|
# open-notebook can dial http://frank-dragoon-halogen:8731 for the NPU
|
|
# endpoints. The engine's unauthenticated 8730 stays private either way (it
|
|
# binds the container's own loopback); only the API port becomes name-resolvable.
|
|
FRANK_DRAGOON_NETWORK=dankclaw_default
|