Frank
af5ddfc54c
Every KEY=value is byte-identical to the previous revision (diffed against git show HEAD: -- 28/25/20 keys). What left is the per-swap narration: the archon header drops 207 lines of era prose, dragoon 127, zealot 104. Each header now keeps the format lock, the digest/identity law, and one line per knob explaining what a reader editing that line cannot recover anywhere. The facts moved to their one owner: LLM/frank-archon.md, LLM/frank-dragoon.md, LLM/frank-zealot.md. One stale claim dies with the trim -- the dragoon header still said the KV pool is ONE 200,704-position budget behind 2 slots, which contradicted its own 401,408 paragraph; the page carries the correct ledger. Also kills two wrong arithmetic claims in the zealot header: the q8_0-V and CTX_SIZE=401408 rungs each free 3,840 MiB at the 524,288 pool, not 8,192 and not ~6.7 GiB.
83 lines
4.8 KiB
Bash
83 lines
4.8 KiB
Bash
# frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
|
|
# Strix Halo iGPU (gfx1151), host port 7770.
|
|
#
|
|
# ONE OWNER: payload identity, shape, eras, laws and every measured number live
|
|
# in serverAppConfigs-docs/LLM/frank-dragoon.md. This file carries the knobs
|
|
# plus only the reason a reader editing THIS line cannot recover from the page.
|
|
# Where the two disagree, this file wins on values; the boot ledger arbitrates
|
|
# the memory figures.
|
|
#
|
|
# The .hgn format is the lock: it loads ONLY in the flash-server line, so the
|
|
# engine moves with the payload, never ahead of it.
|
|
# Image digest-pinned (a tag pull silently re-points; the digest is what was
|
|
# qualified — 0.16.2 is the version the model card names as its quickstart pin).
|
|
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
|
# Container identity kept from the 27B era: the unit's ExecStop names it.
|
|
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
|
|
# Predecessor identity the serve script clears if it is still holding the GPUs.
|
|
FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
|
|
# host:container — the frontend listens on 8731 inside; the engine's own port
|
|
# (8730) has no auth and is NEVER published.
|
|
FRANK_DRAGOON_PUBLISH=7770:8731
|
|
|
|
FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-v2-abliterated
|
|
# Same filename family, DIFFERENT artifact: this is the native v2 conversion,
|
|
# 32 bytes smaller than the writer transplant that used to carry the name.
|
|
# Never infer the payload from the filename — this size and sha256 are the gate.
|
|
FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-abliterated.hgn
|
|
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678432
|
|
FRANK_DRAGOON_CHECKPOINT_SHA256=b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
|
|
# The n-gram table is REQUIRED (the engine reads it beside the checkpoint) and
|
|
# disk-backed: mmap'd page cache, never pinned. It is a stock peonist companion
|
|
# hardlinked from the previous payload dir — a hardlink is a regular file, and
|
|
# the serve gate refuses symlinks — with the engine's own NIAH/PPL verdicts on
|
|
# the page as the referee.
|
|
FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
|
|
FRANK_DRAGOON_NGRAM_BYTES=51200246144
|
|
FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
|
# Vision tower: optional on the engine, ENABLED here (image input).
|
|
FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
|
|
FRANK_DRAGOON_VISION_BYTES=897916416
|
|
FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
|
# The tokenizer is a FLAT DIRECTORY ($FRANK_DRAGOON_MODEL_DIR/tokenizer) mounted
|
|
# whole. The serve gate and the deployer refuse a missing file or ANY symlink:
|
|
# an HF cache snapshot symlinks each file into blobs/ and those links dangle
|
|
# inside the container mount.
|
|
FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
|
|
|
|
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
|
# stays at the image default, which is what was qualified) ------------------
|
|
# Served id. OpenWebUI api_configs model_ids for the :7770 backend MUST name it.
|
|
HALOGEN_MODEL_ID=frank-dragoon
|
|
# PER-REQUEST ceiling: a longer prompt is a 400 naming the limit, never a
|
|
# truncation. The harness window must sit ~45k UNDER this (the 400-zone law on
|
|
# the page) — a window equal to the ceiling makes every long request a 400.
|
|
HALOGEN_CTX=200704
|
|
# ONE pool behind both slots, sized so BOTH can carry a maximum-length request
|
|
# at once. Pool positions buy concurrency; the per-request window never moves
|
|
# with them. Legal band: ctx <= pool <= slots * ctx (the seam enforces it), and
|
|
# RAM the pool takes is RAM the n-gram table's page cache loses.
|
|
HALOGEN_KV_POOL_POSITIONS=401408
|
|
# Two concurrent conversations: the MTP drafter is not slot-locked on this
|
|
# engine, so the second slot is capacity, not a speculation tax.
|
|
HALOGEN_KV_SLOTS=2
|
|
HALOGEN_HOST_RESERVE_GIB=20
|
|
# The MTP head is INSIDE the checkpoint — do NOT set HALOGEN_MTP_HEAD.
|
|
# 0 is mandatory: the adaptive default disarms the head for whole requests after
|
|
# low acceptance and halves decode. Depth 2 is the card's best-all-round default.
|
|
HALOGEN_SPEC_ADAPT=0
|
|
# Coupled trio: MAX_TOK is only the default for requests that omit max_tokens;
|
|
# the cap bounds how long ONE request holds the GPU and the queue timeout bounds
|
|
# how long the NEXT client waits. Move all three together or none.
|
|
HALOGEN_MAX_TOK=8192
|
|
HALOGEN_MAX_TOKENS_CAP=65536
|
|
HALOGEN_QUEUE_TIMEOUT=7200
|
|
# The engine has a server-side sampler (an absent temperature is NOT greedy
|
|
# here); models.yml extraBody carries the same recipe.
|
|
HALOGEN_TEMPERATURE=1.0
|
|
HALOGEN_TOP_P=0.95
|
|
HALOGEN_TOP_K=20
|
|
# Weights stay pinned: the engine pins by default and only the retired w4b build
|
|
# honored HALOGEN_FLASH_PIN_TRUNK=0, which this payload rejects.
|
|
HALOGEN_WEIGHTS_LOCK=1
|