chore(scripts): env files carry knobs plus the reason, not the swap ledger
Every KEY=value is byte-identical to the previous revision (diffed against git show HEAD: -- 28/25/20 keys). What left is the per-swap narration: the archon header drops 207 lines of era prose, dragoon 127, zealot 104. Each header now keeps the format lock, the digest/identity law, and one line per knob explaining what a reader editing that line cannot recover anywhere. The facts moved to their one owner: LLM/frank-archon.md, LLM/frank-dragoon.md, LLM/frank-zealot.md. One stale claim dies with the trim -- the dragoon header still said the KV pool is ONE 200,704-position budget behind 2 slots, which contradicted its own 401,408 paragraph; the page carries the correct ledger. Also kills two wrong arithmetic claims in the zealot header: the q8_0-V and CTX_SIZE=401408 rungs each free 3,840 MiB at the 524,288 pool, not 8,192 and not ~6.7 GiB.
This commit is contained in:
1 parent
61011b0a81
commit
af5ddfc54c
3 files changed
+148
-586
No files matched your search
+45
-172
@@ -1,209 +1,82 @@
|
||||
# frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
|
||||
# Strix Halo iGPU (gfx1151), host port 7770.
|
||||
#
|
||||
# RUNTIME (2026-10-17, Frank): halogen-flash-server 0.16.2 — the
|
||||
# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
|
||||
# digest-pinned below) serving Quat3rnion/halogen-qwen3.8-flash-next-v2-
|
||||
# abliterated — the NATIVE v2 conversion (2026-10-11) of orcarouter's pinned
|
||||
# BF16 Qwen3.8-Flash-Next, abliterated — as a podman container under a root
|
||||
# unit, driven by scripts/serve-frank-dragoon.sh. This CUTS OVER the
|
||||
# 2026-10-06 Swift1.5-finetune shape; rollback = revert the swap commit and
|
||||
# redeploy: the swift15 payload stays on disk at
|
||||
# /home/dank/models/halogen-flashnext-v2/, the 27B-era payload beside it.
|
||||
# ONE OWNER: payload identity, shape, eras, laws and every measured number live
|
||||
# in serverAppConfigs-docs/LLM/frank-dragoon.md. This file carries the knobs
|
||||
# plus only the reason a reader editing THIS line cannot recover from the page.
|
||||
# Where the two disagree, this file wins on values; the boot ledger arbitrates
|
||||
# the memory figures.
|
||||
#
|
||||
# MEASURED A/B, both boots on this box, identical battery traffic (engine
|
||||
# tools with the service stopped, behavior probes against it, 2026-10-17):
|
||||
# NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls and the
|
||||
# red-PNG vision probe identical, needle hit at both depths of a 21.5k-token
|
||||
# haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s,
|
||||
# wikitext-2 PPL 3.968 vs 3.877 (the finetune's edge is gone: +2.3 %) and
|
||||
# thinking tokens 1,476 vs 784 mean — the finetune compressed reasoning, the
|
||||
# base thinks ~1.9x longer per answer.
|
||||
#
|
||||
# The A/B ran at the card's ctx/pool of 262,144. Afterwards Frank lowered ctx to
|
||||
# 200,704 (RAM + the harness's compaction budget) and, on 2026-10-18, raised the
|
||||
# pool to 2 x ctx = 401,408 so BOTH slots can carry a full-length request at
|
||||
# once; the rates above are the ceiling-262,144 record and the production ledger
|
||||
# is the ctx-200,704 one.
|
||||
#
|
||||
# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
|
||||
# card it "will not load in transformers, vLLM, or llama.cpp" — and the v2
|
||||
# .hgn loads ONLY in the flash-server line (the old halogen engine is for
|
||||
# the 27B p1w4d file; the v2 quickstart pins flash-server). What the move
|
||||
# buys (card + live probe evidence): MTP spec decode (53-57 t/s greedy,
|
||||
# 3-10x serial), ~1,640 t/s prefill at 34k, ~72% fewer thinking tokens than
|
||||
# the censored 27B (905 mean), the ABLITERATED base (refusals should NOT
|
||||
# return), IMAGE INPUT (the v2 vision tower loads beside the checkpoint —
|
||||
# red-PNG probe verified on the card; the 27B engine ignored its encoder),
|
||||
# tool calls verified, and the native 262,144 context — served at 200,704 by
|
||||
# choice from 2026-10-17 (see SHAPE: the ceiling is a RAM decision, not a
|
||||
# capability limit).
|
||||
#
|
||||
# SHAPE — the knobs that are NOT free, with the arithmetic:
|
||||
# HALOGEN_CTX=200704 is the PER-REQUEST ceiling and the per-conversation
|
||||
# hard limit — a prompt over it is a 400 naming the limit, never a silent
|
||||
# truncation. LOWERED from 262,144 on 2026-10-17 at Frank's direction (the
|
||||
# checkpoint's native window is 262,144 and the engine serves anything
|
||||
# below it): every position costs KV pool RAM and shrinks the prompt-cache
|
||||
# entry (111 MiB at 262,144, ~85 MiB here). The 400-zone law is INVARIANT
|
||||
# to the value — it says the harness window must sit ~45k UNDER the engine
|
||||
# ceiling, because the harness clamps max_tokens to contextWindow − est −
|
||||
# 4096, its chars/4 est UNDER-counts the rendered prompt by ~3 % on
|
||||
# code/agent traffic, and compaction only fires at 85 % (measured 2026-10-07
|
||||
# at the old 200,192 ceiling: est 152,938, server 157,388, over by 354).
|
||||
# Harness contract (models.yml): contextWindow 155648 (= ceiling − 45056,
|
||||
# the gap that absorbs the estimate error), maxTokens 45056.
|
||||
# Compaction fires at 85 % = 132,301 est ≈ 136k measured — the 180k target
|
||||
# of the 262,144 era is unreachable at this ceiling: the window itself is
|
||||
# 155,648. Worst-case request: prompt + budget <= 155648 − 4096 + 3 %
|
||||
# est-error ≈ 156.2k <= 200,704 — ~9.5× the 3 % error of margin (robust to a
|
||||
# ~25 % under-count). Compaction rides on frank-zealot: input ≈ threshold −
|
||||
# keepRecent(20k) ≈ 112k, inside its 200,704 slot.
|
||||
# HALOGEN_KV_POOL_POSITIONS=401408 = SLOTS x CTX (2 x 200,704), the top of the
|
||||
# legal band (the seam enforces ctx <= pool <= slots * ctx). Still ONE pool
|
||||
# behind HALOGEN_KV_SLOTS=2 (ledger form: "2 slots over ONE <pool>-position KV
|
||||
# pool, each request up to HALOGEN_CTX"), but sized so BOTH slots can hold a
|
||||
# MAXIMUM-length request at the same time — the parity with frank-zealot's
|
||||
# 401,408-position pool that Frank ordered on 2026-10-18. The PER-REQUEST
|
||||
# ceiling does NOT move with it: HALOGEN_CTX stays 200,704, because the
|
||||
# checkpoint's native window is 262,144 and the engine refuses to go past it
|
||||
# without HALOGEN_ROPE_YARN (its own 400 text says so). ~28 KiB/position
|
||||
# (FLAGS.md) ⇒ the DEPLOYED 2026-10-18 boot RESERVES 11.0 GiB (`reserving the
|
||||
# KV pool, 401408 positions (about 11.0 GiB)`), where pool == ctx (200,704)
|
||||
# cost 5.5, 262,144 cost 7.2 and the 2026-10-06 shape's 524,288 measured 14.6.
|
||||
# The preflight formula bills 11.3 (block scratch + ~115 MiB/slot of O(1)
|
||||
# state) — quote the RESERVATION. FLAGS.md law: "RAM the pool
|
||||
# takes is RAM that table loses" — the 47.68 GiB n-gram table surrenders this
|
||||
# much page cache, so the warm prompt-cache ratio is what to re-measure.
|
||||
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
||||
# drafter is NOT slot-locked on this engine (batching and speculation
|
||||
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
|
||||
# quickstart ships 2 slots WITH the head), so the second slot is
|
||||
# concurrent capacity, not a speculation tax.
|
||||
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
||||
# mmap'd page cache beside the checkpoint, not a pinned charge — the boot
|
||||
# ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
|
||||
# weights 62.9 GiB locked (0.8 of them the vision tower) + KV 11.0 +
|
||||
# working 5.7 (HALOGEN_MAX_TOK=8192) = 79.6 GiB engine-side (the boot
|
||||
# ledger of the 401408-position deploy, 2026-10-18), with the 1.3 GiB
|
||||
# prompt cache (12 entries x 111 MiB at ctx 200704) booked separately by
|
||||
# the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host honest. Against
|
||||
# the 121.2 GiB box that leaves ~20 GiB over the reserve (MemAvailable
|
||||
# read 22 GiB right after the raise) for baseline co-tenancy
|
||||
# (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot ledger is the
|
||||
# referee — read it after deploy and re-size if the engine prints a
|
||||
# clamp.
|
||||
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
||||
# turns the MTP head off for whole requests after low acceptance and
|
||||
# HALVES decode as a result. Frank accepted the head always-on trade
|
||||
# (temp 1.0 traffic costs acceptance; the card still ships the recipe).
|
||||
#
|
||||
# CLIENT CONTRACT deltas vs the 27B shape (harness side lives in
|
||||
# ~/.omp/agent/models.yml):
|
||||
# * SERVER-SIDE SAMPLER EXISTS NOW: HALOGEN_TEMPERATURE/TOP_P/TOP_K set
|
||||
# the card's thinking recipe (1.0 / 0.95 / 20) as engine defaults, so
|
||||
# an absent temperature is no longer greedy. models.yml extraBody
|
||||
# carries the same values — consistent, not load-bearing.
|
||||
# * HALOGEN_MAX_TOK=8192 is only the DEFAULT for requests that omit
|
||||
# max_tokens; the harness sends at most its maxTokens 45056. Requests
|
||||
# over HALOGEN_MAX_TOKENS_CAP (65536) are a 400 naming the cap. Cap and
|
||||
# queue timeout are COUPLED (FLAGS.md table): 65536 at ~10 t/s serial
|
||||
# worst case needs > 6550 s; 7200 honors it. Do not move one alone.
|
||||
# * max_tokens INCLUDES thinking; a reasoning reply over a small budget
|
||||
# returns empty content with finish_reason "length".
|
||||
# * Prompt cache is ON by default (PROMPT_CACHE=2): a follow-up turn at
|
||||
# 100k context re-prefills ~2 s instead of ~88 s cold. No pin needed —
|
||||
# unlike the 27B engine, auto-sizing fits this box's ledger.
|
||||
# * Thinking lands in reasoning_content (same as the 27B engine); the
|
||||
# v2 chat template rejects reasoning_effort outside xhigh|medium|low,
|
||||
# so the models.yml fold (high->xhigh) still applies.
|
||||
# * Vision: OpenAI image_url content parts only (the fleet's standard
|
||||
# contract). >=256 px probe images or the model answers "video".
|
||||
# * Concurrency at depth is tight now: the KV pool is ONE 200,704-position
|
||||
# budget behind 2 slots — exactly one maximum-length request fills it. Two
|
||||
# conversations coexist only while they stay under ~100k each. The
|
||||
# PER-REQUEST ceiling moved WITH the pool (262144 -> 200704), so the harness
|
||||
# contract moves too: contextWindow 155648 = ceiling − 45k, never higher.
|
||||
# * response_format json_object/json_schema unsupported in this line
|
||||
# (no constrained decoding) — unchanged from the 27B engine.
|
||||
# * The engine's token protocol has no auth and binds loopback INSIDE
|
||||
# the container; only the frontend port (8731) is published. Same as
|
||||
# before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
|
||||
# amdgpu.noretry=0 (check at deploy).
|
||||
|
||||
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
|
||||
# is what was qualified). 0.16.2 = the version the model card names as its
|
||||
# quickstart pin; RepoDigest captured at pull on 2026-10-06.
|
||||
# The .hgn format is the lock: it loads ONLY in the flash-server line, so the
|
||||
# engine moves with the payload, never ahead of it.
|
||||
# Image digest-pinned (a tag pull silently re-points; the digest is what was
|
||||
# qualified — 0.16.2 is the version the model card names as its quickstart pin).
|
||||
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
||||
# Container identity: kept from the 27B era (unit ExecStop names it).
|
||||
# Container identity kept from the 27B era: the unit's ExecStop names it.
|
||||
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
|
||||
# Old quadlet-era container identity; the serve script clears stale names.
|
||||
# Predecessor identity the serve script clears if it is still holding the GPUs.
|
||||
FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
|
||||
# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
|
||||
# default); the engine port (8730) is never published.
|
||||
# host:container — the frontend listens on 8731 inside; the engine's own port
|
||||
# (8730) has no auth and is NEVER published.
|
||||
FRANK_DRAGOON_PUBLISH=7770:8731
|
||||
|
||||
FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-v2-abliterated
|
||||
# The checkpoint: Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated
|
||||
# qwen38-flash-next-v2-abliterated.hgn — the NATIVE 2026-10-11 v2 conversion
|
||||
# of orcarouter's pinned BF16 Qwen3.8-Flash-Next, abliterated. NOT the writer
|
||||
# transplant this filename used to carry: 66,687,678,432 B where the swift15
|
||||
# r2 measured 66,687,678,464 B — same filename family, different artifact.
|
||||
# Hub LFS sha256 verified at fetch by scripts/download-frank-dragoon-v2abl.sh.
|
||||
# Same filename family, DIFFERENT artifact: this is the native v2 conversion,
|
||||
# 32 bytes smaller than the writer transplant that used to carry the name.
|
||||
# Never infer the payload from the filename — this size and sha256 are the gate.
|
||||
FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-abliterated.hgn
|
||||
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678432
|
||||
FRANK_DRAGOON_CHECKPOINT_SHA256=b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
|
||||
# The n-gram lookup table: REQUIRED (the engine reads it beside the
|
||||
# checkpoint; the card's quickstart names it explicitly). From
|
||||
# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
|
||||
# The card does NOT establish this stock table as matched to the NATIVE
|
||||
# conversion and no source-matched table is published, so the engine's own
|
||||
# tools are the referee: NIAH 18/18 and PPL 3.968 on it (2026-10-17). It is
|
||||
# HARDLINKED from the swift15 payload dir (a hardlink is a regular file; the
|
||||
# serve gate refuses symlinks), i.e. byte-identical to the file that era
|
||||
# served.
|
||||
# The n-gram table is REQUIRED (the engine reads it beside the checkpoint) and
|
||||
# disk-backed: mmap'd page cache, never pinned. It is a stock peonist companion
|
||||
# hardlinked from the previous payload dir — a hardlink is a regular file, and
|
||||
# the serve gate refuses symlinks — with the engine's own NIAH/PPL verdicts on
|
||||
# the page as the referee.
|
||||
FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
|
||||
FRANK_DRAGOON_NGRAM_BYTES=51200246144
|
||||
FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
||||
# The vision tower: OPTIONAL on the engine, ENABLED here (image input).
|
||||
# From peonist-ai/halogen-qwen3.8-flash-next @ e053f488.
|
||||
# Vision tower: optional on the engine, ENABLED here (image input).
|
||||
FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
|
||||
FRANK_DRAGOON_VISION_BYTES=897916416
|
||||
FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
||||
# The tokenizer is a FLAT DIRECTORY, $FRANK_DRAGOON_MODEL_DIR/tokenizer,
|
||||
# mounted whole at /tokenizer. The two repos ship the same one; fetched
|
||||
# fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to
|
||||
# the 27B-era copy; the rest are plain git blobs — size-checked only). The
|
||||
# serve gate and the deployer refuse a missing file or ANY symlink: an HF
|
||||
# cache snapshot symlinks each file into blobs/, and those links dangle
|
||||
# The tokenizer is a FLAT DIRECTORY ($FRANK_DRAGOON_MODEL_DIR/tokenizer) mounted
|
||||
# whole. The serve gate and the deployer refuse a missing file or ANY symlink:
|
||||
# an HF cache snapshot symlinks each file into blobs/ and those links dangle
|
||||
# inside the container mount.
|
||||
FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
|
||||
|
||||
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
||||
# stays at the image default, which is what was qualified) ------------------
|
||||
# Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
|
||||
# scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
|
||||
# model_ids for the :7770 backend MUST name this value.
|
||||
# Served id. OpenWebUI api_configs model_ids for the :7770 backend MUST name it.
|
||||
HALOGEN_MODEL_ID=frank-dragoon
|
||||
# PER-REQUEST ceiling: a longer prompt is a 400 naming the limit, never a
|
||||
# truncation. The harness window must sit ~45k UNDER this (the 400-zone law on
|
||||
# the page) — a window equal to the ceiling makes every long request a 400.
|
||||
HALOGEN_CTX=200704
|
||||
# ONE pool behind both slots, sized so BOTH can carry a maximum-length request
|
||||
# at once. Pool positions buy concurrency; the per-request window never moves
|
||||
# with them. Legal band: ctx <= pool <= slots * ctx (the seam enforces it), and
|
||||
# RAM the pool takes is RAM the n-gram table's page cache loses.
|
||||
HALOGEN_KV_POOL_POSITIONS=401408
|
||||
# Two concurrent conversations: the MTP drafter is not slot-locked on this
|
||||
# engine, so the second slot is capacity, not a speculation tax.
|
||||
HALOGEN_KV_SLOTS=2
|
||||
HALOGEN_HOST_RESERVE_GIB=20
|
||||
# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
|
||||
# depth 2 is the documented "best all-round" default. HALOGEN_DRAFTER /
|
||||
# HALOGEN_DFLASH_* from the 27B engine DO NOT EXIST here.
|
||||
# The MTP head is INSIDE the checkpoint — do NOT set HALOGEN_MTP_HEAD.
|
||||
# 0 is mandatory: the adaptive default disarms the head for whole requests after
|
||||
# low acceptance and halves decode. Depth 2 is the card's best-all-round default.
|
||||
HALOGEN_SPEC_ADAPT=0
|
||||
# Coupled pair (FLAGS.md table): the cap bounds how long ONE request holds
|
||||
# the GPU, the timeout bounds how long the NEXT client waits. Do not move
|
||||
# one alone (see the client-contract note above).
|
||||
# Coupled trio: MAX_TOK is only the default for requests that omit max_tokens;
|
||||
# the cap bounds how long ONE request holds the GPU and the queue timeout bounds
|
||||
# how long the NEXT client waits. Move all three together or none.
|
||||
HALOGEN_MAX_TOK=8192
|
||||
HALOGEN_MAX_TOKENS_CAP=65536
|
||||
HALOGEN_QUEUE_TIMEOUT=7200
|
||||
# Server-side sampler = the card's thinking recipe (the engine now HAS one;
|
||||
# requests may still override per-call). Matches models.yml extraBody.
|
||||
# The engine has a server-side sampler (an absent temperature is NOT greedy
|
||||
# here); models.yml extraBody carries the same recipe.
|
||||
HALOGEN_TEMPERATURE=1.0
|
||||
HALOGEN_TOP_P=0.95
|
||||
HALOGEN_TOP_K=20
|
||||
# Card's belt-and-braces: the engine pins weights by default
|
||||
# (HALOGEN_FLASH_PIN_TRUNK=1; only the retired w4b build honored 0).
|
||||
# Weights stay pinned: the engine pins by default and only the retired w4b build
|
||||
# honored HALOGEN_FLASH_PIN_TRUNK=0, which this payload rejects.
|
||||
HALOGEN_WEIGHTS_LOCK=1
|
||||
Reference in new issue
Block a user