frank-dragoon: cut the endpoint over to the native v2 abliterated checkpoint

Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated is now the NATIVE
2026-10-11 v2 conversion of orcarouter's pinned BF16 (66,687,678,432 B,
LFS sha256 b5bd925f...) — 32 bytes smaller than the writer transplant it
used to ship under the same filename. The n-gram table, vision tower and
tokenizer are the stock peonist companions, hardlinked out of the swift15
payload dir: the release publishes only the main file and no source-matched
pair exists, so the engine's own tools are the referee (niah 18/18, ppl
3.968 with mean KL 0.155 against the swift15 logits, 2026-10-17).

Memory saving, per the card: HALOGEN_KV_POOL_POSITIONS 524288 -> 262144,
one pool behind both slots (boot ledger: 7.2 GiB, where the old pool
measured 14.6). HALOGEN_CTX stays 262144 so the per-request ceiling and the
harness contract (contextWindow 217088) do not move; only two MAXIMUM-length
conversations lose the ability to coexist.

Measured against the swift15 boot on identical traffic: cold prefill
+12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking
tokens x1.9 (the finetune's reasoning compression is gone), refusals 0/5 on
both.
This commit is contained in:
Frank committed 2026-10-17 22:18:00 -07:00
1 parent 3f5ad507c2
commit 061e61baa6
5 files changed
+111 -70

No files matched your search

+23 -23
View File
@@ -1093,18 +1093,18 @@ deploy_anubis() {
} }
# frank-dragoon is the halogen-flash-server CONTAINER (0.16.2, digest-pinned # frank-dragoon is the halogen-flash-server CONTAINER (0.16.2, digest-pinned
# through scripts/frank-dragoon.env) serving the Quat3rnion ABLITERATED # through scripts/frank-dragoon.env) serving Quat3rnion's NATIVE v2
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint — # conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next —
# qwen38-flash-next-v2-swift15-abliterated.hgn (66,687,678,464 B, ~62 GiB # qwen38-flash-next-v2-abliterated.hgn (66,687,678,432 B, ~62 GiB resident)
# resident) + the REQUIRED n-gram lookup table + the vision tower — # + the REQUIRED n-gram lookup table + the vision tower — MTP spec decode
# MTP spec decode (HALOGEN_SPEC_ADAPT=0), 2 slots x 200192, image input — # (HALOGEN_SPEC_ADAPT=0), 2 slots over ONE 262,144-position KV pool, image
# on the Strix Halo iGPU. It stays a ROOT unit because the serve script # input — on the Strix Halo iGPU. It stays a ROOT unit because the serve
# drives `podman run` directly (the frank-archon pattern): relink_unit # script drives `podman run` directly (the frank-archon pattern):
# points /etc/systemd/system/frank-dragoon.service at the repo file, boot # relink_unit points /etc/systemd/system/frank-dragoon.service at the repo
# persistence is that unit's [Install], and LimitMEMLOCK=infinity is # file, boot persistence is that unit's [Install], and LimitMEMLOCK=infinity
# fleet parity. 2026-10-06 cutover from the halogen 27B-dense shape — # is fleet parity. 2026-10-17 cutover from the Swift1.5-finetune shape —
# rollback = revert the swap commit's trio; the 27B payload stays on # rollback = revert the swap commit and redeploy; the swift15 payload
# disk (~/models/halogen-qwen3.8-27b). # (~/models/halogen-flashnext-v2) and the 27B one both stay on disk.
# #
# The gates mirror serve-frank-dragoon.sh's predicates EXACTLY (the # The gates mirror serve-frank-dragoon.sh's predicates EXACTLY (the
# 2026-10-01 crash-loop lesson): image digest in dank's podman storage, # 2026-10-01 crash-loop lesson): image digest in dank's podman storage,
@@ -1115,13 +1115,13 @@ deploy_anubis() {
# port gate is the same as every cutover since 2026-09-16: nothing # port gate is the same as every cutover since 2026-09-16: nothing
# starts until 7770 is released. # starts until 7770 is released.
LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}" LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
LLM_DRAGOON_CKPT="${LLM_DRAGOON_CKPT:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-v2-swift15-abliterated.hgn}" LLM_DRAGOON_CKPT="${LLM_DRAGOON_CKPT:-/home/dank/models/halogen-v2-abliterated/qwen38-flash-next-v2-abliterated.hgn}"
LLM_DRAGOON_CKPT_BYTES="${LLM_DRAGOON_CKPT_BYTES:-66687678464}" LLM_DRAGOON_CKPT_BYTES="${LLM_DRAGOON_CKPT_BYTES:-66687678432}"
LLM_DRAGOON_NGRAM="${LLM_DRAGOON_NGRAM:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-ngram.hgn}" LLM_DRAGOON_NGRAM="${LLM_DRAGOON_NGRAM:-/home/dank/models/halogen-v2-abliterated/qwen38-flash-next-ngram.hgn}"
LLM_DRAGOON_NGRAM_BYTES="${LLM_DRAGOON_NGRAM_BYTES:-51200246144}" LLM_DRAGOON_NGRAM_BYTES="${LLM_DRAGOON_NGRAM_BYTES:-51200246144}"
LLM_DRAGOON_VISION="${LLM_DRAGOON_VISION:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-vision.hgn}" LLM_DRAGOON_VISION="${LLM_DRAGOON_VISION:-/home/dank/models/halogen-v2-abliterated/qwen38-flash-next-vision.hgn}"
LLM_DRAGOON_VISION_BYTES="${LLM_DRAGOON_VISION_BYTES:-897916416}" LLM_DRAGOON_VISION_BYTES="${LLM_DRAGOON_VISION_BYTES:-897916416}"
LLM_DRAGOON_TOK_DIR="${LLM_DRAGOON_TOK_DIR:-/home/dank/models/halogen-flashnext-v2/tokenizer}" LLM_DRAGOON_TOK_DIR="${LLM_DRAGOON_TOK_DIR:-/home/dank/models/halogen-v2-abliterated/tokenizer}"
LLM_DRAGOON_TOK_FILES="${LLM_DRAGOON_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}" LLM_DRAGOON_TOK_FILES="${LLM_DRAGOON_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
retire_halogen_dragoon_quadlet() { retire_halogen_dragoon_quadlet() {
@@ -1166,12 +1166,12 @@ deploy_llm_dragoon() {
fi fi
if [[ ! -f "$LLM_DRAGOON_CKPT" ]]; then if [[ ! -f "$LLM_DRAGOON_CKPT" ]]; then
echo "ERROR: halogen checkpoint absent at $LLM_DRAGOON_CKPT" >&2 echo "ERROR: halogen checkpoint absent at $LLM_DRAGOON_CKPT" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2 echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-v2abl.sh" >&2
return 1 return 1
fi fi
if [[ "$(stat -c %s "$LLM_DRAGOON_CKPT")" != "$LLM_DRAGOON_CKPT_BYTES" ]]; then if [[ "$(stat -c %s "$LLM_DRAGOON_CKPT")" != "$LLM_DRAGOON_CKPT_BYTES" ]]; then
echo "ERROR: halogen checkpoint is not $LLM_DRAGOON_CKPT_BYTES bytes (partial download?)" >&2 echo "ERROR: halogen checkpoint is not $LLM_DRAGOON_CKPT_BYTES bytes (partial download?)" >&2
echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2 echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-v2abl.sh --verify-only" >&2
return 1 return 1
fi fi
for gate in "$LLM_DRAGOON_NGRAM:$LLM_DRAGOON_NGRAM_BYTES:n-gram table" \ for gate in "$LLM_DRAGOON_NGRAM:$LLM_DRAGOON_NGRAM_BYTES:n-gram table" \
@@ -1179,24 +1179,24 @@ deploy_llm_dragoon() {
IFS=: read -r gpath gbytes gname <<<"$gate" IFS=: read -r gpath gbytes gname <<<"$gate"
if [[ ! -f "$gpath" ]]; then if [[ ! -f "$gpath" ]]; then
echo "ERROR: halogen $gname absent at $gpath" >&2 echo "ERROR: halogen $gname absent at $gpath" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2 echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-v2abl.sh" >&2
return 1 return 1
fi fi
if [[ "$(stat -c %s "$gpath")" != "$gbytes" ]]; then if [[ "$(stat -c %s "$gpath")" != "$gbytes" ]]; then
echo "ERROR: halogen $gname is not $gbytes bytes (partial download?)" >&2 echo "ERROR: halogen $gname is not $gbytes bytes (partial download?)" >&2
echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2 echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-v2abl.sh --verify-only" >&2
return 1 return 1
fi fi
done done
if [[ -L "$LLM_DRAGOON_TOK_DIR" || ! -d "$LLM_DRAGOON_TOK_DIR" ]]; then if [[ -L "$LLM_DRAGOON_TOK_DIR" || ! -d "$LLM_DRAGOON_TOK_DIR" ]]; then
echo "ERROR: tokenizer dir missing or a symlink at $LLM_DRAGOON_TOK_DIR" >&2 echo "ERROR: tokenizer dir missing or a symlink at $LLM_DRAGOON_TOK_DIR" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2 echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-v2abl.sh" >&2
return 1 return 1
fi fi
for tf in $LLM_DRAGOON_TOK_FILES; do for tf in $LLM_DRAGOON_TOK_FILES; do
if [[ ! -f "$LLM_DRAGOON_TOK_DIR/$tf" || -L "$LLM_DRAGOON_TOK_DIR/$tf" ]]; then if [[ ! -f "$LLM_DRAGOON_TOK_DIR/$tf" || -L "$LLM_DRAGOON_TOK_DIR/$tf" ]]; then
echo "ERROR: tokenizer file missing or a symlink: $LLM_DRAGOON_TOK_DIR/$tf" >&2 echo "ERROR: tokenizer file missing or a symlink: $LLM_DRAGOON_TOK_DIR/$tf" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2 echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-v2abl.sh" >&2
return 1 return 1
fi fi
done done
+54 -23
View File
@@ -1,14 +1,24 @@
# frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's # frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
# Strix Halo iGPU (gfx1151), host port 7770. # Strix Halo iGPU (gfx1151), host port 7770.
# #
# RUNTIME (2026-10-06, Frank): halogen-flash-server 0.16.2 — the # RUNTIME (2026-10-17, Frank): halogen-flash-server 0.16.2 — the
# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server, # closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
# digest-pinned below) serving the Quat3rnion ABLITERATED Swift1.5 finetune # digest-pinned below) serving Quat3rnion/halogen-qwen3.8-flash-next-v2-
# of peonist's Qwen3.8-Flash-Next v2 checkpoint, as a podman container under # abliterated — the NATIVE v2 conversion (2026-10-11) of orcarouter's pinned
# a root unit, driven by scripts/serve-frank-dragoon.sh. This CUTS OVER the # BF16 Qwen3.8-Flash-Next, abliterated — as a podman container under a root
# 2026-10-05 halogen 27B-dense shape (censored base, DFlash2, 31.7 t/s): # unit, driven by scripts/serve-frank-dragoon.sh. This CUTS OVER the
# rollback = revert the swap commit's trio; the 27B payload stays on disk at # 2026-10-06 Swift1.5-finetune shape; rollback = revert the swap commit and
# /home/dank/models/halogen-qwen3.8-27b/. # redeploy: the swift15 payload stays on disk at
# /home/dank/models/halogen-flashnext-v2/, the 27B-era payload beside it.
#
# MEASURED A/B, both boots on this box, identical battery traffic (engine
# tools with the service stopped, behavior probes against it, 2026-10-17):
# NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls and the
# red-PNG vision probe identical, needle hit at both depths of a 21.5k-token
# haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s,
# wikitext-2 PPL 3.968 vs 3.877 (the finetune's edge is gone: +2.3 %) and
# thinking tokens 1,476 vs 784 mean — the finetune compressed reasoning, the
# base thinks ~1.9x longer per answer.
# #
# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own # WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
# card it "will not load in transformers, vLLM, or llama.cpp" — and the v2 # card it "will not load in transformers, vLLM, or llama.cpp" — and the v2
@@ -38,9 +48,16 @@
# engine ceiling absorbs the estimate error with ~7x margin (robust to a # engine ceiling absorbs the estimate error with ~7x margin (robust to a
# 10 % under-count). Compaction rides on frank-zealot: input ≈ # 10 % under-count). Compaction rides on frank-zealot: input ≈
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot. # threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot.
# HALOGEN_KV_POOL_POSITIONS=524288 = KV_SLOTS(2) x CTX — the explicit # HALOGEN_KV_POOL_POSITIONS=262144 is the card's recommended pool and the
# form of the engine's default, so the seam test can pin the product. # memory saving of this swap. One budget behind HALOGEN_KV_SLOTS=2 (boot
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB. # ledger: "2 slots over ONE 262144-position KV pool, each request up to
# 262144"), so the per-request ceiling stays HALOGEN_CTX while two
# MAXIMUM-length conversations can no longer coexist — two moderate ones
# can. ~28 KiB/position (FLAGS.md) => 7.2 GiB, and the ledger said exactly
# that: "reserving the KV pool, 262144 positions (about 7.2 GiB)". The
# 2026-10-06 shape's 524,288 cost 14.6 GiB; FLAGS.md law — "RAM the pool
# takes is RAM that table loses" — makes the 7.4 GiB handed back page
# cache for the 47.7 GiB n-gram table, which is what long prompts read.
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP # HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
# drafter is NOT slot-locked on this engine (batching and speculation # drafter is NOT slot-locked on this engine (batching and speculation
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own # coexist — DFlash-era slots>1 law does NOT carry over; the card's own
@@ -48,12 +65,13 @@
# concurrent capacity, not a speculation tax. # concurrent capacity, not a speculation tax.
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is # Memory ledger against the box's 121.2 GiB unified (the n-gram table is
# mmap'd page cache beside the checkpoint, not a pinned charge — the # mmap'd page cache beside the checkpoint, not a pinned charge — the
# card's own quickstart arithmetic does not bill it): # card's own quickstart arithmetic does not bill it, and the boot ledger
# weights ~62 GiB resident + KV 14.0 + arena (HALOGEN_MAX_TOK=8192 => # agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
# weights ~62 GiB resident + KV 7.2 + arena (HALOGEN_MAX_TOK=8192 =>
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2 # ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2
# = ~86 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host # = ~79 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
# honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ) # honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ)
# ~10-15. ~15 GiB margin. The boot ledger is the referee — read it # ~10-15. ~25 GiB margin. The boot ledger is the referee — read it
# after deploy and re-size if the engine prints a clamp. # after deploy and re-size if the engine prints a clamp.
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy # HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
# turns the MTP head off for whole requests after low acceptance and # turns the MTP head off for whole requests after low acceptance and
@@ -81,6 +99,11 @@
# so the models.yml fold (high->xhigh) still applies. # so the models.yml fold (high->xhigh) still applies.
# * Vision: OpenAI image_url content parts only (the fleet's standard # * Vision: OpenAI image_url content parts only (the fleet's standard
# contract). >=256 px probe images or the model answers "video". # contract). >=256 px probe images or the model answers "video".
# * TWO maximum-length conversations no longer share the endpoint: the KV
# pool is one 262,144-position budget behind 2 slots. The PER-REQUEST
# ceiling is unchanged (HALOGEN_CTX), so the harness contract below
# (contextWindow 217088) does not move; only concurrency at full depth
# does, and this endpoint carries the slow + security-reviewer roles.
# * response_format json_object/json_schema unsupported in this line # * response_format json_object/json_schema unsupported in this line
# (no constrained decoding) — unchanged from the 27B engine. # (no constrained decoding) — unchanged from the 27B engine.
# * The engine's token protocol has no auth and binds loopback INSIDE # * The engine's token protocol has no auth and binds loopback INSIDE
@@ -100,17 +123,25 @@ FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
# default); the engine port (8730) is never published. # default); the engine port (8730) is never published.
FRANK_DRAGOON_PUBLISH=7770:8731 FRANK_DRAGOON_PUBLISH=7770:8731
FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-flashnext-v2 FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-v2-abliterated
# The checkpoint: Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated # The checkpoint: Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated
# qwen38-flash-next-v2-swift15-abliterated.hgn — compacted r2 of the card's # qwen38-flash-next-v2-abliterated.hgn — the NATIVE 2026-10-11 v2 conversion
# append-only original (byte-identical live tensors, padding dropped: # of orcarouter's pinned BF16 Qwen3.8-Flash-Next, abliterated. NOT the writer
# 66,687,678,464 B vs 89,775,868,032 B). LFS sha256 verified at fetch. # transplant this filename used to carry: 66,687,678,432 B where the swift15
FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-swift15-abliterated.hgn # r2 measured 66,687,678,464 B — same filename family, different artifact.
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678464 # Hub LFS sha256 verified at fetch by scripts/download-frank-dragoon-v2abl.sh.
FRANK_DRAGOON_CHECKPOINT_SHA256=9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037 FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-abliterated.hgn
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678432
FRANK_DRAGOON_CHECKPOINT_SHA256=b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
# The n-gram lookup table: REQUIRED (the engine reads it beside the # The n-gram lookup table: REQUIRED (the engine reads it beside the
# checkpoint; the card's quickstart names it explicitly). From # checkpoint; the card's quickstart names it explicitly). From
# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea. # peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
# The card does NOT establish this stock table as matched to the NATIVE
# conversion and no source-matched table is published, so the engine's own
# tools are the referee: NIAH 18/18 and PPL 3.968 on it (2026-10-17). It is
# HARDLINKED from the swift15 payload dir (a hardlink is a regular file; the
# serve gate refuses symlinks), i.e. byte-identical to the file that era
# served.
FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
FRANK_DRAGOON_NGRAM_BYTES=51200246144 FRANK_DRAGOON_NGRAM_BYTES=51200246144
FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
@@ -135,7 +166,7 @@ FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_conf
# model_ids for the :7770 backend MUST name this value. # model_ids for the :7770 backend MUST name this value.
HALOGEN_MODEL_ID=frank-dragoon HALOGEN_MODEL_ID=frank-dragoon
HALOGEN_CTX=262144 HALOGEN_CTX=262144
HALOGEN_KV_POOL_POSITIONS=524288 HALOGEN_KV_POOL_POSITIONS=262144
HALOGEN_KV_SLOTS=2 HALOGEN_KV_SLOTS=2
HALOGEN_HOST_RESERVE_GIB=20 HALOGEN_HOST_RESERVE_GIB=20
# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD); # Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
+1 -1
View File
@@ -1,5 +1,5 @@
[Unit] [Unit]
Description=halogen-flash-server container (Quat3rnion abliterated Swift1.5 of Qwen3.8-Flash-Next v2, MTP spec decode, 2 slots x 200192, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU) Description=halogen-flash-server container (Quat3rnion NATIVE v2 abliterated Qwen3.8-Flash-Next, MTP spec decode, 2 slots over ONE 262144-position pool, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
+8 -8
View File
@@ -1,11 +1,11 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.16.2 # serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.16.2
# (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server, # (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server,
# digest-pinned in frank-dragoon.env) serving the Quat3rnion abliterated # digest-pinned in frank-dragoon.env) serving Quat3rnion's NATIVE v2
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint on # conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next on Dankclaw's
# Dankclaw's Strix Halo iGPU, host :7770. 2026-10-06 cutover from the # Strix Halo iGPU, host :7770. 2026-10-17 cutover from the Swift1.5-finetune
# halogen 27B-dense shape — revert the swap commit's trio to roll back # shape — revert the swap commit to roll back (the swift15 payload and the
# (the 27B payload stays on disk). # 27B one both stay on disk).
# #
# The engine is driven by `podman run` DIRECTLY (the fleet's frank-archon # The engine is driven by `podman run` DIRECTLY (the fleet's frank-archon
# pattern): the image starts engine + OpenAI frontend in one container. # pattern): the image starts engine + OpenAI frontend in one container.
@@ -75,7 +75,7 @@ ck() {
# ck <env-name> <path> <expected-bytes> # ck <env-name> <path> <expected-bytes>
if [ ! -f "$2" ]; then if [ ! -f "$2" ]; then
echo "frank-dragoon: $1 $2 missing;" >&2 echo "frank-dragoon: $1 $2 missing;" >&2
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2 echo " fetch it: bash scripts/download-frank-dragoon-v2abl.sh" >&2
exit 1 exit 1
fi fi
local sz local sz
@@ -96,13 +96,13 @@ ck vision-tower "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_VISION_TOWER" "$FRANK_D
TOK_DIR="$FRANK_DRAGOON_MODEL_DIR/tokenizer" TOK_DIR="$FRANK_DRAGOON_MODEL_DIR/tokenizer"
if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then
echo "frank-dragoon: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2 echo "frank-dragoon: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2 echo " fetch it: bash scripts/download-frank-dragoon-v2abl.sh" >&2
exit 1 exit 1
fi fi
for TF in $FRANK_DRAGOON_TOKENIZER_FILES; do for TF in $FRANK_DRAGOON_TOKENIZER_FILES; do
if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then
echo "frank-dragoon: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2 echo "frank-dragoon: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2 echo " fetch it: bash scripts/download-frank-dragoon-v2abl.sh" >&2
exit 1 exit 1
fi fi
done done
+25 -15
View File
@@ -1,10 +1,11 @@
#!/usr/bin/env bats #!/usr/bin/env bats
# Seam: frank-dragoon serves the Quat3rnion ABLITERATED Swift1.5 finetune of # Seam: frank-dragoon serves Quat3rnion's NATIVE v2 conversion (2026-10-11)
# peonist's Qwen3.8-Flash-Next v2 checkpoint (a .hgn flash-server payload) # of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the
# on the Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2, # Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2,
# digest-pinned), driven by the root unit's serve script — two 262,144-token # digest-pinned), driven by the root unit's serve script — 2 slots over ONE
# slots with the MTP head always on (HALOGEN_SPEC_ADAPT=0) and the vision # 262,144-position KV pool with the MTP head always on (HALOGEN_SPEC_ADAPT=0)
# tower mounted. 2026-10-06 cutover from the halogen 27B-dense shape. # and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune
# shape, which itself cut over the halogen 27B-dense shape on 2026-10-06.
# #
# This seam exists because the artifacts are a triangle that must agree on # This seam exists because the artifacts are a triangle that must agree on
# exact strings the engine reads: the image DIGEST, the checkpoint/n-gram/ # exact strings the engine reads: the image DIGEST, the checkpoint/n-gram/
@@ -81,7 +82,7 @@ good_env() {
"FRANK_DRAGOON_TOKENIZER_FILES=tok.json" "FRANK_DRAGOON_TOKENIZER_FILES=tok.json"
"HALOGEN_MODEL_ID=frank-dragoon" "HALOGEN_MODEL_ID=frank-dragoon"
"HALOGEN_CTX=262144" "HALOGEN_CTX=262144"
"HALOGEN_KV_POOL_POSITIONS=524288" "HALOGEN_KV_POOL_POSITIONS=262144"
"HALOGEN_KV_SLOTS=2" "HALOGEN_KV_SLOTS=2"
"HALOGEN_SPEC_ADAPT=0" "HALOGEN_SPEC_ADAPT=0"
"HALOGEN_MAX_TOK=8192" "HALOGEN_MAX_TOK=8192"
@@ -136,9 +137,10 @@ good_env() {
[[ "$output" == *"no gfx1151 to serve on"* ]] [[ "$output" == *"no gfx1151 to serve on"* ]]
} }
@test "the pool product and memory ledger fit unified memory with margin" { @test "the pool and memory ledger fit unified memory with margin" {
# Pool arithmetic is the engine's documented product: the explicit # Pool arithmetic is the engine's documented shape: the explicit
# HALOGEN_KV_POOL_POSITIONS must equal slots x ctx, and KV costs # HALOGEN_KV_POOL_POSITIONS must back at least one full window, and KV
# costs
# ~28 KiB/position (FLAGS.md). Against it: ~62 GiB resident weights, # ~28 KiB/position (FLAGS.md). Against it: ~62 GiB resident weights,
# the arena scratch 3.8 + 8.4*(MAX_TOK/16384) GiB, the vision tower # the arena scratch 3.8 + 8.4*(MAX_TOK/16384) GiB, the vision tower
# ~2 GiB pool charge, the 20 GiB host reserve and the host baseline # ~2 GiB pool charge, the 20 GiB host reserve and the host baseline
@@ -151,7 +153,15 @@ good_env() {
pool="$(sed -nE 's/^HALOGEN_KV_POOL_POSITIONS=([0-9]+)$/\1/p' "$ENV_FILE")" pool="$(sed -nE 's/^HALOGEN_KV_POOL_POSITIONS=([0-9]+)$/\1/p' "$ENV_FILE")"
mt="$(sed -nE 's/^HALOGEN_MAX_TOK=([0-9]+)$/\1/p' "$ENV_FILE")" mt="$(sed -nE 's/^HALOGEN_MAX_TOK=([0-9]+)$/\1/p' "$ENV_FILE")"
reserve="$(sed -nE 's/^HALOGEN_HOST_RESERVE_GIB=([0-9]+)$/\1/p' "$ENV_FILE")" reserve="$(sed -nE 's/^HALOGEN_HOST_RESERVE_GIB=([0-9]+)$/\1/p' "$ENV_FILE")"
[ "$pool" -eq $(( slots * ctx )) ] # The pool is ONE budget behind the slots (boot ledger: "2 slots over ONE
# 262144-position KV pool, each request up to 262144"), NOT slots x ctx:
# the card ships it at one window and that IS the memory saving of this
# payload. What still binds is that the pool back ONE full-length
# request, or the advertised per-request ceiling would be unreachable —
# and that it stay inside unified memory, which the ledger sum below
# checks.
[ "$pool" -ge "$ctx" ]
[ "$pool" -le $(( slots * ctx )) ]
kv_gib=$(( pool * 28 * 1024 / 1073741824 )) kv_gib=$(( pool * 28 * 1024 / 1073741824 ))
arena_tenths=$(( 38 + 84 * mt / 16384 )) arena_tenths=$(( 38 + 84 * mt / 16384 ))
total_gib=$(( 62 + kv_gib + arena_tenths / 10 + 2 + reserve + 15 )) total_gib=$(( 62 + kv_gib + arena_tenths / 10 + 2 + reserve + 15 ))
@@ -174,7 +184,7 @@ good_env() {
FRANK_DRAGOON_CHECKPOINT=absent.hgn "$SERVE" FRANK_DRAGOON_CHECKPOINT=absent.hgn "$SERVE"
[ "$status" -ne 0 ] [ "$status" -ne 0 ]
[[ "$output" == *"checkpoint"*"missing"* ]] [[ "$output" == *"checkpoint"*"missing"* ]]
[[ "$output" == *"download-frank-dragoon-flashnext.sh"* ]] [[ "$output" == *"download-frank-dragoon-v2abl.sh"* ]]
} }
@test "the serve script refuses a checkpoint of the wrong byte count" { @test "the serve script refuses a checkpoint of the wrong byte count" {
@@ -264,7 +274,7 @@ $(cat "$TEST_TMP/serve.log")"
[[ "$output" == *"--publish 7770:8731"* ]] [[ "$output" == *"--publish 7770:8731"* ]]
[[ "$output" == *"-e HALOGEN_MODEL_ID=frank-dragoon"* ]] [[ "$output" == *"-e HALOGEN_MODEL_ID=frank-dragoon"* ]]
[[ "$output" == *"-e HALOGEN_CTX=262144"* ]] [[ "$output" == *"-e HALOGEN_CTX=262144"* ]]
[[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=524288"* ]] [[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=262144"* ]]
[[ "$output" == *"-e HALOGEN_KV_SLOTS=2"* ]] [[ "$output" == *"-e HALOGEN_KV_SLOTS=2"* ]]
[[ "$output" == *"-e HALOGEN_SPEC_ADAPT=0"* ]] [[ "$output" == *"-e HALOGEN_SPEC_ADAPT=0"* ]]
[[ "$output" == *"-e HALOGEN_CHECKPOINT=/models/qwen.hgn"* ]] [[ "$output" == *"-e HALOGEN_CHECKPOINT=/models/qwen.hgn"* ]]
@@ -328,8 +338,8 @@ $(cat "$TEST_TMP/serve.log")"
grep -qF "LLM_DRAGOON_CKPT_BYTES=\"\${LLM_DRAGOON_CKPT_BYTES:-$bytes}\"" "$DEPLOYER" grep -qF "LLM_DRAGOON_CKPT_BYTES=\"\${LLM_DRAGOON_CKPT_BYTES:-$bytes}\"" "$DEPLOYER"
} }
@test "the deployer points a missing artifact at the flashnext fetch script" { @test "the deployer points a missing artifact at the v2abl fetch script" {
grep -qE 'download-frank-dragoon-flashnext\.sh' "$DEPLOYER" grep -qE 'download-frank-dragoon-v2abl\.sh' "$DEPLOYER"
grep -qE 'podman pull' "$DEPLOYER" grep -qE 'podman pull' "$DEPLOYER"
} }