serverAppConfigs/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md

frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 NATIVE abliterated (Dankclaw, Strix Halo iGPU)

ACTIVE since 2026-10-17. Rollback = revert the swap commit and redeploy (scripts/frank-dragoon.env, scripts/serve-frank-dragoon.sh, scripts/frank-dragoon.service); the Swift1.5 payload stays on disk at /home/dank/models/halogen-flashnext-v2/ (its fetch script scripts/download-frank-dragoon-flashnext.sh stays the door back), and the 27B-era payload beside that at /home/dank/models/halogen-qwen3.8-27b/. Era write-up: frank-dragoon-flash-server-v2-swift15.

Why this payload

Frank's ask: serve https://huggingface.co/Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated with the card's recommended settings, and save memory.

This is the native 2026-10-11 v2 conversion of orcarouter's pinned BF16 Qwen3.8-Flash-Next, abliterated — the base itself, converted straight into the v2 .hgn format. It is not the writer transplant that carried this filename before: the artifact measures 66,687,678,432 B where the Swift1.5 r2 measured 66,687,678,464 B — 32 bytes apart, same filename family, different model. Byte counts and the Hub LFS sha256 are the only way to tell them apart, which is why the fetch script and both gates check them.

What the swap buys, measured (table below): retrieval at depth, decode, prefill, and 7.4 GiB of RAM handed back. What it costs: the Swift1.5 finetune's compressed reasoning and its small perplexity edge.

The companion law (the reason this swap needed a referee)

The release publishes only the main file. The n-gram lookup table (REQUIRED — the engine reads it beside the checkpoint), the vision tower, and the tokenizer are peonist's, and the card does not establish them as interchangeable with this native conversion. No source-matched pair is published anywhere, so the only available companions are the stock ones.

They are hardlinked out of the verified Swift1.5 payload directory (scripts/download-frank-dragoon-v2abl.sh): a hardlink is a regular file, so it satisfies the serve gate's no-symlink law, and it costs no disk and no copy next to a 62 GB payload. The engine's own checkpoint tools are the referee, and they answered: NIAH 18/18 (vs 17/18 for the payload those files were published with), needle retrieval exact at both depths, the red-PNG vision probe exact, perplexity in the normal band. A wrong PLE table does not whisper — see Verification.

The payload — one new file, three inherited ones

/home/dank/models/halogen-v2-abliterated/, fetched by scripts/download-frank-dragoon-v2abl.sh (-f + exact byte counts + LFS sha256 + --verify-only; no token on the command line):

File	Bytes	sha256	Source
qwen38-flash-next-v2-abliterated.hgn	66,687,678,432	b5bd925f…	Quat3rnion @main, fetched
qwen38-flash-next-ngram.hgn	51,200,246,144	3450acada…	peonist @5cc17cea, hardlinked, REQUIRED
qwen38-flash-next-vision.hgn	897,916,416	d62e0ae5…	peonist @e053f488, hardlinked, vision ON
tokenizer/ (6 files)	—	tokenizer.json LFS	peonist @e053f488, hardlinked

The engine stays halogen-flash-server 0.16.2, digest-pinned (the card tested 0.15.0; 0.16.2 is newer in the same line and is what the v2 format was qualified on in this fleet):

ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a

Shape — the card's settings, and the three deliberate deviations

HALOGEN_CTX=200704 (per-request ceiling — the card's 262,144 lowered the same evening, see below), HALOGEN_KV_SLOTS=2, HALOGEN_KV_POOL_POSITIONS=401408 (one pool behind both slots, raised to 2 × ctx on 2026-10-18 — see Pool doubled for slot parity below; the card recommends it equal to ctx, and that equality is where the memory is), HALOGEN_SPEC_ADAPT=0, HALOGEN_MAX_TOK=8192, HALOGEN_MAX_TOKENS_CAP=65536 + HALOGEN_QUEUE_TIMEOUT=7200 (coupled pair), sampler 1.0 / 0.95 / 20, HALOGEN_WEIGHTS_LOCK=1, HALOGEN_HOST_RESERVE_GIB=20.

Deviations from the card, each on purpose:

• HALOGEN_MODEL_ID=frank-dragoon, not the card's halogen-v2-abliterated — the fleet naming law (service = unit = served id).
• HALOGEN_SPEC_ADAPT=0 — the card is silent; this is a measured 0.16.2 law (the adaptive default disables the drafter per request and halves decode).
• the sampler values are peonist's engine recipe; Quat3rnion publishes none.

The pool is one budget, not a partition. Boot ledger form: listening on 127.0.0.1:8730, 2 slots over ONE <pool>-position KV pool, each request up to <HALOGEN_CTX>. Both numbers were lowered to 200,704 the same evening (the card ships 262,144; Frank took the RAM — nothing else in the fleet asks for the bigger window), so models.yml contextWindow moved to 155,648 = ceiling − 45k with it. With pool == ctx, one maximum-length request fills the pool: two long conversations can no longer share the endpoint (two of ≤100k can). That is acceptable here because frank-dragoon carries the slow (medium-thinking) role and the security-reviewer subagent, not the interactive workhorse (that is frank-archon). Frank lifted exactly that concurrency limit on 2026-10-18 and the lift is deployed (section below): the pool is again slots × ctx, so two maximum-length conversations fit the endpoint at once.

Memory ledger (121.2 GiB unified), from the boot ledgers. The 22:20 boot printed pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s) trunk, 0.84 GiB vision tower, reserving the KV pool, 262144 positions (about 7.2 GiB), and the engine's own sum: memory: 62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …, 75.8 GiB in all. The cut boot printed the same shape at the new size: reserving the KV pool, 200704 positions (about 5.5 GiB) and … 5.5 GiB of KV pool, 5.7 GiB of working memory, 74.1 GiB in all. The n-gram table rides as page cache — mapped 51.2 GB, NOT pinned (Pin::None) — which is exactly why every pool cut is a double saving. FLAGS.md law: RAM the pool takes is RAM that table loses; the ~9 GiB the two cuts together handed back is page cache for the 47.68 GiB table that long prompts read.

HALOGEN_FLASH_PIN_TRUNK=0 is rejected by the v2 loader — the trunk stays pinned. Disk-backing the weights is not a lever on this format.

Pool doubled for slot parity — committed AND deployed 2026-10-18

HALOGEN_KV_POOL_POSITIONS 200,704 → 401,408 (commit 2b12716), Frank's call: frank-dragoon's total in-flight context was half frank-zealot's — the llama.cpp endpoint runs one 401,408-position pool split into two 200,704 conversations, while dragoon ran one 200,704-position pool behind its two slots, so one maximum-length request filled it and any second long conversation had to stay under ~100k (commit message; scripts/frank-dragoon.env:26-27, :183-186).

• The per-request ceiling does NOT move. HALOGEN_CTX stays 200,704, the native wall stays 262,144 (HALOGEN_ROPE_YARN law), and models.yml contextWindow stays 155,648 — the pool is the shared budget behind the slots, not the window. The serve seam enforces HALOGEN_CTX ≤ HALOGEN_KV_POOL_POSITIONS ≤ HALOGEN_KV_SLOTS × HALOGEN_CTX (scripts/serve-frank-dragoon.sh:85-95); 401,408 is slots × ctx, the top of the legal band, and exactly frank-zealot's pool number — parity, not approximation.
• Measured reservation 11.0 GiB — the boot ledger prints reserving the KV pool, 401408 positions (about 11.0 GiB). The preflight KV-budget line bills 11.3 GiB (~28 KiB/position incl. block scratch + ~115 MiB/slot) plus 1.3 GiB of prompt cache. The measured points beside it: 5.5 GiB at 200,704, 7.2 GiB at 262,144, and 14.6 GiB at the 2026-10-06 shape's 524,288 (scripts/frank-dragoon.env:49-52, :74-76).
• That RAM comes out of the n-gram table's page cache — FLAGS.md law: RAM the pool takes is RAM that table loses.

Deployed 2026-10-18 (./scripts/deploy.sh dankclaw --service frank-dragoon, Playwright gate 33/33). The boot ledger confirms the shape: reserving the KV pool, 401408 positions (about 11.0 GiB), 2 slots over ONE 401408-position KV pool, each request up to 200704 (it reserves prompt + max_tokens), and the sum 62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB of KV pool, 5.7 GiB of working memory, 79.6 GiB in all. Prompt cache ON (12 entries × 111 MiB at ctx 200704 = the 1.3 GiB in the KV-budget line), n-gram 47.68 GiB still disk-backed (Pin::None), no clamp line. Host MemAvailable fell 26 → 22 GiB across the raise, which is the FLAGS.md collision being paid in page cache. Warm reuse survived it — measured after the deploy. One 101,987-token wikitext prompt through /v1/chat/completions took 81.5 s cold (1,251 t/s of prefill) and 0.3 s on each of two repeats: HALOGEN_CACHE_INPLACE=1 keeps the KV rows in the pool, so a warm turn prefills nothing and the doubling evicted nothing the cache needed. (The pre-raise 136.7× figure was a ttft reading; this engine sends no Server-Timing header and has no /completion path — /ping, /props, /tokenize are 404 too — so wall-clock around the POST plus usage.prompt_tokens is the only instrument available here.)

Measured A/B — same battery, both boots (2026-10-17)

Behavior probes ran against the live service (tagged swift15 / v2abl); the engine tools ran with the service stopped, per the fleet benchmarking law, under one identical knob set (scripts/census-frank-dragoon.sh).

Probe	Swift1.5 r2 (out)	native v2 (in)
verify on the file	—	OK
NIAH 18 cases (4k/32k × 3 depths × 3 positions)	17/18 (94.4 %)	18/18 (100 %)
wikitext-2 PPL (297,053 ids, 2 × 262,144 seqs)	3.8772	3.9679 (+2.3 %)
KL of the new model against the old model's logits	—	mean 0.155 / median 0.057, top-1 85.1 %
refusal census (5 hostile prompts)	0/5 refusing	0/5 refusing
thinking tokens, mean of 6 questions (temp 1.0)	784	1,476 (×1.9)
tool call, 1-turn / 2-turn	{"city": "Seattle"} / answer	{"city": "Seattle"} / answer
vision (512 px solid red, image_url part)	Red	Red
needle @21,568 tokens, planted 0.85 / 0.10	hit / hit	hit / hit
decode, greedy, 512 tokens	46.8 t/s (spread 25.6–46.8)	50.0 t/s (steady 50.0/50.2)
prefill, cold, 59,024 tokens	1,424.8 t/s	1,604.3 t/s (+12.6 %)
prompt-cache ratio (same prompt, second pass)	136.7×	131.6×
KV pool	14.6 GiB (524,288 positions)	7.2 GiB (262,144)

Read the KL honestly: it is a model-vs-model divergence, not the quant-vs-source comparison the tool was built for. Two different models on the same text should differ; median 0.057 with 85 % top-1 agreement is the signature of two sound models that disagree on content, and the worst rows are date/name completions. A n-gram table mismatched to the checkpoint corrupts the forward pass and shows up as elevated KL everywhere plus degraded retrieval — which NIAH (18/18) and the needle probes rule out.

The decode line needs care: the baseline's own spread was 25.6–46.8 t/s because its samples were taken while the 62 GB fetch was still streaming into page cache, and the candidate's first sample after the battery's long prompts read 32.8 before settling at 50.0/50.2. Honest claim: no decode regression, plausibly faster; never quote either single number.

Verification — how to reproduce

# 1. payload: bytes + LFS sha256 + the hardlinked trio
bash scripts/download-frank-dragoon-v2abl.sh --verify-only
# 2. engine tools on a STOPPED service (fleet law: no GPU experiment beside
#    a live endpoint; stop the unit first — mask fails on a relinked unit)
bash scripts/census-frank-dragoon.sh verify    # no GPU, safe while serving
sudo systemctl stop frank-dragoon
bash scripts/census-frank-dragoon.sh ref       # reference dump, old payload
bash scripts/census-frank-dragoon.sh cand      # PPL + KL/top-1 vs that dump
bash scripts/census-frank-dragoon.sh niah-cur  # baseline retrieval
bash scripts/census-frank-dragoon.sh niah-new  # candidate retrieval
# 3. boot the candidate through a transient drop-in, then the battery
bash scripts/trial-frank-dragoon.sh candidate
python3 scripts/battery-frank-dragoon.py v2abl all
bash scripts/trial-frank-dragoon.sh restore    # back onto the repo env

The battery needs two staged assets in /tmp/halogen-eval/ (a 512 px solid red PNG as base64, red512.b64, and the wikitext-2 raw test split, wikitext-2-raw-test.txt). Laws it encodes:

• the engine bills wikitext at ~4.33 characters/token — the harness chars/4 estimate under-bills by ~8 %, and a prompt over HALOGEN_CTX is refused with a 400 that names the exact overage (never truncated). Size haystacks in characters against a token budget.
• the wikitext-2 raw test split as staged joins rows with a single newline (2,891 lines, zero blank lines), so a blank-line split returns one 1.29 MB "paragraph" and every budget silently becomes "everything".
• the first request after a boot is a warmup artifact; the battery discards it (warmup row) and never measures it.
• ppl/niah load the model → stopped service. verify reads the file back with no GPU and can run while serving.

Contract (unchanged from the Swift1.5 era)

reasoning_content carries the thinking and counts against max_tokens (≥512-token budget on any vision probe or the endpoint looks blind while it is mid-thought); vision via OpenAI image_url content parts only, ≥256 px probes; response_format unsupported; the chat template rejects reasoning_effort outside xhigh|medium|low (the models.yml fold stays mandatory); unknown sampling params are a 400; the prompt cache is on by default, so a warm 100k follow-up costs seconds (measured after the pool raise: 81.5 s cold vs 0.3 s warm at 101,987 prompt tokens). The HTTP surface is thin — /health, /metrics, /v1/models and POST /v1/chat/completions answer, while /ping, /props, /completion and /tokenize are 404 and no response carries Server-Timing: llama.cpp's timing instruments do not exist on this engine, so measure wall-clock plus usage.prompt_tokens. See frank-dragoon-flash-server-v2-swift15 for the engine-flag detail that carries verbatim, and deployment for the deploy/health gate.

Production boot — the deploy ledger (2026-10-17 22:20, fcd7630)

./scripts/deploy.sh dankclaw --service frank-dragoon: healthy after ~12 s, deploy marker fcd7630, e2e gate 33/33. The trial drop-in went to ~/.AGENTTRASH/20261017-v2abl-trial/ FIRST, so this boot is governed by the repo env — systemctl show -p DropInPaths is empty.

Ledger lines that settle the shape (engine stdout, not the journal):

Line	Figure
preflight	MemAvailable now 105.4 GiB against the engine's own estimate of 62.1 weights + 6.0 working + 7.4 KV + 1.3 prompt cache
weights	62.9 GiB of weights locked in RAM (0.8 of them the vision tower's)
KV pool	reserving the KV pool, 262144 positions (about 7.2 GiB); the KV is ONE pool of 262144 positions (6656.0 MiB) carved into per-request regions; slots 1 -> 2 POOLED, 111.0 MiB each of O(1) state
working memory	5.7 GiB at HALOGEN_MAX_TOK 8192
total	75.8 GiB in all — ~25 GiB of the 121.2 GiB box left for co-tenants
n-gram	47.68 GiB from /models/qwen38-flash-next-ngram.hgn, checkpoint: mapped 51.2 GB, NOT pinned (Pin::None) — disk-backed exactly as designed
prompt cache	ON, resume-anywhere, 12 entries, chunk 32768, HALOGEN_CACHE_INPLACE=1 (the KV rows are never copied)
spec decode	MTP depth-2 selectable (drafter 1, speculates while alone)
clamps	none — the requested pool is the delivered pool

Read the two KV figures together: the engine's preflight FORMULA bills 7.4 GiB (~28 KiB/position including block scratch and ~115 MiB/slot of O(1) state) while the RESERVATION is 7.2 GiB / 6,656 MiB. Quote the reservation for the RAM ledger and the formula when comparing pool sizes.

Live probes on the deployed endpoint: 27*59 → 1593 with a warm thinking channel, tool call {"city": "Seattle"}, 512 px solid-red PNG → Red (320 prompt tokens — the image reached the model), greedy 512-token decode 46.9 t/s on the FIRST request after boot, i.e. the warmup artifact; the trial's settled figure is 50.0. OpenWebUI's frank-dragoon row description re-keyed to the native payload (backup webui.db.bak-20261017b; no container restart forced — forcing it logs every family session out for one cosmetic string).

Ceiling lowered to 200,704 — the second production boot (7c4dbeb)

The same evening, after the ledger above was read: HALOGEN_CTX and HALOGEN_KV_POOL_POSITIONS 262144 → 200704 (49 × 4096, the per-slot rung the llama.cpp endpoint frank-zealot already runs). The checkpoint's native window is 262,144 and the engine serves anything below it, so this is a RAM decision, not a capability limit. Nothing in the fleet asks for the bigger window; the harness moved WITH the ceiling (400-zone law), models.yml contextWindow: 155648.

Line	Figure
marker	revision=7c4dbeb…, status=deployed; unit active, /health 200
KV pool	reserving the KV pool, 200704 positions (about 5.5 GiB)
shape	2 slots over ONE 200704-position KV pool, each request up to 200704 (it reserves prompt + max_tokens), max_tok 8192 … MTP depth-2 selectable (drafter 1…)
working	5.7 GiB (unchanged — it scales with HALOGEN_MAX_TOK, not the window)
total	62.9 GiB of weights locked … 5.5 GiB of KV pool, 5.7 GiB of working memory, 74.1 GiB in all
prompt cache	ON, resume-anywhere, 12 entries, chunk 32768, 111 MiB each at ctx 200704
clamps	none — requested pool == delivered pool

The refusal contract at the new ceiling, asked for explicitly (the 1.29 MB wikitext corpus through /v1/chat/completions):

HTTP 400 {"error":{"message":"the prompt does not fit: it is 297109 tokens and
the context is 200704, 96405 tokens over. Shorten the prompt, or raise the
context (HALOGEN_CTX, and HALOGEN_ROPE_YARN for more than the native 262144).",
"type":"invalid_request_error"}}

Refused in 0.86 s — the preflight rejects before any prefill, so probing the ceiling costs the endpoint nothing. OpenWebUI's frank-dragoon row description re-keyed to "ONE 200,704-position KV pool at ctx 200,704" (still no container restart; webui.db.bak-20261017b holds the pre-cutover row).

What the cut cost: with pool == ctx one maximum-length request fills the pool, so two 200k conversations never coexist (two of ≤100 k do), and the harness window is now 155,648 — the 180 k compaction target of the 262,144 era is unreachable because the window itself is smaller. (That pool == ctx concurrency limit IS superseded: the doubling deployed the next day — see Pool doubled for slot parity above. The window math stands, though: the pool never moves the per-request ceiling.) What it bought: 1.7 GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the 2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry.

Third production boot — the 401408 pool, deployed 2026-10-18. Same ctx, same working set, same 111 MiB prompt-cache entry; the pool line reads reserving the KV pool, 401408 positions (about 11.0 GiB) and the total 79.6 GiB in all. Ledger and the measured cost in Pool doubled for slot parity above; host MemAvailable after that boot: 22 GiB.