serverAppConfigs/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md

# frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 NATIVE abliterated (Dankclaw, Strix Halo iGPU)

**ACTIVE since 2026-10-17.** Rollback = revert the swap commit and redeploy
(`scripts/frank-dragoon.env`, `scripts/serve-frank-dragoon.sh`,
`scripts/frank-dragoon.service`); the Swift1.5 payload stays on disk at
`/home/dank/models/halogen-flashnext-v2/` (its fetch script
`scripts/download-frank-dragoon-flashnext.sh` stays the door back), and the
27B-era payload beside that at `/home/dank/models/halogen-qwen3.8-27b/`.
Era write-up: [[frank-dragoon-flash-server-v2-swift15]].

## Why this payload

Frank's ask: serve
`https://huggingface.co/Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated`
with the card's recommended settings, and **save memory**.

This is the **native 2026-10-11 v2 conversion** of orcarouter's pinned BF16
Qwen3.8-Flash-Next, abliterated — the base itself, converted straight into
the v2 `.hgn` format. It is **not** the writer transplant that carried this
filename before: the artifact measures **66,687,678,432 B** where the
Swift1.5 r2 measured 66,687,678,464 B — 32 bytes apart, same filename
family, different model. Byte counts and the Hub LFS sha256 are the only way
to tell them apart, which is why the fetch script and both gates check them.

What the swap buys, measured (table below): retrieval at depth, decode,
prefill, and **7.4 GiB of RAM handed back**. What it costs: the Swift1.5
finetune's compressed reasoning and its small perplexity edge.

## The companion law (the reason this swap needed a referee)

The release publishes **only the main file**. The n-gram lookup table
(REQUIRED — the engine reads it beside the checkpoint), the vision tower,
and the tokenizer are peonist's, and the card does **not** establish them as
interchangeable with this native conversion. No source-matched pair is
published anywhere, so the only available companions are the stock ones.

They are **hardlinked** out of the verified Swift1.5 payload directory
(`scripts/download-frank-dragoon-v2abl.sh`): a hardlink is a regular file, so
it satisfies the serve gate's no-symlink law, and it costs no disk and no
copy next to a 62 GB payload. The engine's own checkpoint tools are the
referee, and they answered: **NIAH 18/18** (vs 17/18 for the payload those
files were published with), needle retrieval exact at both depths, the red-PNG
vision probe exact, perplexity in the normal band. A wrong PLE table does not
whisper — see Verification.

## The payload — one new file, three inherited ones

`/home/dank/models/halogen-v2-abliterated/`, fetched by
`scripts/download-frank-dragoon-v2abl.sh` (`-f` + exact byte counts + LFS
sha256 + `--verify-only`; no token on the command line):

|File|Bytes|sha256|Source|
|---|---|---|---|
|`qwen38-flash-next-v2-abliterated.hgn`|66,687,678,432|`b5bd925f…`|Quat3rnion `@main`, fetched|
|`qwen38-flash-next-ngram.hgn`|51,200,246,144|`3450acada…`|peonist `@5cc17cea`, **hardlinked**, REQUIRED|
|`qwen38-flash-next-vision.hgn`|897,916,416|`d62e0ae5…`|peonist `@e053f488`, hardlinked, vision ON|
|`tokenizer/` (6 files)| — |`tokenizer.json` LFS|peonist `@e053f488`, hardlinked|

The engine stays **halogen-flash-server 0.16.2**, digest-pinned (the card
tested 0.15.0; 0.16.2 is newer in the same line and is what the v2 format was
qualified on in this fleet):

```
ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
```

## Shape — the card's settings, and the three deliberate deviations

`HALOGEN_CTX=200704` (per-request ceiling — the card's 262,144 lowered the
same evening, see below), `HALOGEN_KV_SLOTS=2`,
**`HALOGEN_KV_POOL_POSITIONS=200704`** (one pool behind both slots; the card
recommends it equal to ctx, and that equality is where the memory is),
`HALOGEN_SPEC_ADAPT=0`, `HALOGEN_MAX_TOK=8192`,
`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` (coupled pair),
sampler `1.0 / 0.95 / 20`, `HALOGEN_WEIGHTS_LOCK=1`,
`HALOGEN_HOST_RESERVE_GIB=20`.

Deviations from the card, each on purpose:

- `HALOGEN_MODEL_ID=frank-dragoon`, not the card's `halogen-v2-abliterated` —
  the fleet naming law (service = unit = served id).
- `HALOGEN_SPEC_ADAPT=0` — the card is silent; this is a measured 0.16.2 law
  (the adaptive default disables the drafter per request and halves decode).
- the sampler values are peonist's engine recipe; Quat3rnion publishes none.

**The pool is one budget, not a partition.** Boot ledger form: `listening on
127.0.0.1:8730, 2 slots over ONE <pool>-position KV pool, each request up to
<HALOGEN_CTX>`. Both numbers were lowered to **200,704** the same evening
(the card ships 262,144; Frank took the RAM — nothing else in the fleet asks
for the bigger window), so `models.yml` `contextWindow` moved to **155,648**
= ceiling − 45k with it. With pool == ctx, one maximum-length request fills
the pool: **two long conversations can no longer share the endpoint** (two of
≤100k can). That is acceptable here because frank-dragoon carries the `slow`
(medium-thinking) role and the `security-reviewer` subagent, not the
interactive workhorse (that is frank-archon).

**Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot
printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
`0.84 GiB` vision tower, `reserving the KV pool, 262144 positions (about
7.2 GiB)`, and the engine's own sum: `memory: 62.9 GiB of weights locked in
RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …, 75.8 GiB in
all`. The cut boot printed the same shape at the new size: `reserving the KV
pool, 200704 positions (about 5.5 GiB)` and `… 5.5 GiB of KV pool, 5.7 GiB of
working memory, 74.1 GiB in all`.
The n-gram table rides as page cache — `mapped 51.2 GB,
NOT pinned (Pin::None)` — which is exactly why every pool cut is a *double*
saving. FLAGS.md law: **RAM the pool takes is RAM that table loses**; the ~9
GiB the two cuts together handed back is page cache for the 47.68 GiB table
that long prompts read.

`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
stays pinned. Disk-backing the weights is not a lever on this format.

## Measured A/B — same battery, both boots (2026-10-17)

Behavior probes ran against the live service (tagged `swift15` / `v2abl`);
the engine tools ran with the service **stopped**, per the fleet benchmarking
law, under one identical knob set (`scripts/census-frank-dragoon.sh`).

|Probe|Swift1.5 r2 (out)|native v2 (in)|
|---|---|---|
|`verify` on the file|—|OK|
|NIAH 18 cases (4k/32k × 3 depths × 3 positions)|17/18 (94.4 %)|**18/18 (100 %)**|
|wikitext-2 PPL (297,053 ids, 2 × 262,144 seqs)|**3.8772**|3.9679 (+2.3 %)|
|KL of the new model against the old model's logits|—|mean 0.155 / median 0.057, top-1 85.1 %|
|refusal census (5 hostile prompts)|0/5 refusing|0/5 refusing|
|thinking tokens, mean of 6 questions (temp 1.0)|**784**|1,476 (×1.9)|
|tool call, 1-turn / 2-turn|`{"city": "Seattle"}` / answer|`{"city": "Seattle"}` / answer|
|vision (512 px solid red, `image_url` part)|Red|Red|
|needle @21,568 tokens, planted 0.85 / 0.10|hit / hit|hit / hit|
|decode, greedy, 512 tokens|46.8 t/s (spread 25.6–46.8)|50.0 t/s (steady 50.0/50.2)|
|prefill, cold, 59,024 tokens|1,424.8 t/s|**1,604.3 t/s (+12.6 %)**|
|prompt-cache ratio (same prompt, second pass)|136.7×|131.6×|
|KV pool|14.6 GiB (524,288 positions)|**7.2 GiB (262,144)**|

Read the KL honestly: it is a **model-vs-model** divergence, not the
quant-vs-source comparison the tool was built for. Two different models on
the same text *should* differ; median 0.057 with 85 % top-1 agreement is the
signature of two sound models that disagree on content, and the worst rows
are date/name completions. A n-gram table mismatched to the checkpoint
corrupts the forward pass and shows up as elevated KL **everywhere** plus
degraded retrieval — which NIAH (18/18) and the needle probes rule out.

The decode line needs care: the baseline's own spread was 25.6–46.8 t/s
because its samples were taken while the 62 GB fetch was still streaming into
page cache, and the candidate's **first** sample after the battery's long
prompts read 32.8 before settling at 50.0/50.2. Honest claim: **no decode
regression, plausibly faster**; never quote either single number.

## Verification — how to reproduce

```
# 1. payload: bytes + LFS sha256 + the hardlinked trio
bash scripts/download-frank-dragoon-v2abl.sh --verify-only
# 2. engine tools on a STOPPED service (fleet law: no GPU experiment beside
#    a live endpoint; stop the unit first — mask fails on a relinked unit)
bash scripts/census-frank-dragoon.sh verify    # no GPU, safe while serving
sudo systemctl stop frank-dragoon
bash scripts/census-frank-dragoon.sh ref       # reference dump, old payload
bash scripts/census-frank-dragoon.sh cand      # PPL + KL/top-1 vs that dump
bash scripts/census-frank-dragoon.sh niah-cur  # baseline retrieval
bash scripts/census-frank-dragoon.sh niah-new  # candidate retrieval
# 3. boot the candidate through a transient drop-in, then the battery
bash scripts/trial-frank-dragoon.sh candidate
python3 scripts/battery-frank-dragoon.py v2abl all
bash scripts/trial-frank-dragoon.sh restore    # back onto the repo env
```

The battery needs two staged assets in `/tmp/halogen-eval/` (a 512 px solid
red PNG as base64, `red512.b64`, and the wikitext-2 raw test split,
`wikitext-2-raw-test.txt`). Laws it encodes:

- **the engine bills wikitext at ~4.33 characters/token** — the harness
  chars/4 estimate under-bills by ~8 %, and a prompt over `HALOGEN_CTX` is
  refused with a 400 that names the exact overage (never truncated). Size
  haystacks in characters against a **token** budget.
- the wikitext-2 raw test split as staged joins rows with a **single**
  newline (2,891 lines, zero blank lines), so a blank-line split returns one
  1.29 MB "paragraph" and every budget silently becomes "everything".
- the first request after a boot is a warmup artifact; the battery discards
  it (`warmup` row) and never measures it.
- `ppl`/`niah` **load the model** → stopped service. `verify` reads the file
  back with no GPU and can run while serving.

## Contract (unchanged from the Swift1.5 era)

`reasoning_content` carries the thinking and counts against `max_tokens`
(≥512-token budget on any vision probe or the endpoint looks blind while it
is mid-thought); vision via OpenAI `image_url` content parts only, ≥256 px
probes; `response_format` unsupported; the chat template rejects
`reasoning_effort` outside `xhigh|medium|low` (the `models.yml` fold stays
mandatory); unknown sampling params are a 400; the prompt cache is on by
default, so a warm 100k follow-up costs seconds. See
[[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that
carries verbatim, and [[deployment]] for the deploy/health gate.

## Production boot — the deploy ledger (2026-10-17 22:20, `fcd7630`)

`./scripts/deploy.sh dankclaw --service frank-dragoon`: healthy after ~12 s,
deploy marker `fcd7630`, e2e gate 33/33. The trial drop-in went to
`~/.AGENTTRASH/20261017-v2abl-trial/` FIRST, so this boot is governed by the
repo env — `systemctl show -p DropInPaths` is empty.

Ledger lines that settle the shape (engine stdout, not the journal):

|Line|Figure|
|---|---|
|preflight|`MemAvailable now 105.4 GiB` against the engine's own estimate of 62.1 weights + 6.0 working + 7.4 KV + 1.3 prompt cache|
|weights|`62.9 GiB of weights locked in RAM (0.8 of them the vision tower's)`|
|KV pool|`reserving the KV pool, 262144 positions (about 7.2 GiB)`; `the KV is ONE pool of 262144 positions (6656.0 MiB) carved into per-request regions`; slots `1 -> 2 POOLED, 111.0 MiB each of O(1) state`|
|working memory|5.7 GiB at `HALOGEN_MAX_TOK 8192`|
|total|`75.8 GiB in all` — ~25 GiB of the 121.2 GiB box left for co-tenants|
|n-gram|`47.68 GiB from /models/qwen38-flash-next-ngram.hgn`, `checkpoint: mapped 51.2 GB, NOT pinned (Pin::None)` — disk-backed exactly as designed|
|prompt cache|ON, resume-anywhere, 12 entries, chunk 32768, `HALOGEN_CACHE_INPLACE=1` (the KV rows are never copied)|
|spec decode|`MTP depth-2 selectable (drafter 1, speculates while alone)`|
|clamps|none — the requested pool is the delivered pool|

Read the two KV figures together: the engine's preflight FORMULA bills 7.4 GiB
(~28 KiB/position **including** block scratch and ~115 MiB/slot of O(1) state)
while the RESERVATION is 7.2 GiB / 6,656 MiB. Quote the reservation for the
RAM ledger and the formula when comparing pool sizes.

Live probes on the deployed endpoint: `27*59` → `1593` with a warm thinking
channel, tool call `{"city": "Seattle"}`, 512 px solid-red PNG → **Red** (320
prompt tokens — the image reached the model), greedy 512-token decode 46.9 t/s
on the FIRST request after boot, i.e. the warmup artifact; the trial's settled
figure is 50.0. OpenWebUI's `frank-dragoon` row description re-keyed to the
native payload (backup `webui.db.bak-20261017b`; no container restart forced —
forcing it logs every family session out for one cosmetic string).

## Ceiling lowered to 200,704 — the second production boot (`7c4dbeb`)

The same evening, after the ledger above was read: `HALOGEN_CTX` and
`HALOGEN_KV_POOL_POSITIONS` 262144 → **200704** (49 × 4096, the per-slot rung
the llama.cpp endpoint frank-zealot already runs). The checkpoint's native
window is 262,144 and the engine serves anything below it, so this is a RAM
decision, not a capability limit. Nothing in the fleet asks for the bigger
window; the harness moved WITH the ceiling (400-zone law), `models.yml`
`contextWindow: 155648`.

|Line|Figure|
|---|---|
|marker|`revision=7c4dbeb…`, `status=deployed`; unit `active`, `/health` 200|
|KV pool|`reserving the KV pool, 200704 positions (about 5.5 GiB)`|
|shape|`2 slots over ONE 200704-position KV pool, each request up to 200704 (it reserves prompt + max_tokens), max_tok 8192 … MTP depth-2 selectable (drafter 1…)`|
|working|5.7 GiB (unchanged — it scales with `HALOGEN_MAX_TOK`, not the window)|
|total|`62.9 GiB of weights locked … 5.5 GiB of KV pool, 5.7 GiB of working memory, 74.1 GiB in all`|
|prompt cache|ON, resume-anywhere, 12 entries, chunk 32768, **111 MiB each at ctx 200704**|
|clamps|none — requested pool == delivered pool|

The refusal contract at the new ceiling, asked for explicitly (the 1.29 MB
wikitext corpus through `/v1/chat/completions`):

```
HTTP 400 {"error":{"message":"the prompt does not fit: it is 297109 tokens and
the context is 200704, 96405 tokens over. Shorten the prompt, or raise the
context (HALOGEN_CTX, and HALOGEN_ROPE_YARN for more than the native 262144).",
"type":"invalid_request_error"}}
```

Refused in 0.86 s — the preflight rejects before any prefill, so probing the
ceiling costs the endpoint nothing. OpenWebUI's `frank-dragoon` row description
re-keyed to "ONE 200,704-position KV pool at ctx 200,704" (still no container
restart; `webui.db.bak-20261017b` holds the pre-cutover row).

**What the cut cost:** with pool == ctx one maximum-length request fills the
pool, so two 200k conversations never coexist (two of ≤100 k do), and the
harness window is now 155,648 — the 180 k compaction target of the 262,144 era
is unreachable because the window itself is smaller. **What it bought:** 1.7
GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the
2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry.