docs(frank-dragoon): native-era runbook; retire the swift15 page
New LLM/frank-dragoon-flash-server-v2-native.md carries the shape with the three deliberate deviations from the card (served id, SPEC_ADAPT=0, engine pin 0.16.2), the memory ledger with the PIN_TRUNK rejection, the full measured A/B table, and the companion law (hardlinked stock peonist files, engine-tool certified). The swift15 page is re-banned RETIRED 2026-10-06 → 10-17 with its rollback terms; both indexes and AGENTS.md (topology table, service section, runbook index) are re-keyed, and the root index also stops describing the halogen-27B page as ACTIVE.
This commit is contained in:
Frank committed
1 parent
e36f6a732c
commit
13162e6615
5 files changed
+294
-69
No files matched your search
@@ -0,0 +1,187 @@
|
||||
# frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 NATIVE abliterated (Dankclaw, Strix Halo iGPU)
|
||||
|
||||
**ACTIVE since 2026-10-17.** Rollback = revert the swap commit and redeploy
|
||||
(`scripts/frank-dragoon.env`, `scripts/serve-frank-dragoon.sh`,
|
||||
`scripts/frank-dragoon.service`); the Swift1.5 payload stays on disk at
|
||||
`/home/dank/models/halogen-flashnext-v2/` (its fetch script
|
||||
`scripts/download-frank-dragoon-flashnext.sh` stays the door back), and the
|
||||
27B-era payload beside that at `/home/dank/models/halogen-qwen3.8-27b/`.
|
||||
Era write-up: [[frank-dragoon-flash-server-v2-swift15]].
|
||||
|
||||
## Why this payload
|
||||
|
||||
Frank's ask: serve
|
||||
`https://huggingface.co/Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated`
|
||||
with the card's recommended settings, and **save memory**.
|
||||
|
||||
This is the **native 2026-10-11 v2 conversion** of orcarouter's pinned BF16
|
||||
Qwen3.8-Flash-Next, abliterated — the base itself, converted straight into
|
||||
the v2 `.hgn` format. It is **not** the writer transplant that carried this
|
||||
filename before: the artifact measures **66,687,678,432 B** where the
|
||||
Swift1.5 r2 measured 66,687,678,464 B — 32 bytes apart, same filename
|
||||
family, different model. Byte counts and the Hub LFS sha256 are the only way
|
||||
to tell them apart, which is why the fetch script and both gates check them.
|
||||
|
||||
What the swap buys, measured (table below): retrieval at depth, decode,
|
||||
prefill, and **7.4 GiB of RAM handed back**. What it costs: the Swift1.5
|
||||
finetune's compressed reasoning and its small perplexity edge.
|
||||
|
||||
## The companion law (the reason this swap needed a referee)
|
||||
|
||||
The release publishes **only the main file**. The n-gram lookup table
|
||||
(REQUIRED — the engine reads it beside the checkpoint), the vision tower,
|
||||
and the tokenizer are peonist's, and the card does **not** establish them as
|
||||
interchangeable with this native conversion. No source-matched pair is
|
||||
published anywhere, so the only available companions are the stock ones.
|
||||
|
||||
They are **hardlinked** out of the verified Swift1.5 payload directory
|
||||
(`scripts/download-frank-dragoon-v2abl.sh`): a hardlink is a regular file, so
|
||||
it satisfies the serve gate's no-symlink law, and it costs no disk and no
|
||||
copy next to a 62 GB payload. The engine's own checkpoint tools are the
|
||||
referee, and they answered: **NIAH 18/18** (vs 17/18 for the payload those
|
||||
files were published with), needle retrieval exact at both depths, the red-PNG
|
||||
vision probe exact, perplexity in the normal band. A wrong PLE table does not
|
||||
whisper — see Verification.
|
||||
|
||||
## The payload — one new file, three inherited ones
|
||||
|
||||
`/home/dank/models/halogen-v2-abliterated/`, fetched by
|
||||
`scripts/download-frank-dragoon-v2abl.sh` (`-f` + exact byte counts + LFS
|
||||
sha256 + `--verify-only`; no token on the command line):
|
||||
|
||||
|File|Bytes|sha256|Source|
|
||||
|---|---|---|---|
|
||||
|`qwen38-flash-next-v2-abliterated.hgn`|66,687,678,432|`b5bd925f…`|Quat3rnion `@main`, fetched|
|
||||
|`qwen38-flash-next-ngram.hgn`|51,200,246,144|`3450acada…`|peonist `@5cc17cea`, **hardlinked**, REQUIRED|
|
||||
|`qwen38-flash-next-vision.hgn`|897,916,416|`d62e0ae5…`|peonist `@e053f488`, hardlinked, vision ON|
|
||||
|`tokenizer/` (6 files)| — |`tokenizer.json` LFS|peonist `@e053f488`, hardlinked|
|
||||
|
||||
The engine stays **halogen-flash-server 0.16.2**, digest-pinned (the card
|
||||
tested 0.15.0; 0.16.2 is newer in the same line and is what the v2 format was
|
||||
qualified on in this fleet):
|
||||
|
||||
```
|
||||
ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
||||
```
|
||||
|
||||
## Shape — the card's settings, and the three deliberate deviations
|
||||
|
||||
`HALOGEN_CTX=262144` (per-request ceiling), `HALOGEN_KV_SLOTS=2`,
|
||||
**`HALOGEN_KV_POOL_POSITIONS=262144`** (the card's recommendation, and the
|
||||
memory saving), `HALOGEN_SPEC_ADAPT=0`, `HALOGEN_MAX_TOK=8192`,
|
||||
`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` (coupled pair),
|
||||
sampler `1.0 / 0.95 / 20`, `HALOGEN_WEIGHTS_LOCK=1`,
|
||||
`HALOGEN_HOST_RESERVE_GIB=20`.
|
||||
|
||||
Deviations from the card, each on purpose:
|
||||
|
||||
- `HALOGEN_MODEL_ID=frank-dragoon`, not the card's `halogen-v2-abliterated` —
|
||||
the fleet naming law (service = unit = served id).
|
||||
- `HALOGEN_SPEC_ADAPT=0` — the card is silent; this is a measured 0.16.2 law
|
||||
(the adaptive default disables the drafter per request and halves decode).
|
||||
- the sampler values are peonist's engine recipe; Quat3rnion publishes none.
|
||||
|
||||
**The pool is one budget, not a partition.** Boot ledger: `listening on
|
||||
127.0.0.1:8730, 2 slots over ONE 262144-position KV pool, each request up to
|
||||
262144`. The per-request ceiling is unchanged, so the harness contract
|
||||
(`contextWindow: 217088`) does not move — what changed is that **two
|
||||
maximum-length conversations can no longer share the endpoint**. That is
|
||||
acceptable here because frank-dragoon carries the `slow` (medium-thinking)
|
||||
role and the `security-reviewer` subagent, not the interactive workhorse
|
||||
(that is frank-archon).
|
||||
|
||||
**Memory ledger** (121.2 GiB unified), from the boot ledger of the live
|
||||
candidate: `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
|
||||
`0.84 GiB` vision tower, `reserving the KV pool, 262144 positions (about
|
||||
7.2 GiB)`, and the engine's own sum: `memory: 62.9 GiB of weights locked in
|
||||
RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …`. The n-gram
|
||||
table rides as page cache — `mapped 51.2 GB, NOT pinned (Pin::None)` — which
|
||||
is exactly why cutting the pool is a *double* saving. FLAGS.md law: **RAM the
|
||||
pool takes is RAM that table loses**; the 7.4 GiB the smaller pool hands back
|
||||
is page cache for the 47.68 GiB table that long prompts read.
|
||||
|
||||
`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
|
||||
stays pinned. Disk-backing the weights is not a lever on this format.
|
||||
|
||||
## Measured A/B — same battery, both boots (2026-10-17)
|
||||
|
||||
Behavior probes ran against the live service (tagged `swift15` / `v2abl`);
|
||||
the engine tools ran with the service **stopped**, per the fleet benchmarking
|
||||
law, under one identical knob set (`scripts/census-frank-dragoon.sh`).
|
||||
|
||||
|Probe|Swift1.5 r2 (out)|native v2 (in)|
|
||||
|---|---|---|
|
||||
|`verify` on the file|—|OK|
|
||||
|NIAH 18 cases (4k/32k × 3 depths × 3 positions)|17/18 (94.4 %)|**18/18 (100 %)**|
|
||||
|wikitext-2 PPL (297,053 ids, 2 × 262,144 seqs)|**3.8772**|3.9679 (+2.3 %)|
|
||||
|KL of the new model against the old model's logits|—|mean 0.155 / median 0.057, top-1 85.1 %|
|
||||
|refusal census (5 hostile prompts)|0/5 refusing|0/5 refusing|
|
||||
|thinking tokens, mean of 6 questions (temp 1.0)|**784**|1,476 (×1.9)|
|
||||
|tool call, 1-turn / 2-turn|`{"city": "Seattle"}` / answer|`{"city": "Seattle"}` / answer|
|
||||
|vision (512 px solid red, `image_url` part)|Red|Red|
|
||||
|needle @21,568 tokens, planted 0.85 / 0.10|hit / hit|hit / hit|
|
||||
|decode, greedy, 512 tokens|46.8 t/s (spread 25.6–46.8)|50.0 t/s (steady 50.0/50.2)|
|
||||
|prefill, cold, 59,024 tokens|1,424.8 t/s|**1,604.3 t/s (+12.6 %)**|
|
||||
|prompt-cache ratio (same prompt, second pass)|136.7×|131.6×|
|
||||
|KV pool|14.6 GiB (524,288 positions)|**7.2 GiB (262,144)**|
|
||||
|
||||
Read the KL honestly: it is a **model-vs-model** divergence, not the
|
||||
quant-vs-source comparison the tool was built for. Two different models on
|
||||
the same text *should* differ; median 0.057 with 85 % top-1 agreement is the
|
||||
signature of two sound models that disagree on content, and the worst rows
|
||||
are date/name completions. A n-gram table mismatched to the checkpoint
|
||||
corrupts the forward pass and shows up as elevated KL **everywhere** plus
|
||||
degraded retrieval — which NIAH (18/18) and the needle probes rule out.
|
||||
|
||||
The decode line needs care: the baseline's own spread was 25.6–46.8 t/s
|
||||
because its samples were taken while the 62 GB fetch was still streaming into
|
||||
page cache, and the candidate's **first** sample after the battery's long
|
||||
prompts read 32.8 before settling at 50.0/50.2. Honest claim: **no decode
|
||||
regression, plausibly faster**; never quote either single number.
|
||||
|
||||
## Verification — how to reproduce
|
||||
|
||||
```
|
||||
# 1. payload: bytes + LFS sha256 + the hardlinked trio
|
||||
bash scripts/download-frank-dragoon-v2abl.sh --verify-only
|
||||
# 2. engine tools on a STOPPED service (fleet law: no GPU experiment beside
|
||||
# a live endpoint; stop the unit first — mask fails on a relinked unit)
|
||||
bash scripts/census-frank-dragoon.sh verify # no GPU, safe while serving
|
||||
sudo systemctl stop frank-dragoon
|
||||
bash scripts/census-frank-dragoon.sh ref # reference dump, old payload
|
||||
bash scripts/census-frank-dragoon.sh cand # PPL + KL/top-1 vs that dump
|
||||
bash scripts/census-frank-dragoon.sh niah-cur # baseline retrieval
|
||||
bash scripts/census-frank-dragoon.sh niah-new # candidate retrieval
|
||||
# 3. boot the candidate through a transient drop-in, then the battery
|
||||
bash scripts/trial-frank-dragoon.sh candidate
|
||||
python3 scripts/battery-frank-dragoon.py v2abl all
|
||||
bash scripts/trial-frank-dragoon.sh restore # back onto the repo env
|
||||
```
|
||||
|
||||
The battery needs two staged assets in `/tmp/halogen-eval/` (a 512 px solid
|
||||
red PNG as base64, `red512.b64`, and the wikitext-2 raw test split,
|
||||
`wikitext-2-raw-test.txt`). Laws it encodes:
|
||||
|
||||
- **the engine bills wikitext at ~4.33 characters/token** — the harness
|
||||
chars/4 estimate under-bills by ~8 %, and a prompt over `HALOGEN_CTX` is
|
||||
refused with a 400 that names the exact overage (never truncated). Size
|
||||
haystacks in characters against a **token** budget.
|
||||
- the wikitext-2 raw test split as staged joins rows with a **single**
|
||||
newline (2,891 lines, zero blank lines), so a blank-line split returns one
|
||||
1.29 MB "paragraph" and every budget silently becomes "everything".
|
||||
- the first request after a boot is a warmup artifact; the battery discards
|
||||
it (`warmup` row) and never measures it.
|
||||
- `ppl`/`niah` **load the model** → stopped service. `verify` reads the file
|
||||
back with no GPU and can run while serving.
|
||||
|
||||
## Contract (unchanged from the Swift1.5 era)
|
||||
|
||||
`reasoning_content` carries the thinking and counts against `max_tokens`
|
||||
(≥512-token budget on any vision probe or the endpoint looks blind while it
|
||||
is mid-thought); vision via OpenAI `image_url` content parts only, ≥256 px
|
||||
probes; `response_format` unsupported; the chat template rejects
|
||||
`reasoning_effort` outside `xhigh|medium|low` (the `models.yml` fold stays
|
||||
mandatory); unknown sampling params are a 400; the prompt cache is on by
|
||||
default, so a warm 100k follow-up costs seconds. See
|
||||
[[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that
|
||||
carries verbatim, and [[deployment]] for the deploy/health gate.
|
||||
Reference in new issue
Block a user