# frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, Strix Halo iGPU)

**ACTIVE since 2026-10-06.** Rollback = revert the swap commit's trio
(`scripts/frank-dragoon.env`, `scripts/serve-frank-dragoon.sh`,
`scripts/frank-dragoon.service`); the retired 27B payload stays on disk at
`/home/dank/models/halogen-qwen3.8-27b/` and its fetch script
(`scripts/download-frank-dragoon-halogen.sh`) stays in the repo.

## Why this shape exists

Frank's ask: replace the frank-dragoon model with
`https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`.
That payload is an **abliterated Swift1.5 finetune of peonist's
Qwen3.8-Flash-Next v2 checkpoint in the v2 `.hgn` format** — a file that
loads ONLY in the **halogen-flash-server** engine line, NOT in the halogen
0.1.4 container that served the 27B dense checkpoint. The model move is
therefore a **full engine migration**, and the accepted-product facts flip
twice in the fleet's favour:

- the base is **abliterated** (the 27B halogen endpoint was a finetune of
  the censored `Qwen/Qwen3.8-27B`; refusals were an accepted regression there),
- **vision is back** — the v2 vision tower loads beside the checkpoint and
  the engine runs it (the 27B halogen engine silently ignored its encoder),
- **MTP speculative decoding returns** with the head inside the checkpoint
  itself (the 27B shipped a DFlash2 drafter instead; comparable purpose,
  different file plumbing),
- the card claims **53–57 t/s greedy decode, ~1,640 t/s prefill at 34k, ~72 %
  fewer thinking tokens** than the censored 27B (905 mean), and **262,144
  native context**. Publisher-card numbers — verify against live traffic
  before quoting further (see Verification).

The engine is **closed-source**, ships as `ghcr.io/peonist-ai/
halogen-flash-server:0.16.2` (the version the model card's quickstart pins),
digest-pinned:

```
ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
```

(3.6 GB image; RepoDigest captured at pull 2026-10-06. Card-tested fallback
if a future pin misbehaves: 0.15.1.)

## The payload — a triangle of files, all sha256-verified at fetch

| File | Bytes | Source repo @ revision | sha256 (LFS) |
|---|---|---|---|
| `qwen38-flash-next-v2-swift15-abliterated.hgn` | 66,687,678,464 | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated @ main (`69a85db`) | `9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037` |
| `qwen38-flash-next-ngram.hgn` (REQUIRED) | 51,200,246,144 | peonist-ai/halogen-qwen3.8-flash-next @ `5cc17cea` | `3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe` |
| `qwen38-flash-next-vision.hgn` (enabled) | 897,916,416 | peonist-ai/halogen-qwen3.8-flash-next @ `e053f488` | `d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10` |
| `tokenizer/` (6 files, flat dir) | 22.4 MB total | peonist @ `e053f488` | `tokenizer.json` is LFS `0997f410…` (byte-identical to the 27B-era copy); the rest are plain git blobs, size-checked |

Payload notes worth keeping:

- The deployed checkpoint is a **compacted r2** of the card's append-only
  original (89,775,868,032 B → 66,687,678,464 B): the abliteration workflow
  appends and the repo's `tooling/` compact drops dead padding; live tensors
  are byte-identical, the LFS sha256 is the contract, and the 24 GiB saving
  is why it fits.
- **`HALOGEN_MTP_HEAD` stays UNSET**: the MTP head is INSIDE the v2
  checkpoint (the separate `qwen38-flash-next-mtp.hgn` sidecar is for a
  different assembly and is deliberately not fetched).
- The **n-gram lookup table is REQUIRED**. The engine reads it beside the
  checkpoint (documented fallback); the card's quickstart also names it
  explicitly via `HALOGEN_NGRAM_TABLE`, and the serve script passes both —
  the env var is belt-and-braces over the documented beside-the-checkpoint
  default, and both mechanisms resolve to the same file.
- The tokenizer is fetched fresh from peonist @ `e053f488` into
  `/home/dank/models/halogen-flashnext-v2/tokenizer/` — same flat-directory,
  no-symlink law as every other payload (an HF cache snapshot's symlinks
  dangle inside the container mount; both the serve gate and the deployer
  refuse them).

Fetch: `scripts/download-frank-dragoon-flashnext.sh` (`--verify-only` to
re-check; `-f` + exact byte counts + LFS sha256 + resolve-the-CDN-then-`-C -`
resume law throughout; no token — the repos are public).

## Shape — the knobs that are not free

`scripts/frank-dragoon.env` carries the full rationale inline; the table:

| Env | Value | Why |
|---|---|---|
| `HALOGEN_CTX` | 262144 | per-slot HARD ceiling (the checkpoint's native window; a longer prompt is a 400 naming the limit, never truncation). **RAISED from 200192 on 2026-10-07 — the 400-zone law:** the harness clamps `max_tokens = min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts the rendered prompt by ~3 % (measured twice: est 152,938 → server 157,388, request over room by 354; est 158,906 → server 165,301, over by 2,300), and compaction only fires at 85 % of the harness window — so with window == engine ceiling, every request in the est ~146k–184k zone was a guaranteed 400. Harness window 217,088 = ceiling − 45k absorbs the error: worst-case request ≈ 219k ≤ 262,144 (≈10 % under-count headroom) |
| `HALOGEN_KV_SLOTS` | 2 | the 27B engine's "slots>1 disables the drafter" law does NOT carry over — on this engine batching and speculation coexist (the card's quickstart ships 2 slots WITH the head). 2 slots = concurrent capacity |
| `HALOGEN_KV_POOL_POSITIONS` | 524288 | explicit form of the default (2 × ctx), so the seam test pins the product; ~28 KiB/position ⇒ 14.0 GiB KV |
| `HALOGEN_HOST_RESERVE_GIB` | 20 | keeps the host honest (the w4b-era lesson) |
| `HALOGEN_SPEC_ADAPT` | **0** | MANDATORY (card): the default adaptive policy turns the MTP head off for whole requests after low acceptance and **halves decode**. The head stays on; temp-1.0 traffic pays some acceptance for it — accepted trade |
| `HALOGEN_MAX_TOK` / `HALOGEN_MAX_TOKENS_CAP` / `HALOGEN_QUEUE_TIMEOUT` | 8192 / 65536 / 7200 | 8192 is only the per-request DEFAULT (the harness sends at most its `maxTokens` 45,056); the cap/timeout pair is COUPLED per FLAGS.md (65536 at ~10 t/s serial needs > 6550 s) |
| `HALOGEN_TEMPERATURE` / `TOP_P` / `TOP_K` | 1.0 / 0.95 / 20 | the card's thinking recipe as SERVER-SIDE defaults — the engine HAS a sampler now, so an absent temperature is no longer greedy; models.yml extraBody matches it (consistent, not load-bearing) |
| `HALOGEN_WEIGHTS_LOCK` | 1 | card's belt-and-braces; the engine pins weights by default on v2 (`HALOGEN_FLASH_PIN_TRUNK=0` is a w4b-only knob and is REFUSED by v2) |
| `HALOGEN_MODEL_ID` / publish | **frank-dragoon** / 7770:8731 | served id renamed 2026-10-09 (service/unit/env keep `frank-dragoon`); engine port 8730 never published |
| MTP depth | default 2 | FLAGS.md: 2 is best all-round, 3 for code — Frank's traffic is agent/prose |

**Memory ledger** (121.2 GiB unified, the referee is the boot ledger):
weights ~62 GiB resident + KV 14.0 + arena ~8 (`3.8 + 8.4 × 8192/16384`)
+ vision ~2 ≈ **86 GiB engine-side**, + 20 host reserve = ~106 with
~15 GiB margin over the box's baseline co-tenancy. The 51.2 GB n-gram
table rides as mmap'd page cache beside the checkpoint — the card's own
quickstart arithmetic does not bill it as resident, but WATCH the ledger
at boot and re-size if the engine prints a clamp. Prompt cache is ON by
default (`PROMPT_CACHE=2`): a follow-up turn at 100k ctx costs ~2 s
instead of ~88 s cold — the big-TTFT win the 27B engine needed a manual
`HALOGEN_CACHE_MB` pin for is built in here.

## Engine contract facts (FLAGS.md of the flash-server repo)

- OpenAI chat **and** `/v1/responses` **and** `/v1/messages` **and**
  `/v1/completions`; tool calls verified on the card; `response_format`
  still unsupported (no constrained decoding).
- Thinking → **`reasoning_content`** (same field as every engine this
  endpoint has run); the v2 chat template rejects `reasoning_effort`
  outside `xhigh|medium|low` — the models.yml fold (high→xhigh) stays
  mandatory.
- Vision: OpenAI `image_url` content parts only (the fleet's standard
  contract; the legacy top-level `images` field is the frank-zealot trap).
  ≥256 px probe images or the model answers "video".
- `HALOGEN_MAX_TOKENS_CAP` over-cap is a 400 naming the limit — never a
  truncation. `max_tokens` INCLUDES thinking.
- All flags read once at startup; every flag is BITWISE except
  `HALOGEN_CK_OVERLAY`, `HALOGEN_MATMUL_TUNING_FILE`,
  `HALOGEN_INDEXER_BUDGET`, `HALOGEN_COMPOSABLE_CONTEXT`. Unset
  `HALOGEN_CHECKPOINT` defaults to `/models/qwen38-flash-next-v2.hgn`
  with the lookup table beside it.
- Prompt-lookup drafting (3,3 window) rides beside the MTP head by
  default and is byte-identical to greedy verification where it fires.
- Watchdog: `HALOGEN_ENGINE_WATCHDOG_S=180` with defer-logic for memory
  stalls — on a shared host the defaults are the safe choice; the
  restart policy stays real through the root unit's `Restart=always`.

Kernel: `CONFIG_HSA_AMD_SVM=y` (CachyOS ships it); `amdgpu.noretry=-1`
(auto) — for this IP auto resolves to retry-on, the same effective state
the 27B engine ran on. Only revisit if the ledger shows SVM complaints.

## Mechanics

- **Root systemd unit** (`scripts/frank-dragoon.service`, `User=dank`,
  `LimitMEMLOCK=infinity`, `Restart=always`, `ExecStop=podman stop -t 60
  frank-dragoon-halogen`), driven by `scripts/serve-frank-dragoon.sh` — the
  fleet's `podman run`-direct pattern. The script runs the container
  FOREGROUND so the boot ledger (weights resident, pool fit, MTP head,
  prompt cache, any clamp) lands in `/home/dank/logs/frank_dragoon.log`.
- Serve gates mirror the deployer EXACTLY (2026-10-01 crash-loop law):
  image digest in podman storage, byte-exact checkpoint + n-gram table +
  vision tower, flat tokenizer with all six files as regular files,
  GPU nodes present.
  - **Host-RAM pin-gate preflight** (added 2026-10-06): the engine
  refuses to pin when `MemAvailable` is below the checkpoint (62.11 GiB)
  plus a **16 GiB floor** — pinned pages cannot be swapped, and under the
  floor the n-gram lookup pages in from disk on every long prompt. A
  RAM-short host crash-loops the unit (each boot dies in ~5 s at the pin
  gate) until the RAM clears; the 2026-10-06 cutover hit this (8 boots,
  MemAvailable 74.77 GiB vs the 78.11 GiB needed, self-resolved in ~2 min
  when the dying 27B engine freed RAM). The serve script now checks
  `MemAvailable` BEFORE `podman run` and WAITS up to `FRANK_DRAGOON_RAM_WAIT`
  (default 600 s) for the RAM to clear, so a transient dip boots once
  instead of crash-looping; if it never clears it refuses loudly. The 16
  GiB floor is the engine's (not `HALOGEN_HOST_RESERVE_GIB=20` — a
  separate, higher, overall-budget knob).
- Engine stdout goes to the log, NOT the journal. The FIRST request after
  a (re)start can sit silent ~45–60 s on this engine class (health
  `engine_unresponsive`, 503s) and self-recovers — do not restart over it.

## Verification checklist (run after any redeploy)

1. Boot ledger greps in `/home/dank/logs/frank_dragoon.log`: resident
   weights (~62 GiB), KV pool line, MTP/drafter line, prompt-cache line,
   **no clamp named**.
2. `curl http://192.168.0.143:7770/health` → 200.
3. Text completion probe: reasoning lands in `reasoning_content`,
   `timings` honest (generate ≥128 tokens; at 8 the inter_token number is
   first-token noise).
4. Vision probe: a 512×512 solid-red PNG through an `image_url` part →
   "Red" (≥256 px, one word expected).
5. Spec decode: `draft_n`/`draft_n_accepted` present in timings on
   real-shaped traffic (SPEC_ADAPT=0 means the head is always engaged).

## Client contract (harness)

`~/.omp/agent/models.yml` — provider `llamacpp (dankclaw)` @
`http://192.168.0.143:7770/v1`: `contextWindow: 217088` — deliberately
BELOW the 262,144 engine ceiling: the 45k gap absorbs the harness's
~3 % chars/4 est under-count (see the `HALOGEN_CTX` row above) —
`maxTokens: 45056`, `input: [text, image]`,
`reasoningContentField: "reasoning_content"`, the `high→xhigh` effort
fold, `extraBody` temperature 1.0 / top_p 0.95. Compaction still rides
frank-zealot (the `compactionModel` steering is unchanged; the input
≈ 170k fits its 200,704 slot).

## Rollback

Revert the swap commit and redeploy — the 27B trio is intact on disk
(payload at `~/models/halogen-qwen3.8-27b/`, fetch script, and the halogen
0.1.4 image digest still in podman storage). The older ROCmFPX rollback
door (`build-rocmfpx.sh` pin) is untouched.