# frank-dragoon — halogen container, Qwen3.8-27B + DFlash2 (Dankclaw, ACTIVE 2026-10-05)

frank-dragoon is the Strix Halo iGPU endpoint on Dankclaw (`:7770`). Since
2026-10-05 it runs the **halogen** engine — `ghcr.io/peonist-ai/halogen` at a
pinned digest, launched by `scripts/serve-frank-dragoon.sh` under a root
systemd unit as a podman container — serving
**`peonist-ai/halogen-qwen3.8-27b`** (`qwen3.8-27b-p1w4d-d2.hgn`,
35,865,565,184 B ≈ 33.4 GiB, sha256 `274c3fc7…`, 1352 tensors) with the
**DFlash2** drafter at **one slot x 200,700 ctx**.

It replaced the ROCmFPX llama.cpp fork endpoint ([[frank-dragoon-rocmfpx-q6agent-uncensored-mtp]]),
which remains the rollback target with its payload on disk.

## Why swap

|Metric|ROCmFPX Q6_0_ROCMFPX_AGENT + MTP (retired)|halogen qwen3.8-27b + DFlash2 (active)|
|---|---|---|
|Prose decode|27.6–28.1 t/s measured (4.8k depth, acceptance 0.868)|31.7 t/s mean across eight text workloads (publisher card, bf16 reference hardware)|
|Copy-echo decode|31.4 t/s|publisher reports echo inside the same 31.7 mean|
|Context|1 x 262,144|1 x 200,700|
|Format|GGUF (fork-private tensor types 102/103)|`.hgn` — loads in NO other engine|

The swap was accepted with eyes open on two product regressions, both
confirmed by Frank before deploy:

1. **Text-only.** halogen loads the language tower only; README "honest
   limits": *"The vision encoder is ignored; image inputs are not supported."*
   The mmproj-era vision role moves wholly to frank-zealot.
2. **Censored base.** The checkpoint is a finetune of the censored
   `Qwen/Qwen3.8-27B` base, built to make its DFlash2 head strong. The
   ROCmFPX AGENT file was an *Uncensored* build; refusals can return.

## Shape and why

`scripts/frank-dragoon.env` carries every tunable with its evidence; the load-
bearing numbers:

- **`HALOGEN_KV_SLOTS=1` + `HALOGEN_DRAFTER=2`** — the engine offers batch OR
  speculation, never both (FLAGS.md: "Batching and speculation are exclusive:
  a step batches when more than one conversation is running, so the drafter
  stays off with more than one slot"). Slots>1 silently loses DFlash2; the
  sibling halogen engine measured 41 -> 14 t/s when slots went up. frank-dragoon
  was a one-conversation endpoint before (PARALLEL=1), so the contract holds.
- **`HALOGEN_SLOT_CTX=200700`** — the per-conversation HARD ceiling: a longer
  prompt is a 400 naming the limit, never a truncation. The harness side is
  wired to it: `models.yml` `contextWindow: 200700`, and OMP compacts at 70%
  = 140,490, leaving 60,210 of the slot for the reply.
- **`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200`** — coupled
  pair (FLAGS.md table): a request at the cap must not outlive the timeout.
  65536 > the harness's 57,344 `maxTokens` so an eager request never meets
  the engine's 400.
- **KV math:** 1 slot x 200,700 x 64 KiB/token (d2 KV = 2x fp8) = 12.2 GiB
  pool, inside the ~65 GiB left after 33.4 GiB mmap'd weights + engine working
  set + host baseline on the 121.2 GiB box. The engine sizes the pool itself
  and **prints a boot ledger naming any clamp** — the post-deploy check reads
  that ledger, not a formula.
- **`HALOGEN_CACHE_MB=16384` (pinned 2026-10-06)** — auto is OFF on this
  box: the floor is 2 x a full-context entry (36.9 GB at 262k; one entry
  is ~13.7 GB at our 200,700-slot x ~70 KB/token) and the ledger had
  35.7 GB free, so the engine refused and every deep agent request
  re-prefilled its whole 79-81k context — 84.9-179.8 s measured
  (serve_api lines, 2026-10-06) for ~500-650 new tokens per turn. The
  README prices the cache at ~20x time-to-first-token on a warm turn;
  entries evict LRU by bytes; the reserve (HALOGEN_CACHE_RESERVE_MB,
  default 8192) is re-checked against live MemAvailable on every store.
- **No server-side sampler:** an absent temperature is GREEDY. The card's
  thinking recipe (temp 1.0 / top_p 0.95) lives in `models.yml` `extraBody`
  — on the wire only. The template defaults `reasoning_effort` to **xhigh**
  when a request sends none, and **thinking tokens consume the
  `max_tokens` budget**: a budget exhausted mid-thought returns
  `finish_reason: "length"` with EMPTY `content` and the partial reasoning
  in `reasoning_content` — clients must read both fields.

## Client contract deltas vs llama.cpp

|Behavior|halogen|
|---|---|
|Prompt over slot|400 naming the limit (no truncation)|
|`max_tokens` over cap|400 naming the limit (never silent truncation)|
|full-length request outliving `HALOGEN_QUEUE_TIMEOUT`|503s for everyone queued|
|`timings`|every response carries llama-server-shape `timings` (`draft_n`, `draft_n_accepted`) — the drafter check reads the RESPONSE, and it proves speculation fired|
|`response_format`|NOT supported (no constrained decoding)|
|thinking field|`reasoning_content` (same as before)|
|`reasoning_effort`|same Qwen3.8 template; outside {xhigh,medium,low} rejected — models.yml fold stays; unset defaults to xhigh, `none` disables thinking|
|tools|server-side tool calls advertised — round-trip MUST be re-probed after deploy|
|image input|ignored/error — route to frank-zealot|

## Deploy mechanics

- **Image:** `ghcr.io/peonist-ai/halogen@sha256:dc0a39a0016d6cfc58a197978febaafdf8d28403f724ded6111d98b5fb7ac0ea`
  (published tag 0.1.4). The deployer gates the exact digest exists in podman
  storage before dispatch. NOT the 2026-09 `halogen-flash-server` repo — that
  was the Flash-Next w4b era, different image, retired.
- **Payload:** `scripts/download-frank-dragoon-halogen.sh` (fetch law: `curl -f`,
  exact byte count, LFS sha256 where the Hub publishes one, `--verify-only`,
  token never in argv) into `/home/dank/models/halogen-qwen3.8-27b/`. The
  tokenizer is a FLAT DIRECTORY (`tokenizer/`, 5 files — tokenizer.json,
  vocab.json, merges.txt, chat_template.jinja, tokenizer_config.json),
  mounted whole at `/tokenizer`; the engine's `HALOGEN_TOKENIZER` default is
  that directory, so the container gets no tokenizer flag (model-card
  docker-run parity). The deployer and serve script refuse a missing file or
  ANY symlink: an HF cache snapshot symlinks each entry into `blobs/` and
  those links dangle inside the container mount.
- **Unit:** `scripts/frank-dragoon.service` (root, `User=dank`, relinked into
  `/etc/systemd/system/`), `ExecStart` = serve script, `ExecStop` = `podman
  stop -t 120 frank-dragoon-halogen`, script runs `podman run` FOREGROUND
  (2026-10-06: the 2026-10-05 `-d` + `exec podman wait` shape captured
  only the container ID — the engine ledger existed nowhere on disk and
  the post-deploy grep below read a one-line hex file; foreground run
  returns the container's exit code so `Restart=always` is real and the
  ledger lands in the log). Engine stdout goes to
  `/home/dank/logs/frank_dragoon.log` (the boot ledger lives there, not in
  the journal).
- **Cutover:** `deploy_llm_dragoon` in `scripts/deploy-remote.sh` sweeps the
  legacy bare `llama-server --model /home/dank/models/kingjones-*` process and
  the legacy `frank-dragoon` container name by NAME (never `pkill -f
  llama-server` — that hits frank-zealot), retires the old quadlet if the 2026-09
  halogen era's unit lingers, waits port 7770 free, relinks, health-gates.
- **Bench law:** stop+mask the unit before any GPU experiment on this box
  (AGENTS.md); `/health` answers 503 while the 33.4 GiB payload loads — that
  is loading, not failure.

## Verification (post-deploy checklist)

1. `grep -iE 'clamp|kv|pool|drafter|dflash|cache' /home/dank/logs/frank_dragoon.log`
   — ledger shows the pool at the asked shape, DFlash2 active, the
   prompt cache line naming the pinned budget, NO clamp line. EXPECT the
   WARNING "smaller than ONE full-context entry (18.4 GB at
   kMaxCtx=262144)": it sizes against the NATIVE window, not our
   200,700 slot — the deepest possible entry (~14 GB at ~70 KB/token)
   fits the 16 GB pin, and the 262k entry it fears cannot exist (over
   slot is a 400). Probe-proven 2026-10-06: the warm turn hit with
   cache_n=16,384.
2. `curl -s http://192.168.0.143:7770/health` -> 200.
3. Real completion with `reasoning_effort` + temp/top_p from the wire recipe;
   `reasoning_content` populated; decode rate from the timings.
4. Spec decode fires: multi-token step behavior visible in rate; compare
   against DRAFTER=0 arm if in doubt (never against gufo — bench law).
5. Harness round-trip: an OMP call through `llamacpp (dankclaw)/frank-dragoon`
   incl. a tool call (V1TOOLS).
6. Prompt > 200,700 tokens returns the 400 naming the limit (contract test).

## Tuning notes (2026-10-06)

Measured on real agent traffic (serve_api lines) the day after cutover:

- **Decode is depth-bound, and the KV path is the reason.** Shallow
  prompts (< 1k tokens) decode 30-50 t/s at commit 4-6.3/round; 46k runs
  5.5, 63-67k runs 4.1-6.5, 79-81k runs 3.3-3.8. The round-time slope is
  ~8 us/token of context: the KV pool reads at ~8 GB/s effective while
  the weights stream ~180-240 GB/s (the engine's README puts batch-1
  weight streaming at the 240 GB/s hardware wall). Commit also falls
  from 4-6 to 2.5-3 accepted/round as the conversation grows.
- **No supported flag touches decode.** `docs/FLAGS.md`: HALOGEN_W4A4 /
  W4A4_EXCL are prefill-only ("decode is unaffected either way"); the
  undocumented kernel levers are declared unsupported and their defaults
  are the measured winners. 0.1.4 is the latest release (CHANGELOG
  head; kernels unchanged since 0.1.0). The publisher's tg bench never
  measured depth — its tg rows run over the ten SHORT eval prompt
  shapes — so the README's "decode barely degrades at depth" claim was
  never exercised at 46-80k.
- **A/B'd the one FA lever with public precedent (2026-10-06 stop-window
  probe): rejected.** Issue #12's `HALOGEN_FA_TQ=256` (the 64 GB-carve
  crash workaround) vs the default tile — same 80k filler prompt,
  128-token essays, the runtime's own `timings`: decode 6.35 vs 6.34 t/s
  cold and 5.74 vs 5.76 warm (±0.3 % = noise; draft acceptance identical,
  53/74 and 58/69), while prefill LOST 9.5 % on this topology (447.5 ->
  404.8 t/s). The depth collapse is not an FA tile artifact — it sits
  deeper than the knob. The engine binary advertises 40+ undocumented
  `HALOGEN_*` kernel levers (FA/DN/MATMUL families, readable via
  `grep -aoE "HALOGEN_[A-Z0-9_]+"` on the binary); `FA_TQ` is the only
  one with public evidence, and it moved nothing that matters here.
- **Routing is the applied mitigation (2026-10-06):** the harness's
  deep-conversation roles (`slow`, `advisor` in `~/.omp/agent/config.yml`)
  moved to frank-archon (radiance on Dankserver: fp8 KV at
  7.32 KiB/token = 8.7x less KV to read per round at depth, MTP depth 3,
  200k window, prefix-cache host+disk tiers). frank-dragoon keeps OpenWebUI
  and shallow bursts, where it runs 30-50 t/s. frank-archon's
  depth-decode rate is designed-for but not yet measured on record.
- **The lever that existed was the prompt cache, and it was OFF** (see
  the shape bullet). The pin turns the 85-180 s re-prefill per deep
  turn into a resume; decode itself is expected UNCHANGED by the pin —
  the cache resumes prefill, it does not touch the round loop.
  Probe-proven (two synthetic 17.3k turns, 2026-10-06): cold prefill
  27.0 s (640 t/s, cache_n 0) -> warm turn cache_n 16,384, prompt_n 946
  (the extension + one 2048-aligned chunk), prefill 1.7 s — 16x on the
  pair, answer correct warm.
- **Everything else stays put.** dflash2 over mtp (publisher 31.71 vs
  26.88 mean; at depth both pay the same KV term and dflash2's higher
  commit wins), KV_SLOTS=1 (batch XOR speculation), SLOT_CTX=200700
  (Frank's window), W4A4 shipped (+9 % prefill, decode unharmed).

## Rollback

Revert the frank-dragoon swap commit and redeploy `dankclaw --service frank-dragoon`:
the ROCmFPX payload (`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/`),
the fork build (`/home/dank/GitPackages/kingjones-ROCmFPX/build`), and its
`scripts/build-rocmfpx.sh` pin (commit `36e9acd`, seam-tested) all remain on
disk. The halogen payload can stay: it costs nothing while the unit points
elsewhere.