serverAppConfigs/serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md

# frank-large — halogen container, Qwen3.8-27B + DFlash2 (Dankclaw, ACTIVE 2026-10-05)

frank-large is the Strix Halo iGPU endpoint on Dankclaw (`:7770`). Since
2026-10-05 it runs the **halogen** engine — `ghcr.io/peonist-ai/halogen` at a
pinned digest, launched by `scripts/serve-frank-large.sh` under a root
systemd unit as a podman container — serving
**`peonist-ai/halogen-qwen3.8-27b`** (`qwen3.8-27b-p1w4d-d2.hgn`,
35,865,565,184 B ≈ 33.4 GiB, sha256 `274c3fc7…`, 1352 tensors) with the
**DFlash2** drafter at **one slot x 200,700 ctx**.

It replaced the ROCmFPX llama.cpp fork endpoint ([[frank-large-rocmfpx-q6agent-uncensored-mtp]]),
which remains the rollback target with its payload on disk.

## Why swap

|Metric|ROCmFPX Q6_0_ROCMFPX_AGENT + MTP (retired)|halogen qwen3.8-27b + DFlash2 (active)|
|---|---|---|
|Prose decode|27.6–28.1 t/s measured (4.8k depth, acceptance 0.868)|31.7 t/s mean across eight text workloads (publisher card, bf16 reference hardware)|
|Copy-echo decode|31.4 t/s|publisher reports echo inside the same 31.7 mean|
|Context|1 x 262,144|1 x 200,700|
|Format|GGUF (fork-private tensor types 102/103)|`.hgn` — loads in NO other engine|

The swap was accepted with eyes open on two product regressions, both
confirmed by Frank before deploy:

1. **Text-only.** halogen loads the language tower only; README "honest
   limits": *"The vision encoder is ignored; image inputs are not supported."*
   The mmproj-era vision role moves wholly to frank-small.
2. **Censored base.** The checkpoint is a finetune of the censored
   `Qwen/Qwen3.8-27B` base, built to make its DFlash2 head strong. The
   ROCmFPX AGENT file was an *Uncensored* build; refusals can return.

## Shape and why

`scripts/frank-large.env` carries every tunable with its evidence; the load-
bearing numbers:

- **`HALOGEN_KV_SLOTS=1` + `HALOGEN_DRAFTER=2`** — the engine offers batch OR
  speculation, never both (FLAGS.md: "Batching and speculation are exclusive:
  a step batches when more than one conversation is running, so the drafter
  stays off with more than one slot"). Slots>1 silently loses DFlash2; the
  sibling halogen engine measured 41 -> 14 t/s when slots went up. frank-large
  was a one-conversation endpoint before (PARALLEL=1), so the contract holds.
- **`HALOGEN_SLOT_CTX=200700`** — the per-conversation HARD ceiling: a longer
  prompt is a 400 naming the limit, never a truncation. The harness side is
  wired to it: `models.yml` `contextWindow: 200700`, and OMP compacts at 70%
  = 140,490, leaving 60,210 of the slot for the reply.
- **`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200`** — coupled
  pair (FLAGS.md table): a request at the cap must not outlive the timeout.
  65536 > the harness's 57,344 `maxTokens` so an eager request never meets
  the engine's 400.
- **KV math:** 1 slot x 200,700 x 64 KiB/token (d2 KV = 2x fp8) = 12.2 GiB
  pool, inside the ~65 GiB left after 33.4 GiB mmap'd weights + engine working
  set + host baseline on the 121.2 GiB box. The engine sizes the pool itself
  and **prints a boot ledger naming any clamp** — the post-deploy check reads
  that ledger, not a formula.
- **No server-side sampler:** an absent temperature is GREEDY. The card's
  thinking recipe (temp 1.0 / top_p 0.95) lives in `models.yml` `extraBody`
  — on the wire only. The template defaults `reasoning_effort` to **xhigh**
  when a request sends none, and **thinking tokens consume the
  `max_tokens` budget**: a budget exhausted mid-thought returns
  `finish_reason: "length"` with EMPTY `content` and the partial reasoning
  in `reasoning_content` — clients must read both fields.

## Client contract deltas vs llama.cpp

|Behavior|halogen|
|---|---|
|Prompt over slot|400 naming the limit (no truncation)|
|`max_tokens` over cap|400 naming the limit (never silent truncation)|
|full-length request outliving `HALOGEN_QUEUE_TIMEOUT`|503s for everyone queued|
|`timings`|every response carries llama-server-shape `timings` (`draft_n`, `draft_n_accepted`) — the drafter check reads the RESPONSE, and it proves speculation fired|
|`response_format`|NOT supported (no constrained decoding)|
|thinking field|`reasoning_content` (same as before)|
|`reasoning_effort`|same Qwen3.8 template; outside {xhigh,medium,low} rejected — models.yml fold stays; unset defaults to xhigh, `none` disables thinking|
|tools|server-side tool calls advertised — round-trip MUST be re-probed after deploy|
|image input|ignored/error — route to frank-small|

## Deploy mechanics

- **Image:** `ghcr.io/peonist-ai/halogen@sha256:dc0a39a0016d6cfc58a197978febaafdf8d28403f724ded6111d98b5fb7ac0ea`
  (published tag 0.1.4). The deployer gates the exact digest exists in podman
  storage before dispatch. NOT the 2026-09 `halogen-flash-server` repo — that
  was the Flash-Next w4b era, different image, retired.
- **Payload:** `scripts/download-frank-large-halogen.sh` (fetch law: `curl -f`,
  exact byte count, LFS sha256 where the Hub publishes one, `--verify-only`,
  token never in argv) into `/home/dank/models/halogen-qwen3.8-27b/`. The
  tokenizer is a FLAT DIRECTORY (`tokenizer/`, 5 files — tokenizer.json,
  vocab.json, merges.txt, chat_template.jinja, tokenizer_config.json),
  mounted whole at `/tokenizer`; the engine's `HALOGEN_TOKENIZER` default is
  that directory, so the container gets no tokenizer flag (model-card
  docker-run parity). The deployer and serve script refuse a missing file or
  ANY symlink: an HF cache snapshot symlinks each entry into `blobs/` and
  those links dangle inside the container mount.
- **Unit:** `scripts/frank-large.service` (root, `User=dank`, relinked into
  `/etc/systemd/system/`), `ExecStart` = serve script, `ExecStop` = `podman
  stop -t 120 frank-large-halogen`, script `exec podman wait` so
  `Restart=always` is real. Engine stdout goes to
  `/home/dank/logs/frank_large.log` (the boot ledger lives there, not in the
  journal).
- **Cutover:** `deploy_llm_large` in `scripts/deploy-remote.sh` sweeps the
  legacy bare `llama-server --model /home/dank/models/kingjones-*` process and
  the legacy `frank-large` container name by NAME (never `pkill -f
  llama-server` — that hits frank-small), retires the old quadlet if the 2026-09
  halogen era's unit lingers, waits port 7770 free, relinks, health-gates.
- **Bench law:** stop+mask the unit before any GPU experiment on this box
  (AGENTS.md); `/health` answers 503 while the 33.4 GiB payload loads — that
  is loading, not failure.

## Verification (post-deploy checklist)

1. `grep -iE 'clamp|kv|pool|drafter|dflash' /home/dank/logs/frank_large.log`
   — ledger shows the pool at the asked shape, DFlash2 active, NO clamp line.
2. `curl -s http://192.168.0.143:7770/health` -> 200.
3. Real completion with `reasoning_effort` + temp/top_p from the wire recipe;
   `reasoning_content` populated; decode rate from the timings.
4. Spec decode fires: multi-token step behavior visible in rate; compare
   against DRAFTER=0 arm if in doubt (never against gufo — bench law).
5. Harness round-trip: an OMP call through `llamacpp (dankclaw)/frank-large`
   incl. a tool call (V1TOOLS).
6. Prompt > 200,700 tokens returns the 400 naming the limit (contract test).

## Rollback

Revert the frank-large swap commit and redeploy `dankclaw --service frank-large`:
the ROCmFPX payload (`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/`),
the fork build (`/home/dank/GitPackages/kingjones-ROCmFPX/build`), and its
`scripts/build-rocmfpx.sh` pin (commit `36e9acd`, seam-tested) all remain on
disk. The halogen payload can stay: it costs nothing while the unit points
elsewhere.