serverAppConfigs/serverAppConfigs-docs/LLM/frank-dragoon-halogen27b-dflash2.md

frank-dragoon — halogen container, Qwen3.8-27B + DFlash2 (Dankclaw, ACTIVE 2026-10-05)

frank-dragoon is the Strix Halo iGPU endpoint on Dankclaw (:7770). Since 2026-10-05 it runs the halogen engine — ghcr.io/peonist-ai/halogen at a pinned digest, launched by scripts/serve-frank-dragoon.sh under a root systemd unit as a podman container — serving peonist-ai/halogen-qwen3.8-27b (qwen3.8-27b-p1w4d-d2.hgn, 35,865,565,184 B ≈ 33.4 GiB, sha256 274c3fc7…, 1352 tensors) with the DFlash2 drafter at one slot x 200,700 ctx.

It replaced the ROCmFPX llama.cpp fork endpoint (frank-dragoon-rocmfpx-q6agent-uncensored-mtp), which remains the rollback target with its payload on disk.

Why swap

Metric	ROCmFPX Q6_0_ROCMFPX_AGENT + MTP (retired)	halogen qwen3.8-27b + DFlash2 (active)
Prose decode	27.6–28.1 t/s measured (4.8k depth, acceptance 0.868)	31.7 t/s mean across eight text workloads (publisher card, bf16 reference hardware)
Copy-echo decode	31.4 t/s	publisher reports echo inside the same 31.7 mean
Context	1 x 262,144	1 x 200,700
Format	GGUF (fork-private tensor types 102/103)	.hgn — loads in NO other engine

The swap was accepted with eyes open on two product regressions, both confirmed by Frank before deploy:

1. Text-only. halogen loads the language tower only; README "honest limits": "The vision encoder is ignored; image inputs are not supported." The mmproj-era vision role moves wholly to frank-zealot.
2. Censored base. The checkpoint is a finetune of the censored Qwen/Qwen3.8-27B base, built to make its DFlash2 head strong. The ROCmFPX AGENT file was an Uncensored build; refusals can return.

Shape and why

scripts/frank-dragoon.env carries every tunable with its evidence; the load- bearing numbers:

• HALOGEN_KV_SLOTS=1 + HALOGEN_DRAFTER=2 — the engine offers batch OR speculation, never both (FLAGS.md: "Batching and speculation are exclusive: a step batches when more than one conversation is running, so the drafter stays off with more than one slot"). Slots>1 silently loses DFlash2; the sibling halogen engine measured 41 -> 14 t/s when slots went up. frank-dragoon was a one-conversation endpoint before (PARALLEL=1), so the contract holds.
• HALOGEN_SLOT_CTX=200700 — the per-conversation HARD ceiling: a longer prompt is a 400 naming the limit, never a truncation. The harness side is wired to it: models.yml contextWindow: 200700, and OMP compacts at 70% = 140,490, leaving 60,210 of the slot for the reply.
• HALOGEN_MAX_TOKENS_CAP=65536 + HALOGEN_QUEUE_TIMEOUT=7200 — coupled pair (FLAGS.md table): a request at the cap must not outlive the timeout. 65536 > the harness's 57,344 maxTokens so an eager request never meets the engine's 400.
• KV math: 1 slot x 200,700 x 64 KiB/token (d2 KV = 2x fp8) = 12.2 GiB pool, inside the ~65 GiB left after 33.4 GiB mmap'd weights + engine working set + host baseline on the 121.2 GiB box. The engine sizes the pool itself and prints a boot ledger naming any clamp — the post-deploy check reads that ledger, not a formula.
• HALOGEN_CACHE_MB=16384 (pinned 2026-10-06) — auto is OFF on this box: the floor is 2 x a full-context entry (36.9 GB at 262k; one entry is ~13.7 GB at our 200,700-slot x ~70 KB/token) and the ledger had 35.7 GB free, so the engine refused and every deep agent request re-prefilled its whole 79-81k context — 84.9-179.8 s measured (serve_api lines, 2026-10-06) for ~500-650 new tokens per turn. The README prices the cache at ~20x time-to-first-token on a warm turn; entries evict LRU by bytes; the reserve (HALOGEN_CACHE_RESERVE_MB, default 8192) is re-checked against live MemAvailable on every store.
• No server-side sampler: an absent temperature is GREEDY. The card's thinking recipe (temp 1.0 / top_p 0.95) lives in models.yml extraBody — on the wire only. The template defaults reasoning_effort to xhigh when a request sends none, and thinking tokens consume the max_tokens budget: a budget exhausted mid-thought returns finish_reason: "length" with EMPTY content and the partial reasoning in reasoning_content — clients must read both fields.

Client contract deltas vs llama.cpp

Behavior	halogen
Prompt over slot	400 naming the limit (no truncation)
max_tokens over cap	400 naming the limit (never silent truncation)
full-length request outliving HALOGEN_QUEUE_TIMEOUT	503s for everyone queued
timings	every response carries llama-server-shape timings (draft_n, draft_n_accepted) — the drafter check reads the RESPONSE, and it proves speculation fired
response_format	NOT supported (no constrained decoding)
thinking field	reasoning_content (same as before)
reasoning_effort	same Qwen3.8 template; outside {xhigh,medium,low} rejected — models.yml fold stays; unset defaults to xhigh, none disables thinking
tools	server-side tool calls advertised — round-trip MUST be re-probed after deploy
image input	ignored/error — route to frank-zealot

Deploy mechanics

• Image: ghcr.io/peonist-ai/halogen@sha256:dc0a39a0016d6cfc58a197978febaafdf8d28403f724ded6111d98b5fb7ac0ea (published tag 0.1.4). The deployer gates the exact digest exists in podman storage before dispatch. NOT the 2026-09 halogen-flash-server repo — that was the Flash-Next w4b era, different image, retired.
• Payload: scripts/download-frank-dragoon-halogen.sh (fetch law: curl -f, exact byte count, LFS sha256 where the Hub publishes one, --verify-only, token never in argv) into /home/dank/models/halogen-qwen3.8-27b/. The tokenizer is a FLAT DIRECTORY (tokenizer/, 5 files — tokenizer.json, vocab.json, merges.txt, chat_template.jinja, tokenizer_config.json), mounted whole at /tokenizer; the engine's HALOGEN_TOKENIZER default is that directory, so the container gets no tokenizer flag (model-card docker-run parity). The deployer and serve script refuse a missing file or ANY symlink: an HF cache snapshot symlinks each entry into blobs/ and those links dangle inside the container mount.
• Unit: scripts/frank-dragoon.service (root, User=dank, relinked into /etc/systemd/system/), ExecStart = serve script, ExecStop = podman stop -t 120 frank-dragoon-halogen, script runs podman run FOREGROUND (2026-10-06: the 2026-10-05 -d + exec podman wait shape captured only the container ID — the engine ledger existed nowhere on disk and the post-deploy grep below read a one-line hex file; foreground run returns the container's exit code so Restart=always is real and the ledger lands in the log). Engine stdout goes to /home/dank/logs/frank_dragoon.log (the boot ledger lives there, not in the journal).
• Cutover: deploy_llm_dragoon in scripts/deploy-remote.sh sweeps the legacy bare llama-server --model /home/dank/models/kingjones-* process and the legacy frank-dragoon container name by NAME (never pkill -f llama-server — that hits frank-zealot), retires the old quadlet if the 2026-09 halogen era's unit lingers, waits port 7770 free, relinks, health-gates.
• Bench law: stop+mask the unit before any GPU experiment on this box (AGENTS.md); /health answers 503 while the 33.4 GiB payload loads — that is loading, not failure.

Verification (post-deploy checklist)

1. grep -iE 'clamp|kv|pool|drafter|dflash|cache' /home/dank/logs/frank_dragoon.log — ledger shows the pool at the asked shape, DFlash2 active, the prompt cache line naming the pinned budget, NO clamp line. EXPECT the WARNING "smaller than ONE full-context entry (18.4 GB at kMaxCtx=262144)": it sizes against the NATIVE window, not our 200,700 slot — the deepest possible entry (~14 GB at ~70 KB/token) fits the 16 GB pin, and the 262k entry it fears cannot exist (over slot is a 400). Probe-proven 2026-10-06: the warm turn hit with cache_n=16,384.
2. curl -s http://192.168.0.143:7770/health -> 200.
3. Real completion with reasoning_effort + temp/top_p from the wire recipe; reasoning_content populated; decode rate from the timings.
4. Spec decode fires: multi-token step behavior visible in rate; compare against DRAFTER=0 arm if in doubt (never against gufo — bench law).
5. Harness round-trip: an OMP call through llamacpp (dankclaw)/frank-dragoon incl. a tool call (V1TOOLS).
6. Prompt > 200,700 tokens returns the 400 naming the limit (contract test).

Tuning notes (2026-10-06)

Measured on real agent traffic (serve_api lines) the day after cutover:

• Decode is depth-bound, and the KV path is the reason. Shallow prompts (< 1k tokens) decode 30-50 t/s at commit 4-6.3/round; 46k runs 5.5, 63-67k runs 4.1-6.5, 79-81k runs 3.3-3.8. The round-time slope is ~8 us/token of context: the KV pool reads at ~8 GB/s effective while the weights stream ~180-240 GB/s (the engine's README puts batch-1 weight streaming at the 240 GB/s hardware wall). Commit also falls from 4-6 to 2.5-3 accepted/round as the conversation grows.
• No supported flag touches decode. docs/FLAGS.md: HALOGEN_W4A4 / W4A4_EXCL are prefill-only ("decode is unaffected either way"); the undocumented kernel levers are declared unsupported and their defaults are the measured winners. 0.1.4 is the latest release (CHANGELOG head; kernels unchanged since 0.1.0). The publisher's tg bench never measured depth — its tg rows run over the ten SHORT eval prompt shapes — so the README's "decode barely degrades at depth" claim was never exercised at 46-80k.
• A/B'd the one FA lever with public precedent (2026-10-06 stop-window probe): rejected. Issue #12's HALOGEN_FA_TQ=256 (the 64 GB-carve crash workaround) vs the default tile — same 80k filler prompt, 128-token essays, the runtime's own timings: decode 6.35 vs 6.34 t/s cold and 5.74 vs 5.76 warm (±0.3 % = noise; draft acceptance identical, 53/74 and 58/69), while prefill LOST 9.5 % on this topology (447.5 -> 404.8 t/s). The depth collapse is not an FA tile artifact — it sits deeper than the knob. The engine binary advertises 40+ undocumented HALOGEN_* kernel levers (FA/DN/MATMUL families, readable via grep -aoE "HALOGEN_[A-Z0-9_]+" on the binary); FA_TQ is the only one with public evidence, and it moved nothing that matters here.
• Routing is the applied mitigation (2026-10-06): the harness's deep-conversation roles (slow, advisor in ~/.omp/agent/config.yml) moved to frank-archon (radiance on Dankserver: fp8 KV at 7.32 KiB/token = 8.7x less KV to read per round at depth, MTP depth 3, 200k window, prefix-cache host+disk tiers). frank-dragoon keeps OpenWebUI and shallow bursts, where it runs 30-50 t/s. frank-archon's depth-decode rate is designed-for but not yet measured on record.
• The lever that existed was the prompt cache, and it was OFF (see the shape bullet). The pin turns the 85-180 s re-prefill per deep turn into a resume; decode itself is expected UNCHANGED by the pin — the cache resumes prefill, it does not touch the round loop. Probe-proven (two synthetic 17.3k turns, 2026-10-06): cold prefill 27.0 s (640 t/s, cache_n 0) -> warm turn cache_n 16,384, prompt_n 946 (the extension + one 2048-aligned chunk), prefill 1.7 s — 16x on the pair, answer correct warm.
• Everything else stays put. dflash2 over mtp (publisher 31.71 vs 26.88 mean; at depth both pay the same KV term and dflash2's higher commit wins), KV_SLOTS=1 (batch XOR speculation), SLOT_CTX=200700 (Frank's window), W4A4 shipped (+9 % prefill, decode unharmed).

Rollback

Revert the frank-dragoon swap commit and redeploy dankclaw --service frank-dragoon: the ROCmFPX payload (/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/), the fork build (/home/dank/GitPackages/kingjones-ROCmFPX/build), and its scripts/build-rocmfpx.sh pin (commit 36e9acd, seam-tested) all remain on disk. The halogen payload can stay: it costs nothing while the unit points elsewhere.