serverAppConfigs/serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md

frank-large — halogen container, Qwen3.8-27B + DFlash2 (Dankclaw, ACTIVE 2026-10-05)

frank-large is the Strix Halo iGPU endpoint on Dankclaw (:7770). Since 2026-10-05 it runs the halogen engine — ghcr.io/peonist-ai/halogen at a pinned digest, launched by scripts/serve-frank-large.sh under a root systemd unit as a podman container — serving peonist-ai/halogen-qwen3.8-27b (qwen3.8-27b-p1w4d-d2.hgn, 35,865,565,184 B ≈ 33.4 GiB, sha256 274c3fc7…, 1352 tensors) with the DFlash2 drafter at one slot x 200,700 ctx.

It replaced the ROCmFPX llama.cpp fork endpoint (frank-large-rocmfpx-q6agent-uncensored-mtp), which remains the rollback target with its payload on disk.

Why swap

Metric	ROCmFPX Q6_0_ROCMFPX_AGENT + MTP (retired)	halogen qwen3.8-27b + DFlash2 (active)
Prose decode	27.6–28.1 t/s measured (4.8k depth, acceptance 0.868)	31.7 t/s mean across eight text workloads (publisher card, bf16 reference hardware)
Copy-echo decode	31.4 t/s	publisher reports echo inside the same 31.7 mean
Context	1 x 262,144	1 x 200,700
Format	GGUF (fork-private tensor types 102/103)	.hgn — loads in NO other engine

The swap was accepted with eyes open on two product regressions, both confirmed by Frank before deploy:

1. Text-only. halogen loads the language tower only; README "honest limits": "The vision encoder is ignored; image inputs are not supported." The mmproj-era vision role moves wholly to frank-small.
2. Censored base. The checkpoint is a finetune of the censored Qwen/Qwen3.8-27B base, built to make its DFlash2 head strong. The ROCmFPX AGENT file was an Uncensored build; refusals can return.

Shape and why

scripts/frank-large.env carries every tunable with its evidence; the load- bearing numbers:

• HALOGEN_KV_SLOTS=1 + HALOGEN_DRAFTER=2 — the engine offers batch OR speculation, never both (FLAGS.md: "Batching and speculation are exclusive: a step batches when more than one conversation is running, so the drafter stays off with more than one slot"). Slots>1 silently loses DFlash2; the sibling halogen engine measured 41 -> 14 t/s when slots went up. frank-large was a one-conversation endpoint before (PARALLEL=1), so the contract holds.
• HALOGEN_SLOT_CTX=200700 — the per-conversation HARD ceiling: a longer prompt is a 400 naming the limit, never a truncation. The harness side is wired to it: models.yml contextWindow: 200700, and OMP compacts at 70% = 140,490, leaving 60,210 of the slot for the reply.
• HALOGEN_MAX_TOKENS_CAP=65536 + HALOGEN_QUEUE_TIMEOUT=7200 — coupled pair (FLAGS.md table): a request at the cap must not outlive the timeout. 65536 > the harness's 57,344 maxTokens so an eager request never meets the engine's 400.
• KV math: 1 slot x 200,700 x 64 KiB/token (d2 KV = 2x fp8) = 12.2 GiB pool, inside the ~65 GiB left after 33.4 GiB mmap'd weights + engine working set + host baseline on the 121.2 GiB box. The engine sizes the pool itself and prints a boot ledger naming any clamp — the post-deploy check reads that ledger, not a formula.
• No server-side sampler: an absent temperature is GREEDY. The card's thinking recipe (temp 1.0 / top_p 0.95) lives in models.yml extraBody — on the wire only. The template defaults reasoning_effort to xhigh when a request sends none, and thinking tokens consume the max_tokens budget: a budget exhausted mid-thought returns finish_reason: "length" with EMPTY content and the partial reasoning in reasoning_content — clients must read both fields.

Client contract deltas vs llama.cpp

Behavior	halogen
Prompt over slot	400 naming the limit (no truncation)
max_tokens over cap	400 naming the limit (never silent truncation)
full-length request outliving HALOGEN_QUEUE_TIMEOUT	503s for everyone queued
timings	every response carries llama-server-shape timings (draft_n, draft_n_accepted) — the drafter check reads the RESPONSE, and it proves speculation fired
response_format	NOT supported (no constrained decoding)
thinking field	reasoning_content (same as before)
reasoning_effort	same Qwen3.8 template; outside {xhigh,medium,low} rejected — models.yml fold stays; unset defaults to xhigh, none disables thinking
tools	server-side tool calls advertised — round-trip MUST be re-probed after deploy
image input	ignored/error — route to frank-small

Deploy mechanics

• Image: ghcr.io/peonist-ai/halogen@sha256:dc0a39a0016d6cfc58a197978febaafdf8d28403f724ded6111d98b5fb7ac0ea (published tag 0.1.4). The deployer gates the exact digest exists in podman storage before dispatch. NOT the 2026-09 halogen-flash-server repo — that was the Flash-Next w4b era, different image, retired.
• Payload: scripts/download-frank-large-halogen.sh (fetch law: curl -f, exact byte count, LFS sha256 where the Hub publishes one, --verify-only, token never in argv) into /home/dank/models/halogen-qwen3.8-27b/. The tokenizer is a FLAT DIRECTORY (tokenizer/, 5 files — tokenizer.json, vocab.json, merges.txt, chat_template.jinja, tokenizer_config.json), mounted whole at /tokenizer; the engine's HALOGEN_TOKENIZER default is that directory, so the container gets no tokenizer flag (model-card docker-run parity). The deployer and serve script refuse a missing file or ANY symlink: an HF cache snapshot symlinks each entry into blobs/ and those links dangle inside the container mount.
• Unit: scripts/frank-large.service (root, User=dank, relinked into /etc/systemd/system/), ExecStart = serve script, ExecStop = podman stop -t 120 frank-large-halogen, script exec podman wait so Restart=always is real. Engine stdout goes to /home/dank/logs/frank_large.log (the boot ledger lives there, not in the journal).
• Cutover: deploy_llm_large in scripts/deploy-remote.sh sweeps the legacy bare llama-server --model /home/dank/models/kingjones-* process and the legacy frank-large container name by NAME (never pkill -f llama-server — that hits frank-small), retires the old quadlet if the 2026-09 halogen era's unit lingers, waits port 7770 free, relinks, health-gates.
• Bench law: stop+mask the unit before any GPU experiment on this box (AGENTS.md); /health answers 503 while the 33.4 GiB payload loads — that is loading, not failure.

Verification (post-deploy checklist)

1. grep -iE 'clamp|kv|pool|drafter|dflash' /home/dank/logs/frank_large.log — ledger shows the pool at the asked shape, DFlash2 active, NO clamp line.
2. curl -s http://192.168.0.143:7770/health -> 200.
3. Real completion with reasoning_effort + temp/top_p from the wire recipe; reasoning_content populated; decode rate from the timings.
4. Spec decode fires: multi-token step behavior visible in rate; compare against DRAFTER=0 arm if in doubt (never against gufo — bench law).
5. Harness round-trip: an OMP call through llamacpp (dankclaw)/frank-large incl. a tool call (V1TOOLS).
6. Prompt > 200,700 tokens returns the 400 naming the limit (contract test).

Rollback

Revert the frank-large swap commit and redeploy dankclaw --service frank-large: the ROCmFPX payload (/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/), the fork build (/home/dank/GitPackages/kingjones-ROCmFPX/build), and its scripts/build-rocmfpx.sh pin (commit 36e9acd, seam-tested) all remain on disk. The halogen payload can stay: it costs nothing while the unit points elsewhere.