serverAppConfigs/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md

frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, Strix Halo iGPU)

RETIRED 2026-10-06 → 2026-10-17, superseded by the native v2 conversion: frank-dragoon-flash-server-v2-native. Rollback target — the payload stays on disk at /home/dank/models/halogen-flashnext-v2/, scripts/download-frank-dragoon-flashnext.sh re-verifies it, and restoring the era is a revert of the 2026-10-17 swap commit. The 27B payload this era replaced stays at /home/dank/models/halogen-qwen3.8-27b/ with scripts/download-frank-dragoon-halogen.sh. Everything below is the era as it shipped; the engine-flag detail still governs the current page.

Why this shape exists

Frank's ask: replace the frank-dragoon model with https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated. That payload is an abliterated Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint in the v2 .hgn format — a file that loads ONLY in the halogen-flash-server engine line, NOT in the halogen 0.1.4 container that served the 27B dense checkpoint. The model move is therefore a full engine migration, and the accepted-product facts flip twice in the fleet's favour:

• the base is abliterated (the 27B halogen endpoint was a finetune of the censored Qwen/Qwen3.8-27B; refusals were an accepted regression there),
• vision is back — the v2 vision tower loads beside the checkpoint and the engine runs it (the 27B halogen engine silently ignored its encoder),
• MTP speculative decoding returns with the head inside the checkpoint itself (the 27B shipped a DFlash2 drafter instead; comparable purpose, different file plumbing),
• the card claims 53–57 t/s greedy decode, ~1,640 t/s prefill at 34k, ~72 % fewer thinking tokens than the censored 27B (905 mean), and 262,144 native context. Publisher-card numbers — verify against live traffic before quoting further (see Verification).

The engine is closed-source, ships as ghcr.io/peonist-ai/ halogen-flash-server:0.16.2 (the version the model card's quickstart pins), digest-pinned:

ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a

(3.6 GB image; RepoDigest captured at pull 2026-10-06. Card-tested fallback if a future pin misbehaves: 0.15.1.)

The payload — a triangle of files, all sha256-verified at fetch

File	Bytes	Source repo @ revision	sha256 (LFS)
qwen38-flash-next-v2-swift15-abliterated.hgn	66,687,678,464	Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated @ main (69a85db)	9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037
qwen38-flash-next-ngram.hgn (REQUIRED)	51,200,246,144	peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea	3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
qwen38-flash-next-vision.hgn (enabled)	897,916,416	peonist-ai/halogen-qwen3.8-flash-next @ e053f488	d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
tokenizer/ (6 files, flat dir)	22.4 MB total	peonist @ e053f488	tokenizer.json is LFS 0997f410… (byte-identical to the 27B-era copy); the rest are plain git blobs, size-checked

Payload notes worth keeping:

• The deployed checkpoint is a compacted r2 of the card's append-only original (89,775,868,032 B → 66,687,678,464 B): the abliteration workflow appends and the repo's tooling/ compact drops dead padding; live tensors are byte-identical, the LFS sha256 is the contract, and the 24 GiB saving is why it fits.
• HALOGEN_MTP_HEAD stays UNSET: the MTP head is INSIDE the v2 checkpoint (the separate qwen38-flash-next-mtp.hgn sidecar is for a different assembly and is deliberately not fetched).
• The n-gram lookup table is REQUIRED. The engine reads it beside the checkpoint (documented fallback); the card's quickstart also names it explicitly via HALOGEN_NGRAM_TABLE, and the serve script passes both — the env var is belt-and-braces over the documented beside-the-checkpoint default, and both mechanisms resolve to the same file.
• The tokenizer is fetched fresh from peonist @ e053f488 into /home/dank/models/halogen-flashnext-v2/tokenizer/ — same flat-directory, no-symlink law as every other payload (an HF cache snapshot's symlinks dangle inside the container mount; both the serve gate and the deployer refuse them).

Fetch: scripts/download-frank-dragoon-flashnext.sh (--verify-only to re-check; -f + exact byte counts + LFS sha256 + resolve-the-CDN-then--C - resume law throughout; no token — the repos are public).

Shape — the knobs that are not free

scripts/frank-dragoon.env carries the full rationale inline; the table:

Env	Value	Why
HALOGEN_CTX	262144	per-slot HARD ceiling (the checkpoint's native window; a longer prompt is a 400 naming the limit, never truncation). RAISED from 200192 on 2026-10-07 — the 400-zone law: the harness clamps max_tokens = min(45056, contextWindow − est − 4096), its chars/4 est UNDER-counts the rendered prompt by ~3 % (measured twice: est 152,938 → server 157,388, request over room by 354; est 158,906 → server 165,301, over by 2,300), and compaction only fires at 85 % of the harness window — so with window == engine ceiling, every request in the est ~146k–184k zone was a guaranteed 400. Harness window 217,088 = ceiling − 45k absorbs the error: worst-case request ≈ 219k ≤ 262,144 (≈10 % under-count headroom)
HALOGEN_KV_SLOTS	2	the 27B engine's "slots>1 disables the drafter" law does NOT carry over — on this engine batching and speculation coexist (the card's quickstart ships 2 slots WITH the head). 2 slots = concurrent capacity
HALOGEN_KV_POOL_POSITIONS	524288	explicit form of the default (2 × ctx), so the seam test pins the product; ~28 KiB/position ⇒ 14.0 GiB KV
HALOGEN_HOST_RESERVE_GIB	20	keeps the host honest (the w4b-era lesson)
HALOGEN_SPEC_ADAPT	0	MANDATORY (card): the default adaptive policy turns the MTP head off for whole requests after low acceptance and halves decode. The head stays on; temp-1.0 traffic pays some acceptance for it — accepted trade
HALOGEN_MAX_TOK / HALOGEN_MAX_TOKENS_CAP / HALOGEN_QUEUE_TIMEOUT	8192 / 65536 / 7200	8192 is only the per-request DEFAULT (the harness sends at most its maxTokens 45,056); the cap/timeout pair is COUPLED per FLAGS.md (65536 at ~10 t/s serial needs > 6550 s)
HALOGEN_TEMPERATURE / TOP_P / TOP_K	1.0 / 0.95 / 20	the card's thinking recipe as SERVER-SIDE defaults — the engine HAS a sampler now, so an absent temperature is no longer greedy; models.yml extraBody matches it (consistent, not load-bearing)
HALOGEN_WEIGHTS_LOCK	1	card's belt-and-braces; the engine pins weights by default on v2 (HALOGEN_FLASH_PIN_TRUNK=0 is a w4b-only knob and is REFUSED by v2)
HALOGEN_MODEL_ID / publish	frank-dragoon / 7770:8731	served id renamed 2026-10-09 (service/unit/env keep frank-dragoon); engine port 8730 never published
MTP depth	default 2	FLAGS.md: 2 is best all-round, 3 for code — Frank's traffic is agent/prose

Memory ledger (121.2 GiB unified, the referee is the boot ledger): weights ~62 GiB resident + KV 14.0 + arena ~8 (3.8 + 8.4 × 8192/16384)

• vision ~2 ≈ 86 GiB engine-side, + 20 host reserve = ~106 with ~15 GiB margin over the box's baseline co-tenancy. The 51.2 GB n-gram table rides as mmap'd page cache beside the checkpoint — the card's own quickstart arithmetic does not bill it as resident, but WATCH the ledger at boot and re-size if the engine prints a clamp. Prompt cache is ON by default (PROMPT_CACHE=2): a follow-up turn at 100k ctx costs ~2 s instead of ~88 s cold — the big-TTFT win the 27B engine needed a manual HALOGEN_CACHE_MB pin for is built in here.

Engine contract facts (FLAGS.md of the flash-server repo)

• OpenAI chat and /v1/responses and /v1/messages and /v1/completions; tool calls verified on the card; response_format still unsupported (no constrained decoding).
• Thinking → reasoning_content (same field as every engine this endpoint has run); the v2 chat template rejects reasoning_effort outside xhigh|medium|low — the models.yml fold (high→xhigh) stays mandatory.
• Vision: OpenAI image_url content parts only (the fleet's standard contract; the legacy top-level images field is the frank-zealot trap). ≥256 px probe images or the model answers "video".
• HALOGEN_MAX_TOKENS_CAP over-cap is a 400 naming the limit — never a truncation. max_tokens INCLUDES thinking.
• All flags read once at startup; every flag is BITWISE except HALOGEN_CK_OVERLAY, HALOGEN_MATMUL_TUNING_FILE, HALOGEN_INDEXER_BUDGET, HALOGEN_COMPOSABLE_CONTEXT. Unset HALOGEN_CHECKPOINT defaults to /models/qwen38-flash-next-v2.hgn with the lookup table beside it.
• Prompt-lookup drafting (3,3 window) rides beside the MTP head by default and is byte-identical to greedy verification where it fires.
• Watchdog: HALOGEN_ENGINE_WATCHDOG_S=180 with defer-logic for memory stalls — on a shared host the defaults are the safe choice; the restart policy stays real through the root unit's Restart=always.

Kernel: CONFIG_HSA_AMD_SVM=y (CachyOS ships it); amdgpu.noretry=-1 (auto) — for this IP auto resolves to retry-on, the same effective state the 27B engine ran on. Only revisit if the ledger shows SVM complaints.

Mechanics

• Root systemd unit (scripts/frank-dragoon.service, User=dank, LimitMEMLOCK=infinity, Restart=always, ExecStop=podman stop -t 60 frank-dragoon-halogen), driven by scripts/serve-frank-dragoon.sh — the fleet's podman run-direct pattern. The script runs the container FOREGROUND so the boot ledger (weights resident, pool fit, MTP head, prompt cache, any clamp) lands in /home/dank/logs/frank_dragoon.log.
• Serve gates mirror the deployer EXACTLY (2026-10-01 crash-loop law): image digest in podman storage, byte-exact checkpoint + n-gram table + vision tower, flat tokenizer with all six files as regular files, GPU nodes present.
  • Host-RAM pin-gate preflight (added 2026-10-06): the engine refuses to pin when MemAvailable is below the checkpoint (62.11 GiB) plus a 16 GiB floor — pinned pages cannot be swapped, and under the floor the n-gram lookup pages in from disk on every long prompt. A RAM-short host crash-loops the unit (each boot dies in ~5 s at the pin gate) until the RAM clears; the 2026-10-06 cutover hit this (8 boots, MemAvailable 74.77 GiB vs the 78.11 GiB needed, self-resolved in ~2 min when the dying 27B engine freed RAM). The serve script now checks MemAvailable BEFORE podman run and WAITS up to FRANK_DRAGOON_RAM_WAIT (default 600 s) for the RAM to clear, so a transient dip boots once instead of crash-looping; if it never clears it refuses loudly. The 16 GiB floor is the engine's (not HALOGEN_HOST_RESERVE_GIB=20 — a separate, higher, overall-budget knob).
• Engine stdout goes to the log, NOT the journal. The FIRST request after a (re)start can sit silent ~45–60 s on this engine class (health engine_unresponsive, 503s) and self-recovers — do not restart over it.

Verification checklist (run after any redeploy)

1. Boot ledger greps in /home/dank/logs/frank_dragoon.log: resident weights (~62 GiB), KV pool line, MTP/drafter line, prompt-cache line, no clamp named.
2. curl http://192.168.0.143:7770/health → 200.
3. Text completion probe: reasoning lands in reasoning_content, timings honest (generate ≥128 tokens; at 8 the inter_token number is first-token noise).
4. Vision probe: a 512×512 solid-red PNG through an image_url part → "Red" (≥256 px, one word expected).
5. Spec decode: draft_n/draft_n_accepted present in timings on real-shaped traffic (SPEC_ADAPT=0 means the head is always engaged).

Client contract (harness)

~/.omp/agent/models.yml — provider llamacpp (dankclaw) @ http://192.168.0.143:7770/v1: contextWindow: 217088 — deliberately BELOW the 262,144 engine ceiling: the 45k gap absorbs the harness's ~3 % chars/4 est under-count (see the HALOGEN_CTX row above) — maxTokens: 45056, input: [text, image], reasoningContentField: "reasoning_content", the high→xhigh effort fold, extraBody temperature 1.0 / top_p 0.95. Compaction still rides frank-zealot (the compactionModel steering is unchanged; the input ≈ 170k fits its 200,704 slot).

Rollback

Revert the swap commit and redeploy — the 27B trio is intact on disk (payload at ~/models/halogen-qwen3.8-27b/, fetch script, and the halogen 0.1.4 image digest still in podman storage). The older ROCmFPX rollback door (build-rocmfpx.sh pin) is untouched.