serverAppConfigs/serverAppConfigs-docs/LLM/frank-dragoon.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)	The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it.	llm frank-dragoon dankclaw strix-halo gfx1151 halogen endpoint	2026-10-18

frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU)

Strix Halo (gfx1151) endpoint http://192.168.0.143:7770, served id frank-dragoon: a podman container under a root User=dank unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18 (2b12716). This page replaces every per-era dragoon runbook; the Eras table is the rollback record.

Current shape

scripts/frank-dragoon.env is authoritative and scripts/serve-frank-dragoon.sh passes exactly those flags and nothing else. Payload /home/dank/models/halogen-v2-abliterated/, verified by scripts/download-frank-dragoon-v2abl.sh --verify-only:

File	Bytes	sha256	Source
qwen38-flash-next-v2-abliterated.hgn	66,687,678,432	b5bd925f…	Quat3rnion @main — the NATIVE 2026-10-11 v2 conversion of orcarouter's BF16 Qwen3.8-Flash-Next, abliterated
qwen38-flash-next-ngram.hgn	51,200,246,144	3450acada…	peonist @5cc17cea, REQUIRED, hardlinked
qwen38-flash-next-vision.hgn	897,916,416	d62e0ae5…	peonist @e053f488, hardlinked — vision ON
tokenizer/ (6 flat files)	~22 MB	tokenizer.json LFS	peonist @e053f488, hardlinked

Image ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a (0.16.2, closed source), container frank-dragoon-halogen, published 7770:8731; the engine's unauthenticated token port 8730 stays on container loopback. Engine stdout → /home/dank/logs/frank_dragoon.log, not the journal.

• Shape: HALOGEN_CTX=200704 hard per-request ceiling · HALOGEN_KV_SLOTS=2 · HALOGEN_KV_POOL_POSITIONS=401408 = slots × ctx, the top of the band tests/frank_dragoon_unit.bats enforces (ctx ≤ pool ≤ slots × ctx) · KV ~28 KiB/position → 11.0 GiB reserved · no KV-dtype knob exists here, the pool is the only KV lever · native window 262,144, HALOGEN_ROPE_YARN beyond it.
• Knobs: HALOGEN_MAX_TOK=8192 sizes the prefill arena (5.7 GiB) and is only the default for requests omitting max_tokens · HALOGEN_MAX_TOKENS_CAP=65536 + HALOGEN_QUEUE_TIMEOUT=7200 are a coupled pair · MTP depth 2 with the head INSIDE the checkpoint (HALOGEN_MTP_HEAD stays unset) and HALOGEN_SPEC_ADAPT=0 · sampler 1.0 / 0.95 / 20 server-side · HALOGEN_WEIGHTS_LOCK=1 · HALOGEN_HOST_RESERVE_GIB=20 · prompt cache ON by image default (PROMPT_CACHE=2, no flag passed): 12 entries, which the boot ledger prices at 111 MiB each at this window = 1.3 GiB (the env header's "~85 MiB here" estimate did not survive that ledger line).
• Deployed boot ledger: 62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB of KV pool, 5.7 GiB of working memory, 79.6 GiB in all; n-gram mapped 51.2 GB, NOT pinned (Pin::None); no clamp line. Host MemAvailable read 22 GiB right after the 2026-10-18 raise — ~20 GiB above the 20 GiB reserve, against 10–15 GiB of baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observability).
• Harness (~/.omp/agent/models.yml, provider llamacpp (dankclaw)): contextWindow: 155648 (= ceiling − 45k per the 400-zone law, never higher), maxTokens: 45056, input: [text, image] reasoningContentField: "reasoning_content", high→xhigh fold, extraBody 1.0/0.95. Compaction rides frank-zealot.
• Deploy ./scripts/deploy.sh dankclaw --service frank-dragoon; gate = boot-ledger greps (weights, pool, MTP, prompt cache, no clamp) + /health 200 + a tool call + the 512 px red-PNG probe. Kernel: CONFIG_HSA_AMD_SVM=y (CachyOS ships it) and amdgpu.noretry=0 — the env says check it at deploy; boot line per dankclaw-iommu-translation-off.

Eras

Rollback = revert that era's swap commit (or repoint the payload vars) and redeploy dankclaw --service frank-dragoon. Newest first; payloads staged under ~/.AGENTTRASH/ are not a rollback plan — recoverable only until the trash is purged.

Window	Payload	Runtime	Shape	Replaced because	Rollback
2026-10-06 → 10-17	Swift1.5 abliterated v2 .hgn r2 66,687,678,464 B 9c880b8b… + same trio	flash-server 0.16.2, same digest	2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on	—	/home/dank/models/halogen-flashnext-v2/ + scripts/download-frank-dragoon-flashnext.sh
2026-10-05 → 10-06	peonist halogen-qwen3.8-27b qwen3.8-27b-p1w4d-d2.hgn 35,865,565,184 B 274c3fc7…	halogen 0.1.4 sha256:dc0a39a0…	1 slot × 200,700, DFlash2 drafter, HALOGEN_CACHE_MB=16384, text-only, censored base	flash-server v2 restored vision, abliteration and MTP	/home/dank/models/halogen-qwen3.8-27b/ + scripts/download-frank-dragoon-halogen.sh; 0.1.4 digest still in podman storage
2026-10-03 → 10-05	kingjones777 Uncensored Q6_0_ROCMFPX_AGENT.gguf 24,856,607,008 B e966deb3… + mtp-Q4_0 051a1764… + BF16 mmproj	kingjones30/ROCmFPX @ 36e9acd (scripts/build-rocmfpx.sh)	1 × 262,144 f16 KV, draft-mtp n-max 4, vision on	—	/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/ + /home/dank/GitPackages/kingjones-ROCmFPX/build + download-frank-dragoon-kingjones-uncensored.sh
2026-10-02 → 10-03	aligned Q8_0_ROCMFPX_AGENT.gguf 28,214,425,952 B f3ef2bdb…	same fork pin	1 × 262,144 f16 KV, draft-mtp 4 (MTP +123 % decode)	the uncensored file was faster and uncensored	/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/ + download-frank-dragoon-kingjones.sh
2026-10-01 (one day)	MiMo-V2.6-Distill-Qwen-9B BF16 17.53 GiB + cross-model MTP from Qwen3.5-9B	vLLM 0.30 TheRock kyuz0/vllm-therock-gfx1151 sha256:98c1f3ac…	4 seqs × 262,144 on a scheduler-shared pool, 2 draft tokens	—	staged /home/dank/.AGENTTRASH/20261002-retired-llm-payloads/ until purged; fetch trio in old/
2026-09-28 → 10-01, again 10-01 → 10-02	Swift-1.5 Uncensored UD-Q8_K_XL 31,457,990,272 B 9076f99a…, then UD-Q4_K_XL; z-lab DFlash2 Q4_K_M	gufo v0.1.1 (b0f8673), source build	1 × 262,144, --draft-policy adaptive --draft-tokens 7, no KV knob, no Jinja	—	payloads deleted 2026-10-03 → re-download download-frank-dragoon-swift.sh (+ download-frank-dragoon.sh for the drafter); tag frank-dragoon-gufo-q4kxl-20261001; scripts/build-gufo.sh
2026-09-21 → 09-28	unsloth Qwen3.8-27B-GGUF:UD-Q8_K_XL 31.46 GiB + DFlash2 Q4_K_M 1.14 GiB	unsloth llama.cpp b11030 prebuilt	1 × 262,144, q8_0 KV, Qwen-Sharp v22.4.0 template	gufo: +37 % prefill shallow, +80 % at 131k, 4.5× faster load	/home/dank/GitPackages/unsloth-llama-b11030/ on disk; payload deleted 2026-10-03 → download-frank-dragoon-unsloth.sh
2026-09-11 → 09-15, again 09-16 → 09-19	peonist-ai/halogen-qwen3.8-flash-next w4b 115.55 GiB + in-place-versioned overlay.hgn	halogen-flash-server 0.5.8, rootless Quadlet	2 × 200,192 over a 400,384 pool, MAX_TOK 24576, HOST_RESERVE 20, vision OFF	displaced by CIRU Orca; the shape starved co-tenants	re-tag 0.5.8 + first-start HALOGEN_DOWNLOAD re-fetch into /home/dank/models/halogen; the Quadlet left the repo 2026-10-05, so the unit must be rewritten
through 2026-08-24	CIRU ActiveFPX PromptForge v3: GGUF 15.44 GiB + 3 .pfs + mmproj ≈ 36.7 GiB	CIRU ROCmFPX build-promptforge	1 × 262,144, native MTP depth 4, ngram off	eemin ROCmI4	/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/ + /home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server; scripts git revert 4c43839 7aee1df 40d50fe or checkout 132e147
≤ 2026-09 — 11 earlier eras: payloads, shapes and every measured number destroyed 2026-10-19 as obsolete	—	—	—	—	—

Port note: this slot settled on :7770 on 2026-08-22 (52625 before it); the 2026-08-17 fleet redo created it — earlier topologies in .

Laws learned here

• KV arithmetic: ~28 KiB/position incl. block scratch plus ~115 MiB/slot of O(1) state; reservations measured 5.5 GiB @200,704, 7.2 @262,144, 11.0 @401,408, 14.6 @524,288. Quote the reserving the KV pool line, not the preflight formula.
• One pool, not a partition: the pool decides how many conversations stay resident; only HALOGEN_CTX sets the window, and over-window is a 400 naming the limit, never a truncation. The 2026-10-18 doubling to 401,408 was bought as pool parity with frank-zealot so each slot can hold a maximum-length request at once — it did not lengthen the window by one token.
• Budget: device budget = 121.2 GiB (unrounded MemTotal 121.2159) − pinned weights − HOST_RESERVE_GIB; need = KV + arena + vision, arena = 3.80 + 8.40 × (MAX_TOK/16384) GiB — it predicts every recorded point to 0.1 GiB. HOST_RESERVE_GIB is a term in that inequality, not an allocation: it cannot return RAM, only take the pool away (ceiling 16 at the 524,288 shape, ≥18 clamped it). Free RAM by cutting the pool or MAX_TOK.
• HALOGEN_KV_POOL_FIT prints its sizing block only when it clamped — silence is not headroom and a green /health is not the pool you asked for.
• RAM the pool takes is RAM the 47.68 GiB n-gram table loses (page cache, Pin::None). The real failure is file-cache starvation, not RAM pressure — long prompts crawl with the disk busy and the process uninterruptible, and the fix is a smaller pool.
• Pinned weights count as reclaimable file cache, so free, MemAvailable and Grafana overstate free RAM by ~63 GiB; believe the engine's … in all line.
• Pin gate: the engine refuses to pin below checkpoint + 16 GiB of MemAvailable (pinned pages cannot swap, and under the floor the n-gram table pages in from disk every long prompt), so a RAM-short host crash-loops the unit every ~5 s per boot — 2026-10-06: 8 boots at MemAvailable 74.77 GiB against the 78.11 GiB needed, self-resolved in ~2 min when the dying 27B engine freed RAM. The serve script waits FRANK_DRAGOON_RAM_WAIT (600 s) and refuses loudly if the RAM never clears.
• HALOGEN_FLASH_PIN_TRUNK=0 is refused by the v2 loader (and cost several times decode on the w4b build): disk-backing the weights is not a lever here.
• MTP is always on: HALOGEN_SPEC_ADAPT=0 is mandatory — the adaptive default disables the head for whole requests and halves decode. Depth 2 all-round, 3 for code.
• Batch XOR speculation was engine-version-specific: halogen 0.1.4 lost the drafter above one slot (41 → 14 t/s); flash-server runs two slots with the head. On the ROCmFPX/llama.cpp line --spec-draft-n-max defaults to 16 and must be pinned to 4 — the measured knee on the dense family, with the draft head offloaded (--spec-draft-ngl 999) and ngram never layered under it.
• Prompt cache is free here and was manual on the last engine: flash-server ships it on and its auto-sizing fits this box's ledger (no pin) — a 100k follow-up re-prefills in ~2 s instead of ~88 s cold (env, 2026-10-17), and the 2026-10-18 A/B measured 81.5 s cold against 0.3 s warm at 101,987 prompt tokens. halogen 0.1.4's auto-sizing refused here and re-prefilled every 79–81k turn in 85–180 s until HALOGEN_CACHE_MB=16384 was pinned. What the pool takes is what the table loses, so re-measure the warm ratio after any pool change.
• 400-zone law: the harness window MUST sit ~45k under the engine ceiling (contextWindow: 155648 = 200,704 − 45,056) because the harness clamps max_tokens to contextWindow − est − 4096 its chars/4 est under-counts code/agent traffic by ~3 %, and compaction only fires at 85 %. The 3 % is measured, not theory: 2026-10-07 at the old 200,192 ceiling, est 152,938 vs server 157,388 — over by 354, a 400. At the live shape compaction fires at 132,301 est ≈ 136k measured (the 180k target of the 262,144 era is unreachable — the window itself is 155,648), and the worst-case request is 155,648 − 4,096 + 3 % ≈ 156.2k ≤ 200,704: ~9.5× the error of margin, robust to a ~25 % under-count. Compaction rides frank-zealot at ≈ threshold − keepRecent(20k) ≈ 112k input.
• Admission reserves prompt + max_tokens: an un-fittable request waits in arrival order, then 503s engine_busy for the whole queue after QUEUE_TIMEOUT — and the cap/timeout pair moves together: 65536 tokens at ~10 t/s serial needs >6550 s, which is why 7200 is pinned (env, 2026-10-17).
• Depth is the cost on this silicon: ~8 µs per token of context per round (KV reads ~8 GB/s against 180–240 GB/s of weight streaming) — 30–50 t/s shallow, 3.3–3.8 at 79–81k, so deep-conversation roles belong on the fp8-KV endpoint frank-archon. BF16 dense decode tops out at 13.1 t/s (≈94 % of the roof): only speculative decode moves it.
• The file picks the runtime, never the reverse: v2 .hgn only in flash-server, the 27B .hgn only in halogen, ROCmFPX types 101/102/103/105/108 in no stock loader; fork builds pin by SHA — pre-36e9acd ROCmFPX loads the AGENT files then SIGSEGVs in ggml_compute_forward_set_rows.
• Same filename ≠ same model: native v2 is 66,687,678,432 B, the Swift1.5 r2 is 66,687,678,464 B. Byte count + LFS sha256 is the only discriminator, and upstream versions sidecars in place, so equal sizes prove nothing.
• Companion law: the n-gram table is REQUIRED beside the checkpoint; if the publisher ships only the main file, hardlink the engine vendor's stock table/tower/tokenizer (a hardlink is a regular file and passes the no-symlink gate) and referee with the engine's own tools — a mismatched table raises KL everywhere and degrades retrieval, which is what NIAH + needle probes rule out.
• Payload gates: flat tokenizer directory, every entry a regular file (HF cache symlinks dangle inside the mount), and the serve gate must mirror the deployer exactly or deploy goes green while the unit crash-loops.
• Serve-script lore (this file has been bitten by all three): capture podman images -q into a variable before grepping it — under pipefail, grep's early exit killed the pipeline and the deploy refused with "no match for the pinned digest" on a box that had the image (2026-09-30); never put a comment inside the podman run backslash chain — a mid-chain # terminates the continuation and runc gets --name run (2026-10-05, exit-127 crash loop); pass only --ulimit memlock, since Dankclaw's nofile hard limit is 524,288 and rootless runc cannot raise it (exit 126), and clear the legacy frank-dragoon container name before starting.
• Bench discipline (this endpoint wrote the fleet law): stop the service for any GPU probe (census-frank-dragoon.sh verify is the only GPU-free one); gufo bench cannot rank speculative settings and inverts them; ≥128 generated tokens or the per-token figure is first-token noise; card acceptance/t-s numbers are file properties, not promises — the live card's 53–57 t/s greedy decode and ~1,640 t/s prefill at 34k measured 50.0 and 1,604 here (2026-10-17), and the 27B card's 31.7 t/s measured 27.6–28.1; health 200 does not prove your pid owns the port; there are no scratch ports on this host — A/B through scripts/trial-frank-dragoon.sh drop-ins; never pkill -f llama-server (frank-zealot is one), sweep by payload name.
• Dead ends — do not re-try: ngram under MTP or DFlash (slower at every depth; on recurrent Gated-DeltaNet layers it also disables MTP's rollback path, −7…−15 %); DFlash across parallel slots (shared draft ctx spuriously emits EOG); -ot/-ncmoe on a hybrid split (disables mmap, page-locks ~108 GiB and stranded the host until a power cycle); --no-mmap on any PLE-table file (silent cgroup OOM-kill); forcing PLE to CPU (23 → 13.4 t/s); HALOGEN_FA_TQ=256 (no decode win, −9.5 % prefill) big ubatches as a decode knob (prefill-only, ub 512 fastest); Flash-Next MTP heads on the qwen4exp loader (only blk.48, never the full per-layer norm set); mradermacher i1 GGUFs (NextN stripped) vLLM quant paths on gfx1151; a custom Jinja template on gufo (no Jinja engine — send the terse text as a client system message instead).
• Build lore if a fork is rebuilt: purge Ubuntu librocwmma-dev (shadows AMD's headers, -I cannot beat it), pin --gcc-install-dir to GCC 15, -DLLAMA_BUILD_WEBUI=OFF; four HIP processes is the device ceiling and ComfyUI is a co-tenant; amdgpu: SVM mapping failed spam is a symptom, not a cause.

Contract (wire)

• Live: /v1/models, /health, /metrics, POST /v1/chat/completions; 404: /ping, /props /completion, /tokenize. No Server-Timing header and no /completion timing object — measure wall-clock plus usage.prompt_tokens.
• Thinking lands in reasoning_content and counts against max_tokens: a budget that dies mid-thought returns finish_reason: "length" with empty content, so clients read both fields; size vision probes at ≥512 tokens or the model looks blind. Budget for the base reasoning long: 1,476 thinking tokens mean on the 2026-10-17 battery, against 784 for the retired Swift1.5 finetune (×1.9) and 905 for the censored 27B.
• reasoning_effort accepts only xhigh|medium|low (anything else is a template error — the models.yml fold stays) and there is no "none": disable thinking with enable_thinking:false which is also what the llama.cpp-era QwenSharp template wanted (reasoning_effort:"none" threw there too). preserve_reasoning is a 400 on this engine; the llama.cpp-era key was chat_template_kwargs.preserve_thinking, which decided whether <think> blocks stayed in recorded assistant messages. Sampler defaults 1.0/0.95/20 are server-side, so an absent temperature is NOT greedy (halogen 0.1.4 and w4b decoded greedily); unknown sampling fields are a 400, out-of-range values are refused not clamped, and response_format is unsupported.
• Over-ceiling prompt → 400 naming the exact overage, rejected by preflight in <1 s, so probing the ceiling is free; over-cap max_tokens → 400 naming the cap; max_tokens, max_completion_tokens and max_output_tokens all work and must agree.
• Vision: OpenAI image_url parts only (top-level images is the frank-zealot dialect), ≥256 px or the model answers "video", ~2,040 tokens per image, http(s) image URLs refused on purpose.
• The first request after a boot is a warmup artifact and may sit silent 45–60 s with 503 engine_unresponsive then self-recover — do not restart over it; /health 503 during load is loading. Flags read once at startup, all bitwise except HALOGEN_CK_OVERLAY, HALOGEN_MATMUL_TUNING_FILE HALOGEN_INDEXER_BUDGET, HALOGEN_COMPOSABLE_CONTEXT.
• Deploy waits for :7770 to be free: a rootless container binds it at CREATE and exits 126, and a leftover Quadlet beside the root unit guarantees the race.

Files

• scripts/frank-dragoon.env · scripts/serve-frank-dragoon.sh (gates, RAM pin-gate wait, foreground podman run, ledger to /home/dank/logs/frank_dragoon.log) · scripts/frank-dragoon.service (User=dank, LimitMEMLOCK=infinity, Restart=always, OOMScoreAdjust=-500, ExecStop=podman stop -t 60) → deploy_llm_dragoon in scripts/deploy-remote.sh.
• Fetchers: download-frank-dragoon-v2abl.sh (live); era rollbacks -flashnext, -halogen -kingjones, -kingjones-uncensored, -swift, -unsloth, -q4, -orca download-frank-dragoon.sh; runtime pins build-rocmfpx.sh, build-gufo.sh.
• Verification: tests/frank_dragoon_unit.bats (payload gates, deployer/serve parity, spec block, KV band), tests/llm_units.bats, tests/llm_secrets.bats, scripts/census-frank-dragoon.sh scripts/battery-frank-dragoon.py, scripts/stage-frank-dragoon-eval.sh scripts/trial-frank-dragoon.sh. -Related: · · · ·

---