Files
serverAppConfigs/serverAppConfigs-docs/LLM/frank-dragoon.md
T
Frank 6215f1a6c8 docs(vault): halogen has an NPU lane on the pin we already run, and it is greedy-only
- frank-dragoon: measured the response_format contract. json_schema and json_object answer
  schema-valid JSON at temperature 0 and draw a 400 at any sampled temperature, and because
  the sampler default is server-side an absent temperature is sampled. The page claimed
  "response_format is unsupported", which sends the next agent past a working feature.
- frank-dragoon: new "The NPU lane" section. The pinned 0.16.2 image already ships the
  five-program AIE catalog including qwen3-embedding-0.6b, so /v1/embeddings is one env var
  plus the accel device, the distro-XRT mount and the fabric-clock hold; embeddinggemma-2
  needs 0.18.3+, the stock-kernel-driver memory-budget fix 0.18.8+. Host census recorded:
  no /opt/xilinx/xrt, distro XRT 1:2.21.75 with libxrt_driver_xdna, clocks unheld.
- Open Notebook: scriptwriter probe verdict — greedy-only, and open-notebook's model record
  is {credential, name, provider, type} with no per-row config, so it cannot ask for greedy;
  frank-archon stays the writer. The embedding gap now has a named door instead of "build a
  new llama.cpp slot".
- Frank TTS: "the NPU is not a door at all" was too broad. It is not a door for the TTS
  family; halogen's precompiled catalog is a door on the same device.
- System Specs: NPU runtime row corrected to the distro XRT actually installed, and the
  stale "frank-zealot serves 27B" line retired in favour of [[frank-zealot]] as sole owner.

Tests: ./tests/run.sh, 455 passed.
2026-10-19 16:40:48 -07:00

279 lines
24 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
type: reference
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it."
tags:
- llm
- frank-dragoon
- dankclaw
- strix-halo
- gfx1151
- halogen
- endpoint
timestamp: 2026-10-18
---
# frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU)
Strix Halo (gfx1151) endpoint `http://192.168.0.143:7770`, served id `frank-dragoon`: a podman container
under a root `User=dank` unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18
(`2b12716`). This page replaces every per-era dragoon runbook; the Eras table is the rollback record.
## Current shape
`scripts/frank-dragoon.env` is authoritative and `scripts/serve-frank-dragoon.sh` passes exactly those
flags and nothing else. Payload `/home/dank/models/halogen-v2-abliterated/`, verified by
`scripts/download-frank-dragoon-v2abl.sh --verify-only`:
| File | Bytes | sha256 | Source |
|---|---|---|---|
| `qwen38-flash-next-v2-abliterated.hgn` | 66,687,678,432 | `b5bd925f…` | Quat3rnion `@main` — the NATIVE 2026-10-11 v2 conversion of orcarouter's BF16 Qwen3.8-Flash-Next, abliterated |
| `qwen38-flash-next-ngram.hgn` | 51,200,246,144 | `3450acada…` | peonist `@5cc17cea`, **REQUIRED**, hardlinked |
| `qwen38-flash-next-vision.hgn` | 897,916,416 | `d62e0ae5…` | peonist `@e053f488`, hardlinked — vision ON |
| `tokenizer/` (6 flat files) | ~22 MB | `tokenizer.json` LFS | peonist `@e053f488`, hardlinked |
Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a`
(0.16.2, closed source), container `frank-dragoon-halogen`, published `7770:8731`; the engine's
unauthenticated token port 8730 stays on container loopback. Engine stdout →
`/home/dank/logs/frank_dragoon.log`, **not** the journal.
- Shape: `HALOGEN_CTX=200704` hard per-request ceiling · `HALOGEN_KV_SLOTS=2` ·
`HALOGEN_KV_POOL_POSITIONS=401408` = slots × ctx, the top of the band `tests/frank_dragoon_unit.bats`
enforces (`ctx ≤ pool ≤ slots × ctx`) · KV ~28 KiB/position → **11.0 GiB reserved** · no KV-dtype knob
exists here, the pool is the only KV lever · native window 262,144, `HALOGEN_ROPE_YARN` beyond it.
- Knobs: `HALOGEN_MAX_TOK=8192` sizes the prefill arena (5.7 GiB) and is only the default for requests
omitting `max_tokens` · `HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled
pair · MTP depth 2 with the head INSIDE the checkpoint (**`HALOGEN_MTP_HEAD` stays unset**) and
`HALOGEN_SPEC_ADAPT=0` · sampler 1.0 / 0.95 / 20 server-side · `HALOGEN_WEIGHTS_LOCK=1` ·
`HALOGEN_HOST_RESERVE_GIB=20` · prompt cache ON by image default (`PROMPT_CACHE=2`, no flag
passed): 12 entries, which the boot ledger prices at 111 MiB each at this window = 1.3 GiB
(the env header's "~85 MiB here" estimate did not survive that ledger line).
- Deployed boot ledger: `62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB
of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`; n-gram `mapped 51.2 GB, NOT pinned
(Pin::None)`; no clamp line. Host `MemAvailable` read 22 GiB right after the 2026-10-18 raise —
~20 GiB above the 20 GiB reserve, against 10–15 GiB of baseline co-tenancy (frank-zealot, ComfyUI,
Caddy/observability).
- Harness (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw)`): `contextWindow: 155648`
(= ceiling − 45k per the 400-zone law, never higher), `maxTokens: 45056`, `input: [text, image]`
`reasoningContentField: "reasoning_content"`, `high→xhigh` fold, `extraBody` 1.0/0.95. Compaction
rides [[frank-zealot]].
- Deploy `./scripts/deploy.sh dankclaw --service frank-dragoon`; gate = boot-ledger greps (weights,
pool, MTP, prompt cache, **no clamp**) + `/health` 200 + a tool call + the 512 px red-PNG probe.
Kernel: `CONFIG_HSA_AMD_SVM=y` (CachyOS ships it) and `amdgpu.noretry=0` — the env says check it at
deploy; boot line per [[dankclaw-iommu-translation-off]].
## Eras
Rollback = revert that era's swap commit (or repoint the payload vars) and redeploy
`dankclaw --service frank-dragoon`. Newest first; payloads staged under `~/.AGENTTRASH/` are **not** a
rollback plan — recoverable only until the trash is purged.
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---|
| 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | — | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` |
| 2026-10-05 → 10-06 | peonist `halogen-qwen3.8-27b` `qwen3.8-27b-p1w4d-d2.hgn` 35,865,565,184 B `274c3fc7…` | halogen 0.1.4 `sha256:dc0a39a0…` | 1 slot × 200,700, DFlash2 drafter, `HALOGEN_CACHE_MB=16384`, text-only, censored base | flash-server v2 restored vision, abliteration and MTP | `/home/dank/models/halogen-qwen3.8-27b/` + `scripts/download-frank-dragoon-halogen.sh`; 0.1.4 digest still in podman storage |
| 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | — | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` |
| 2026-10-02 → 10-03 | aligned `Q8_0_ROCMFPX_AGENT.gguf` 28,214,425,952 B `f3ef2bdb…` | same fork pin | 1 × 262,144 f16 KV, `draft-mtp` 4 (MTP +123 % decode) | the uncensored file was faster *and* uncensored | `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` + `download-frank-dragoon-kingjones.sh` |
| 2026-10-01 (one day) | MiMo-V2.6-Distill-Qwen-9B BF16 17.53 GiB + cross-model MTP from `Qwen3.5-9B` | vLLM 0.30 TheRock `kyuz0/vllm-therock-gfx1151` `sha256:98c1f3ac…` | 4 seqs × 262,144 on a scheduler-shared pool, 2 draft tokens | — | staged `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` until purged; fetch trio in `old/` |
| 2026-09-28 → 10-01, again 10-01 → 10-02 | Swift-1.5 Uncensored `UD-Q8_K_XL` 31,457,990,272 B `9076f99a…`, then `UD-Q4_K_XL`; z-lab DFlash2 Q4_K_M | gufo v0.1.1 (`b0f8673`), source build | 1 × 262,144, `--draft-policy adaptive --draft-tokens 7`, no KV knob, no Jinja | — | payloads **deleted 2026-10-03** → re-download `download-frank-dragoon-swift.sh` (+ `download-frank-dragoon.sh` for the drafter); tag `frank-dragoon-gufo-q4kxl-20261001`; `scripts/build-gufo.sh` |
| 2026-09-21 → 09-28 | unsloth `Qwen3.8-27B-GGUF:UD-Q8_K_XL` 31.46 GiB + DFlash2 Q4_K_M 1.14 GiB | unsloth llama.cpp `b11030` prebuilt | 1 × 262,144, q8_0 KV, Qwen-Sharp v22.4.0 template | gufo: +37 % prefill shallow, +80 % at 131k, 4.5× faster load | `/home/dank/GitPackages/unsloth-llama-b11030/` on disk; payload deleted 2026-10-03 → `download-frank-dragoon-unsloth.sh` |
| 2026-09-11 → 09-15, again 09-16 → 09-19 | `peonist-ai/halogen-qwen3.8-flash-next` w4b 115.55 GiB + in-place-versioned `overlay.hgn` | halogen-flash-server 0.5.8, rootless Quadlet | 2 × 200,192 over a 400,384 pool, `MAX_TOK 24576`, `HOST_RESERVE 20`, vision OFF | displaced by CIRU Orca; the shape starved co-tenants | re-tag 0.5.8 + first-start `HALOGEN_DOWNLOAD` re-fetch into `/home/dank/models/halogen`; the Quadlet left the repo 2026-10-05, so the unit must be rewritten |
| through 2026-08-24 | CIRU ActiveFPX PromptForge v3: GGUF 15.44 GiB + 3 `.pfs` + mmproj ≈ 36.7 GiB | CIRU ROCmFPX `build-promptforge` | 1 × 262,144, native MTP depth 4, ngram off | eemin ROCmI4 | `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` + `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`; scripts `git revert 4c43839 7aee1df 40d50fe` or checkout `132e147` |
| ≤ 2026-09 — 11 earlier eras: payloads, shapes and every measured number destroyed 2026-10-19 as obsolete | — | — | — | — | —
Port note: this slot settled on **:7770** on 2026-08-22 (52625 before it); the 2026-08-17 fleet redo
created it — earlier topologies in .
## Laws learned here
- **KV arithmetic:** ~28 KiB/position incl. block scratch plus ~115 MiB/slot of O(1) state; reservations
measured 5.5 GiB @200,704, 7.2 @262,144, 11.0 @401,408, 14.6 @524,288. Quote the `reserving the KV
pool` line, not the preflight formula.
- **One pool, not a partition:** the pool decides how many conversations stay resident; only
`HALOGEN_CTX` sets the window, and over-window is a 400 naming the limit, never a truncation. The
2026-10-18 doubling to 401,408 was bought as pool parity with [[frank-zealot]] so each slot can
hold a maximum-length request at once — it did not lengthen the window by one token.
- **Budget:** device budget = 121.2 GiB (unrounded `MemTotal` 121.2159) − pinned weights −
`HOST_RESERVE_GIB`; need = KV + arena + vision, arena = 3.80 + 8.40 × (`MAX_TOK`/16384) GiB — it
predicts every recorded point to 0.1 GiB. `HOST_RESERVE_GIB` is a term in that inequality, **not an
allocation**: it cannot return RAM, only take the pool away (ceiling 16 at the 524,288 shape, ≥18
clamped it). Free RAM by cutting the pool or `MAX_TOK`.
- **`HALOGEN_KV_POOL_FIT` prints its sizing block only when it clamped** — silence is not headroom and a
green `/health` is not the pool you asked for.
- **RAM the pool takes is RAM the 47.68 GiB n-gram table loses** (page cache, `Pin::None`). The real
failure is file-cache starvation, not RAM pressure — long prompts crawl with the disk busy and the
process uninterruptible, and the fix is a *smaller* pool.
- Pinned weights count as reclaimable file cache, so `free`, `MemAvailable` and Grafana overstate free
RAM by ~63 GiB; believe the engine's `… in all` line.
- **Pin gate:** the engine refuses to pin below checkpoint + 16 GiB of `MemAvailable` (pinned pages
cannot swap, and under the floor the n-gram table pages in from disk every long prompt), so a
RAM-short host crash-loops the unit every ~5 s per boot — 2026-10-06: 8 boots at `MemAvailable`
74.77 GiB against the 78.11 GiB needed, self-resolved in ~2 min when the dying 27B engine freed RAM.
The serve script waits `FRANK_DRAGOON_RAM_WAIT` (600 s) and refuses loudly if the RAM never clears.
- `HALOGEN_FLASH_PIN_TRUNK=0` is refused by the v2 loader (and cost several times decode on the w4b
build): disk-backing the weights is not a lever here.
- **MTP is always on:** `HALOGEN_SPEC_ADAPT=0` is mandatory — the adaptive default disables the head for
whole requests and **halves decode**. Depth 2 all-round, 3 for code.
- **Batch XOR speculation was engine-version-specific:** halogen 0.1.4 lost the drafter above one slot
(41 → 14 t/s); flash-server runs two slots *with* the head. On the ROCmFPX/llama.cpp line
`--spec-draft-n-max` defaults to **16** and must be pinned to 4 — the measured knee on the dense
family, with the draft head offloaded (`--spec-draft-ngl 999`) and ngram never layered under it.
- **Prompt cache is free here and was manual on the last engine:** flash-server ships it on and its
auto-sizing fits this box's ledger (no pin) — a 100k follow-up re-prefills in ~2 s instead of ~88 s
cold (env, 2026-10-17), and the 2026-10-18 A/B measured 81.5 s cold against 0.3 s warm at 101,987
prompt tokens. halogen 0.1.4's auto-sizing refused here and re-prefilled every 79–81k turn in
85–180 s until `HALOGEN_CACHE_MB=16384` was pinned. What the pool takes is what the table loses, so
re-measure the warm ratio after any pool change.
- **400-zone law:** the harness window MUST sit ~45k under the engine ceiling (`contextWindow:
155648` = 200,704 − 45,056) because the harness clamps `max_tokens` to `contextWindow − est − 4096`
its chars/4 `est` under-counts code/agent traffic by ~3 %, and compaction only fires at 85 %. The
3 % is measured, not theory: 2026-10-07 at the old 200,192 ceiling, est 152,938 vs server 157,388 —
over by 354, a 400. At the live shape compaction fires at 132,301 est ≈ 136k measured (the 180k
target of the 262,144 era is unreachable — the window itself is 155,648), and the worst-case request
is 155,648 − 4,096 + 3 % ≈ 156.2k ≤ 200,704: ~9.5× the error of margin, robust to a ~25 %
under-count. Compaction rides [[frank-zealot]] at ≈ threshold − keepRecent(20k) ≈ 112k input.
- **Admission reserves prompt + `max_tokens`:** an un-fittable request waits in arrival order, then
503s `engine_busy` for the whole queue after `QUEUE_TIMEOUT` — and the cap/timeout pair moves
together: 65536 tokens at ~10 t/s serial needs >6550 s, which is why 7200 is pinned (env, 2026-10-17).
- **Depth is the cost on this silicon:** ~8 µs per token of context per round (KV reads ~8 GB/s against
180–240 GB/s of weight streaming) — 30–50 t/s shallow, 3.3–3.8 at 79–81k, so deep-conversation roles
belong on the fp8-KV endpoint [[frank-archon]]. BF16 dense decode tops out at 13.1 t/s (≈94 % of the
roof): only speculative decode moves it.
- **The file picks the runtime, never the reverse:** v2 `.hgn` only in flash-server, the 27B `.hgn` only
in halogen, ROCmFPX types 101/102/103/105/108 in no stock loader; fork builds pin by SHA —
pre-`36e9acd` ROCmFPX loads the AGENT files then SIGSEGVs in `ggml_compute_forward_set_rows`.
- **Same filename ≠ same model:** native v2 is 66,687,678,432 B, the Swift1.5 r2 is 66,687,678,464 B.
Byte count + LFS sha256 is the only discriminator, and upstream versions sidecars in place, so equal
sizes prove nothing.
- **Companion law:** the n-gram table is REQUIRED beside the checkpoint; if the publisher ships only the
main file, hardlink the engine vendor's stock table/tower/tokenizer (a hardlink is a regular file and
passes the no-symlink gate) and referee with the engine's own tools — a mismatched table raises KL
*everywhere* and degrades retrieval, which is what NIAH + needle probes rule out.
- **Payload gates:** flat tokenizer directory, every entry a regular file (HF cache symlinks dangle
inside the mount), and the serve gate must mirror the deployer exactly or deploy goes green while the
unit crash-loops.
- Serve-script lore (this file has been bitten by all three): capture `podman images -q` into a
variable *before* grepping it — under `pipefail`, grep's early exit killed the pipeline and the
deploy refused with "no match for the pinned digest" on a box that had the image (2026-09-30); never
put a comment inside the `podman run` backslash chain — a mid-chain `#` terminates the continuation
and runc gets `--name run` (2026-10-05, exit-127 crash loop); pass only `--ulimit memlock`, since
Dankclaw's nofile hard limit is 524,288 and rootless runc cannot raise it (exit 126), and clear the
legacy `frank-dragoon` container name before starting.
- **Bench discipline (this endpoint wrote the fleet law):** stop the service for any GPU probe
(`census-frank-dragoon.sh verify` is the only GPU-free one); `gufo bench` cannot rank speculative
settings and inverts them; ≥128 generated tokens or the per-token figure is first-token noise; card
acceptance/t-s numbers are file properties, not promises — the live card's 53–57 t/s greedy decode
and ~1,640 t/s prefill at 34k measured 50.0 and 1,604 here (2026-10-17), and the 27B card's 31.7
t/s measured 27.6–28.1; health 200 does not prove your pid owns the port; there are no scratch ports
on this host — A/B through `scripts/trial-frank-dragoon.sh` drop-ins; never `pkill -f llama-server`
(frank-zealot is one), sweep by payload name.
- **Dead ends — do not re-try:** ngram under MTP or DFlash (slower at every depth; on recurrent
Gated-DeltaNet layers it also disables MTP's rollback path, −7…−15 %); DFlash across parallel slots
(shared draft ctx spuriously emits EOG); `-ot`/`-ncmoe` on a hybrid split (disables mmap, page-locks
~108 GiB and stranded the host until a power cycle); `--no-mmap` on any PLE-table file (silent cgroup
OOM-kill); forcing PLE to CPU (23 → 13.4 t/s); `HALOGEN_FA_TQ=256` (no decode win, −9.5 % prefill)
big ubatches as a decode knob (prefill-only, ub 512 fastest); Flash-Next MTP heads on the qwen4exp
loader (only `blk.48`, never the full per-layer norm set); mradermacher i1 GGUFs (NextN stripped)
vLLM quant paths on gfx1151; a custom Jinja template on gufo (no Jinja engine — send the `terse` text
as a client system message instead).
- Build lore if a fork is rebuilt: purge Ubuntu `librocwmma-dev` (shadows AMD's headers, `-I` cannot beat
it), pin `--gcc-install-dir` to GCC 15, `-DLLAMA_BUILD_WEBUI=OFF`; four HIP processes is the device
ceiling and ComfyUI is a co-tenant; `amdgpu: SVM mapping failed` spam is a symptom, not a cause.
## Contract (wire)
- Live: `/v1/models`, `/health`, `/metrics`, POST `/v1/chat/completions`; `404`: `/ping`, `/props`
`/completion`, `/tokenize`. **No `Server-Timing` header and no `/completion` timing object** — measure
wall-clock plus `usage.prompt_tokens`.
- Thinking lands in `reasoning_content` and **counts against `max_tokens`**: a budget that dies
mid-thought returns `finish_reason: "length"` with empty `content`, so clients read both fields; size
vision probes at ≥512 tokens or the model looks blind. Budget for the base reasoning long: 1,476
thinking tokens mean on the 2026-10-17 battery, against 784 for the retired Swift1.5 finetune (×1.9)
and 905 for the censored 27B.
- `reasoning_effort` accepts only `xhigh|medium|low` (anything else is a template error — the
`models.yml` fold stays) and there is **no `"none"`**: disable thinking with `enable_thinking:false`
which is also what the llama.cpp-era QwenSharp template wanted (`reasoning_effort:"none"` threw
there too). `preserve_reasoning` is a 400 on this engine; the llama.cpp-era key was
`chat_template_kwargs.preserve_thinking`, which decided whether `<think>` blocks stayed in recorded
assistant messages. Sampler defaults 1.0/0.95/20 are server-side, so an absent temperature is NOT
greedy (halogen 0.1.4 and w4b decoded greedily); unknown sampling fields are a 400, out-of-range
values are refused not clamped.
- **`response_format` works, but only greedy** (measured 2026-10-19 on this pin): `json_schema` and `json_object`
both answer schema-valid JSON at `temperature: 0`; at any sampled temperature both draw `400 "response_format
json_schema / json_object decodes greedy in this release: send temperature 0 (or omit it). Sampling under a schema
is not implemented yet"`. **Omitting `temperature` does not save you** — the sampler default is server-side 1.0, so
"absent" means sampled. Clients that pattern-match that string for "endpoint cannot do schemas" (esperanto's
`is_json_schema_unsupported_error` fires on the words *not implemented*) downgrade to `json_object`, draw the same
400, and fail outright; a structured-output consumer MUST send `temperature: 0`. Escape hatch that keeps sampling:
ask for JSON in the prompt and send no `response_format` — the model returns parseable JSON anyway, and the
tolerant client-side extractor handles a prose wrapper.
- Over-ceiling prompt → 400 naming the exact overage, rejected by preflight in <1 s, so probing the
ceiling is free; over-cap `max_tokens` → 400 naming the cap; `max_tokens`, `max_completion_tokens` and
`max_output_tokens` all work and must agree.
- Vision: OpenAI `image_url` parts only (top-level `images` is the frank-zealot dialect), ≥256 px or the
model answers "video", ~2,040 tokens per image, `http(s)` image URLs refused on purpose.
- The first request after a boot is a warmup artifact and may sit silent 45–60 s with 503
`engine_unresponsive` then self-recover — do not restart over it; `/health` 503 during load is loading.
Flags read once at startup, all bitwise except `HALOGEN_CK_OVERLAY`, `HALOGEN_MATMUL_TUNING_FILE`
`HALOGEN_INDEXER_BUDGET`, `HALOGEN_COMPOSABLE_CONTEXT`.
- Deploy waits for :7770 to be free: a rootless container binds it at CREATE and exits 126, and a
leftover Quadlet beside the root unit guarantees the race.
## The NPU lane (present in the image, not enabled)
XDNA2 executes only vendor-precompiled AIE partition programs — that is the wall the TTS attempt died at
([[Frank TTS]]) — and flash-server ships those programs. `HALOGEN_NPU_MODELS=<names>` starts a second engine **in the
same container behind the same port**, and a request picks one by its `model`; leave the variable out and the server
is exactly what runs today. The pinned 0.16.2 image already carries the five-program catalog
(`/opt/halogen/npu/models.txt`: `decider-0.8b`, `qwen3-embedding-0.6b`, `qwen3-reranker-0.6b`, `qwen3guard-gen-0.6b`,
`qwen3.5-2b`) and `docs/NPU.md` at tag `v0.16.2` documents it, so **`/v1/embeddings` is one env var and three mounts
away — no runtime bump needed.** Wider catalog: `qwen3-embedding-4b` and `decider-4b` from 0.17.4, `embeddinggemma-2`
(text ≤ 8,192 tokens, pictures and audio in one vector space) and `wemm-embedding-4b` from 0.18.3.
Embedding contract on that lane: `POST /v1/embeddings`, `model: "qwen3-embedding-0.6b"`, `input` a list of **texts**
(not token ids), 1024 unit-length floats back, the Qwen instruct prefix goes inside the string
(`Instruct: …\nQuery:…`), `base64` accepted, and LangChain's `OpenAIEmbeddings` needs
`check_embedding_ctx_length=False`.
Four prerequisites the trio does not satisfy today, from the vendor's own host list — and this box is one of the two
configurations they test (Ubuntu 26.04's XRT 2.21 `libxrt2`+`libxrt-npu2`, the kernel's `amdxdna`, `iommu=pt`):
1. `--device /dev/accel/accel0` plus `--group-add keep-groups` (the pattern ComfyUI uses for `/dev/kfd`; `dank` is
already in `render`).
2. The host's XRT reachable inside: there is **no `/opt/xilinx/xrt`** here — distro packages put the three files the
container needs (`libxrt_coreutil.so.2`, `libxrt_core.so.2`, `libxrt_driver_xdna.so.2`) in
`/usr/lib/x86_64-linux-gnu`, where all three exist.
3. **The GPU fabric clock held at its top level, or the engine refuses to start.** GPU and NPU work share the die:
while `pp_dpm_fclk` is changing speed, concurrent GPU work can hang the machine and corrupt NPU results. The
engine holds the clock itself only when it runs as root with `/sys` mounted; `frank-dragoon` is a **rootless**
`podman run` under `User=dank`, so the path is the vendor's root host unit (`deploy/host/halogen-fabric-clock`) —
which is exactly the `pp_dpm_fclk` level-7 pin deferred in [[dankclaw-iommu-translation-off]] with the words "it
goes in when an NPU endpoint actually exists". Held: about a watt. Unheld: the hang. Measured 2026-10-19 nothing
is held (fabric DPM lists 400–2000 MHz with no forced level, graphics clock sitting at 857 of 2900 MHz).
4. `HALOGEN_DOWNLOAD=peonist-ai/halogen-qwen3.8-flash-next` on a boot with a **writable** `/models/npu` — the payload
mount is `:ro` by law, so the programs land via a throwaway writable run or a dedicated read-write bind for that
one subdirectory; the vendor notes a read-only volume works but re-runs the conversion at every start.
Price: the programs and their weights sit in host RAM beside the 62.9 GiB of locked weights ("a few GB in all" for
the default set; ~1.1 GiB for `embeddinggemma-2`), and **the Flash model decodes slower while the NPU works** — same
memory bus, same power budget. Under this page's RAM law the question is pool versus n-gram table, so read the boot
ledger after the first NPU boot instead of modelling it. If the lane is opened, prefer **0.18.8 or newer**: that
release budgets NPU memory before asking for it, fixing kernel errors from several NPU models under a concurrent mix
**on the stock kernel driver** — the driver this box runs.
## Files
- `scripts/frank-dragoon.env` · `scripts/serve-frank-dragoon.sh` (gates, RAM pin-gate wait, foreground
`podman run`, ledger to `/home/dank/logs/frank_dragoon.log`) · `scripts/frank-dragoon.service`
(`User=dank`, `LimitMEMLOCK=infinity`, `Restart=always`, `OOMScoreAdjust=-500`, `ExecStop=podman stop
-t 60`) → `deploy_llm_dragoon` in `scripts/deploy-remote.sh`.
- Fetchers: `download-frank-dragoon-v2abl.sh` (live); era rollbacks `-flashnext`, `-halogen`
`-kingjones`, `-kingjones-uncensored`, `-swift`, `-unsloth`, `-q4`, `-orca`
`download-frank-dragoon.sh`; runtime pins `build-rocmfpx.sh`, `build-gufo.sh`.
- Verification: `tests/frank_dragoon_unit.bats` (payload gates, deployer/serve parity, spec block, KV
band), `tests/llm_units.bats`, `tests/llm_secrets.bats`, `scripts/census-frank-dragoon.sh`
`scripts/battery-frank-dragoon.py`, `scripts/stage-frank-dragoon-eval.sh`
`scripts/trial-frank-dragoon.sh`.
-Related: · · · ·
---