Frank
846f81ddb8
log.md drops 466 lines of per-swap ledger prose (1,169 lines out, 703 back) and keeps the causal stories: why the host went down, why a knob resets at boot, why a stale drop-in served a different model for 24 minutes, why the RAM was the fault. Date sections are unchanged (65 before, 65 after) -- this is compression inside sections, not deletion of days. Every number the compression would have orphaned was ported to its owner first, and a census of the deleted lines against the whole vault is what found them: - frank-zealot: the two boots that priced non-KV (20,190/KV 12,800 => 7,390 without the projector; 22,972/KV 14,113 => 8,859 with it, so the vision tower costs ~1.4 GiB for a 918 MB file), the 1,153 MiB-free 3-slot boot that made the seam price non-KV at 9 GiB, and the f16 wall's own figures (fit 33,198 vs 23,558 free on a 25,088.00 MiB KV request). - frank-archon: the pool-cut-is-headroom-not-arena law (2.75 GiB/rank bought zero slots; d(reserve) = d(arena) = +248 slots; 14.87 GiB / 6,496 slots against a 398,658-token pool), the MNBT compile-RANGE vs SEQS+1 law, and the 2 x 196,919 prefix-cache proof (393,837 tokens resident above the pool's 386,904; 94.1/102.3/117.6 t/s, acceptance 0.7284/0.7525). - frank-dragoon: the KL arm of the 2026-10-17 A/B (mean 0.155 / median 0.057, top-1 agreement 85.1 %) joins the PPL and NIAH figures already on the era row. - Payload and Disk Reclaim: the 121 GB layer census lesson -- a huge image layer can be the engine image, not a payload. Also fixes a wikilink the rewrite had hard-wrapped across two lines, which the link checker reads as a target that does not exist. New seam: 'one page per live LLM endpoint'. The live set is derived from scripts/ (an endpoint has both an env and a unit), and any <endpoint>-*.md page in the vault fails it -- the enforcement of the one-page-per-endpoint law, so a re-created era page cannot pass tests.
237 lines
25 KiB
Markdown
237 lines
25 KiB
Markdown
---
|
||
type: reference
|
||
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
|
||
description: "The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env, every retired era with its rollback path, the durable laws this endpoint paid for, the wire contract, and the files that own it."
|
||
tags:
|
||
- llm
|
||
- frank-dragoon
|
||
- dankclaw
|
||
- strix-halo
|
||
- gfx1151
|
||
- halogen
|
||
timestamp: 2026-10-18
|
||
---
|
||
|
||
# frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU)
|
||
|
||
Strix Halo (gfx1151) endpoint `http://192.168.0.143:7770`, served id `frank-dragoon`: a podman container
|
||
under a root `User=dank` unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18
|
||
(`2b12716`). This page replaces every per-era dragoon runbook; the Eras table is the rollback record.
|
||
|
||
## Current shape
|
||
|
||
`scripts/frank-dragoon.env` is authoritative and `scripts/serve-frank-dragoon.sh` passes exactly those
|
||
flags and nothing else. Payload `/home/dank/models/halogen-v2-abliterated/`, verified by
|
||
`scripts/download-frank-dragoon-v2abl.sh --verify-only`:
|
||
|
||
| File | Bytes | sha256 | Source |
|
||
|---|---|---|---|
|
||
| `qwen38-flash-next-v2-abliterated.hgn` | 66,687,678,432 | `b5bd925f…` | Quat3rnion `@main` — the NATIVE 2026-10-11 v2 conversion of orcarouter's BF16 Qwen3.8-Flash-Next, abliterated |
|
||
| `qwen38-flash-next-ngram.hgn` | 51,200,246,144 | `3450acada…` | peonist `@5cc17cea`, **REQUIRED**, hardlinked |
|
||
| `qwen38-flash-next-vision.hgn` | 897,916,416 | `d62e0ae5…` | peonist `@e053f488`, hardlinked — vision ON |
|
||
| `tokenizer/` (6 flat files) | ~22 MB | `tokenizer.json` LFS | peonist `@e053f488`, hardlinked |
|
||
|
||
Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a`
|
||
(0.16.2, closed source), container `frank-dragoon-halogen`, published `7770:8731`; the engine's
|
||
unauthenticated token port 8730 stays on container loopback. Engine stdout →
|
||
`/home/dank/logs/frank_dragoon.log`, **not** the journal.
|
||
|
||
- Shape: `HALOGEN_CTX=200704` hard per-request ceiling · `HALOGEN_KV_SLOTS=2` ·
|
||
`HALOGEN_KV_POOL_POSITIONS=401408` = slots × ctx, the top of the band `tests/frank_dragoon_unit.bats`
|
||
enforces (`ctx ≤ pool ≤ slots × ctx`) · KV ~28 KiB/position → **11.0 GiB reserved** · no KV-dtype knob
|
||
exists here, the pool is the only KV lever · native window 262,144, `HALOGEN_ROPE_YARN` beyond it.
|
||
- Knobs: `HALOGEN_MAX_TOK=8192` sizes the prefill arena (5.7 GiB) and is only the default for requests
|
||
omitting `max_tokens` · `HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled
|
||
pair · MTP depth 2 with the head INSIDE the checkpoint (**`HALOGEN_MTP_HEAD` stays unset**) and
|
||
`HALOGEN_SPEC_ADAPT=0` · sampler 1.0 / 0.95 / 20 server-side · `HALOGEN_WEIGHTS_LOCK=1` ·
|
||
`HALOGEN_HOST_RESERVE_GIB=20` · prompt cache ON by image default (`PROMPT_CACHE=2`, no flag
|
||
passed): 12 entries, which the boot ledger prices at 111 MiB each at this window = 1.3 GiB
|
||
(the env header's "~85 MiB here" estimate did not survive that ledger line).
|
||
- Deployed boot ledger: `62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB
|
||
of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`; n-gram `mapped 51.2 GB, NOT pinned
|
||
(Pin::None)`; no clamp line. Host `MemAvailable` read 22 GiB right after the 2026-10-18 raise —
|
||
~20 GiB above the 20 GiB reserve, against 10–15 GiB of baseline co-tenancy (frank-zealot, ComfyUI,
|
||
Caddy/observability).
|
||
- Harness (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw)`): `contextWindow: 155648`
|
||
(= ceiling − 45k per the 400-zone law, never higher), `maxTokens: 45056`, `input: [text, image]`,
|
||
`reasoningContentField: "reasoning_content"`, `high→xhigh` fold, `extraBody` 1.0/0.95. Compaction
|
||
rides [[frank-zealot]].
|
||
- Deploy `./scripts/deploy.sh dankclaw --service frank-dragoon`; gate = boot-ledger greps (weights,
|
||
pool, MTP, prompt cache, **no clamp**) + `/health` 200 + a tool call + the 512 px red-PNG probe.
|
||
Kernel: `CONFIG_HSA_AMD_SVM=y` (CachyOS ships it) and `amdgpu.noretry=0` — the env says check it at
|
||
deploy; boot line per [[dankclaw-iommu-translation-off]].
|
||
|
||
## Eras
|
||
|
||
Rollback = revert that era's swap commit (or repoint the payload vars) and redeploy
|
||
`dankclaw --service frank-dragoon`. Newest first; payloads staged under `~/.AGENTTRASH/` are **not** a
|
||
rollback plan — recoverable only until the trash is purged.
|
||
|
||
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|
||
|---|---|---|---|---|---|
|
||
| 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | 2026-10-17 A/B at that shape: NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls + red-PNG probe identical, needle hit at both depths of a 21.5k haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s, PPL 3.968 vs 3.877 (finetune's edge gone: +2.3 %), KL vs the swift15 logits mean 0.155 / median 0.057 with top-1 agreement 85.1 % — two sound models disagreeing on content, not a corrupt forward pass, thinking 1,476 vs 784 mean, 7.4 GiB RAM back | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` |
|
||
| 2026-10-05 → 10-06 | peonist `halogen-qwen3.8-27b` `qwen3.8-27b-p1w4d-d2.hgn` 35,865,565,184 B `274c3fc7…` | halogen 0.1.4 `sha256:dc0a39a0…` | 1 slot × 200,700, DFlash2 drafter, `HALOGEN_CACHE_MB=16384`, text-only, censored base | flash-server v2 restored vision, abliteration and MTP | `/home/dank/models/halogen-qwen3.8-27b/` + `scripts/download-frank-dragoon-halogen.sh`; 0.1.4 digest still in podman storage |
|
||
| 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | halogen card claimed 31.7 t/s vs 27.6–28.1 measured | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` |
|
||
| 2026-10-02 → 10-03 | aligned `Q8_0_ROCMFPX_AGENT.gguf` 28,214,425,952 B `f3ef2bdb…` | same fork pin | 1 × 262,144 f16 KV, `draft-mtp` 4 (MTP +123 % decode) | the uncensored file was faster *and* uncensored | `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` + `download-frank-dragoon-kingjones.sh` |
|
||
| 2026-10-01 (one day) | MiMo-V2.6-Distill-Qwen-9B BF16 17.53 GiB + cross-model MTP from `Qwen3.5-9B` | vLLM 0.30 TheRock `kyuz0/vllm-therock-gfx1151` `sha256:98c1f3ac…` | 4 seqs × 262,144 on a scheduler-shared pool, 2 draft tokens | 6.19 t/s with MTP / 9.01 without vs gufo ~15, and no ≥6-bit vLLM quant path boots on gfx1151 | staged `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` until purged; fetch trio in `old/` |
|
||
| 2026-09-28 → 10-01, again 10-01 → 10-02 | Swift-1.5 Uncensored `UD-Q8_K_XL` 31,457,990,272 B `9076f99a…`, then `UD-Q4_K_XL`; z-lab DFlash2 Q4_K_M | gufo v0.1.1 (`b0f8673`), source build | 1 × 262,144, `--draft-policy adaptive --draft-tokens 7`, no KV knob, no Jinja | ROCmFPX gave 8.39 bpw + MTP at 17 t/s | payloads **deleted 2026-10-03** → re-download `download-frank-dragoon-swift.sh` (+ `download-frank-dragoon.sh` for the drafter); tag `frank-dragoon-gufo-q4kxl-20261001`; `scripts/build-gufo.sh` |
|
||
| 2026-09-21 → 09-28 | unsloth `Qwen3.8-27B-GGUF:UD-Q8_K_XL` 31.46 GiB + DFlash2 Q4_K_M 1.14 GiB | unsloth llama.cpp `b11030` prebuilt | 1 × 262,144, q8_0 KV, Qwen-Sharp v22.4.0 template | gufo: +37 % prefill shallow, +80 % at 131k, 4.5× faster load | `/home/dank/GitPackages/unsloth-llama-b11030/` on disk; payload deleted 2026-10-03 → `download-frank-dragoon-unsloth.sh` |
|
||
| 2026-09-21 (same day) | unsloth `Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS` 87.3 GiB in 3 split parts | same prebuilt | 1 × 200,000, MTP depth 2 via explicit `-md` shared Q8_0 head | swapped to the 27B dense hours later | payload + prebuilt on disk at retirement; the parts ARE the payload (concatenating → `invalid split file name`) |
|
||
| 2026-09-21 (same day) | `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` + 3 `.pfs` = 26.05 GiB | ciru-ai/ROCmFPX `kairic-edge-qwen38-27b-v1.2` @ `205a3e5` | 1 × 262,144, native MTP 4 + n-gram | same-day swap to the 27B dense | payload + runtime intact on disk at retirement |
|
||
| 2026-09-19 → 09-21 | `jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca` | CIRU v4.4.1 (QSA window hotfix) | never written down — that runbook is a frontmatter stub | the stock llama.cpp prebuilt won | `scripts/download-frank-dragoon-orca.sh` |
|
||
| 2026-09-11 → 09-15, again 09-16 → 09-19 | `peonist-ai/halogen-qwen3.8-flash-next` w4b 115.55 GiB + in-place-versioned `overlay.hgn` | halogen-flash-server 0.5.8, rootless Quadlet | 2 × 200,192 over a 400,384 pool, `MAX_TOK 24576`, `HOST_RESERVE 20`, vision OFF | displaced by CIRU Orca; the shape starved co-tenants | re-tag 0.5.8 + first-start `HALOGEN_DOWNLOAD` re-fetch into `/home/dank/models/halogen`; the Quadlet left the repo 2026-10-05, so the unit must be rewritten |
|
||
| 2026-09-15 → 09-16 (one day) | `mradermacher/Qwen3.8-Flash-Next-Uncensored` i1-Q4_K_M 111 GiB + f16 mmproj | ik_llama.cpp `d85a1d7`, 3090 + CPU hybrid | 1 × 262,144, `-ot exps=CPU`, `GGML_CUDA_NO_PINNED_WEIGHTS=1` | 3.8–4.0 t/s: ~100 CPU↔GPU graph splits per token, GPU at 5 % SM | `/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M`, `/home/dank/GitPackages/ik_llama.cpp/build`, trio `scripts/.AGENTTRASH/frank-dragoon.{env,service}.ik-llama.cpp` + `serve-frank-dragoon.sh.ik-llama.cpp` |
|
||
| 2026-08-30 → 09-11 | kingjones STRIX `Q4_0_ROCMFP4-STRIX.gguf` 113.47 GiB (51.2 B PLE via mmap) | ROCmFPX @ `36e9acd` | 2 × 200,704 over one 401,408 f16 window, spec off, f16 mmproj | halogen-flash-server ≈ 4× prefill | payload dir + `/home/dank/GitPackages/kingjones-ROCmFPX/build`; trio `.AGENTTRASH/frank-dragoon.{env,service}.llama.cpp` + `serve-frank-dragoon.sh.llama.cpp` |
|
||
| 2026-08-25 → 08-29 (restored 08-29 → 08-30 after the Q8_0 AGENT file crashed before it ever served) | `kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP`: `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` 23.15 GiB (private ftype 114) + a **SEPARATE** MTP head `mtp-Qwen3.8-27B-Q4_0.gguf` 1.56 GiB via `--model-draft` + `mmproj-Qwen3.8-27B-Q8_0.gguf` 0.59 GiB | CIRU ROCmFPX llama.cpp (`build-promptforge`) | 1 slot at `-c 196608`, `-b 8192 -ub 8192 -ctxcp 32`, `-fa on --no-mmap`, f16 K+V, `draft-mtp` n-max 4 with the head on ROCm1 at f16 KV, QwenSharp template (`enable_thinking`, `preserve_thinking`) behind the `peg-native` + `deepseek` chat/reasoning pairing the Pi agent expected | the Q8_0 AGENT experiment, then the STRIX Q4_0 tier | `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/` (six card files + projector), runtime `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` |
|
||
| 2026-08-29 → 08-30 (one day) | agentionai FAST-imatrix **`v2/` joined-PLE** file 87.06 GiB | charlie12345/ROCmFPX `build-qwen38` + `scripts/patches/qwen4exp-qsa-f32-mask.patch` | 2 × 200,704 / 401,408 f16 KV, no MTP head loads | kingjones STRIX on its own pinned fork | `/home/dank/GitPackages/ROCmFPX/build-qwen38` + the agentionai payload, both on disk |
|
||
| 2026-08-18 → 08-25 | caf-eemin `Q4_0_ROCMI4.gguf` 13.50 GiB `ad51ecbe…` with embedded NextN/MTP | charlie12345/ROCmFPX, ROCmI4 W4A4 | one window, MTP `n4 / p0.60` | the 27B STRIX tier took the slot | re-fetch into `/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP/` |
|
||
| through 2026-08-24 | CIRU ActiveFPX PromptForge v3: GGUF 15.44 GiB + 3 `.pfs` + mmproj ≈ 36.7 GiB | CIRU ROCmFPX `build-promptforge` | 1 × 262,144, native MTP depth 4, ngram off | eemin ROCmI4 | `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` + `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`; scripts `git revert 4c43839 7aee1df 40d50fe` or checkout `132e147` |
|
||
| 2026-08-22 (same day) | `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` 15.48 GiB + 3 `.pfs` | ciru-ai/ROCmFPX tag `kairic-edge-qwen38-27b-v1` @ `e97b324` | one window, IU4 W4A4 lane, ubatch 512 | prefill 1.5–2.3× faster but prose decode 15–26 % slower → back to v3 the same day | `/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/` + `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server` |
|
||
| 2026-08-21 (same day) | unsloth `UD-Q8_K_XL` 31.4 GiB + z-lab DFlash2 Q4_K_M 2.0 GiB | upstream llama.cpp PR #27342, `GGML_VULKAN=ON` | Vulkan0 — HIP has no `TOP_K`, so DFlash could not run there | slower than the CIRU runtime it replaced | both files in `/home/dank/models/`; the PR build must be rebuilt |
|
||
|
||
Port note: this slot settled on **:7770** on 2026-08-22 (52625 before it); the 2026-08-17 fleet redo
|
||
created it — earlier topologies in [[frank-redo-2026-08-17]].
|
||
|
||
## Laws learned here
|
||
|
||
- **KV arithmetic:** ~28 KiB/position incl. block scratch plus ~115 MiB/slot of O(1) state; reservations
|
||
measured 5.5 GiB @200,704, 7.2 @262,144, 11.0 @401,408, 14.6 @524,288. Quote the `reserving the KV
|
||
pool` line, not the preflight formula.
|
||
- **One pool, not a partition:** the pool decides how many conversations stay resident; only
|
||
`HALOGEN_CTX` sets the window, and over-window is a 400 naming the limit, never a truncation. The
|
||
2026-10-18 doubling to 401,408 was bought as pool parity with [[frank-zealot]] so each slot can
|
||
hold a maximum-length request at once — it did not lengthen the window by one token.
|
||
- **Budget:** device budget = 121.2 GiB (unrounded `MemTotal` 121.2159) − pinned weights −
|
||
`HOST_RESERVE_GIB`; need = KV + arena + vision, arena = 3.80 + 8.40 × (`MAX_TOK`/16384) GiB — it
|
||
predicts every recorded point to 0.1 GiB. `HOST_RESERVE_GIB` is a term in that inequality, **not an
|
||
allocation**: it cannot return RAM, only take the pool away (ceiling 16 at the 524,288 shape, ≥18
|
||
clamped it). Free RAM by cutting the pool or `MAX_TOK`.
|
||
- **`HALOGEN_KV_POOL_FIT` prints its sizing block only when it clamped** — silence is not headroom and a
|
||
green `/health` is not the pool you asked for.
|
||
- **RAM the pool takes is RAM the 47.68 GiB n-gram table loses** (page cache, `Pin::None`). The real
|
||
failure is file-cache starvation, not RAM pressure — long prompts crawl with the disk busy and the
|
||
process uninterruptible, and the fix is a *smaller* pool.
|
||
- Pinned weights count as reclaimable file cache, so `free`, `MemAvailable` and Grafana overstate free
|
||
RAM by ~63 GiB; believe the engine's `… in all` line.
|
||
- **Pin gate:** the engine refuses to pin below checkpoint + 16 GiB of `MemAvailable` (pinned pages
|
||
cannot swap, and under the floor the n-gram table pages in from disk every long prompt), so a
|
||
RAM-short host crash-loops the unit every ~5 s per boot — 2026-10-06: 8 boots at `MemAvailable`
|
||
74.77 GiB against the 78.11 GiB needed, self-resolved in ~2 min when the dying 27B engine freed RAM.
|
||
The serve script waits `FRANK_DRAGOON_RAM_WAIT` (600 s) and refuses loudly if the RAM never clears.
|
||
- `HALOGEN_FLASH_PIN_TRUNK=0` is refused by the v2 loader (and cost several times decode on the w4b
|
||
build): disk-backing the weights is not a lever here.
|
||
- **MTP is always on:** `HALOGEN_SPEC_ADAPT=0` is mandatory — the adaptive default disables the head for
|
||
whole requests and **halves decode**. Depth 2 all-round, 3 for code.
|
||
- **Batch XOR speculation was engine-version-specific:** halogen 0.1.4 lost the drafter above one slot
|
||
(41 → 14 t/s); flash-server runs two slots *with* the head. On the ROCmFPX/llama.cpp line
|
||
`--spec-draft-n-max` defaults to **16** and must be pinned to 4 — the measured knee on the dense
|
||
family, with the draft head offloaded (`--spec-draft-ngl 999`) and ngram never layered under it.
|
||
- **Prompt cache is free here and was manual on the last engine:** flash-server ships it on and its
|
||
auto-sizing fits this box's ledger (no pin) — a 100k follow-up re-prefills in ~2 s instead of ~88 s
|
||
cold (env, 2026-10-17), and the 2026-10-18 A/B measured 81.5 s cold against 0.3 s warm at 101,987
|
||
prompt tokens. halogen 0.1.4's auto-sizing refused here and re-prefilled every 79–81k turn in
|
||
85–180 s until `HALOGEN_CACHE_MB=16384` was pinned. What the pool takes is what the table loses, so
|
||
re-measure the warm ratio after any pool change.
|
||
- **400-zone law:** the harness window MUST sit ~45k under the engine ceiling (`contextWindow:
|
||
155648` = 200,704 − 45,056) because the harness clamps `max_tokens` to `contextWindow − est − 4096`,
|
||
its chars/4 `est` under-counts code/agent traffic by ~3 %, and compaction only fires at 85 %. The
|
||
3 % is measured, not theory: 2026-10-07 at the old 200,192 ceiling, est 152,938 vs server 157,388 —
|
||
over by 354, a 400. At the live shape compaction fires at 132,301 est ≈ 136k measured (the 180k
|
||
target of the 262,144 era is unreachable — the window itself is 155,648), and the worst-case request
|
||
is 155,648 − 4,096 + 3 % ≈ 156.2k ≤ 200,704: ~9.5× the error of margin, robust to a ~25 %
|
||
under-count. Compaction rides [[frank-zealot]] at ≈ threshold − keepRecent(20k) ≈ 112k input.
|
||
- **Admission reserves prompt + `max_tokens`:** an un-fittable request waits in arrival order, then
|
||
503s `engine_busy` for the whole queue after `QUEUE_TIMEOUT` — and the cap/timeout pair moves
|
||
together: 65536 tokens at ~10 t/s serial needs >6550 s, which is why 7200 is pinned (env, 2026-10-17).
|
||
- **Depth is the cost on this silicon:** ~8 µs per token of context per round (KV reads ~8 GB/s against
|
||
180–240 GB/s of weight streaming) — 30–50 t/s shallow, 3.3–3.8 at 79–81k, so deep-conversation roles
|
||
belong on the fp8-KV endpoint [[frank-archon]]. BF16 dense decode tops out at 13.1 t/s (≈94 % of the
|
||
roof): only speculative decode moves it.
|
||
- **The file picks the runtime, never the reverse:** v2 `.hgn` only in flash-server, the 27B `.hgn` only
|
||
in halogen, ROCmFPX types 101/102/103/105/108 in no stock loader; fork builds pin by SHA —
|
||
pre-`36e9acd` ROCmFPX loads the AGENT files then SIGSEGVs in `ggml_compute_forward_set_rows`.
|
||
- **Same filename ≠ same model:** native v2 is 66,687,678,432 B, the Swift1.5 r2 is 66,687,678,464 B.
|
||
Byte count + LFS sha256 is the only discriminator, and upstream versions sidecars in place, so equal
|
||
sizes prove nothing.
|
||
- **Companion law:** the n-gram table is REQUIRED beside the checkpoint; if the publisher ships only the
|
||
main file, hardlink the engine vendor's stock table/tower/tokenizer (a hardlink is a regular file and
|
||
passes the no-symlink gate) and referee with the engine's own tools — a mismatched table raises KL
|
||
*everywhere* and degrades retrieval, which is what NIAH + needle probes rule out.
|
||
- **Payload gates:** flat tokenizer directory, every entry a regular file (HF cache symlinks dangle
|
||
inside the mount), and the serve gate must mirror the deployer exactly or deploy goes green while the
|
||
unit crash-loops.
|
||
- Serve-script lore (this file has been bitten by all three): capture `podman images -q` into a
|
||
variable *before* grepping it — under `pipefail`, grep's early exit killed the pipeline and the
|
||
deploy refused with "no match for the pinned digest" on a box that had the image (2026-09-30); never
|
||
put a comment inside the `podman run` backslash chain — a mid-chain `#` terminates the continuation
|
||
and runc gets `--name run` (2026-10-05, exit-127 crash loop); pass only `--ulimit memlock`, since
|
||
Dankclaw's nofile hard limit is 524,288 and rootless runc cannot raise it (exit 126), and clear the
|
||
legacy `frank-dragoon` container name before starting.
|
||
- **Bench discipline (this endpoint wrote the fleet law):** stop the service for any GPU probe
|
||
(`census-frank-dragoon.sh verify` is the only GPU-free one); `gufo bench` cannot rank speculative
|
||
settings and inverts them; ≥128 generated tokens or the per-token figure is first-token noise; card
|
||
acceptance/t-s numbers are file properties, not promises — the live card's 53–57 t/s greedy decode
|
||
and ~1,640 t/s prefill at 34k measured 50.0 and 1,604 here (2026-10-17), and the 27B card's 31.7
|
||
t/s measured 27.6–28.1; health 200 does not prove your pid owns the port; there are no scratch ports
|
||
on this host — A/B through `scripts/trial-frank-dragoon.sh` drop-ins; never `pkill -f llama-server`
|
||
(frank-zealot is one), sweep by payload name.
|
||
- **Dead ends — do not re-try:** ngram under MTP or DFlash (slower at every depth; on recurrent
|
||
Gated-DeltaNet layers it also disables MTP's rollback path, −7…−15 %); DFlash across parallel slots
|
||
(shared draft ctx spuriously emits EOG); `-ot`/`-ncmoe` on a hybrid split (disables mmap, page-locks
|
||
~108 GiB and stranded the host until a power cycle); `--no-mmap` on any PLE-table file (silent cgroup
|
||
OOM-kill); forcing PLE to CPU (23 → 13.4 t/s); `HALOGEN_FA_TQ=256` (no decode win, −9.5 % prefill);
|
||
big ubatches as a decode knob (prefill-only, ub 512 fastest); Flash-Next MTP heads on the qwen4exp
|
||
loader (only `blk.48`, never the full per-layer norm set); mradermacher i1 GGUFs (NextN stripped);
|
||
vLLM quant paths on gfx1151; a custom Jinja template on gufo (no Jinja engine — send the `terse` text
|
||
as a client system message instead).
|
||
- Build lore if a fork is rebuilt: purge Ubuntu `librocwmma-dev` (shadows AMD's headers, `-I` cannot beat
|
||
it), pin `--gcc-install-dir` to GCC 15, `-DLLAMA_BUILD_WEBUI=OFF`; four HIP processes is the device
|
||
ceiling and ComfyUI is a co-tenant; `amdgpu: SVM mapping failed` spam is a symptom, not a cause.
|
||
|
||
## Contract (wire)
|
||
|
||
- Live: `/v1/models`, `/health`, `/metrics`, POST `/v1/chat/completions`; `404`: `/ping`, `/props`,
|
||
`/completion`, `/tokenize`. **No `Server-Timing` header and no `/completion` timing object** — measure
|
||
wall-clock plus `usage.prompt_tokens`.
|
||
- Thinking lands in `reasoning_content` and **counts against `max_tokens`**: a budget that dies
|
||
mid-thought returns `finish_reason: "length"` with empty `content`, so clients read both fields; size
|
||
vision probes at ≥512 tokens or the model looks blind. Budget for the base reasoning long: 1,476
|
||
thinking tokens mean on the 2026-10-17 battery, against 784 for the retired Swift1.5 finetune (×1.9)
|
||
and 905 for the censored 27B.
|
||
- `reasoning_effort` accepts only `xhigh|medium|low` (anything else is a template error — the
|
||
`models.yml` fold stays) and there is **no `"none"`**: disable thinking with `enable_thinking:false`,
|
||
which is also what the llama.cpp-era QwenSharp template wanted (`reasoning_effort:"none"` threw
|
||
there too). `preserve_reasoning` is a 400 on this engine; the llama.cpp-era key was
|
||
`chat_template_kwargs.preserve_thinking`, which decided whether `<think>` blocks stayed in recorded
|
||
assistant messages. Sampler defaults 1.0/0.95/20 are server-side, so an absent temperature is NOT
|
||
greedy (halogen 0.1.4 and w4b decoded greedily); unknown sampling fields are a 400, out-of-range
|
||
values are refused not clamped, and `response_format` is unsupported.
|
||
- Over-ceiling prompt → 400 naming the exact overage, rejected by preflight in <1 s, so probing the
|
||
ceiling is free; over-cap `max_tokens` → 400 naming the cap; `max_tokens`, `max_completion_tokens` and
|
||
`max_output_tokens` all work and must agree.
|
||
- Vision: OpenAI `image_url` parts only (top-level `images` is the frank-zealot dialect), ≥256 px or the
|
||
model answers "video", ~2,040 tokens per image, `http(s)` image URLs refused on purpose.
|
||
- The first request after a boot is a warmup artifact and may sit silent 45–60 s with 503
|
||
`engine_unresponsive` then self-recover — do not restart over it; `/health` 503 during load is loading.
|
||
Flags read once at startup, all bitwise except `HALOGEN_CK_OVERLAY`, `HALOGEN_MATMUL_TUNING_FILE`,
|
||
`HALOGEN_INDEXER_BUDGET`, `HALOGEN_COMPOSABLE_CONTEXT`.
|
||
- Deploy waits for :7770 to be free: a rootless container binds it at CREATE and exits 126, and a
|
||
leftover Quadlet beside the root unit guarantees the race.
|
||
|
||
## Files
|
||
|
||
- `scripts/frank-dragoon.env` · `scripts/serve-frank-dragoon.sh` (gates, RAM pin-gate wait, foreground
|
||
`podman run`, ledger to `/home/dank/logs/frank_dragoon.log`) · `scripts/frank-dragoon.service`
|
||
(`User=dank`, `LimitMEMLOCK=infinity`, `Restart=always`, `OOMScoreAdjust=-500`, `ExecStop=podman stop
|
||
-t 60`) → `deploy_llm_dragoon` in `scripts/deploy-remote.sh`.
|
||
- Fetchers: `download-frank-dragoon-v2abl.sh` (live); era rollbacks `-flashnext`, `-halogen`,
|
||
`-kingjones`, `-kingjones-uncensored`, `-swift`, `-unsloth`, `-q4`, `-orca`,
|
||
`download-frank-dragoon.sh`; runtime pins `build-rocmfpx.sh`, `build-gufo.sh`.
|
||
- Verification: `tests/frank_dragoon_unit.bats` (payload gates, deployer/serve parity, spec block, KV
|
||
band), `tests/llm_units.bats`, `tests/llm_secrets.bats`, `scripts/census-frank-dragoon.sh`,
|
||
`scripts/battery-frank-dragoon.py`, `scripts/stage-frank-dragoon-eval.sh`,
|
||
`scripts/trial-frank-dragoon.sh`.
|
||
- Related: [[deployment]] · [[Model Swap and Rollback]] · [[frank-zealot]] · [[frank-archon]] ·
|
||
[[dankclaw-iommu-translation-off]] · [[frank-redo-2026-08-17]] · [[Dankclaw System Specs]] ·
|
||
[[DFlash DSpark Speculative Decoding]] · [[qwen3.8-spec-matrix]] · [[qwen3.8-strix-optimizations]].
|