dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
52dbe18486ac0630aa0e51ebc8ee0f403a1eacf3
serverAppConfigs/serverAppConfigs-docs/LLM/frank-dragoon.md
T
Frank a5fe045b58 docs(vault): one fact, one owner — repair the doubled pages, single-source the facts 
Structural repair:
- 13 pages were the same document twice (each exactly 2x its HEAD size),
  a second frontmatter block whose closing fence had been glued onto a body
  line by a frontmatter rewrite that re-read its own output. Un-doubled by
  cutting at the second frontmatter and keeping the copy whose body is a
  superset; the orphan census reports 0 lost tokens against HEAD.
- log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first —
  66 dated headings merged into 34, titles demoted to bold leads, every
  content line verified present. The file had been descending at the head
  and ascending at the tail.

Ownership moves:
- New `LLM/frank-medium.md` is the locator for the retired Dankserver
  endpoint: era windows quoted from each era page's own banner, no numbers
  restated, and the two unresolved date conflicts recorded instead of
  silently resolved. `frank-archon.md` links to it instead of restating the
  pre-rename window.
- The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone;
  `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor
  verdicts and cross-split spec decode, and the four links that pointed at
  the stubs were repointed at it or at the drafter page.
- Authentik's three stub pages fold into `Authentik.md`, with the process
  detail and the outpost reference split out so the parent stays under the
  length hint.

Verified corrections (host census + live probe, 2026-10-19):
- The MXFP4 checkpoint two pages called the "on-disk rollback" is gone:
  /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on
  the era page that owns that payload; the swap page now links to it.
- frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2
  arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run
  census is kept once (155 requests) instead of twice (~120 and 155).
- Fleet shapes re-confirmed against the live endpoints before touching any
  page: 3x200000 fp8 KV, 2x200704, 2x262144.

Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
2026-10-19 05:00:43 -07:00

25 KiB
Raw Blame History

type, title, description, tags, timestamp
type title description tags timestamp
reference frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical) The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it.
llm
frank-dragoon
dankclaw
strix-halo
gfx1151
halogen
endpoint
2026-10-18

frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU)

Strix Halo (gfx1151) endpoint http://192.168.0.143:7770, served id frank-dragoon: a podman container under a root User=dank unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18 (2b12716). This page replaces every per-era dragoon runbook; the Eras table is the rollback record.

Current shape

scripts/frank-dragoon.env is authoritative and scripts/serve-frank-dragoon.sh passes exactly those flags and nothing else. Payload /home/dank/models/halogen-v2-abliterated/, verified by scripts/download-frank-dragoon-v2abl.sh --verify-only:

File Bytes sha256 Source
qwen38-flash-next-v2-abliterated.hgn 66,687,678,432 b5bd925f… Quat3rnion @main — the NATIVE 2026-10-11 v2 conversion of orcarouter's BF16 Qwen3.8-Flash-Next, abliterated
qwen38-flash-next-ngram.hgn 51,200,246,144 3450acada… peonist @5cc17cea, REQUIRED, hardlinked
qwen38-flash-next-vision.hgn 897,916,416 d62e0ae5… peonist @e053f488, hardlinked — vision ON
tokenizer/ (6 flat files) ~22 MB tokenizer.json LFS peonist @e053f488, hardlinked

Image ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a (0.16.2, closed source), container frank-dragoon-halogen, published 7770:8731; the engine's unauthenticated token port 8730 stays on container loopback. Engine stdout → /home/dank/logs/frank_dragoon.log, not the journal.

  • Shape: HALOGEN_CTX=200704 hard per-request ceiling · HALOGEN_KV_SLOTS=2 · HALOGEN_KV_POOL_POSITIONS=401408 = slots × ctx, the top of the band tests/frank_dragoon_unit.bats enforces (ctx ≤ pool ≤ slots × ctx) · KV ~28 KiB/position → 11.0 GiB reserved · no KV-dtype knob exists here, the pool is the only KV lever · native window 262,144, HALOGEN_ROPE_YARN beyond it.
  • Knobs: HALOGEN_MAX_TOK=8192 sizes the prefill arena (5.7 GiB) and is only the default for requests omitting max_tokens · HALOGEN_MAX_TOKENS_CAP=65536 + HALOGEN_QUEUE_TIMEOUT=7200 are a coupled pair · MTP depth 2 with the head INSIDE the checkpoint (HALOGEN_MTP_HEAD stays unset) and HALOGEN_SPEC_ADAPT=0 · sampler 1.0 / 0.95 / 20 server-side · HALOGEN_WEIGHTS_LOCK=1 · HALOGEN_HOST_RESERVE_GIB=20 · prompt cache ON by image default (PROMPT_CACHE=2, no flag passed): 12 entries, which the boot ledger prices at 111 MiB each at this window = 1.3 GiB (the env header's "~85 MiB here" estimate did not survive that ledger line).
  • Deployed boot ledger: 62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB of KV pool, 5.7 GiB of working memory, 79.6 GiB in all; n-gram mapped 51.2 GB, NOT pinned (Pin::None); no clamp line. Host MemAvailable read 22 GiB right after the 2026-10-18 raise — ~20 GiB above the 20 GiB reserve, against 10–15 GiB of baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observability).
  • Harness (~/.omp/agent/models.yml, provider llamacpp (dankclaw)): contextWindow: 155648 (= ceiling − 45k per the 400-zone law, never higher), maxTokens: 45056, input: [text, image], reasoningContentField: "reasoning_content", high→xhigh fold, extraBody 1.0/0.95. Compaction rides frank-zealot.
  • Deploy ./scripts/deploy.sh dankclaw --service frank-dragoon; gate = boot-ledger greps (weights, pool, MTP, prompt cache, no clamp) + /health 200 + a tool call + the 512 px red-PNG probe. Kernel: CONFIG_HSA_AMD_SVM=y (CachyOS ships it) and amdgpu.noretry=0 — the env says check it at deploy; boot line per dankclaw-iommu-translation-off.

Eras

Rollback = revert that era's swap commit (or repoint the payload vars) and redeploy dankclaw --service frank-dragoon. Newest first; payloads staged under ~/.AGENTTRASH/ are not a rollback plan — recoverable only until the trash is purged.

Window Payload Runtime Shape Replaced because Rollback
2026-10-06 → 10-17 Swift1.5 abliterated v2 .hgn r2 66,687,678,464 B 9c880b8b… + same trio flash-server 0.16.2, same digest 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on 2026-10-17 A/B at that shape: NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls + red-PNG probe identical, needle hit at both depths of a 21.5k haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s, PPL 3.968 vs 3.877 (finetune's edge gone: +2.3 %), KL vs the swift15 logits mean 0.155 / median 0.057 with top-1 agreement 85.1 % — two sound models disagreeing on content, not a corrupt forward pass, thinking 1,476 vs 784 mean, 7.4 GiB RAM back /home/dank/models/halogen-flashnext-v2/ + scripts/download-frank-dragoon-flashnext.sh
2026-10-05 → 10-06 peonist halogen-qwen3.8-27b qwen3.8-27b-p1w4d-d2.hgn 35,865,565,184 B 274c3fc7… halogen 0.1.4 sha256:dc0a39a0… 1 slot × 200,700, DFlash2 drafter, HALOGEN_CACHE_MB=16384, text-only, censored base flash-server v2 restored vision, abliteration and MTP /home/dank/models/halogen-qwen3.8-27b/ + scripts/download-frank-dragoon-halogen.sh; 0.1.4 digest still in podman storage
2026-10-03 → 10-05 kingjones777 Uncensored Q6_0_ROCMFPX_AGENT.gguf 24,856,607,008 B e966deb3… + mtp-Q4_0 051a1764… + BF16 mmproj kingjones30/ROCmFPX @ 36e9acd (scripts/build-rocmfpx.sh) 1 × 262,144 f16 KV, draft-mtp n-max 4, vision on halogen card claimed 31.7 t/s vs 27.6–28.1 measured /home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/ + /home/dank/GitPackages/kingjones-ROCmFPX/build + download-frank-dragoon-kingjones-uncensored.sh
2026-10-02 → 10-03 aligned Q8_0_ROCMFPX_AGENT.gguf 28,214,425,952 B f3ef2bdb… same fork pin 1 × 262,144 f16 KV, draft-mtp 4 (MTP +123 % decode) the uncensored file was faster and uncensored /home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/ + download-frank-dragoon-kingjones.sh
2026-10-01 (one day) MiMo-V2.6-Distill-Qwen-9B BF16 17.53 GiB + cross-model MTP from Qwen3.5-9B vLLM 0.30 TheRock kyuz0/vllm-therock-gfx1151 sha256:98c1f3ac… 4 seqs × 262,144 on a scheduler-shared pool, 2 draft tokens 6.19 t/s with MTP / 9.01 without vs gufo ~15, and no ≥6-bit vLLM quant path boots on gfx1151 staged /home/dank/.AGENTTRASH/20261002-retired-llm-payloads/ until purged; fetch trio in old/
2026-09-28 → 10-01, again 10-01 → 10-02 Swift-1.5 Uncensored UD-Q8_K_XL 31,457,990,272 B 9076f99a…, then UD-Q4_K_XL; z-lab DFlash2 Q4_K_M gufo v0.1.1 (b0f8673), source build 1 × 262,144, --draft-policy adaptive --draft-tokens 7, no KV knob, no Jinja ROCmFPX gave 8.39 bpw + MTP at 17 t/s payloads deleted 2026-10-03 → re-download download-frank-dragoon-swift.sh (+ download-frank-dragoon.sh for the drafter); tag frank-dragoon-gufo-q4kxl-20261001; scripts/build-gufo.sh
2026-09-21 → 09-28 unsloth Qwen3.8-27B-GGUF:UD-Q8_K_XL 31.46 GiB + DFlash2 Q4_K_M 1.14 GiB unsloth llama.cpp b11030 prebuilt 1 × 262,144, q8_0 KV, Qwen-Sharp v22.4.0 template gufo: +37 % prefill shallow, +80 % at 131k, 4.5× faster load /home/dank/GitPackages/unsloth-llama-b11030/ on disk; payload deleted 2026-10-03 → download-frank-dragoon-unsloth.sh
2026-09-21 (same day) unsloth Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS 87.3 GiB in 3 split parts same prebuilt 1 × 200,000, MTP depth 2 via explicit -md shared Q8_0 head swapped to the 27B dense hours later payload + prebuilt on disk at retirement; the parts ARE the payload (concatenating → invalid split file name)
2026-09-21 (same day) jcbtc/Qwen3.8-27B-IU4-Kairic-Edge + 3 .pfs = 26.05 GiB ciru-ai/ROCmFPX kairic-edge-qwen38-27b-v1.2 @ 205a3e5 1 × 262,144, native MTP 4 + n-gram same-day swap to the 27B dense payload + runtime intact on disk at retirement
2026-09-19 → 09-21 jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca CIRU v4.4.1 (QSA window hotfix) never written down — that runbook is a frontmatter stub the stock llama.cpp prebuilt won scripts/download-frank-dragoon-orca.sh
2026-09-11 → 09-15, again 09-16 → 09-19 peonist-ai/halogen-qwen3.8-flash-next w4b 115.55 GiB + in-place-versioned overlay.hgn halogen-flash-server 0.5.8, rootless Quadlet 2 × 200,192 over a 400,384 pool, MAX_TOK 24576, HOST_RESERVE 20, vision OFF displaced by CIRU Orca; the shape starved co-tenants re-tag 0.5.8 + first-start HALOGEN_DOWNLOAD re-fetch into /home/dank/models/halogen; the Quadlet left the repo 2026-10-05, so the unit must be rewritten
2026-09-15 → 09-16 (one day) mradermacher/Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 111 GiB + f16 mmproj ik_llama.cpp d85a1d7, 3090 + CPU hybrid 1 × 262,144, -ot exps=CPU, GGML_CUDA_NO_PINNED_WEIGHTS=1 3.8–4.0 t/s: ~100 CPU↔GPU graph splits per token, GPU at 5 % SM /home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M, /home/dank/GitPackages/ik_llama.cpp/build, trio scripts/.AGENTTRASH/frank-dragoon.{env,service}.ik-llama.cpp + serve-frank-dragoon.sh.ik-llama.cpp
2026-08-30 → 09-11 kingjones STRIX Q4_0_ROCMFP4-STRIX.gguf 113.47 GiB (51.2 B PLE via mmap) ROCmFPX @ 36e9acd 2 × 200,704 over one 401,408 f16 window, spec off, f16 mmproj halogen-flash-server ≈ 4× prefill payload dir + /home/dank/GitPackages/kingjones-ROCmFPX/build; trio .AGENTTRASH/frank-dragoon.{env,service}.llama.cpp + serve-frank-dragoon.sh.llama.cpp
2026-08-25 → 08-29 (restored 08-29 → 08-30 after the Q8_0 AGENT file crashed before it ever served) kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP: Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf 23.15 GiB (private ftype 114) + a SEPARATE MTP head mtp-Qwen3.8-27B-Q4_0.gguf 1.56 GiB via --model-draft + mmproj-Qwen3.8-27B-Q8_0.gguf 0.59 GiB CIRU ROCmFPX llama.cpp (build-promptforge) 1 slot at -c 196608, -b 8192 -ub 8192 -ctxcp 32, -fa on --no-mmap, f16 K+V, draft-mtp n-max 4 with the head on ROCm1 at f16 KV, QwenSharp template (enable_thinking, preserve_thinking) behind the peg-native + deepseek chat/reasoning pairing the Pi agent expected the Q8_0 AGENT experiment, then the STRIX Q4_0 tier /home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/ (six card files + projector), runtime /home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server
2026-08-29 → 08-30 (one day) agentionai FAST-imatrix v2/ joined-PLE file 87.06 GiB charlie12345/ROCmFPX build-qwen38 + scripts/patches/qwen4exp-qsa-f32-mask.patch 2 × 200,704 / 401,408 f16 KV, no MTP head loads kingjones STRIX on its own pinned fork /home/dank/GitPackages/ROCmFPX/build-qwen38 + the agentionai payload, both on disk
2026-08-18 → 08-25 caf-eemin Q4_0_ROCMI4.gguf 13.50 GiB ad51ecbe… with embedded NextN/MTP charlie12345/ROCmFPX, ROCmI4 W4A4 one window, MTP n4 / p0.60 the 27B STRIX tier took the slot re-fetch into /home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP/
through 2026-08-24 CIRU ActiveFPX PromptForge v3: GGUF 15.44 GiB + 3 .pfs + mmproj ≈ 36.7 GiB CIRU ROCmFPX build-promptforge 1 × 262,144, native MTP depth 4, ngram off eemin ROCmI4 /home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/ + /home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server; scripts git revert 4c43839 7aee1df 40d50fe or checkout 132e147
2026-08-22 (same day) jcbtc/Qwen3.8-27B-IU4-Kairic-Edge 15.48 GiB + 3 .pfs ciru-ai/ROCmFPX tag kairic-edge-qwen38-27b-v1 @ e97b324 one window, IU4 W4A4 lane, ubatch 512 prefill 1.5–2.3× faster but prose decode 15–26 % slower → back to v3 the same day /home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/ + /home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server
2026-08-21 (same day) unsloth UD-Q8_K_XL 31.4 GiB + z-lab DFlash2 Q4_K_M 2.0 GiB upstream llama.cpp PR #27342, GGML_VULKAN=ON Vulkan0 — HIP has no TOP_K, so DFlash could not run there slower than the CIRU runtime it replaced both files in /home/dank/models/; the PR build must be rebuilt

Port note: this slot settled on :7770 on 2026-08-22 (52625 before it); the 2026-08-17 fleet redo created it — earlier topologies in frank-redo-2026-08-17.

Laws learned here

  • KV arithmetic: ~28 KiB/position incl. block scratch plus ~115 MiB/slot of O(1) state; reservations measured 5.5 GiB @200,704, 7.2 @262,144, 11.0 @401,408, 14.6 @524,288. Quote the reserving the KV pool line, not the preflight formula.
  • One pool, not a partition: the pool decides how many conversations stay resident; only HALOGEN_CTX sets the window, and over-window is a 400 naming the limit, never a truncation. The 2026-10-18 doubling to 401,408 was bought as pool parity with frank-zealot so each slot can hold a maximum-length request at once — it did not lengthen the window by one token.
  • Budget: device budget = 121.2 GiB (unrounded MemTotal 121.2159) − pinned weights − HOST_RESERVE_GIB; need = KV + arena + vision, arena = 3.80 + 8.40 × (MAX_TOK/16384) GiB — it predicts every recorded point to 0.1 GiB. HOST_RESERVE_GIB is a term in that inequality, not an allocation: it cannot return RAM, only take the pool away (ceiling 16 at the 524,288 shape, ≥18 clamped it). Free RAM by cutting the pool or MAX_TOK.
  • HALOGEN_KV_POOL_FIT prints its sizing block only when it clamped — silence is not headroom and a green /health is not the pool you asked for.
  • RAM the pool takes is RAM the 47.68 GiB n-gram table loses (page cache, Pin::None). The real failure is file-cache starvation, not RAM pressure — long prompts crawl with the disk busy and the process uninterruptible, and the fix is a smaller pool.
  • Pinned weights count as reclaimable file cache, so free, MemAvailable and Grafana overstate free RAM by ~63 GiB; believe the engine's … in all line.
  • Pin gate: the engine refuses to pin below checkpoint + 16 GiB of MemAvailable (pinned pages cannot swap, and under the floor the n-gram table pages in from disk every long prompt), so a RAM-short host crash-loops the unit every ~5 s per boot — 2026-10-06: 8 boots at MemAvailable 74.77 GiB against the 78.11 GiB needed, self-resolved in ~2 min when the dying 27B engine freed RAM. The serve script waits FRANK_DRAGOON_RAM_WAIT (600 s) and refuses loudly if the RAM never clears.
  • HALOGEN_FLASH_PIN_TRUNK=0 is refused by the v2 loader (and cost several times decode on the w4b build): disk-backing the weights is not a lever here.
  • MTP is always on: HALOGEN_SPEC_ADAPT=0 is mandatory — the adaptive default disables the head for whole requests and halves decode. Depth 2 all-round, 3 for code.
  • Batch XOR speculation was engine-version-specific: halogen 0.1.4 lost the drafter above one slot (41 → 14 t/s); flash-server runs two slots with the head. On the ROCmFPX/llama.cpp line --spec-draft-n-max defaults to 16 and must be pinned to 4 — the measured knee on the dense family, with the draft head offloaded (--spec-draft-ngl 999) and ngram never layered under it.
  • Prompt cache is free here and was manual on the last engine: flash-server ships it on and its auto-sizing fits this box's ledger (no pin) — a 100k follow-up re-prefills in ~2 s instead of ~88 s cold (env, 2026-10-17), and the 2026-10-18 A/B measured 81.5 s cold against 0.3 s warm at 101,987 prompt tokens. halogen 0.1.4's auto-sizing refused here and re-prefilled every 79–81k turn in 85–180 s until HALOGEN_CACHE_MB=16384 was pinned. What the pool takes is what the table loses, so re-measure the warm ratio after any pool change.
  • 400-zone law: the harness window MUST sit ~45k under the engine ceiling (contextWindow: 155648 = 200,704 − 45,056) because the harness clamps max_tokens to contextWindow − est − 4096, its chars/4 est under-counts code/agent traffic by ~3 %, and compaction only fires at 85 %. The 3 % is measured, not theory: 2026-10-07 at the old 200,192 ceiling, est 152,938 vs server 157,388 — over by 354, a 400. At the live shape compaction fires at 132,301 est ≈ 136k measured (the 180k target of the 262,144 era is unreachable — the window itself is 155,648), and the worst-case request is 155,648 − 4,096 + 3 % ≈ 156.2k ≤ 200,704: ~9.5× the error of margin, robust to a ~25 % under-count. Compaction rides frank-zealot at ≈ threshold − keepRecent(20k) ≈ 112k input.
  • Admission reserves prompt + max_tokens: an un-fittable request waits in arrival order, then 503s engine_busy for the whole queue after QUEUE_TIMEOUT — and the cap/timeout pair moves together: 65536 tokens at ~10 t/s serial needs >6550 s, which is why 7200 is pinned (env, 2026-10-17).
  • Depth is the cost on this silicon: ~8 µs per token of context per round (KV reads ~8 GB/s against 180–240 GB/s of weight streaming) — 30–50 t/s shallow, 3.3–3.8 at 79–81k, so deep-conversation roles belong on the fp8-KV endpoint frank-archon. BF16 dense decode tops out at 13.1 t/s (≈94 % of the roof): only speculative decode moves it.
  • The file picks the runtime, never the reverse: v2 .hgn only in flash-server, the 27B .hgn only in halogen, ROCmFPX types 101/102/103/105/108 in no stock loader; fork builds pin by SHA — pre-36e9acd ROCmFPX loads the AGENT files then SIGSEGVs in ggml_compute_forward_set_rows.
  • Same filename ≠ same model: native v2 is 66,687,678,432 B, the Swift1.5 r2 is 66,687,678,464 B. Byte count + LFS sha256 is the only discriminator, and upstream versions sidecars in place, so equal sizes prove nothing.
  • Companion law: the n-gram table is REQUIRED beside the checkpoint; if the publisher ships only the main file, hardlink the engine vendor's stock table/tower/tokenizer (a hardlink is a regular file and passes the no-symlink gate) and referee with the engine's own tools — a mismatched table raises KL everywhere and degrades retrieval, which is what NIAH + needle probes rule out.
  • Payload gates: flat tokenizer directory, every entry a regular file (HF cache symlinks dangle inside the mount), and the serve gate must mirror the deployer exactly or deploy goes green while the unit crash-loops.
  • Serve-script lore (this file has been bitten by all three): capture podman images -q into a variable before grepping it — under pipefail, grep's early exit killed the pipeline and the deploy refused with "no match for the pinned digest" on a box that had the image (2026-09-30); never put a comment inside the podman run backslash chain — a mid-chain # terminates the continuation and runc gets --name run (2026-10-05, exit-127 crash loop); pass only --ulimit memlock, since Dankclaw's nofile hard limit is 524,288 and rootless runc cannot raise it (exit 126), and clear the legacy frank-dragoon container name before starting.
  • Bench discipline (this endpoint wrote the fleet law): stop the service for any GPU probe (census-frank-dragoon.sh verify is the only GPU-free one); gufo bench cannot rank speculative settings and inverts them; ≥128 generated tokens or the per-token figure is first-token noise; card acceptance/t-s numbers are file properties, not promises — the live card's 53–57 t/s greedy decode and ~1,640 t/s prefill at 34k measured 50.0 and 1,604 here (2026-10-17), and the 27B card's 31.7 t/s measured 27.6–28.1; health 200 does not prove your pid owns the port; there are no scratch ports on this host — A/B through scripts/trial-frank-dragoon.sh drop-ins; never pkill -f llama-server (frank-zealot is one), sweep by payload name.
  • Dead ends — do not re-try: ngram under MTP or DFlash (slower at every depth; on recurrent Gated-DeltaNet layers it also disables MTP's rollback path, −7…−15 %); DFlash across parallel slots (shared draft ctx spuriously emits EOG); -ot/-ncmoe on a hybrid split (disables mmap, page-locks ~108 GiB and stranded the host until a power cycle); --no-mmap on any PLE-table file (silent cgroup OOM-kill); forcing PLE to CPU (23 → 13.4 t/s); HALOGEN_FA_TQ=256 (no decode win, −9.5 % prefill); big ubatches as a decode knob (prefill-only, ub 512 fastest); Flash-Next MTP heads on the qwen4exp loader (only blk.48, never the full per-layer norm set); mradermacher i1 GGUFs (NextN stripped); vLLM quant paths on gfx1151; a custom Jinja template on gufo (no Jinja engine — send the terse text as a client system message instead).
  • Build lore if a fork is rebuilt: purge Ubuntu librocwmma-dev (shadows AMD's headers, -I cannot beat it), pin --gcc-install-dir to GCC 15, -DLLAMA_BUILD_WEBUI=OFF; four HIP processes is the device ceiling and ComfyUI is a co-tenant; amdgpu: SVM mapping failed spam is a symptom, not a cause.

Contract (wire)

  • Live: /v1/models, /health, /metrics, POST /v1/chat/completions; 404: /ping, /props, /completion, /tokenize. No Server-Timing header and no /completion timing object — measure wall-clock plus usage.prompt_tokens.
  • Thinking lands in reasoning_content and counts against max_tokens: a budget that dies mid-thought returns finish_reason: "length" with empty content, so clients read both fields; size vision probes at ≥512 tokens or the model looks blind. Budget for the base reasoning long: 1,476 thinking tokens mean on the 2026-10-17 battery, against 784 for the retired Swift1.5 finetune (×1.9) and 905 for the censored 27B.
  • reasoning_effort accepts only xhigh|medium|low (anything else is a template error — the models.yml fold stays) and there is no "none": disable thinking with enable_thinking:false, which is also what the llama.cpp-era QwenSharp template wanted (reasoning_effort:"none" threw there too). preserve_reasoning is a 400 on this engine; the llama.cpp-era key was chat_template_kwargs.preserve_thinking, which decided whether <think> blocks stayed in recorded assistant messages. Sampler defaults 1.0/0.95/20 are server-side, so an absent temperature is NOT greedy (halogen 0.1.4 and w4b decoded greedily); unknown sampling fields are a 400, out-of-range values are refused not clamped, and response_format is unsupported.
  • Over-ceiling prompt → 400 naming the exact overage, rejected by preflight in <1 s, so probing the ceiling is free; over-cap max_tokens → 400 naming the cap; max_tokens, max_completion_tokens and max_output_tokens all work and must agree.
  • Vision: OpenAI image_url parts only (top-level images is the frank-zealot dialect), ≥256 px or the model answers "video", ~2,040 tokens per image, http(s) image URLs refused on purpose.
  • The first request after a boot is a warmup artifact and may sit silent 45–60 s with 503 engine_unresponsive then self-recover — do not restart over it; /health 503 during load is loading. Flags read once at startup, all bitwise except HALOGEN_CK_OVERLAY, HALOGEN_MATMUL_TUNING_FILE, HALOGEN_INDEXER_BUDGET, HALOGEN_COMPOSABLE_CONTEXT.
  • Deploy waits for :7770 to be free: a rootless container binds it at CREATE and exits 126, and a leftover Quadlet beside the root unit guarantees the race.

Files

  • scripts/frank-dragoon.env · scripts/serve-frank-dragoon.sh (gates, RAM pin-gate wait, foreground podman run, ledger to /home/dank/logs/frank_dragoon.log) · scripts/frank-dragoon.service (User=dank, LimitMEMLOCK=infinity, Restart=always, OOMScoreAdjust=-500, ExecStop=podman stop -t 60) → deploy_llm_dragoon in scripts/deploy-remote.sh.
  • Fetchers: download-frank-dragoon-v2abl.sh (live); era rollbacks -flashnext, -halogen, -kingjones, -kingjones-uncensored, -swift, -unsloth, -q4, -orca, download-frank-dragoon.sh; runtime pins build-rocmfpx.sh, build-gufo.sh.
  • Verification: tests/frank_dragoon_unit.bats (payload gates, deployer/serve parity, spec block, KV band), tests/llm_units.bats, tests/llm_secrets.bats, scripts/census-frank-dragoon.sh, scripts/battery-frank-dragoon.py, scripts/stage-frank-dragoon-eval.sh, scripts/trial-frank-dragoon.sh.
  • Related: deployment · Model Swap and Rollback · frank-zealot · frank-archon · dankclaw-iommu-translation-off · frank-redo-2026-08-17 · Dankclaw System Specs · DFlash DSpark Speculative Decoding · qwen3.8-spec-matrix · qwen3.8-strix-optimizations.
Reference in New Issue View Git Blame Copy Permalink