Compare commits

...
Author SHA1 Message Date
Frank 39b8396794 governance: bench is removed from the model-swap procedure
Model changes verify by driving the live service — boot ledger,
per-request timings, one real request per harness role. No bench window
is ever scheduled for a swap. The stop-and-mask policy stays for
deliberate standalone bench windows (it is host protection, not a swap
step). Codified in AGENTS.md (policy lead + summary row), Model Swap and
Rollback (No bench step law, §8 close-out source, Related rescope).
2026-10-15 03:07:46 -07:00
Frank a9d45e7012 frank-archon: the slab is a threshold, and decode is where slots cost
Measured, 512 tokens, greedy, cold, at the deployed 3 x 200,000 fp8 shape:
prefill 22,208 -> 3,397, 24,000 -> 3,290, 25,600 -> 3,290 t/s at 43.5k —
the third slab step bought nothing, so the slab is a threshold whose cliff is
the 2026-10-11 arm at 26,071 (4,379), not a slope. Decode ladder: 99.0 t/s
single-stream at 43.5k, 93.3 at 197k (nearly window-free), and 3 concurrent
streams 113.9/43.3/10.9 = 18.4 t/s aggregate — the slot count is paid in
decode concurrency, the axis the shape hunt never measured because probe leg
B generates 24 tokens. probe-frank-archon-kv.py gains leg D (N-stream decode,
512 tokens, per-stream + aggregate) so no resize is priced on prefill alone.
2026-10-13 20:51:32 -07:00
Frank 2581b8d4b6 docs(frank-archon): the slab is a slope, and the ceiling was mine not the engine's
Measured A/B on the shipped pair 25600/5120: A-cold 43.5k prefill 3,290 ->
3,584 t/s (+8.9 %) and the 3-way 197k aggregate 3,051 -> 3,589 t/s (+17.6 %,
wall 193.7 -> 164.5 s). The prediction I shipped with the arm -- slab is a
threshold below ~26 GiB, so 1,600 MiB buys ~0 -- is falsified by the measurement
it asked for. Two slab points inside noise of EACH OTHER (22,208 / 24,000) are
not a plateau. The cliff is above 25,600: 26,071 measured 4,379 t/s.

Ceiling re-derived from the engine's own budget table (card 32,624 MiB, pair at
29,120 settling 2,058 MiB free against its 209.19 MiB floor => 30,969 claimable).
The 29,120 bound was my 1 GiB \`keep free\` reserve; the seam now derives the
number, so the next resize starts from the engine's arithmetic.

Ledger for the record: slab 16,235 -> 17,548 slots while the pinned host pool
hands the same experts back (8,893 -> 7,580) -- the transfer is the mechanism,
the pinned pool is the slow path. Capacity contract live-verified at 3 slots:
3 x 199,984-token prompts accepted (599,952 tokens), fourth refused as too
large to ever fit.

Probe law learned: the oversized-prompt refusal names the prompt's token count
BEFORE prefill (free calibration) and fires on prompt tokens ALONE, not on
prompt + max_tokens; the window is ~514,286 chars at 2.5714 chars/token.
2026-10-13 20:33:02 -07:00
Frank 09a0251d2d llm(frank-archon): claim the card's settling slack, slab 24000 -> 25600
The 2 GiB/card that \`rocm-smi\` shows free is not idle: the engine's budget
table calls it driver context + the 896 MiB activation arena + 96 MiB headroom,
and says the rest is `throughput on the floor`. Its own settle line is the
number that matters — `card 0 settled at 2.01 GiB free against 209.19 MiB (the
96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)` — so
1,849 MiB above the ENGINE's floor is claimable and the real ceiling is 30,969
MiB, not the 29,120 my 1 GiB reserve invented on 2026-10-11.

Shipped pair 25600/5120 = 30,720: +1,600 MiB of slab, ~458 MiB above the floor.
The slab is already saturated at 24,000 (mover took all of it, 451 KiB slack),
so this is the only direction left — KV is at its 3-slot floor (5,063 needed of
5,120).

Named prediction, because today's two points say the slab is a THRESHOLD and not
a slope (22,208 -> 3,397 t/s, 24,000 -> 3,290 t/s at 43.5k single-stream): this
arm buys ~1,600 resident expert slots and ~0 prefill. The point that moves
prefill is 26,071 (2026-10-11: 4,379 t/s), fundable only at two slots. Prefill
moving > +-3 % here falsifies the threshold reading.

Seam: the ceiling gate now derives from the engine's budget table (30,969) and
is the single owner of the pair arithmetic; the duplicate I put in the shape
test is removed.
2026-10-13 20:18:41 -07:00
Frank 066813ea95 llm(frank-archon): 3 x 200,000, pool cut to the contract, slab +1,792 MiB
Frank's call: three slots, and spend the KV the third slot doesn't need on
resident experts. `--max-num-seqs 3`, `--vram-kv-mib 6912 -> 5120`,
`--vram-weights-mib 22208 -> 24000` — the pair is still exactly the measured
29,120 MiB/card ceiling. The pool funds 608,027 tokens against the 600,000-token
contract (3 x 200,000, +1.3 %), and the slab gains 8.1 %.

The direction is the 2026-10-11 A/B: prefill is slab-limited, KV beyond the
contract buys nothing. At fp8 the slab curve measured on this host is
22,208 -> 3,397 t/s and 26,071 -> 4,379 t/s at 43.5k, i.e. ~0.25 t/s per MiB, so
24,000 predicts ~3,850 t/s single-stream. Falsifiable; the probe is the check.

The cost is named, not hidden: a FOURTH full-height conversation no longer fits
(4 x 200,000 needs 6,601 MiB of pool) and is refused as too large to ever fit —
radiance never truncates. Reverting to four slots is one env pair.

Seam: two new arithmetic gates, because both failures are boot-then-400 shape
faults that no flag check catches — the pair must respect the per-card ceiling,
and the pool must back seqs x max-model-len at the dtype's measured rate plus
the dtype-independent terms (110 MiB a sequence of recurrent state, 448 MiB of
prefix checkpoints).
2026-10-13 20:00:39 -07:00
Frank 7daa886dd6 docs(frank-archon): work-log entry and the 4-way capacity probe
Probe on the live service: four concurrent 196,960-token chats, 4/4 exact
marker retrieval, 787,842 tokens in 238.4 s (3,305 t/s aggregate prefill = the
single-stream ceiling, so the four streams shared one pipeline and the KV pool
never ran short); prefix-cache repeat answered in 1.0 s with cache_n=43008.
Recorded with the caveat that 8 generated tokens do not make a decode rate.
2026-10-13 19:30:46 -07:00
Frank bae78e7dae docs(frank-archon): the ttm reset outage, and the 4 x 200,000 fp8 shape
Runbook gains the 2026-10-13 section (symptom signature, the runtime-knob law,
the shape arithmetic, the boot ledger that confirms it: 843,276 tokens =
4 x 200,000, 12 all-reduce rows, 5.87 GiB of KV lent to the slab). AGENTS.md
gains the fleet-row update, the KV dtype law at four slots, the sysfs-raise
lesson under Key Lessons, and the pinned-RAM paragraph now points at
scripts/ttm-preflight.sh as the repair. LLM index entry + topology node match.
2026-10-13 19:28:25 -07:00
Frank a6f55f4cc6 fix(frank-archon): raise the ttm knob from a root ExecStartPre — User=dank cannot write it
a9da65e put the raise inside the serve script, which runs as `User=dank`: the
knob is root-owned 0644, so `-w` is false and the write silently failed — the
gate printed its refusal on a cap it had just failed to move. The privileged
path is systemd's `+` prefix: `ExecStartPre=+scripts/ttm-preflight.sh --raise`
runs as root even under `User=`, before ExecStart, so `Restart=always` can
never outrun it. The lib is now also executable (`--raise` sources the env file
and verifies), and keeps a `sudo -n` fallback for a human running the gate from
a shell.

Seam: the refusal cases run with a PATH that has `cat`+`bash` and NO `sudo`
(root writes a 444 file, so a passwordless dev box would have tested nothing),
and both now assert the gate's own exit 1, not merely non-zero. New seam covers
the unit line and the `--raise` entry point end to end (writable knob -> raised,
no args -> usage exit 2).
2026-10-13 19:20:33 -07:00
Frank a9da6589a6 fix(frank-archon): :" guard needs a space — :"${VAR:?}"` runs :204800
The TTM_TARGET_MIB guard I added on 1a6a78e was written `:"...` (no space), so
bash parsed the expansion as one word and executed `cmd :204800` as the
command `:204800`: exit 127 `command not found` on every start, before the
ttm gate ran. Caught on the deploy: the unit crash-looped 34 times in ~4 min
with exactly that journal line.

probe-frank-archon-kv.py now reads the slot count, window and KV dtype from the
deployed env on the host, so probe B exercises the capacity contract that is
actually live (4 x 200,000 today) instead of a hardcoded pair, and each probe's
marker starts with a unique word because show() scores the hit on that word.
2026-10-13 19:16:48 -07:00
Frank 1a6a78ef73 fix(frank-archon): raise the ttm pages cap in the serve path; reshape to 4 x 200,000 fp8 KV
The 2026-10-12 23:17 power failure reset /sys/module/ttm/parameters/pages_limit
from the manual 2026-10-05 raise (200 GiB) back to the kernel default (half of
RAM = 125,988 MiB), below the pinned-RAM ledger floor of 150,000 MiB. The serve
script's preflight named the refusal and exited 1; Restart=always re-ran it
6,919 times over ~20 hours and frank-archon served nobody.

scripts/ttm-preflight.sh (sourced by the serve script) now RAISES the knob
itself before refusing: the unit is root, the knob is a 0644 root module
parameter, and the cap is a runtime knob whose default is half of RAM — so a
manual sysfs write is not a repair procedure, it is a countdown to the next
reboot. FRANK_ARCHON_TTM_TARGET_MIB=204800 records the host ceiling; the gate
re-reads the cap after writing and only then refuses, so a knob the kernel
will not move is still a named failure.

Shape change (Frank's call): FOUR decode slots x 200,000 tokens with fp8
(quantized) KV. The pool must back seqs x max-model-len, so fp8 is what funds
the fourth slot: at bf16 the four slots cost 11,679 MiB of pool and would cut
the resident expert slab to 17,441 MiB (-33 %); at fp8 they cost 6,601 MiB, so
the pool stays 6,912 MiB (predicts 843,566 tokens), the VRAM pair stays at the
measured 29,120 MiB/card ceiling, the weights slab and the 44 GiB host pool are
untouched, and the fused TP all-reduce rides 4 x (1+2) = 12 rows, inside the
25-row bound where the wht6 wire is bit-exact.
2026-10-13 19:09:58 -07:00
Frank c52ba781e3 docs(frank-archon): record the bf16 KV shape, its measured cost, and the probe
Runbook section `2026-10-11 — unquantized KV at 2 x 200,000`: the dtype law
(only bf16|fp8, no K/V split; the QSA block key is dtype-fixed, which is why
bf16 costs 1.89x and not 2x), the pool formula that reproduces both live
boots within 0.1%, the seqs-3-to-2 provenance (the tcclaviger fork's
torch.compile constraint, not radiance's), the boot ledger, and the A/B.

The A/B, fp8 control arm through a transient FRANK_ARCHON_ENV_FILE drop-in:
prefill -29% (4,379 -> 3,126 cold), TTFT at the full window +25% (the 2 x 197k
pair: 110.5 s -> 138.1 s), decode inside the +-3% band on two of three passes,
draft acceptance UP 0.661 -> 0.740. The price is prefill, not decode. Open
call for Frank: keep exact KV, or take the cheap fp8 2 x 200k fit (pool 3,648).

Also corrects two stale index entries (w4dn8 said STAGED, fp8-native said
ACTIVE), adds the worklog entry, and commits the verification tool as
scripts/probe-frank-archon-kv.py — it carries the probe law that cost two
false negatives: radiance answers in reasoning_content, so a chat probe with a
small budget reads as empty content + finish_reason stop on a healthy server.
Tests 368/368.
2026-10-11 01:36:48 -07:00
Frank 0bf26b0741 llm(frank-archon): unquantized bf16 KV at 2 x 200,000
The KV pool at the fp8 shape backed 386,904 tokens — 1.93 sessions of the
served 200,000 window, so a second full-height conversation preempted to the
host/disk prefix tiers. Moving the cache to bf16 (the engine's only
unquantized rung; `--kv-cache-dtype` takes bf16|fp8 and has no per-tensor
K/V split) doubles what a token costs: 7.3125 -> 14.625 KiB per rank
(kv_attn 26.00 -> 52.00 KiB per 4-token page + kv_qsa_bkey 3.25 -> 6.50).

Pool 3543 -> 6912 MiB (437k tokens, >= 2 x 200,000), and the weights slab
pays for all of it: 26071 -> 22208 MiB, the pair pinned to the measured
card ceiling 29,120 MiB (card 31,856 - 1,710 consumed outside both budgets
- 1,024 free). The resident set that leaves VRAM spills into the pinned host
pool, so 40960 -> 44928 MiB, and the prefix host tier doubles with the entry
size (58.50 -> 117 KiB): 4096 -> 8192 MiB. Pinned ledger ~147 GiB, ttm floor
140000 -> 150000 (cap 200 GiB).

--max-num-seqs 3 -> 2: two slots x the window is the capacity contract, and
the "seqs 2 cannot boot" law is the retired tcclaviger fork's torch.compile
query_start_loc constraint, not radiance's (radiance booted seqs 2 in the
fp8-native era; the 2026-10-06 sweep had 2 and 3 tied inside +-3%). Bonus:
the fused all-reduce is now declared over 2 x 3 = 6 rows, under the 25-row
bound at which the wht6 wire is bit-exact.

Cost, paid twice on this engine: idle KV VRAM is also the expert cache's
spillover (lends 2.69 GiB + 740 MiB of arena), so the pool steals weight
residency from the slab AND the lendable cache. Priors: 2.14 GiB of stolen
weights cost 2.5% decode (MAX_MODEL_LEN 262144), the EVC ladder's other
direction cost 8.9%. Prediction -3..-8% vs the fp8 control (136 t/s at 44.5k,
acceptance 0.7246); the deploy boot ledger and live probes are the referee.

Seams: MAX_SEQS/HOST_POOL/KV_DTYPE/PREFIX_HOST literals follow the shape, and
two new invariants guard it — the KV pool must back seqs x max-model-len at
the declared dtype (an engine hard limit: an over-long request is accepted
then refused as too large to ever fit; the formula reproduces the fp8 boot's
386,904 to within 0.1%), and the VRAM pair must stay inside the ceiling.
2026-10-11 01:01:58 -07:00
Frank 394fd8535b docs(log): record the service-rename cutover outcome and the zealot --metrics gap 2026-10-10 17:17:59 -07:00
Frank 55f87fcdce llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot
The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.

Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.
2026-10-10 17:03:14 -07:00
Frank 5ced474eb9 docs(log): wire-rename wave — three served ids, OpenWebUI + Hermes re-key, bench drop-in retired 2026-10-09 17:10:25 -07:00
Frank 2992a7ddd6 docs: record the wire rename, the OpenWebUI re-key, and the frank-primary bench drop-in trap
- AGENTS.md Model Config: the Protoss names are now the ids on the wire
  (per-endpoint knob named); verify a rename on /v1/models, never on a chat
  response — radiance echoes the requested name
- OMP Harness.md: harness id == served id; one id per model everywhere
- Homepage.md: the docs-page "api id" annotations and the switchover promise
  are retired
- OpenWebUI.md: the id-keyed trio (model row, access_grant, api_configs
  model_ids) is a foreign-key set — re-key it in the same deploy; plus the
  contenteditable-composer probe note
- frank-secondary/frank-large/frank-primary runbooks: served ids
  frank-zealot/frank-dragoon/frank-archon, and the bench.conf drop-in that
  made every frank-primary deploy since 2026-10-08 04:09 restart the staging
  worktree behind a green health gate
2026-10-09 17:06:44 -07:00
Frank 7c77326c33 llm: complete the Protoss rename on the wire — served ids frank-archon/dragoon/zealot
The 2026-10-08 rename moved the harness model ids and the homepage cards;
the served aliases on the wire stayed frank-primary/frank-large/
frank-secondary. This closes the gap: the alias each endpoint announces is
now the family name, while the services, units, scripts, env files and
runbooks keep the old names (container identity stays stable).

- scripts/frank-secondary.env ALIAS, scripts/frank-large.env
  HALOGEN_MODEL_ID, scripts/frank-primary.env FRANK_PRIMARY_SERVED_NAME
  (+ the two serve-script defaults/gate messages)
- seams re-pinned: frank_secondary/frank_large/frank_primary unit bats
- fleet harnesses retargeted: bench-probe, deep-probe, calib-push,
  fleet-probes-swift15, autoresearch(+measure), bench-threads-batch-http,
  bench-decode
- shared/homepage/docs.html: api-id annotations and the "switchover will
  rename the ids" footnote removed — the page's names are now the ids;
  opencode/Pi/curl examples send frank-zealot
- dankclaw/hermes/config.yaml.example: frank-large -> frank-dragoon and the
  live 262144 context (Hermes routes by served id; see Hermes.md)

Consumers with live state re-keyed out-of-band in the same window:
OpenWebUI (model row, access_grant, openai.api_configs model_ids) and the
Hermes config.yaml.
2026-10-09 16:47:19 -07:00
Frank 65dec6a80e docs(authentik): provision kirby/ry/nic/debo/freya (pk 17-21); OWU pending-role first-login flip law 2026-10-09 11:49:58 -07:00
Frank f99b698d25 docs(LLM): 3090 worked example — archon w4dn8 methodology ports to GGUF-native quants; 24 GiB size law for 35B-A3B 2026-10-08 23:44:13 -07:00
Frank 757a90057f docs(LLM): recipe portability audit — qwen35moe plugin carries the calibration tap; 35B-A3B w4 fits one R9700 resident 2026-10-08 23:42:58 -07:00
Frank 009e8a885d docs(LLM): Quantization Production — 3090 as the unlocked card (runtime menu, Ampere ceilings, Playbook B safetensors ladder) 2026-10-08 23:38:49 -07:00
Frank 6b677139ad docs(LLM): Quantization Production reference — per-card self-quant playbooks, REAP/Heretic/abliteration/K_P prior art 2026-10-08 22:39:18 -07:00
Frank 184a5c665a homepage+docs: frank-archon runs our own GPTQ-w4 build, link our HF publication 2026-10-08 22:15:33 -07:00
Frank 958b10545e homepage+docs: bottom-pinned privacy disclosure chips, live OWU api-keys screenshot, docs steps for 0.6 UI, drop house-DNS/Pi-hole bullet 2026-10-08 18:07:31 -07:00
Frank c4dca68a03 homepage: link Model Information panel to the docs AI section
The panel title carries a right-aligned 'AI guide ->' link to
/docs#models (the running-models section), at the point of use.
Bare mode recolors the link; layout is flex baseline, doctrine-minimal.

Seam: homepage_config.bats 6/6; headless smoke census + screenshot.
2026-10-08 18:05:50 -07:00
Frank f29dbdf7c7 homepage: rounded corners, aligned title glyphs, greyed closed items, spliced dragoon tagline, inline privacy notes 2026-10-08 17:29:45 -07:00
Frank 98f11a19c1 docs+AGENTS: model ids officially frank-archon/frank-dragoon/frank-zealot (services keep old names) 2026-10-08 17:16:21 -07:00
Frank a30ac9457a merge: swift15 runbook LIVE 2026-10-08 05:02:10 -07:00
Frank 75695c312a swift15: AGENTS.md aligned to deployed shape (fp8 KV, spec 2, seqs 3, PLE fp8 n-gram) 2026-10-08 05:02:10 -07:00
Frank c507f91438 swift15: runbook LIVE + deployed verification + vision probe law 2026-10-08 05:02:10 -07:00
Frank b175f4afd1 swift15: vision probe reads reasoning_content fallback, 256-token budget 2026-10-08 04:57:31 -07:00
Frank 971e23a25c swift15: fix fleet-probes vision heredoc (quoted 'PY' ate $B); probe header 2026-10-08 04:57:10 -07:00
Frank 3ca64e14fc merge: swift15 KLD re-PASS at production shape 2026-10-08 04:50:28 -07:00
Frank f3dfea13ad swift15: KLD re-PASS at production MNBT 2048 (KL 0.0131, top-1 97.92%) 2026-10-08 04:50:23 -07:00
Frank 02fa1e22ae merge: swift15-radiance-w4dn8 — frank-primary GPTQ w4 (Swift1.5)
Conflict resolved: frank-primary row from the branch (w4 chain),
frank-secondary row from main (Qwen3.5-9B swap of 2026-10-07).
2026-10-08 04:46:11 -07:00
Frank 1aef7418ad swift15 cutover: AGENTS.md + runbook + log + fleet probes + KLD gate shape fix
- AGENTS.md: fleet table row + frank-primary section rewritten for the
  GPTQ w4 chain (1.0.13 serve/convert, 1.2.0 calib, int8 KV, vram pair).
- runbook: throughput gate PASS (136.0 t/s median @ 44.5k, 4.5x bar),
  fleet probes PASS, control baselines recorded.
- log.md: 2026-10-08 campaign entry with the four measured engine laws.
- kld-gate: shape matched to the reference recording, then reshaped for
  the w4 MNBT<=2048 file-tier law (pool 40960).
- fleet-probes-swift15.sh: tools/reasoning/response_format/vision walk.
2026-10-08 04:41:07 -07:00
Frank fa5a444b64 homepage: simplified console — flat navy, static down-state, no scanlines/lattice/frames/glows 2026-10-08 04:31:30 -07:00
Frank 90f678582e homepage: navy Protoss console — maximal octagon cornering, Share Tech Mono + Rajdhani, pictograms for CW/RomM/Copyparty/Marreta, dragoon teal 2026-10-08 04:17:33 -07:00
Frank cc0fdd15a1 swift15: env -> candidate payload, RAD floor 110000 (measured 116,416 MiB) 2026-10-08 04:11:02 -07:00
Frank f148ecab4a homepage: HF source links per model; docs reorder (AI last), arr stack out of family docs 2026-10-08 04:00:26 -07:00
Frank c2f3411342 swift15: campaign decode probe (43.6k greedy, timings-based) + runbook gate results 2026-10-08 03:58:17 -07:00
Frank 8c5bde7034 homepage: Protoss theme, brand-color icons, dragoon portrait, GameBoy RomM, OWU white art + search warn 2026-10-08 03:51:50 -07:00
Frank 22a7331100 homepage: icon swaps, clickable tiles, down pulse, header nav, compaction callout
- Icons: Copyparty -> improvised amber folder, Marreta -> crumbling
  wall (bricks + red crack), Calibre-Web -> book, OpenWebUI -> official
  logo.png from openwebui.com, Seerr -> official seerr.dev
  os_logo_filled.svg.
- Tiles with an open link are clickable whole (.panel.clickable: hover
  lift + glow; inner link clicks pass through).
- Any DOWN panel (app tile or model card) gets a red gradient wash and a
  slow pulsing glow (.panel.down, .mi-card:has(.badge.down)).
- docs / admin / account-settings buttons moved from the footer to the
  header (nav.top-links); footer keeps the deployment stamp.
- Zealot compaction advice promoted from the Use paragraph to a bordered
  glowing model.compaction callout (.mi-comp); component seam pinned to
  the new field.
2026-10-08 03:37:13 -07:00
Frank c1e192df00 homepage: app-card art, zealot-first order, common uncensored warning, admin identity gate
- Every app card carries hex-clipped art: 19 amber-tinted brand marks
  from the Simple Icons CDN plus 19 improvised monogram SVGs, wired via
  a per-service "art" field; homepage_config.bats pins art presence for
  all 32 apps.
- Model cards render zealot -> dragoon -> archon (registry order pinned
  in the seam).
- Dragoon and zealot share one common uncensored legal warning (adult
  content, harmful/hateful/illegal descriptions, sensitive offensive
  cybersecurity topics, severe legal risk, "don't ruin this server for
  everybody"), hover label on card and docs.
- Archon gets a bordered red permission callout: ask Frank before use.
- SearXNG obfuscation-not-anonymity note moved to a card-level hover
  warning (.svc-warn) and a docs hover note.
- Gitea: self-service signup flow; no need to ask Frank.
- Footer docs/admin/account-settings become glowing bordered buttons;
  the admin button is shown only to homepage-admins members via a new
  Caddy /whoami endpoint echoing the outpost-injected X-Authentik-*
  headers (spoof-proof; the /admin* 403 stays the real gate). e2e pins
  both sides.
- Docs naming swept: Protoss names primary, bracketed api ids, explicit
  note that a future switchover renames the ids themselves.
2026-10-08 03:21:26 -07:00
Frank 8375567de3 swift15: restore expert_tiered + disk-tier flags dropped in the gate rewrite 2026-10-08 03:18:04 -07:00
Frank 79c2e64c2c swift15: gate MNBT 3072 per Chief (w4 arena scales hard with MNBT; 32k step is absurd here) 2026-10-08 03:17:33 -07:00
Frank f20da6a6ff swift15: gate shape MNBT 16384 (w4 arena at 32768 costs 33 GiB), kv 2048, headroom 512 2026-10-08 03:16:45 -07:00
Frank 058ea6cca8 swift15: gate boots at the production pool (w4 gate_up experts are RAM-resident; disk tier refuses non-serving stored forms) 2026-10-08 03:14:20 -07:00
Frank 3f8531179e swift15: gate flags per engine (--kld-ref takes no corpus; --kld-out JSON parsed) 2026-10-08 03:12:18 -07:00
Frank 3c5d77d1fa e2e: count fused Model Information names in the user homepage spec
The Protoss model cards render as .mi-name inside one full-width panel;
the panel-name census now counts both render styles, keeping services.json
the single source of truth.
2026-10-08 02:51:09 -07:00
Frank 203b0be145 homepage: Protoss rename + fused Model Information component
The model cards become the StarCraft trio: frank-large -> frank-dragoon,
frank-primary -> frank-archon, frank-secondary -> frank-zealot (family
names only; endpoint/API ids unchanged). The three cards fuse into one
Model Information component: per-unit Protoss accent colours and hex-
clipped unit art, 15px type, clickable StarCraft unit-voice quotations
(original .ogg files from the fandom wiki, served from /sfx/), supervision
notes (dragoon/archon safe for autonomous use, zealot tight supervision),
compaction-model recommendation for frank-zealot, and uncensored-model
legal notes rendered as hover tooltips instead of wall text. docs.html
carries the same renames plus api-id mapping.

App-card updates: Seerr, Calibre-Web, RomM, Jellyfin set to WIP: DOWN
with Open links removed (new wip "down" badge state); Copyparty notes it
serves all other media apps content including WIP apps; Gitea and Plex
noted as needing logins separate from SSO; SearXNG noted as obfuscated,
not anonymous (upstream API keys tie to Frank).

Plumbing: Caddy @data allowlist grew /sfx/*; deploy_homepage copies sfx/;
homepage_config.bats pins the new probe names, the component fields,
art/voice-file presence, and the wip-down schema.
2026-10-08 02:40:45 -07:00
Frank 74da746c51 docs(homepage): record the capability equivalence notes 2026-10-08 01:32:35 -07:00
Frank 7066accc25 homepage: capability equivalence on the AI cards
Every fleet model card (and the #models docs section) now states, in
plain family-facing terms, what the model is equivalent to:
frank-large ~ Claude Sonnet 5 / OpenAI-Luna, frank-primary ~ Claude
Opus 4.8 / OpenAI-Terra (both Summer 2026 frontier), frank-secondary ~
last year (2025) frontier. Rendered as a new details.equiv line
(desc-equiv, styled in the head color, bare-mode clean); pinned in
homepage_config.bats.
2026-10-08 01:30:50 -07:00
Frank 76aaa2543b frank-large: raise ctx to 262144 to close the harness 400-zone
The harness clamps max_tokens to min(45056, contextWindow - est - 4096),
its chars/4 est under-counts the rendered prompt by ~3 % (measured:
152,938 -> 157,388 and 158,906 -> 165,301), and compaction fires only at
85 % -- so with window == engine ceiling 200192, every request in the
est ~146k-184k zone was a guaranteed 400. HALOGEN_CTX=262144 (native) +
KV pool 524288; harness window 217088 = ceiling - 45k absorbs the
estimate error (worst-case request ~219k <= 262144). models.yml updated
out-of-repo: contextWindow 217088, maxTokens 45056.
2026-10-08 00:40:12 -07:00
Frank 8220030ba1 swift15: canonical recipe = host copy the convert loaded (all 24 protected experts; committed copy was a partial splice) 2026-10-07 23:45:45 -07:00
Frank 5191c49967 swift15: KLD gate script (candidate vs bf16 ref, KL<=0.03 / top-1>=95%) 2026-10-07 23:43:06 -07:00
Frank f9330fa201 docs(homepage): record the @data allowlist law for new assets 2026-10-07 23:41:58 -07:00
Frank 4b71c014e9 caddy: serve /img/* from the homepage root
The @data path allowlist on the post-auth listener did not include /img/*,
so docs screenshots fell through to the index.html catch-all and the
browser got 200 text/html for every PNG (docs images broken live
2026-10-08). Pinned by the homepage_caddy seam.
2026-10-07 23:38:18 -07:00
Frank cee90eea52 homepage: amber CRT theme, VT323, current-fleet docs + API access guide
- style.css: green phosphor -> amber CRT-AMBIOS palette; docs styles for
  screenshots, code blocks, key warning (full + bare mode)
- index/admin/docs: VT323 from the Google Fonts CDN
- docs.html: rewrite #chat/#models for the live fleet (flash-server v2,
  radiance native FP8, Qwen3.5-9B Q6_K); new #api-access with OpenWebUI
  key walkthrough + Opencode and Pi configs; three OpenWebUI screenshots
- services.json: AI card details match the current fleet
- deploy-remote.sh: ship shared/homepage/img/ to /srv/homepage/
- docs: Homepage.md theme wave + OpenWebUI.md picker access-control law
2026-10-07 23:27:23 -07:00
Frank 73c37f4301 swift15: all experts through GPTQ w4 (plugin one-table-one-encoding law)
rad_arch_declare rejects the split tier (down block-fp8 + gate_up w4): the
qwen4exp_fp8 kernel serves one encoding per expert table (w4nl64a8h) with only
whole bf16 experts beside it. Recipe now sends gate_up AND down through the
GPTQ w4 rule (down calibrated by the Grams, gate_up uncalibrated), drops the
down-fp8 rule, and lowers the size floor 126000->108000 MiB for the ~19 GiB
lighter container. Convert script: checkpoint lives under models/ukisai/, and
the in-container path must keep the subdir (basename stripped it).
2026-10-07 21:31:10 -07:00
Frank 6632d0a2f6 docs(openwebui,authentik): frank-secondary-only picker, API gating, artem account
OpenWebUI model picker narrowed to exactly frank-secondary (connections
frank-large/frank-primary disabled in openai.api_configs, model table rebuilt,
stale frank-small model_id retargeted to the served alias). Documented the
auth-gated API surface (anonymous probes all 401; keys per account).
Authentik roster gains artem (pk 16) in the human-user template.
2026-10-07 18:49:22 -07:00
Frankandomp caf5e1e0ca docs(authentik): record the human-user template and alex's recreation
kassie (pk 14) and alex (pk 15, deleted pk 7 then recreated) now share the
ridley shape: <user>@franksdank.cloud + Non-Admin Users + jellyfin_users +
openwebui_users. Audit before destroy: the old alex row carried no MFA
device, token, entitlement, or proxy session - checked across the MFA
device tables, authentik_core_token, and providers_proxy_proxysession
(the fleet's column names differ from the ORM names: events.user is
denormalized JSON, certificatekeypair has no user_id). Events survive the
delete, so the audit trail keeps its shape.

Co-Authored-By: omp <omp@franksdank.cloud>
2026-10-07 18:20:11 -07:00
Frankandomp 5278f5be26 frank-secondary: record the deployed boot, keep the unit description honest
Deploy verified: nvidia-smi 21,436 MiB used / 2,689 free at ubatch 2048 -
the MiMo-priced projection (~21,430/~2,700) landed within 11 MiB, and the
verbosity-4 ledger confirms the KV law to the MiB (CUDA0 KV buffer
12,544.00 MiB = K(f16) 6,272 + V(f16) 6,272 = 32 KiB/token over the
401,408 pool), compute buffers 1,136 MiB CUDA0 + 848 MiB host, mmproj
879 + 248 MiB. Playwright gate: 33 passed.

Two lessons recorded: the 203/EXEC boot loop my shebang-less rewrite
caused (journal signature + the head -1 check), and that llama.cpp's
"n_ctx_seq < n_ctx_train" line is informational when the context is BELOW
the trained window - the Gemma4 drafter warning was the opposite case.

The unit's Description named the Swift-1.5 era (and said 2x131072/q8_0/
draft-mtp); it now names the Q6_K shape. Cosmetic, so it is NOT deployed
as its own restart: the next frank-secondary deploy relinks it.

Co-Authored-By: omp <omp@franksdank.cloud>
2026-10-07 17:33:21 -07:00
Frankandomp ec6184de9a frank-secondary: restore the shebang my rewrite dropped (unit failed 203/EXEC)
Co-Authored-By: omp <omp@franksdank.cloud>
2026-10-07 14:50:49 -07:00
Frankandomp 1d062f06a3 frank-secondary: swap Gemma4-26B-A4B Q4_K_M -> Qwen3.5-9B-Uncensored Q6_K 2x200,704
Capacity trade: the 9B dense-hybrid (qwen35: 32 layers = 8 full-attention
+ 24 Gated DeltaNet, 4 KV heads x 256 dim, native 262,144, MTP-less,
natively multimodal) costs ~7 GiB of weights where the 26B MoE cost
~16.6 GiB. The freed room buys back the second conversation slot and
exact KV: --ctx-size 401408 --parallel 2 (two 200,704 windows), f16 K +
f16 V, MTP off (the file carries no nextn/mtp tensors - flag-only enable
is impossible), vision via the publisher's BF16 mmproj, ubatch 1024
(the CUDA decode-optimal value), verbosity 4, publisher sampler recipe
(0.6/0.95/20/0) in models.yml.

Hosts with the MiMo-V2.6-Distill Q6_K boots at this geometry: 21,276 MiB
used / 2,849 free at ubatch 1024, 40,960 tokens of fp16 KV at ubatch
2048. Compaction stays routed here; window math is unchanged (maxTokens
16,384 keeps the worst-case harness request inside the slot).

Gemma4 payload (Q4_K_M + mmproj + MTP assistant) stays on disk as the
rollback; its runbook is retired in place, keeping the attention_k_eq_v
pair-billing law.

Co-Authored-By: omp <omp@franksdank.cloud>
2026-10-07 14:47:32 -07:00
Frank b51e078e36 frank-secondary: record the deployed f16-K boot ledger, correct the k_eq_v billing law
Verbosity 4 paid for itself: the engine's own llama_kv_cache lines give the
exact accounting — the cache bills the K+V pair TWICE under attention_k_eq_v
(6.125 B/elem at f16 K + q8_0 V), reproducing 3,001.25 MiB (5 full layers at
n_head_kv 2) + 306.25 MiB (25 SWA rings at n_head_kv 8, window 1024) to the
byte. The earlier "bill K-only" note was wrong; full layers have 2 KV heads,
not 8.

Deployed boot: 21,868 used / 2,257 free — the f16-K raise (+2,161 MiB KV) is
funded by the ubatch 2048 -> 1024 cut (~616 MiB of compute buffer) plus
variance, so exact K came essentially free. Measured decode 139.9 t/s temp 0;
vision probe returns Red at prompt_tokens 151.
2026-10-07 13:58:30 -07:00
Frank 68a525740d frank-secondary: swap Swift-1.5 RCO-IQ3_XXS -> Gemma4-26B-A4B QAT Q4_K_M
Payload: HauhauCS/Gemma4-26B-A4B-QAT-Uncensored Q4_K_M (16,796,015,520 B)
+ the publisher's BF16 mmproj, /home/dank/models/Gemma4-26B-A4B/.

Shape: ONE 200,704-token slot (--parallel 1; 2x200,704 does not fit the
24 GiB card), f16 K + q8_0 V under flash attention, batch 4096 / ubatch
1024, --log-verbosity 4, MTP OFF (the gemma4-assistant file is on disk,
not loaded: its draft KV is built against the target's full window,
measured +1.9 GiB -> 23,806/319 free, no margin beside the f16-K raise).

Measured boots (nvidia-smi): q8_0 K + q8_0 V / ubatch 2048 / MTP off =
21,890 used / 2,235 free; with the MTP assistant = 23,806 / 319.

Engine facts recorded: gemma4 hybrid attention (25 sliding-window + 5
full-attention layers) so KV scales with 5 layers only, and
attention_k_eq_v is honoured by build 5fc4f3c (pool bills K-only; the
measured q8_0 pool = 2.05 GiB = the K-only figure).

Serve script gained VERBOSITY (+ --log-verbosity in the argv), the
SPEC_DRAFT_FILE drafter path, and defaults/lore retargeted to Gemma4.
deploy-remote.sh payload gates follow the env. The frank-secondary bats
seam is rewritten for the new contract (argv-based) and fixes a
pre-existing vacuity: serve_argv regenerates the test env, so
mutations made before it were silently undone - tests that mutate now
exec through run_stage. frank-secondary is the fleet compaction hub
(200,704 is the only window that fits a worst-case compaction input).
2026-10-07 13:48:54 -07:00
Frank 5b7b3ccb90 frank-secondary: swap to Gemma4-26B-A4B Q4_K_M, 1x200704, q8_0 KV, MTP via separate assistant file
- MODEL_FILE: Swift-1.5-Qwen3.8-27B RCO-IQ3_XXS -> Gemma4-26B-A4B Q4_K_M
- CTX_SIZE: 229376 (2x114,688) -> 200704 (1x200,704)
- PARALLEL: 2 -> 1
- CACHE_TYPE_K/V: q8_0 (unchanged, FP8 class)
- SPEC_TYPE: draft-mtp (unchanged)
- SPEC_DRAFT_FILE: new — separate MTP assistant file (mtp-gemma-4-26B-A4B-it.gguf)
- serve script: add --spec-draft-model support for separate draft files
2026-10-07 12:41:07 -07:00
Frank 2137080d1c docs(index): top-level link for the staged Swift15 runbook (seam 135) 2026-10-07 04:42:36 -07:00
Frank f747cb0dab docs(runbook): staged Swift15 w4dn8 runbook skeleton + index link
Contract facts frozen (payload chain, recipe tiers, shape, gates);
MEASURED@cutover slots for bench/KLD numbers.
2026-10-07 04:40:31 -07:00
Frank db8faa68ff docs(worklog): KLD reference complete (rows 50235), script-swap law third occurrence 2026-10-07 02:49:14 -07:00
Frank cf6c62d44a fix(kld-then-calib): restore S= path clobbered by the SKIP_KLD edit 2026-10-07 02:45:28 -07:00
Frank 0aafad5f2c fix(kld-then-calib): SKIP_KLD resume + rows-50235 receipt gate
The KLD engine exited clean with the record complete (kld.json rows:
50235, 24 GiB, 5 files), but the wrapper died rc=127: a mid-run scp of
the calib-shape edit shifted bash's lazy-read offset — the fleet's own
script-swap law, biting on the same class as the autoresearch collision.
Resume path: SKIP_KLD=1 verifies the receipt (rows + size + files) and
goes straight to calib. Never scp a script a running process owns.
2026-10-07 02:44:48 -07:00
Frank a3f45ce028 docs(campaign): interim work log for the Swift1.5 w4dn8 campaign
Execution facts, blocker trail (plugin-tap probe law, weights-slab
formula, KLD_MOUNT), engine verdict, and the post-chain order. Linked
from the docs index per the completeness seam; the cutover runbook
replaces it.
2026-10-06 23:42:32 -07:00
Frank 45de2fd47f perf(run-swift15-bf16): calib at the vendor big-step shape
MNBT 65536 / seqs 8: MOE-W4.md:244 measures 640 tok/s at a 32k step
vs 930 at 55k — the KLD record's own rate confirmed step size drives
disk-tier prefill. 131k resident tokens fits the 3072 pool (~237k
tokens/rank at fp8). Cuts the 10M-token calibration from ~4.3 h to
~3 h.
2026-10-06 23:39:10 -07:00
Frank fa858f4c10 feat(campaign): protected-cast applier, recipe markers, KLD gate runner
apply-protected-cast.sh splices derive-protected-experts.py output into
the recipe's PROTECTED-BEGIN/END section (refuses to re-derive, trashes
the pre-edit recipe). kld-gate-swift15.sh boots the candidate on the
serving pin in --kld-ref mode (pool fully resident, no disk tier) and
enforces mean KL<=0.03 / top-1>=95% from the JSON report.
2026-10-06 22:41:18 -07:00
Frank 07af1a9366 feat(kld-then-calib): chain KLD record -> 1.2.0 calib serve -> 10M push
One-GPU serialization: the chain waits for the kld driver log to close
rc=0, sanity-checks kld-ref, boots the calibration server, pushes the
corpus at parallel 16 (4 engine slots; queue absorbs the rest), then
kills the server so its trap restores production.
2026-10-06 22:39:14 -07:00
Frank f0d0614ce6 fix(run-swift15-bf16): define KLD mounts (were unbound-empty)
kld died: 'cannot read --kld-corpus /data/kld.jsonl' — KLD_MOUNT was
referenced but never defined, so the corpus/ref-dir mounts silently
expanded to nothing. Add them (nested order: after the /data mount).

feat(convert-swift15-w4dn8): final recipe convert on 1.0.13

Convert with the SERVING pin so the .rad crosses no version boundary;
only the Gram files cross (RADGRAM2 magic, gram.r%d.%s.bin naming and
calib_gram/calib_resolve/calib_stem are byte-identical between the
1.0.13 and 1.2.0 libquant). Host-backend run: prod can stay up.
Gates: grams present (>=48), bf16 container deleted, disk floor,
rad-info gptq receipt, size floor 126000 MiB.
2026-10-06 22:37:16 -07:00
Frank 9aed9987b6 fix(run-swift15-bf16): kv pool 3072 both offline arms
The weights slab derives as card - headroom - kv, and the disk tier
refuses under 9.57 GiB (4.21 layer buffers + 5.36 fixed). 4608 starved
it (8.68 budget, DID NOT FIT). 3072 leaves 10.2 for weights; 4x16k fp8
slots fit the pool, recording is prefill-bound so slot count only
queues pushes.
2026-10-06 22:33:19 -07:00
Frank 4048dc8630 fix(run-swift15-bf16): per-mode engine pins + explicit KV pools
kld stays on 1.0.13 (record on the scoring engine) with --vram-kv-mib
4608 + 16 slots: the ratio-derived pool came up SHORT by 217 MiB at
32x16k. calib moves to 1.2.0: the RADIANCE_CALIB_TOKENS tap is a 1.2.0
qwen4exp plugin feature (1.0.13 plugin has the DIR half only), and
Grams are bf16 activations - engine-invariant. --vram-kv-mib 9984
covers 64x16k fp8 slots.
2026-10-06 22:31:17 -07:00
Frank 9bc40fd899 fix(run-swift15-bf16): drop duplicated radiance arg (image entrypoint already runs radiance) 2026-10-06 22:25:01 -07:00
Frank bb60e8524f run-swift15-bf16: boot the bf16 container for KLD record / calibration (vendor disk-tier shape) 2026-10-06 21:46:05 -07:00
Frank 319dd0f7da derive: restore the header read 2026-10-06 21:42:16 -07:00
Frank 650dfff5a2 derive: RADGRAM2 header is magic,k,rank,rows,domain (28 B) 2026-10-06 21:42:00 -07:00
Frank de70320cde derive-protected-experts: RADGRAM2 scan + 3%/3x law + parity fill from calib recording 2026-10-06 21:41:16 -07:00
Frank 910c54b6ff frank-primary: keep 1.0.13 engine pin — 1.2.0 bench rejected (accept 0.7123 vs 0.7549, decode -2.9%; vram pair exonerated, ledger-exact) 2026-10-06 21:35:39 -07:00
Frank 8c887e6fc9 calib-push: drive calib.jsonl through scratch prefill until the tap writes Hessians 2026-10-06 21:32:56 -07:00
Frank 49dd2be4cd corpus: handle codeparrot content column + per-class caps
- load_parquet_texts: read codeparrot-clean 'content' column (schema-robust)
- per-class HF caps (prose 60k for short ~66-tok rows, code 40k for long files)
- pyarrow 18: ParquetFile.iter_row_groups removed -> read_table().to_pydict()

Produces 10,014,932 calib tokens (4.5M code/3.0M prose/2.5M tools) and
1,800,816 KLD tokens (197 prompts, all >=8192, score_from=n-256) from real
HF sources (codeparrot-clean + wikitext-103).
2026-10-06 21:16:16 -07:00
Frank a11f703c49 swift15: corpus generator v2 — HF code/prose ingest, richer tool JSON, 8k KLD prompts 2026-10-06 20:58:32 -07:00
Frank 6cdf88f611 swift15: restore the numpy import the census edit clobbered 2026-10-06 20:41:33 -07:00
Frank 259fc5505d swift15: PLE audit adds over-cap census + next-bf16 scale suggestion 2026-10-06 20:41:07 -07:00
Frank 3e46d51df8 swift15: BF16 absmax bits belong in the HIGH half of the float32 2026-10-06 20:39:21 -07:00
Frank d062a376b2 swift15: corpus generator (10M calib + 1.8M KLD, Froggeric-rendered, disjoint pools) 2026-10-06 20:38:26 -07:00
Frank 95219cf4c1 swift15: release the numpy view before closing the PLE audit mmap 2026-10-06 20:35:51 -07:00
Frank fee66187a1 swift15: PLE fixed-scale audit (streaming BF16 absmax vs 448*scale cap) 2026-10-06 20:33:55 -07:00
Frank 0258ae0897 frank-primary: pin radiance v1.2.0, translate expert-vs-cache-ratio to vram knobs
Engine v1.2.0 (Codeberg e95d397, Docker Hub tag 1.2.0, index digest
665775ed...) removes --expert-vs-cache-ratio. The explicit
--vram-weights-mib 26071 / --vram-kv-mib 3543 pair transcribes the live
1.0.13 boot ledger at ratio 0.82 (weights 25.46 GiB, KV 3.46 GiB per
card); bracket arms scale KV linearly in (1-r): 0.90 -> 1968, 0.95 -> 984.
Serve script, autoresearch harness, and bats gates follow; the bats
shape asserts are realigned to the deployed seqs=3/spec=2 sweep result
(95923d3, df6b5bb) they failed to track.
2026-10-06 20:31:46 -07:00
Frank 9aee00f2b3 swift15: fetch script + w4dn8 expert recipe for the frank-primary campaign
- download-frank-primary-swift15.sh: ukisai/Swift1.5-Qwen3.8-Flash-Next
  (BF16, 131 shards + index + configs, 360037355038 B = 335.3 GiB, ungated)
  via the vendored hfget.sh; verify = listed sizes + 131-shard weight-map
  count + every shard's sha256 against the Hub LFS oid. Pattern cloned
  from scripts/download-frank-primary.sh.
- qwen4exp-swift15-w4dn8-i8-hc8m.recipe: routed experts with DOWN at
  block-fp8 (rtn codes=fp8_e4m3 block=128x128 scale=bf16, the served
  FP8-native tile form; moe_gemm_fp8a8 reads it; minicpm5-2b is the
  shipped bf16->fp8a8 precedent) and gate_up at w4nl64 GPTQ cd=3;
  hc/MTP-fc fp8, trunk i8, lm_head i8, PLE fixed scale, draft_head u2
  (inert). Protected-expert cast lines carry the BASE checkpoint's names
  as a warm start; the derivation step rewrites them from Swift's own
  calibration recording before the recipe is ever run (law documented
  in the file).
2026-10-06 19:19:40 -07:00
278 changed files with 8068 additions and 3373 deletions
@@ -1,4 +1,4 @@
# Tunables for frank-large (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
# Tunables for frank-dragoon (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
# STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw,
# Strix Halo gfx1151, unified memory).
#
@@ -56,7 +56,7 @@ CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
# Alias + network.
ALIAS=frank-large
ALIAS=frank-dragoon
BIND_ADDR=0.0.0.0
LISTEN_PORT=7770
@@ -1,5 +1,5 @@
[Unit]
Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-large)
Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-dragoon)
After=network-online.target
Wants=network-online.target
@@ -7,8 +7,8 @@ Wants=network-online.target
Type=simple
User=dank
Group=dank
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
Restart=on-failure
RestartSec=10
+1 -1
View File
@@ -6,7 +6,7 @@
# is reachable on this port regardless of the boot tag. The NPU is a
# single-lane device: generations serialize FIFO, and the first request after a
# model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
# pool shared with frank-large (iGPU).
# pool shared with frank-dragoon (iGPU).
FLM_MODEL=nuextract3:4b
FLM_HOST=0.0.0.0
FLM_PORT=7771
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as
# frank-large via the kingjones30/ROCmFPX llama.cpp runtime on
# frank-dragoon via the kingjones30/ROCmFPX llama.cpp runtime on
# Dankclaw (Strix Halo gfx1151, unified memory).
#
# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
@@ -29,7 +29,7 @@
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
if [[ -f "$ENV_FILE" ]]; then
set -a
# shellcheck disable=SC1090
@@ -44,7 +44,7 @@ fi
: "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}"
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
: "${CHAT_TEMPLATE_KWARGS:={}}"
: "${ALIAS:=frank-large}"
: "${ALIAS:=frank-dragoon}"
: "${BIND_ADDR:=0.0.0.0}"
: "${LISTEN_PORT:=7770}"
: "${CTX_SIZE:=401408}"
+1 -1
View File
@@ -4,7 +4,7 @@
# Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a
# single-lane device: generations serialize in FIFO order, so this is a
# background/triage endpoint, not an interactive multi-user chat. The NPU
# draws from the unified RAM pool shared with frank-large (the iGPU
# draws from the unified RAM pool shared with frank-dragoon (the iGPU
# llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md
# for the memory budget and the model-selection tradeoff.
#
+309 -233
View File
@@ -9,11 +9,11 @@ services run from `/home/dank/serverAppConfigs` on each host (the 2026-08-22
merge absorbed the former `vllmScripts` repo, history preserved).
- **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
Calibre-Web, Jellyfin, Plex) plus the `frank-primary` LLM service.
Calibre-Web, Jellyfin, Plex) plus the `frank-archon` LLM service.
- **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
Alloy, Pi-hole) plus TWO LLM services: `frank-large` (flash-server
container on the gfx1151 iGPU) and `frank-secondary` (llama.cpp CUDA on the
Alloy, Pi-hole) plus TWO LLM services: `frank-dragoon` (flash-server
container on the gfx1151 iGPU) and `frank-zealot` (llama.cpp CUDA on the
RTX 3090). Caddy on
Dankclaw reverse-proxies both hosts.
- **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
@@ -25,9 +25,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).
| Target | Service | Runtime | Model | Port |
|--------|---------|---------|-------|------|
| Dankserver | `frank-primary` | **radiance** (StillDeadcode v1.0.7 line; image `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned INDEX digest, 287 MiB, `libr4d` gfx1201 kernels, plain `podman run`, root systemd unit) | `Qwen/Qwen3.8-Flash-Next-FP8` official (172.8 GB, 131 shards, sha256-verified) converted **no-recipe** to `qwen4exp-fp8-native.rad` (177,109 MiB measured = 173.0 GiB) — **native modelopt block-FP8, no requantisation** — on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, `--expert-vs-cache-ratio 0.82` = the 99 %-hit point), **PLE 47.7 GiB pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV 7.32 KiB/token**, **MTP spec decode depth 3**, *…
| Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the head per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 200,192 ctx** (KV pool 400,384 positions ≈ 10.7 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold) | 7770 |
| Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF **RCO-IQ3_XXS -mtp** (27B dense qwen3_5 hybrid, 10,442,827,392 B; a per-tensor allocation mix of stock types; the MTP head in the file **drafted** — **spec ON** (draft-mtp depth 3) since 2026-10-06 evening, ngram skipped on the recurrent-rollback law) + the publisher's own **BF16 mmproj** for **vision**, on the RTX 3090, **2 × 114,688** (Frank's 112k cap each since the 2026-10-06 late reshape, cut from 131,072 to fund the **ubatch ladder 512 → 1024 → 2048** — **1,005 MiB free at boot, 763 settled after a vision probe** (encoder buffers resident)) with **q8_0 K + q8_0 V (FP8 class, 34 KiB/token)** under flash attention, batch 4096 / **ubatch 2048** (the ceiling for this shape — 4096 would not fit); **prefill 1,209 t/s @ 3,760 tokens; ~52–58 t/s recipe decode** (MTP acceptance 34–54 %, temp-1.0 noise; decode is ubatch-invariant). Renamed from `frank-small` 2026-10-06 (87721ab). Supersedes the same-morning IQ3_S-mtp payloads: the two-slot no-spec shape and the one-slot MTP shape (both 2026-10-06; payload on disk).|7771|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **44 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the **30,969 MiB** the engine's own budget table makes claimable: card 32,624 MiB, settle floor 209 MiB), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (pool 5,120 MiB = the 3-slot contract + 1.3 % — the only dtype that funds 3 × 200,000 without gutting the slab, since bf16 bills 13.8 KiB/token), **MTP spec decode depth 2**, `--max-num-seqs 3` (the capacity contract is 3 × 200,000; 3 × (1+2) = 9 all-reduce rows, inside the 25-row wht6 exactness bound), `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8); prefill is **slab-limited and pays for it** — 22,208 → 3,397, 24,000 → 3,290, 25,600 → **3,584** t/s at 43.5k single-stream, 3-way 197k aggregate 3,051 → **3,589** t/s, with a cliff at ~26 GiB (26,071 → 4,379). Cutover 2026-10-08 from fp8-native (payload on disk = rollback); 3-slot fp8-KV shape 2026-10-13, re-cut twice in the slab's favour (22208/6912 at 4 slots → 24000/5120 → 25600/5120); the 4-slot shape (22208/6912, `--max-num-seqs 4`) is the documented rollback. | 7770 |
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
Retired endpoints (`frank-npu`, and every prior runtime) are in
the [runbook index](#runbook-index). The full swap history lives in
@@ -66,14 +66,14 @@ All edits flow through git. Never edit files on a remote machine directly.
./scripts/deploy.sh dankclaw
./scripts/deploy.sh dankserver
./scripts/deploy.sh dankclaw --service searxng
./scripts/deploy.sh dankserver --service frank-primary
./scripts/deploy.sh dankserver --service frank-archon
./scripts/deploy.sh dankclaw --dry-run
```
The planner maps changed paths to services; the remote runner restarts only
the selected service and records its revision outside the checkout.
Documentation-only changes are a no-op. LLM services are selected only by
their own files — every endpoint (`frank-large`/`frank-secondary`/`frank-primary`)
their own files — every endpoint (`frank-dragoon`/`frank-zealot`/`frank-archon`)
by its unit/env/serve trio in `scripts/` — so
unrelated
container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
@@ -85,7 +85,7 @@ restarts each service whose marker is behind HEAD *and* whose paths changed
since that marker — so a backlog accumulated by service-scoped deploys fires
all at once, and one failing health gate aborts the run with every later
service left pending. Name the service unless a drain is what you want
(`./scripts/deploy.sh dankserver --service frank-primary`).
(`./scripts/deploy.sh dankserver --service frank-archon`).
### Deployment completeness (MANDATORY)
@@ -226,20 +226,27 @@ archive.
## Benchmarking Policy (MANDATORY)
**Benchmarking is NOT part of the model-swap procedure** (2026-10-15). A
model change is verified by driving the live service — boot ledger,
per-request `timings`/`Server-Timing`, one real request per harness role
(`serverAppConfigs-docs/LLM/Model Swap and Rollback.md` §6/§8). No bench
window is scheduled for a swap, ever. The rest of this policy governs
*deliberate* bench windows (flag A/Bs, shape sweeps), which are separate
operations on a live host:
**STOP the production service BEFORE benchmarking** — these servers serve
real users, and a bench beside a live service starves the GPU and corrupts
results. Use the runtime's own harness: `llama-bench` for the llama.cpp
endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-large and frank-secondary were measured. Rolling your own bench
endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-dragoon and frank-zealot were measured. Rolling your own bench
requires asking first.
Scratch ports are not used on LLM hosts — a
test process cannot share the single GPU anyway; validate against the service
itself.
(A gufo-era law; frank-large ran ROCmFPX llama.cpp 2026-10-02 → 10-05, the
(A gufo-era law; frank-dragoon ran ROCmFPX llama.cpp 2026-10-02 → 10-05, the
halogen 27B container 2026-10-05 → 10-06, and is flash-server since, but the
rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode
settings, and will invert the ranking.**
Measured 2026-09-28 on frank-large at depth 32768: its corpus
Measured 2026-09-28 on frank-dragoon at depth 32768: its corpus
reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
draft widths, i.e. the same highly-predictable text every run, and 93 % of
every proposal accepted. It reported 29–37 t/s decode where real traffic
@@ -264,12 +271,12 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
merely wedged kernel usually still answers ARP.
1. Stop and mask (self-healing would restart it):
- Dankclaw `frank-large` (root unit, halogen podman on the iGPU):
`sudo systemctl stop frank-large && sudo systemctl mask frank-large`
- Dankclaw `frank-secondary` (root unit, llama.cpp on the 3090):
`sudo systemctl stop frank-secondary && sudo systemctl mask frank-secondary`
- Dankserver `frank-primary` (root unit, radiance on both R9700s, TP=2):
`sudo systemctl stop frank-primary && sudo systemctl mask frank-primary`
- Dankclaw `frank-dragoon` (root unit, halogen podman on the iGPU):
`sudo systemctl stop frank-dragoon && sudo systemctl mask frank-dragoon`
- Dankclaw `frank-zealot` (root unit, llama.cpp on the 3090):
`sudo systemctl stop frank-zealot && sudo systemctl mask frank-zealot`
- Dankserver `frank-archon` (root unit, radiance on both R9700s, TP=2):
`sudo systemctl stop frank-archon && sudo systemctl mask frank-archon`
2. Bench.
NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
`/etc/systemd/system/` is itself a symlink into the checkout — mask
@@ -286,9 +293,18 @@ deletes the unit file from `/etc/systemd/system/`, not just the symlink.
The harness reaches the fleet through `.omp/agent/models.yml` in the
operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
where the harness runs; there is no copy on the servers. Each provider is one
HOST:PORT; each model `id` is the served alias: `frank-primary` →
`192.168.0.235:7770`, `frank-large` → `192.168.0.143:7770`, `frank-secondary` →
`modelRoles`.
HOST:PORT; each model `id` is the served alias. **Official model ids are
the StarCraft unit names (2026-10-08), on the wire since 2026-10-09, and
since 2026-10-10 they are also the SERVICE names:** `frank-archon` →
`192.168.0.235:7770` (`FRANK_ARCHON_SERVED_NAME`), `frank-dragoon` →
`192.168.0.143:7770` (`HALOGEN_MODEL_ID`), `frank-zealot` →
`192.168.0.143:7771` (`ALIAS`). The 2026-10-10 cutover renamed the units,
scripts, env files, containers, deploy services and runbooks to match —
the three former service names are fully retired and exist nowhere in
the repo; historical text in runbooks uses the current names. Verify a
rename on `/v1/models`, never
on a chat response: radiance echoes the **requested** model name, so a 200
proves nothing about the served id (llama.cpp and halogen ignore it).
Resolution rules: the default model resolves as a `defaultProvider` +
`defaultModel` pair (the provider must serve the model); a subagent override
@@ -301,18 +317,20 @@ provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
`models.yml`, then fix every `modelRoles` entry that named the old endpoint —
a role pointing at a provider the file does not define fails at call time. As
of 2026-10-06 **all three endpoints accept images** (`input: [text,
image]`) — `frank-large` returned to vision with the flash-server v2 swap
image]`) — `frank-dragoon` returned to vision with the flash-server v2 swap
(the v2 vision tower loads beside the checkpoint; the text-only halogen
27B window closed 2026-10-06) — while
`frank-primary` takes them natively (radiance runs the checkpoint's
`frank-archon` takes them natively (radiance runs the checkpoint's
27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
are the engine core; `--mm-max-patches` available),
and `frank-secondary` through
the publisher's own `mmproj-BF16.gguf` passed as `--mmproj` — the Swift
GGUF itself carries no vision tensors. `frank-secondary`'s `contextWindow` is
**114688** (cut from 131072 on 2026-10-06 late to fund the ubatch ladder to 2048), the
per-slot window, not the model's native 262144: the harness must compact
before llama.cpp answers a longer prompt with a 400. Note also
and `frank-zealot` through
the publisher's own BF16 mmproj passed as `--mmproj` — the model is natively
multimodal but the Q6_K GGUF carries no vision tensors (same arrangement the
Gemma4 era used). `frank-zealot`'s `contextWindow` is
**200704** — the PER-SLOT window (`--ctx-size 401408 --parallel 2` since the
2026-10-07 Qwen3.5-9B swap), not the pool and not the model's native 262144:
the harness must compact before llama.cpp answers a longer prompt with a 400.
Note also
that
the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
parser, live-verified) emit `reasoning_content`, vLLM emits `reasoning` —
@@ -321,22 +339,22 @@ There is **no audio**
input anywhere in the fleet: that capability left with the FLM NPU
endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
(the MiMo tower is vision-only).
(neither the MiMo nor the Qwen3.5 tower carries audio).
### frank-large — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
### frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
driven by `scripts/serve-frank-large.sh` under a **root systemd unit**
(`scripts/frank-large.service`, `User=dank`, `Restart=always`,
`ExecStop=podman stop -t 60 frank-large-halogen`, the script runs `podman
driven by `scripts/serve-frank-dragoon.sh` under a **root systemd unit**
(`scripts/frank-dragoon.service`, `User=dank`, `Restart=always`,
`ExecStop=podman stop -t 60 frank-dragoon-halogen`, the script runs `podman
run` FOREGROUND — the container's exit code is the service's exit so
`Restart=always` stays real, and the engine ledger lands on disk) on port
7770 from the Strix Halo iGPU. Engine stdout goes to
`/home/dank/logs/frank_large.log` — **the boot ledger (weights resident, KV
`/home/dank/logs/frank_dragoon.log` — **the boot ledger (weights resident, KV
pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
Cutover 2026-10-06 from the halogen 27B-dense era
([[frank-large-halogen27b-dflash2]], retired the same window).
([[frank-dragoon-halogen27b-dflash2]], retired the same window).
Serving `Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`:
`qwen38-flash-next-v2-swift15-abliterated.hgn` (66,687,678,464 B ≈ 62.1
@@ -358,22 +376,29 @@ sidecar is deliberately not fetched).
**Product facts vs the 27B era, both regressions FIXED:** the base is
**abliterated** (the 27B was a censored-base finetune; refusals were an
accepted trade there) and **vision is ON** (the 27B engine silently
ignored its encoder — frank-secondary owned images for exactly one day at this
ignored its encoder — frank-zealot owned images for exactly one day at this
endpoint).
**Engine laws that shape the config** (all in `scripts/frank-large.env`
**Engine laws that shape the config** (all in `scripts/frank-dragoon.env`
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
policy disables the MTP head for whole requests after low acceptance and
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
for it (accepted). `HALOGEN_CTX=200192` is the PER-SLOT hard ceiling (400
naming the limit, never truncation); `models.yml` `contextWindow: 200192`
must match, and compaction at 70 % = 140,134 + 57,344 budget = 197,478 fits.
for it (accepted). `HALOGEN_CTX=262144` is the PER-SLOT hard ceiling (400
naming the limit, never truncation) — RAISED from 200192 on 2026-10-07
under the 400-zone law: the harness clamps max_tokens to
`min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts
the rendered prompt by ~3 % (measured: est 152,938 → server 157,388,
request 43,158 vs room 42,804), so a harness window == engine ceiling
made every request over ~146k est a guaranteed 400. `models.yml`
`contextWindow: 217088` leaves the 45k gap that absorbs the estimate
error (worst-case request ≈ 219k ≤ 262144); compaction fires at 85 % of
the harness window = 184,524 est ≈ 190k measured (Frank's 180k target).
`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
law does NOT carry over; batching and speculation coexist (the card's
quickstart ships 2 slots with the head), so 2 is concurrent capacity.
`HALOGEN_KV_POOL_POSITIONS=400384` (explicit 2 × ctx; ~28 KiB/position ⇒
~10.7 GiB). `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
harness always sends 57,344); `HALOGEN_MAX_TOKENS_CAP=65536` +
`HALOGEN_KV_POOL_POSITIONS=524288` (explicit 2 × ctx; ~28 KiB/position ⇒
14.0 GiB). `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
**A server-side sampler EXISTS** (`HALOGEN_TEMPERATURE/TOP_P/TOP_K` =
1.0/0.95/20, the card's thinking recipe) — an absent temperature is no
@@ -389,14 +414,14 @@ stays mandatory. Thinking tokens COUNT against the budget (empty content +
"video".
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger):
weights ~62 + KV ~10.7 + arena ~8 (`3.8 + 8.4 × MAX_TOK/16384`) + vision
~2 ≈ 82.7 GiB engine-side + `HALOGEN_HOST_RESERVE_GIB=20` → ~18 GiB
weights ~62 + KV 14.0 + arena ~8 (`3.8 + 8.4 × MAX_TOK/16384`) + vision
~2 ≈ 86 GiB engine-side + `HALOGEN_HOST_RESERVE_GIB=20` → ~15 GiB
margin over baseline co-tenancy. The 51.2 GB n-gram table rides as mmap'd
page cache beside the checkpoint — the card's quickstart arithmetic does
not bill it as resident, but READ the boot ledger and re-size if the
engine prints a clamp.
Fetch: `scripts/download-frank-large-flashnext.sh` into
Fetch: `scripts/download-frank-dragoon-flashnext.sh` into
`/home/dank/models/halogen-flashnext-v2/` (`-f` + exact byte counts + LFS
sha256 + `--verify-only`, no token on the command line; resume goes
against the RESOLVED CDN URL because curl drops `Range` across HF's 302).
@@ -405,164 +430,202 @@ n-gram + vision tower, flat tokenizer with all six files as regular
files) — the 2026-10-01 crash-loop lesson.
Runbook:
`serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md`.
`serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md`.
Rollback: revert the swap commit and redeploy — the 27B payload
(`/home/dank/models/halogen-qwen3.8-27b/`), its fetch script and the
halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
### frank-secondary — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
### frank-zealot — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
**llama.cpp master** (0.5.0-dev, commit `5fc4f3c`; CUDA sm_86 build
`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`,
`GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit**
(`scripts/frank-secondary.service`, `relink_unit`ed into
`/etc/systemd/system/`), serving `ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`
at **RCO-IQ3_XXS -mtp** (10,442,827,392 B, sha256 `e13d3e74…`) **plus
the publisher's own `mmproj-BF16.gguf`** (931,145,888 B, sha256
`19acd7fc…`), both `/home/dank/models/Swift15-UncIQ3XXS/`, on port 7771.
Cutover 2026-10-05 (evening) on Frank's explicit contract: **128k
context cap, KV at FP8, vision on, as many 128k contexts as fit**;
reshaped three times on 2026-10-06 — morning to two 131,072 slots
no-spec, evening to this UNCERSED RCO-IQ3_XXS payload with MTP back
on, late the RENAME to **frank-secondary** (87721ab) at two 114,688
slots with the ubatch ladder climbing 512 → 1024 → 2048 (f2575a1,
d8da7a9). The
same-day payloads it supersedes: the LIVE MiMo-V2.6 Q6_K shape and the
Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but never
deployed (blocked at the docs gate; the docs debt is what this section
settles).
(`scripts/frank-zealot.service`, `relink_unit`ed into
`/etc/systemd/system/`), serving
`HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive` at **Q6_K**
(7,359,259,008 B, sha256 `c0ba7beb…` computed locally — the Hub carries no
LFS oid) **plus the publisher's own BF16 mmproj** (921,704,448 B), both
`/home/dank/models/Qwen35-9B-HauhauCS-Q6K/`, on port 7771. Cutover
2026-10-07 (late) from the Gemma4-26B-A4B Q4_K_M 1×200,704 shape (same day).
**Architecture: qwen3_5 hybrid, bigger** — block_count 65 (64 real layers +
the blk.64 MTP head), `full_attention_interval 4` ⇒ **16 full-attention +
48 Gated DeltaNet**, native 262,144 window, dense. KV exists only in the 16
attention layers (4 KV heads × 256 dim): 64 KiB/token at f16, **34 KiB/token
at q8_0**. The DeltaNet state is a fixed per-slot buffer (~150 MiB/slot at
48 linear layers × 48 value heads). Header facts were read from the Hub
file by HTTP range + struct parse before anything was downloaded.
**Architecture: qwen35 dense hybrid** — read from the GGUF's own header, not
the model card: 32 layers, `full_attention_interval = 4` ⇒ **8 full-attention
layers (3, 7, 11, …, 31) + 24 Gated DeltaNet linear layers**, 16 attention
heads / **4 KV heads** × 256 dim, `ssm` conv_kernel 4 / state_size 128 /
group_count 16 / time_step_rank 32 / inner_size 4096, native 262,144, vocab
248,320. Tensor types are Q6_K × 250 + F32 × 177 — standard quants only, so
the stock master build loads it (arch `qwen35` is registered in this build's
`src/llama-arch.cpp`). **KV accounting:** the pool scales with the 8 full
layers only — 8 × 4 × 256 = 8,192 elements/token per cache, so f16 K + f16 V
bills **32 KiB/token** = 12.25 GiB at the 401,408 pool; the 24 DeltaNet
layers bill a fixed recurrent state per SLOT, not per token, which is exactly
why two 200k slots fit where the 26B MoE fit one.
**The shape (2026-10-06 midnight; Frank's tight-fit order, MTP back on
in the evening, per-slot cut + ubatch climbed to 2048):** `--ctx-size
229376 --parallel 2` = **TWO 114,688-token conversations** (Frank's
112k cap each; `n_ctx_slot = CTX_SIZE / PARALLEL`; cut from 131,072 to
fund the ubatch rungs), `--flash-attn on` pinned, **q8_0 K + q8_0 V**
(llama.cpp's FP8 class — no true fp8 KV dtype exists; 8.5 effective
bits; supersedes the same-day f16/f16 rule written for the Ornith 9B
fit), all layers on the GPU, `--mmproj` for vision, batch 4096 /
ubatch 2048. **Speculative decoding is ON** (draft-mtp, depth 3): the
-mtp file's head is drafted against the main model's own weights;
ngram-mod skipped on purpose — on a qwen35 hybrid ngram disables
recurrent-state rollback, so a rejected draft corrupts the DeltaNet
state.
**The shape (2026-10-07):** `--ctx-size 401408 --parallel 2` = TWO
conversations of 200,704 (Frank's cap; the harness `contextWindow` quotes
the per-slot number), `--flash-attn on`, **f16 K + f16 V** (exact KV),
batch 4096 / **ubatch 2048** (the measured CUDA optimum: +4.6 % prefill,
decode flat, 616 MiB — MiMo-era A/B on this card), `--log-verbosity 4`,
`--mmproj` for vision, **speculative decoding OFF** (`SPEC_TYPE=none`).
**Measured at the midnight boot (2026-10-06, 2 × 114,688 / ubatch
2048): 23,120 used / 1,005 MiB free, settling 763 MiB after a vision
probe** (`nvidia-smi` free; the encoder buffers stay resident on this
boot) — the per-slot cut freed ~1.05 GiB of KV and both ubatch rungs
spent it back (~0.3 + ~0.75 GiB measured; the 1024 rung had landed
22,374/1,751, the 2048 rung cost 746 MiB, ~130 over the 616 the rung
measured on the MiMo shape). The MTP draft ctx costs **~2.4 GiB
measured** (23,636 − 22,728 + 1,561 at the 262,144 draft window) — ~6×
the ~0.4 GiB the one-slot era assumed, because the drafter's KV is
built against the target's FULL context, not the 3-token proposal
depth; the lighter RCO-IQ3_XXS quant (−1.56 GiB vs IQ3_S-mtp) funds it.
**Prefill 1,209 t/s @ 3,760 tokens; decode 52.0–58.2 tok/s (recipe
temp 1.0) / 66.9 (temp 0)**, acceptance 34.2–54.4 % — decode is
ubatch-invariant, the spread is temp-1.0 acceptance noise.
`reasoning_effort` probe unchanged: `xhigh` 200, `high` → 500 Jinja
raise (the models.yml fold stays mandatory).
**Why MTP is off**: it is not a choice — **this Q6_K file carries no
`nextn`/`mtp` tensors** (verified from the tensor-name list), and MTP is a
property of the file, not of the flags. The publisher card advertises "MTP
support" for the base architecture; the quantized export dropped the head.
**Vision with the publisher's own projector**: ajgazin's repo SHIPS
`mmproj-BF16.gguf` in the same repo (the IQ3_S era's unsloth mmproj was
built from the SHARED BASE `Qwen/Qwen3.8-27B`, legal because Swift-1.5's
vision stack was hash-proven byte-identical to the base). The probe
traps carry over: ≥256 px probe images or the model answers "video";
only the OpenAI `image_url` content part reaches the model. **New on
this payload: the uncensored model thinks more than the censored
Swift 1.5** — a 64-token budget is eaten entirely by
`reasoning_content` (empty `content`); 128 tokens clears the vision
probe.
**Measured boots** (nvidia-smi, the referee): the deployed boot is
**21,436 used / 2,689 free** (ubatch 2048) — the pricing boots were the
MiMo-V2.6-Distill-Qwen-9B pair at this exact geometry (**21,276 / 2,849** at
ubatch 1024, **21,846 / 2,279** at 2048), so the 415 MiB lighter weights
bought back what the projection predicted. The engine ledger confirms the KV
law to the MiB: `CUDA0 KV buffer size = 12544.00 MiB` (`K (f16): 6272.00`,
`V (f16): 6272.00`), compute buffers 1,136 MiB on CUDA0 + 848 MiB host, the
mmproj 879 MiB + 248 MiB. Decode is ubatch-invariant (Swift-era 3090 law). The
Q6_K rung costs ~10 % decode vs Q5_K_M (weight bandwidth; MiMo-measured 94.4
vs 104.8 t/s).
The RCO-IQ3_XXS tier is a **per-tensor allocation mix** of stock quant
types (IQ3_S 97 / BF16 96 / IQ3_XXS 83 / IQ2_S 72 / IQ4_XS 33 / IQ2_XS 32
/ IQ2_XXS 30 / Q2_K 28 / Q4_K 19 / Q6_K 8 / IQ1_S 4 / IQ1_M 4) — stock
llama.cpp loads it; the tensor-type histogram is the law (see Key
Lessons). Publisher KLD vs Swift BF16: mean 0.002631 / 90th 0.042136 /
max 34.49 — RCO is the publisher's "more experimental" family, and the
payload is the UNCECERSED Swift 1.5 (23/100 refusals vs Swift 1.5's
98/100, KL 0.0884).
**Vision**: the publisher's own BF16 mmproj via `--mmproj` (the model is
natively multimodal; the Q6_K file carries no vision tensors). Probe traps
carry over: ≥256 px probe images or the model answers "video", and only the
OpenAI `image_url` content part reaches the model.
**Compaction hub**: `models.yml` routes `compactionModel` for
`frank-archon`, `frank-dragoon` and `frank-zealot` (the 3090 endpoint) here — 200,704 is the
only window in the fleet that fits a worst-case compaction input (~170k
tokens). With `--parallel 2` a compaction request no longer serializes behind
a live generation (the Gemma4 era's `--parallel 1` law).
Runbook:
`serverAppConfigs-docs/LLM/frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md`.
Rollback: `/home/dank/models/Swift15-IQ3S/` (the IQ3_S-mtp payload live
until the 2026-10-06 evening swap — revert da36fa3 for the two-slot
no-spec shape) or `/home/dank/models/MiMo-Q6K/` (revert to the
2×200,704 f16 shape — [[frank-secondary-llama-mimo-q6k-2x200704]]) or
`/home/dank/models/Ornith-Q8/` (the committed-never-deployed Q8_0;
revert to 380aafa's tree).
`serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md`.
Rollback: revert the swap commit — the Gemma4 payload
(`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload
(`/home/dank/models/MiMo-Q6K/`), the Swift payload
(`/home/dank/models/Swift15-UncIQ3XXS/`) and the Ornith Q8_0 payload
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
### frank-primary — Radiance (StillDeadcode), Qwen3.8-Flash-Next FP8 native, TP=2 (Dankserver, ACTIVE)
### frank-archon — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
The **radiance engine** (StillDeadcode, Apache-2.0; image
`stilldeadcode/radiance@sha256:4c3f484564bb2872784e4…` digest-pinned,
287 MiB, `libr4d@0.5.0` kernels prebuilt for gfx1201, engine as the image
entrypoint) serving the **official `Qwen/Qwen3.8-Flash-Next-FP8`
checkpoint at NATIVE precision** — modelopt block-FP8 throughout, no
requantisation, no calibration. Plain `podman run` through
`scripts/serve-frank-primary.sh` under the root systemd unit
(`frank-primary.service`, `User=dank`, `LimitMEMLOCK=infinity`); the
container name is the unit name (`frank-primary`). Cutover LIVE 2026-10-06
00:55 (chain built 2026-10-05/06);
rollback tag `frank-primary-freetoken-pr587-20261005` (the FreeToken trio
+ Dockerfile are in the tag; the image and the orcarouter payload stay on
disk until the window closes), older rollback `frank-primary-vllm-shapeG-20261003`.
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
for serve/convert/KLD; `libr4d` gfx1201 kernels, plain `podman run`, root
systemd unit, container name = unit name) serving
**ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, 131 shards,
397/397 sha256-verified)** converted by `scripts/convert-swift15-w4dn8.sh`
with recipe `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` to
`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ
w4** (act-order, damp 0.05, 10,000,000-token calibration captured on engine
**1.2.0** via `RADIANCE_CALIB_DIR`/`RADIANCE_CALIB_TOKENS` — the only pin
with the token knob; the RADGRAM2 grams are format-identical across pins and
cross the boundary by design), **24 whole experts per layer cast bf16** (the
PROTECTED set, derived from the calibration grams' energy share),
**int8 trunk/lm_head**, **fp8 HC mixtures** (`--hc-mix-dtype`), PLE n-gram
table at fp8 fixed scale (47.7 GiB in the container, from 95.4 GiB bf16).
The plugin's **one-table-one-encoding law** (measured at rad-convert
declare): one expert table takes ONE encoding; only whole bf16 experts are
legal beside it — no w4/fp8 split of the down/gate_up pair (24-value
`rad_arch_quant_desc` array, one per layer). KLD gate vs the BF16
reference (recorded on the same 1.0.13 pin) is the precision referee;
the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar
(122–173 t/s at 44.5k depth). Measured decode/prefill in the runbook.
Rollback: revert the swap commit — the fp8-native `.rad`
(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk.
**Payload chain**: official checkpoint → staging dir whose `config.json`
drops `quantization_config` → `stack-qwen38-official-fp8.py` (two
value-preserving reshapes: experts byte-stacked into the plugin's stacked
`gate_up_proj`/`down_proj` layout with `_scale_inv` at the same index — 49
stacks × 512 experts, no requantisation; PLE's 128 scaleless fp8 shards
decoded to one bf16 shard with the checkpoint's own bf16 scale
1.9931793212890625e-04) → `rad-convert` with ONE `--quant` rule — pure
copy for every weight, and the rtn rule re-encodes the bf16 PLE at that
same fixed scale, an exact round-trip that restores the original codes and
lands the plugin's one-element scale plane →
`/home/dank/models/rad/qwen4exp-fp8-native.rad` (177,109 MiB measured,
floor `FRANK_PRIMARY_RAD_MIN_MIB=177000`). The staging drop is the plugin
SELECTION law (spec §2.4): the descriptor is `(arch_id, quant)` matched
exactly, no shipped plugin claims `(qwen4exp, "fp8")`, the shipped
`qwen4exp_fp8` plugin claims `(qwen4exp, "")` — and the descriptor is only
the selection key, per-weight encodings are data, so nothing else is
requantised.
Raw serve AND unmodified convert both refuse; that is the design. The
checkpoint itself is sha256-verified against the Hub (131/131 shards,
2026-10-05). Built by `scripts/build-frank-primary-rad.sh` (`--plan-only`
first — its declare phase is the engine's own, so a resolving plan boots).
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
`--max-model-len 200000` (the harness contextWindow must quote this),
`--placement expert_tiered --host-pool-mib 44928` + **explicit VRAM
partition `--vram-weights-mib 25600 --vram-kv-mib 5120`** (the vram-pair
replaced `--expert-vs-cache-ratio`; the two are ONE budget —
`weights + kv ≤ 30,969 MiB`, derived from the engine's own budget table: card
32,624 MiB, the pair at 29,120 settling 2,058 MiB free against the engine's
209.19 MiB floor (`--gpu-headroom-mib 96` + 113.19 static), so 1,849 MiB is
claimable and the old 29,120 bound was my hand-picked 1 GiB reserve, not a
hardware wall), `--kv-cache-dtype` **`fp8` — quantized KV, restored 2026-10-13
because it is the only dtype that funds the slot count at this window**,
`--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`, prefix cache
8 GiB pinned host + 64 GiB disk at `/home/dank/radiance-state`,
**`--ngram-placement ram`**, Froggeric v22.5 chat template vendored
(`scripts/qwen38-froggeric-v22.5.jinja`, `--override-chat-template`). Boot
ledger (2026-10-13, pair 25600/5120): `kv cache 5.00 GiB … backs 3 sequences
all at 200000 tokens at once, which is the worst case` — `kv_attn` page
26.00 KiB (fp8) at 607,740 tokens (the sizing law predicted 608,027),
`kv_qsa_bkey` 3.25 KiB at the same token count, `kv_gdn_state` 54 MiB × 6 slots
(3 sequences × 2 copies) = 324 MiB; the sampler prints **9 rows**
(3 × (1+2), inside the 25-row `wht6` exactness bound); the slab filled to
563 KiB of slack with **17,548 slots** (16,235 at 24,000, 14,753 at 22,208 —
every 1,792 MiB buys ~1,300 resident experts, and the pinned host pool hands
the same experts back: 8,893 → 7,580 slots); `unclaimed 477.30 MiB`, cards
settled ~336 MiB free against the engine's 209.19 MiB floor, KV lending
4.22 GiB to the expert cache. Prefill tracks this slab only up to a
**threshold**: 22,208 → 3,397 and 24,000 → 3,290 t/s at 43.5k are one band
(±3 %), and the 25,600 arm measured **3,290** — the third step bought nothing
(the 3,584 arm did not reproduce at 3 slots); the cliff is at ~26 GiB
(26,071 → 4,379 on 2026-10-11's arm), so below that the slab is a threshold,
not a slope.
**Shape (both R9700s, TP=2)**: `--tp-wire wht6` (fused lossy all-reduce,
+5.6–6.3 % prefill / ~−1.3 % 8-stream decode at TP2), `--max-num-seqs 2`
(Frank's first boot; raise from the ledger + slot occupancy), `--max-model-len
200000` (native 262,144; the harness contextWindow must quote this — a longer
prompt is refused, never truncated), `--placement expert_tiered
--host-pool-mib 40960` (native FP8 experts are 112.8 GiB total, 56.4 GiB
per rank slice — the pinned pool is the price of native precision),
`--expert-vs-cache-ratio 0.82` (the measured 99 %-hit point; 0.87 collapses
to 0 %), `--kv-cache-dtype fp8` (**7.32 KiB/token**; a 200K sequence is
1.43 GiB), `--num-speculative-tokens 3` (the checkpoint's own MTP head —
3,101 `mtp.*` tensors, probe depth 3), `--max-num-batched-tokens 2048`,
prefix cache 4 GiB pinned host + 64 GiB disk at
`/home/dank/radiance-state`, and **`--ngram-placement ram`**: the 47.7 GiB
PLE n-gram table (51.2 GB = 29 % of all parameters) serves from pinned
host RAM. Froggeric v22.5 chat template vendored
(`scripts/qwen38-froggeric-v22.5.jinja`, `--override-chat-template`).
**Decode ladder (2026-10-13, 512 tokens, greedy, this shape)**: 99.0 t/s
single-stream at 43.5k, 93.3 at 197k — decode is nearly window-free (−6 % for
4.5× depth); **3 concurrent streams give 113.9/43.3/10.9 t/s = 18.4 t/s
aggregate**, so the slot count is paid in decode concurrency (a fused-step
lottery), not in prefill; leg D of `probe-frank-archon-kv.py` measures it.
**The pinned-RAM ledger is why this is a root unit**: 2 × 40 GiB pools +
47.7 GiB PLE + 4 GiB prefix host ≈ **134 GiB** charged against
`RLIMIT_MEMLOCK`. `ttm.pages_limit`/`pages_max` cap GPU-mappable pinned
pages (pre-cutover cap 123 GiB — BELOW the plan); the serve script's
TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to start until
both knobs are raised in the stop window (sysfs write first, kernel
cmdline + reboot only if read-only).
Shape notes carried from the fp8-native era and unchanged: wht6 gains,
MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
prefix cache, and the rule that a prompt over `--max-model-len` is
refused, never truncated. What changed: the fp8-native era's NVMe expert
streaming (the 3.5 GB/s wall) is gone — the w4 resident set fits the
host pool, so decode ran 122–173 t/s in the campaign arms and ~190 t/s
on the fp8 shape (512-token streamed probe). The campaign also
measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth,
accept 0.7246).
**The KV dtype law (measured 2026-10-11, applied at 4 slots 2026-10-13)**:
`--kv-cache-dtype` takes ONLY `bf16|fp8` — one dtype for the whole cache, so
llama.cpp's exact-K/quantized-V pair (what frank-zealot runs) has NO radiance
equivalent; the rung under bf16 is fp8 on both halves. Measured per rank from
the two boot ledgers: `kv_attn` (13 layers, 4-token blocks) pages 26.00 → 52.00
KiB, while `kv_qsa_bkey` (13 layers) stays 3.25 KiB at BOTH dtypes (the QSA
block key is an index, not attention state) — so a token costs 7.3125 KiB at fp8
and 13.8125 KiB at bf16: **1.89×, not 2×**. Pool sizing must also pay the
dtype-independent terms that live in the same pool (~110 MiB a sequence of
recurrent state + 448 MiB of prefix checkpoints):
`(vram_kv − 110·seqs − 448) MiB / rate` — bf16 at 6,912 gives 462,946 where the
engine reported 462,752 (0.04 % page rounding); fp8 at 6,912 with four slots
gives 843,566, i.e. the deployed capacity contract 4 × 200,000. **That is why
the dtype is fp8: the pool must back `seqs × max-model-len` as a hard limit, and
bf16 at four slots costs 11,679 MiB — the slab would fall to 17,441 MiB (−33 %),
where fp8 leaves it at 22,208.** The cost is paid twice on this engine, and
half-refunded: idle KV VRAM is also the expert cache's spillover, and the bigger
pool *lends* 6.08 GiB to the slab where the 3,543 MiB fp8 pool lent 2.69 GiB.
`--max-num-seqs` rides 4 × (1 + spec) = 12 fused-all-reduce rows, under the
25-row bound where wht6 is bit-exact (the "seqs 2 cannot boot" law belongs to
the retired tcclaviger fork's torch.compile, not radiance). Disk-tier sizing
law: the engine forces ~9.6 GiB of disk-tier weight buffers + arena onto the
weights side, so `weights + kv` is capped near 29.6 GiB/card; boot with the
pair, never `--kv-cache-size-mib`.
Shape notes carried from the fp8-native era and unchanged: wht6 gains,
MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
prefix cache, and the rule that a prompt over `--max-model-len` is
refused, never truncated. What changed: the fp8-native era's NVMe expert
streaming (the 3.5 GB/s wall) is gone — the w4 resident set fits the
host pool, so decode ran 122–173 t/s in the campaign arms and ~190 t/s
on the fp8 shape (512-token streamed probe). The campaign also
measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth,
accept 0.7246).
**The pinned-RAM ledger is why this is a root unit**: 2 × 44 GiB expert
pools + the 47.7 GiB n-gram copy (`--ngram-placement ram` is pinned RAM
the card reads) + 8 GiB prefix host ≈ **147 GiB** charged against
`RLIMIT_MEMLOCK` — the TTM floor is set just above it.
`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages, and **the knob is
runtime state whose default is half of RAM**: `scripts/ttm-preflight.sh` (sourced
by the serve script) therefore RAISES it to `FRANK_ARCHON_TTM_TARGET_MIB=204800`
before refusing, because the 2026-10-12 power failure reset it to 125,988 MiB
and the preflight re-refused 6,919 times. A manual sysfs write in the stop
window is a convenience, not the repair; kernel cmdline persistence
(`ttm.pages_limit=52428800`) is the belt to it.
**Contract facts** (source-verified): the HTTP listener binds AFTER the
full load, so a bare `curl /health` gate is honest (`/health` == `/ping`,
@@ -574,10 +637,11 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`.
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`.
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`.
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
## Key Lessons Learned
Durable operational lessons from the fleet's history. Full detail in the
@@ -739,11 +803,22 @@ linked runbooks.
- Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff`
of the old one, or live pages must land in RAM that `MemAvailable`
overstates. Dankclaw swap (64 GiB at `/swap.img`, fstab-driven) protects
the co-tenants, not frank-large — pinned weights don't swap, and paging
the co-tenants, not frank-dragoon — pinned weights don't swap, and paging
the KV pool is worse than the OOM it prevents.
- Host swap state is invisible to `deploy.sh` (no repo file) — check
`swapon --show` before reasoning about headroom.
- **Silent RAM corruption outranks every software theory.** frank-primary's
- **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults
to half of RAM, so the manual raise to 200 GiB (2026-10-05) vanished with the
2026-10-12 23:17 POWER FAILURE on dankserver: the cap fell to 125,988 MiB,
frank-archon's pinned-ledger preflight (floor 150,000 MiB) refused the start,
and `Restart=always` re-ran that refusal **6,919 times in ~20 h** — a unit in
`activating (auto-restart)` that exits 1 in 50 ms with no container and no
engine log is a PREFLIGHT refusal, not an engine fault. Repo law now:
`scripts/ttm-preflight.sh` (sourced by `serve-frank-archon.sh`) raises the knob
to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only
after a write the kernel rejected. Never prescribe a manual sysfs write as the
recovery procedure for a service that can boot before a human reaches it.
- **Silent RAM corruption outranks every software theory.** frank-archon's
~40k-token output corruption (2026-10-04/05, repetition/garbage in long
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and
@@ -766,7 +841,7 @@ linked runbooks.
`LimitMEMLOCK=infinity` (systemd is privileged; the value flows through
podman to runc without any raise). Upstream halogen's own run lines pass
NO `--ulimit` at all — copy vendor run lines before adding flags
(measured 2026-10-05 on the frank-large cutover).
(measured 2026-10-05 on the frank-dragoon cutover).
- **Kairic Edge** (if ever re-deployed): the pinned CK commit needs
`third_party/composable_kernel/include/ck/config.h` generated manually
(untracked; `git clean` removes it), and its greedy-argmax fastpath rejects
@@ -792,7 +867,7 @@ linked runbooks.
121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
Record the child pid and kill it in a `finally`, then prove `free` came back
before believing a number. Kill by pid — never `pkill -f llama-server`,
which also hits frank-secondary.
which also hits frank-zealot.
**llama.cpp serving**
@@ -816,7 +891,7 @@ linked runbooks.
builds its context against whatever model the draft context holds. Two
context-backed types cannot share one process, and MTP's head only loads
when MTP **is** the draft model. **A LEGAL chain can still lose on a
recurrent model:** `draft-mtp,ngram-mod` on frank-large's qwen35 hybrid
recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35 hybrid
measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer
logs `recurrent state rollback is not compatible with 'ngram-mod' -
disabling rollback support`, and MTP verification depends on that rollback,
@@ -858,7 +933,7 @@ linked runbooks.
free. That was a CONTIGUITY failure, not a total-bytes one, and no totals
model — the sweep's or the seam's — can see it.)
- **On the CUDA 3090 the same knob has NO interior optimum up to 2048, so
frank-secondary ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the
frank-zealot ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the
live service: at 8,928 prompt tokens ubatch 1024 gives 3,972 / 4,077 t/s and
ubatch 2048 gives 4,207 / 4,212 — **+4.6 %** — for **616 MiB** (free 2,849 →
2,279). Decode did not move (91.9–93.0 vs 93.8–94.3, i.e. noise), and a
@@ -867,9 +942,9 @@ linked runbooks.
than the ubatch cannot see it. The logical `--batch-size` was again free. And
the predecessor's "ubatch 2048 failed the scratch boot" was a shape
observation, not a law: the same file boots 2048 clean at 2 × 200,704. Bench
it by restarting the SERVICE through a transient `FRANK_SECONDARY_ENV_FILE`
it by restarting the SERVICE through a transient `FRANK_ZEALOT_ENV_FILE`
drop-in, never a scratch port — one card holds one server. See
[[frank-secondary-llama-mimo-q6k-2x200704]].
[[frank-zealot-llama-mimo-q6k-2x200704]].
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
size.** qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is
worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on a full-attention one
@@ -891,7 +966,7 @@ linked runbooks.
CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT
(`400 exceed_context_size_error: request (184001 tokens) exceeds the
available context size (180224 tokens)`) even when the pool has room. So
frank-secondary's 2 → 3 reslice moved each conversation from the model's native
frank-zealot's 2 → 3 reslice moved each conversation from the model's native
262,144 down to 180,224, and `models.yml` `contextWindow` had to follow or
the harness would discover the ceiling as a 400 on a long conversation. Side
effect worth knowing: the shorter slot made decode ~5 % FASTER (106.8 vs
@@ -902,9 +977,9 @@ linked runbooks.
day the freed space also bought the Q6_K rung (now 94.4 tok/s — the rung
itself is a further measured −10 %, weight-bandwidth, not slot geometry).
All moves measured; both slot directions cost or buy window, never pool.
See [[frank-secondary-llama-mimo-q6k-2x200704]].
See [[frank-zealot-llama-mimo-q6k-2x200704]].
- **Price a VRAM budget from two boots, never from a formula — and an mmproj
costs ~1.5x its file size.** The frank-secondary 3090 fit was solved from two
costs ~1.5x its file size.** The frank-zealot 3090 fit was solved from two
measurements: 2 × 262,144 with no projector = 20,190 MiB used (KV 12,800 ⇒
**non-KV 7,390**), 3 × 196,608 with the f16 projector = 22,972 used (KV
14,113 ⇒ **non-KV 8,859**). The difference is the vision tower: a 918 MB file
@@ -912,24 +987,24 @@ linked runbooks.
ride with it. Subtracting the two rows is exact; extrapolating weights+KV+
"a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7. Then
solve `KV ≤ card − non-KV − headroom` and round to a multiple of
`PARALLEL × 4096`. Recorded in `scripts/frank-secondary.env` so the next resize
`PARALLEL × 4096`. Recorded in `scripts/frank-zealot.env` so the next resize
starts from measurement.
- **`images` and `image_url` are NOT interchangeable: only the content part
reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
model still invents an answer if the client used the legacy top-level
`images` field on `/v1/chat/completions` — frank-secondary answered "Brown." for
`images` field on `/v1/chat/completions` — frank-zealot answered "Brown." for
a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
failure is silent because a hallucinated color is a plausible answer. Probe
vision with a solid color and expect an exact word.
- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
`500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
then served frank-large: every tool-using agent on that endpoint was broken
then served frank-dragoon: every tool-using agent on that endpoint was broken
and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults
`--jinja` on and the serve script pins it; halogen also accepts `tools` and
returns `tool_calls`. Check tool support before routing a role at an endpoint.
- **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
option and the same ~12x prefill collapse described in the ROCm note above.
f16 KV needs nothing; frank-secondary ran f16 for that reason until
f16 KV needs nothing; frank-zealot ran f16 for that reason until
2026-09-17, when `build-cuda` was rebuilt with the option ON so the
endpoint could take q8_0 KV.
- **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both
@@ -947,7 +1022,7 @@ linked runbooks.
raises on anything else — HTTP 500, not a clamp — so the harness's levels
must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`).
A role whose level is unmapped fails on its FIRST request, and the failure
names Jinja, not the model: every subagent on `frank-secondary` died with
names Jinja, not the model: every subagent on `frank-zealot` died with
`Unexpected reasoning effort high` until `high: xhigh` was added on
2026-09-16. Halogen accepts the wider set, so the two entries legitimately
differ.
@@ -967,7 +1042,7 @@ linked runbooks.
times (`RuntimeError: No HIP GPUs are available`).
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
ports off the gluetun service.
- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-large,
- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-dragoon,
2026-09-11) permanently retired the XDNA2 NPU: the driver's SVA/PASID bind
needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). Revert path
in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
@@ -1021,9 +1096,9 @@ Active:
| Service | Runbook |
|---------|---------|
| frank-large (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×200,192, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md` |
| frank-secondary (llama.cpp CUDA, Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV, Dankclaw) | `serverAppConfigs-docs/LLM/frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
| frank-primary (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md` |
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
| frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` |
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
@@ -1031,40 +1106,41 @@ Retired (rollback targets and history):
| Runtime | Host | Window | Runbook |
|---------|------|--------|---------|
| halogen 0.1.4 Qwen3.8-27B p1w4d-d2 + DFlash2 (`frank-large`) | Dankclaw | 2026-10-05 → 10-06 (rollback target — payload + fetch script + image digest on disk; owns the prompt-cache pin and FA-tile lessons) | `frank-large-halogen27b-dflash2.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-primary`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-primary-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-primary`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-primary-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-primary-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-primary-freetoken-pr587.md` |
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-secondary-llama-mimo-q6k-2x200704.md` |
| llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` |
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` |
| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-secondary`) | Dankclaw | 2026-10-01 → 10-02 (the rung returned ACTIVE 2026-10-03 at the 2×200,704 shape — `frank-secondary-llama-mimo-q6k-2x200704.md`) | `frank-secondary-llama-mimo-q6k.md` |
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q5_K_M 2×200,704 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-02 → 10-03 (rollback target — one `MODEL_FILE` line, buys back 875 MiB + the −10 % rung tax; owns the boot-pair table) | `frank-secondary-llama-mimo-q5km.md` |
| SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-secondary`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-secondary-sglang-mimo-v26.md` |
|tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-primary`)|Dankserver|2026-09-30 → 10-01|`frank-primary-glm53-flash-w4a16.md`|
| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-21 → 09-28 | `frank-large-unsloth-qwen38-27b-q8xl.md` |
| R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-primary`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-primary-r9v-radiance-qwen38-uncensored.md` |
| llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-primary`) | Dankserver | 2026-09-26 → 09-27 | `frank-primary-llamacpp-qwen38-flash-next-q8.md` |
| ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-large`) | Dankclaw | 2026-09-15 → 09-16 | `frank-large-ik-uncensored.md` |
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-zealot-llama-mimo-q6k-2x200704.md` |
| llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md` |
| llama.cpp CUDA Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV (`frank-zealot`) | Dankclaw | 2026-10-06 → 2026-10-07 (rollback target — payload on disk; owns the 3090 ubatch ladder A/B and the 34 KiB/token qwen35 KV law) | `frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-dragoon-rocmfpx-q8agent-mtp.md` |
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-dragoon`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-dragoon-gufo-qwen38-27b.md` |
| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-dragoon`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-dragoon-vllm-mimo-v26-bf16.md` |
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-zealot`) | Dankclaw | 2026-10-01 → 10-02 (the rung returned ACTIVE 2026-10-03 at the 2×200,704 shape — `frank-zealot-llama-mimo-q6k-2x200704.md`) | `frank-zealot-llama-mimo-q6k.md` |
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q5_K_M 2×200,704 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-02 → 10-03 (rollback target — one `MODEL_FILE` line, buys back 875 MiB + the −10 % rung tax; owns the boot-pair table) | `frank-zealot-llama-mimo-q5km.md` |
| SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-zealot`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-zealot-sglang-mimo-v26.md` |
|tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-archon`)|Dankserver|2026-09-30 → 10-01|`frank-archon-glm53-flash-w4a16.md`|
| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-dragoon`) | Dankclaw | 2026-09-21 → 09-28 | `frank-dragoon-unsloth-qwen38-27b-q8xl.md` |
| R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-archon`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-archon-r9v-radiance-qwen38-uncensored.md` |
| llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-archon`) | Dankserver | 2026-09-26 → 09-27 | `frank-archon-llamacpp-qwen38-flash-next-q8.md` |
| ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-dragoon`) | Dankclaw | 2026-09-15 → 09-16 | `frank-dragoon-ik-uncensored.md` |
| vllm-radiance TP=2 MXFP4 + DFlash2 (`frank-medium`) | Dankserver | 2026-09-20 → 09-26 | `frank-medium-radiance-mxfp4.md` |
| Lucebox DS4.1 DeepSeek-V4.1-Flash ROCmFP23 (`frank-primary`) | Dankserver | 2026-09-26 (same-day swap) | `frank-primary-lucebox-ds4.md` |
| Lucebox DS4.1 DeepSeek-V4.1-Flash ROCmFP23 (`frank-archon`) | Dankserver | 2026-09-26 (same-day swap) | `frank-archon-lucebox-ds4.md` |
| kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
| halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-large`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` |
| CIRU v4.4.1 Orca Qwen3.8-Flash (`frank-large`) | Dankclaw | 2026-09-19 → 09-21 | `frank-large-ciru-orca.md` |
| unsloth llama.cpp b11030 Qwen3.8-Flash-Next UD-IQ4_XS (`frank-large`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-large-unsloth-qwen38-flash-next.md` |
| Kairic Edge Qwen3.8-27B IU4 W4A4 (`frank-large`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-large-kairic-edge-qwen38-27b.md` |
| ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-secondary`) | Dankclaw | 2026-09-16 → 09-17 | `frank-secondary-qwen38-gsq-rco.md` |
| vLLM Gemma4-12B w4a16 (`frank-secondary`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-secondary-vllm-gemma4.md` |
| llama.cpp Gemma4-12B QAT Q4_K_M (`frank-secondary`) | Dankclaw | 2026-09-17 → 10-01 | `frank-secondary-gemma4-12b.md` |
| halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-dragoon`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` |
| CIRU v4.4.1 Orca Qwen3.8-Flash (`frank-dragoon`) | Dankclaw | 2026-09-19 → 09-21 | `frank-dragoon-ciru-orca.md` |
| unsloth llama.cpp b11030 Qwen3.8-Flash-Next UD-IQ4_XS (`frank-dragoon`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-dragoon-unsloth-qwen38-flash-next.md` |
| Kairic Edge Qwen3.8-27B IU4 W4A4 (`frank-dragoon`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-dragoon-kairic-edge-qwen38-27b.md` |
| ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-zealot`) | Dankclaw | 2026-09-16 → 09-17 | `frank-zealot-qwen38-gsq-rco.md` |
| vLLM Gemma4-12B w4a16 (`frank-zealot`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-zealot-vllm-gemma4.md` |
| llama.cpp Gemma4-12B QAT Q4_K_M (`frank-zealot`) | Dankclaw | 2026-09-17 → 10-01 | `frank-zealot-gemma4-12b.md` |
| agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` |
| eemin Qwen3.8-27B ROCmI4 W4A4 MTP | Dankclaw | 2026-08-18 → 08-25 | `eemin-rocmi4-mtp.md` |
| Kairic Edge IU4 | Dankclaw | 2026-08-22 (same-day revert) | `kairic-edge.md` |
| CIRU ActiveFPX v3 / PromptForge | Dankclaw | through 2026-08-22 | `ciru-activefpx-promptforge.md` |
| DFlash2 Vulkan | Dankclaw | 2026-08-21 (same-day revert) | `dflash2-large-vulkan.md` |
| FastFlowLM NPU (`frank-npu`) | Dankclaw | twice, retired 2026-09-11 | `frank-npu.md`, `FastFlowLM NPU Inference.md` |
| FreeToken (official + Huihui FP8, `frank-secondary`) | Dankserver | 2026-09-02 → 09-15 | `freetoken-frank-small.md` |
| FreeToken (official + Huihui FP8, `frank-zealot`) | Dankserver | 2026-09-02 → 09-15 | `freetoken-frank-small.md` |
| FastMTP patched llama.cpp | Dankserver | retired 2026-08-18 | `qwen3.8-spec-matrix.md` |
| ik_llama.cpp Ornith CRACK | Dankserver | retired 2026-09-02 | `ikllama-ornith-dankserver.md` |
| llama.cpp layer split + DFlash2 (`frank-medium`) | Dankserver | 2026-09-16 → 09-20 | `frank-medium-llamacpp-dflash2.md` |
@@ -1088,7 +1164,7 @@ history.
| Unstaged local changes | **Commit or discard** — check `git status` before starting work; ask if unsure. |
| Deploying a committed change | **`scripts/deploy.sh <target>`** — pushes, pulls ff-only, plans, dispatches, health-checks. |
| Container without a Quadlet unit | **Give it one** — boot-persistent, self-healing `.container` via git. |
| Benchmarking | **Stop and mask the service first**, bench, unmask, restart, verify health. |
| Benchmarking | **Not part of a model swap** — swaps verify on the live service. A deliberate bench window: **stop and mask the service first**, bench, unmask, restart, verify health. |
| Deleting a file | **Move it to trash** — `.AGENTTRASH/` (in-repo) or `~/.AGENTTRASH/YYYYMMDD-topic`. Never `rm`. |
| Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
| Real secrets | **Never commit** — host `.env` files; `.env.example` holds placeholders only. |
+2 -2
View File
@@ -13,7 +13,7 @@ one deploy script.
| Host | User | IP | Podman stack | LLM service | LLM port |
|------|------|----|--------------|-------------|----------|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-dragoon (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
## Active Services
@@ -30,7 +30,7 @@ one deploy script.
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-large | dankclaw | — (bare endpoint) | 7770 |
| frank-dragoon | dankclaw | — (bare endpoint) | 7770 |
## Structure
+9 -9
View File
@@ -1,26 +1,26 @@
#!/usr/bin/env bash
# autoresearch.sh — canonical frank-primary benchmark entry point (dev box).
# autoresearch.sh — canonical frank-archon benchmark entry point (dev box).
#
# The service runs on dankserver (192.168.0.235:7770), the two R9700s live
# there, and the benchmark must own those cards, so the whole stop -> boot
# scratch -> bench -> restore sequence runs ON dankserver (see
# scripts/autoresearch-frank-primary.sh). This wrapper:
# 1. copies the candidate config (the repo's scripts/frank-primary.env, the
# scripts/autoresearch-frank-archon.sh). This wrapper:
# 1. copies the candidate config (the repo's scripts/frank-archon.env, the
# file the autoresearch loop edits) and the bench script to dankserver,
# 2. runs the bench over ssh, relaying its stdout (the METRIC lines) and
# exit code back here verbatim.
# It deliberately does NOT push to origin: the scp dirties dankserver's
# working tree, and the loop's uncommitted candidate config must not be
# pushed. Only this file and scripts/autoresearch-frank-primary.sh are
# pushed. Only this file and scripts/autoresearch-frank-archon.sh are
# committed; the candidate env rides over ssh each run.
set -uo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REMOTE=dankserver
REMOTE_REPO=/home/dank/serverAppConfigs
scp -q "$DIR/scripts/frank-primary.env" \
"$REMOTE:$REMOTE_REPO/scripts/frank-primary.env" || { echo "scp env failed" >&2; exit 1; }
scp -q "$DIR/scripts/autoresearch-frank-primary.sh" \
"$REMOTE:$REMOTE_REPO/scripts/autoresearch-frank-primary.sh" || { echo "scp script failed" >&2; exit 1; }
scp -q "$DIR/scripts/frank-archon.env" \
"$REMOTE:$REMOTE_REPO/scripts/frank-archon.env" || { echo "scp env failed" >&2; exit 1; }
scp -q "$DIR/scripts/autoresearch-frank-archon.sh" \
"$REMOTE:$REMOTE_REPO/scripts/autoresearch-frank-archon.sh" || { echo "scp script failed" >&2; exit 1; }
ssh "$REMOTE" "bash $REMOTE_REPO/scripts/autoresearch-frank-primary.sh $REMOTE_REPO/scripts/frank-primary.env"
ssh "$REMOTE" "bash $REMOTE_REPO/scripts/autoresearch-frank-archon.sh $REMOTE_REPO/scripts/frank-archon.env"
+10 -1
View File
@@ -337,7 +337,16 @@ http://:8017 {
try_files /docs.html
file_server
}
@data path /services.json /status.json /app.js /style.css /deployed.js
# /whoami echoes the outpost-injected identity headers so the page
# can gate the admin button client-side (the /admin* 403 above stays
# the real gate; this only decides what is shown). The outpost strips
# client-supplied X-Authentik-* before injecting, so this is spoof-proof.
handle /whoami {
templates
header Content-Type application/json
respond "{\"user\":\"{header.X-Authentik-Username}\",\"groups\":\"{header.X-Authentik-Groups}\"}"
}
@data path /services.json /status.json /app.js /style.css /deployed.js /img/* /sfx/*
handle @data {
file_server
}
+6 -6
View File
@@ -10,15 +10,15 @@ model:
provider: dankclaw-large
base_url: http://192.168.0.143:7770/v1
api_key: not-needed
default: frank-large
default: frank-dragoon
providers:
dankclaw-large:
api: http://192.168.0.143:7770/v1
discover_models: false
models:
frank-large:
context_length: 114688
frank-dragoon:
context_length: 262144
dankserver-small:
api: http://192.168.0.235:7770/v1
discover_models: false
@@ -29,7 +29,7 @@ providers:
# Auxiliary routing from the live configuration. Named providers route via
# their `api:` URL; `custom` entries route by their explicit `base_url`.
auxiliary:
vision: {provider: dankclaw-large, model: frank-large}
vision: {provider: dankclaw-large, model: frank-dragoon}
web_extract: {provider: dankserver-small, model: frank-medium}
compression: {provider: dankserver-small, model: frank-medium}
skills_hub: {provider: dankserver-small, model: frank-medium}
@@ -50,8 +50,8 @@ auxiliary:
effort: none
memory_query_rewrite: {provider: custom, model: frank-medium, base_url: http://192.168.0.235:7770/v1}
tts_audio_tags: {provider: custom, model: frank-medium, base_url: http://192.168.0.235:7770/v1}
triage_specifier: {provider: dankclaw-large, model: frank-large}
kanban_decomposer: {provider: dankclaw-large, model: frank-large}
triage_specifier: {provider: dankclaw-large, model: frank-dragoon}
kanban_decomposer: {provider: dankclaw-large, model: frank-dragoon}
profile_describer: {provider: dankserver-small, model: frank-medium}
curator: {provider: dankserver-small, model: frank-medium}
+1 -1
View File
File diff suppressed because one or more lines are too long
@@ -1,9 +1,9 @@
#!/usr/bin/env bash
# download-frank-primary-fp8.sh — fetch/verify the Qwen3.8-Flash-Next-FP8
# download-frank-archon-fp8.sh — fetch/verify the Qwen3.8-Flash-Next-FP8
# payload (131 safetensors shards, 172.78 GiB total, plus aux files) for the
# vLLM frank-primary deployment on dankserver.
# vLLM frank-archon deployment on dankserver.
#
# bash scripts/download-frank-primary-fp8.sh [dest_dir]
# bash scripts/download-frank-archon-fp8.sh [dest_dir]
#
# The manifest (scripts/manifests/qwen3.8-flash-next-fp8.manifest) carries
# per-file sizes + LFS sha256s from the Hub; hf-fetch.sh is idempotent — an
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Download the z-lab Qwen3.5-9B-DFlash drafter for frank-large's DFlash
# Download the z-lab Qwen3.5-9B-DFlash drafter for frank-dragoon's DFlash
# speculative-decode arm (vLLM --speculative-config method=dflash). Public
# repo — no auth header on the command line.
#
@@ -10,7 +10,7 @@
# the draft for that target.
#
# Usage:
# scripts/download-frank-large-dflash.sh [DEST_DIR] [--verify-only]
# scripts/download-frank-dragoon-dflash.sh [DEST_DIR] [--verify-only]
set -euo pipefail
REPO="z-lab/Qwen3.5-9B-DFlash"
@@ -1,14 +1,14 @@
#!/usr/bin/env bash
# Download the BF16 MiMo-V2.6-Distill-Qwen-9B payload for frank-large
# Download the BF16 MiMo-V2.6-Distill-Qwen-9B payload for frank-dragoon
# (vLLM/kyuz0 gfx1151, Strix Halo iGPU). Public repo — no auth header on the
# command line (a Bearer token in argv is world-readable via /proc).
#
# Per-file FRESH downloads, no resume: the house rule (see
# download-frank-secondary.sh); -f/--fail-with-body plus size+sha256 against the
# download-frank-zealot.sh); -f/--fail-with-body plus size+sha256 against the
# Hub LFS oid is what makes an access error fail instead of "succeeding".
#
# Usage:
# scripts/download-frank-large-mimo.sh [DEST_DIR] [--verify-only]
# scripts/download-frank-dragoon-mimo.sh [DEST_DIR] [--verify-only]
#
# The verified set below is exactly what the 2026-10-01 bench window served:
# 17.53 GiB of BF16 weights (4 shards, 18,819,720,848 B total with the rest).
@@ -1,14 +1,14 @@
#!/usr/bin/env bash
# Download the source Qwen3.5-9B checkpoint — the ONLY place this model
# family's MTP head (15 mtp.* tensors) exists. frank-large runs it as the
# family's MTP head (15 mtp.* tensors) exists. frank-dragoon runs it as the
# vLLM cross-model MTP draft (MiMo-V2.6-Distill-Qwen-9B itself ships zero
# mtp tensors; that is why SGLang's NEXTN on frank-secondary draws 0.00
# mtp tensors; that is why SGLang's NEXTN on frank-zealot draws 0.00
# acceptance). Public repo — no auth header on the command line.
#
# NOTE the odd upstream shard names (model.safetensors-0000N-of-00004...).
#
# Usage:
# scripts/download-frank-large-qwen35.sh [DEST_DIR] [--verify-only]
# scripts/download-frank-dragoon-qwen35.sh [DEST_DIR] [--verify-only]
set -euo pipefail
REPO="Qwen/Qwen3.5-9B"
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-secondary
# Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-zealot
# (SGLang, RTX 3090). Public repo — no auth header on the command line.
#
# Per-file FRESH downloads, no resume: `curl -C -` once produced a 213 MB
@@ -7,7 +7,7 @@
# the house rule.
#
# Usage:
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only]
# scripts/download-frank-zealot.sh [DEST_DIR] [--verify-only]
#
# LFS shards are verified against the Hub LFS sha256; non-LFS text files
# against their declared byte size. Idempotent: complete files are skipped.
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# bench-frank-large.sh — 64k-context benchmark for the frank-default-large stack.
# bench-frank-dragoon.sh — 64k-context benchmark for the frank-default-large stack.
#
# Measures generation throughput of Qwen3.6-27B Q8_K_P + DFlash + ngram-mod on
# a single backend at 64k context. It starts llama-server on a scratch port,
@@ -7,15 +7,15 @@
# prompt, and prints tokens/sec.
#
# Usage:
# ./scripts/bench-frank-large.sh <ROCm0|Vulkan0> [--generations N]
# ./scripts/bench-frank-dragoon.sh <ROCm0|Vulkan0> [--generations N]
#
# Example:
# ./scripts/bench-frank-large.sh ROCm0
# ./scripts/bench-frank-large.sh Vulkan0 --generations 3
# ./scripts/bench-frank-dragoon.sh ROCm0
# ./scripts/bench-frank-dragoon.sh Vulkan0 --generations 3
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DEVICE="${1:?usage: bench-frank-large.sh <ROCm0|Vulkan0> [--generations N]}"
DEVICE="${1:?usage: bench-frank-dragoon.sh <ROCm0|Vulkan0> [--generations N]}"
GENERATIONS=1
if [ "${2:-}" = "--generations" ]; then
GENERATIONS="${3:?--generations needs a count}"
+1 -1
View File
@@ -41,7 +41,7 @@ SERVICE_HEALTH[dankclaw]="http://127.0.0.1:7770/health"
# Drop page caches before restart: with the model's pages fully cached, the
# KV/graph allocation stalls at load (VRAM stuck at 28 GiB). See AGENTS.md.
DROP_CACHES[dankclaw]=1
# Passwordless restart helper (root-owned, installed via /etc/sudoers.d/frank-large).
# Passwordless restart helper (root-owned, installed via /etc/sudoers.d/frank-dragoon).
# When present, deploy.sh uses it instead of inline sudo commands.
HELPER[dankclaw]="/usr/local/sbin/frank-restart-large.sh"
# Health wait in iterations of 2s. Cold loads after the cache drop measured
+2 -2
View File
@@ -1,6 +1,6 @@
# Tunables for the Ling-3.0-flash TRAIL on dankclaw (frank-large-ling).
# Tunables for the Ling-3.0-flash TRAIL on dankclaw (frank-dragoon-ling).
# This REPLACES frank-default-large (Qwen3.6-27B) temporarily. To switch
# back: systemctl stop frank-large-ling && systemctl start frank-default-large.
# back: systemctl stop frank-dragoon-ling && systemctl start frank-default-large.
# Runtime: llama.cpp FORK bailingmoe3-support; Vulkan + AMD GTT.
#
# Memory model (amdgpu Strix Halo gfx1151, 8x16GB LPDDR5 = 128 GiB physical):
@@ -1,5 +1,5 @@
[Unit]
Description=llama.cpp Ling-3.0-flash Q6_K Vulkan+MTP (frank-large-ling TRIAL)
Description=llama.cpp Ling-3.0-flash Q6_K Vulkan+MTP (frank-dragoon-ling TRIAL)
After=network.target
[Service]
+1 -1
View File
@@ -8,7 +8,7 @@
# Vulkan exposes as an ~111 GiB heap). MTP drafter is bundled in the same
# GGUF (block 42) - no separate --spec-draft-model needed.
#
# Tie-back: to restore the Qwen3.6-27B service, stop frank-large-ling and
# Tie-back: to restore the Qwen3.6-27B service, stop frank-dragoon-ling and
# start frank-default-large (its env/unit are committed and unchanged).
#
# Tunables come from frank-default-large.ling.env in this directory unless
@@ -4,11 +4,11 @@
# Runtime: llama.cpp llama-server (>= b10411).
#
# Tunables come from frank-default-large.env in this directory unless the
# FRANK_LARGE_ENV_FILE environment variable overrides that path.
# FRANK_DRAGOON_ENV_FILE environment variable overrides that path.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-default-large.env}"
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-default-large.env}"
if [[ -f "$ENV_FILE" ]]; then
set -a
# shellcheck disable=SC1090
+1 -1
View File
File diff suppressed because one or more lines are too long
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
"""
Decode throughput benchmark for frank-large.
Decode throughput benchmark for frank-dragoon.
Starts the server with a given --threads-batch value, sends an 8K+ token prompt,
generates 256 tokens, measures generation tok/s.
@@ -13,12 +13,12 @@ SERVE = "/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server"
PORT = 7770
N_GEN = 256
PROMPT_FILE = "/tmp/bench-prompts/prompt_code.txt"
ENV_FILE = os.path.join(os.path.dirname(__file__), "frank-large.env")
ENV_FILE = os.path.join(os.path.dirname(__file__), "frank-dragoon.env")
SERVER_DIR = os.path.dirname(SERVE)
ROCM_PATH = "/opt/rocm"
def make_env():
"""Build the same env as serve-frank-large.sh."""
"""Build the same env as serve-frank-dragoon.sh."""
env = {**os.environ, "HSA_OVERRIDE_GFX_VERSION": "11.5.1", "HIP_VISIBLE_DEVICES": "0"}
# Source env file for tunables
@@ -59,7 +59,7 @@ def wait_ready():
def send_request(prompt, max_tokens=N_GEN):
"""Send chat completion, measure generation throughput."""
body = json.dumps({
"model": "frank-large",
"model": "frank-dragoon",
"messages": [{"role": "user", "content": prompt}],
"max_tokens": max_tokens,
"stream": False,
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
"""
Benchmark n-gram speculative decoding vs no-spec for frank-large.
Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
Tests prompt-processing throughput with 16K token prompts (manageable request size).
Code-heavy prompt (high n-gram redundancy) vs prose prompt (low redundancy).
@@ -103,7 +103,7 @@ def start_server(port, spec_type):
def run_test(prompt_text, spec_type, port):
"""Send a request and extract prompt processing metrics."""
data = json.dumps({
"model": "frank-large",
"model": "frank-dragoon",
"messages": [{"role": "user", "content": prompt_text}],
"max_tokens": TEST_TOKENS,
"stream": False,
@@ -189,7 +189,7 @@ def main():
print(f" Prose: {prose_prompt_file.stat().st_size:,} bytes")
print("=" * 60)
print(" N-gram Mod Benchmark — frank-large")
print(" N-gram Mod Benchmark — frank-dragoon")
print(f" STRIX ROCmFP4 · 64K context · 16K prompts")
print("=" * 60)
@@ -1,19 +1,19 @@
#!/usr/bin/env bash
# Benchmark n-gram speculative decoding vs no-spec for frank-large.
# Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
#
# This tests prompt-processing throughput at 64k context with two distinct
# prompts: one code-heavy and one prose-heavy. Uses llama-server on scratch
# ports with the OpenAI-compatible API to measure prompt_eval_duration_ms.
#
# Run on Dankclaw after stopping the production service:
# sudo systemctl stop frank-large
# sudo systemctl stop frank-dragoon
# ./scripts/bench-ngram-mod.sh
# sudo systemctl start frank-large
# sudo systemctl start frank-dragoon
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
if [[ -f "$ENV_FILE" ]]; then
set -a; source "$ENV_FILE"; set +a
fi
@@ -184,7 +184,7 @@ run_single_test() {
response=$(curl -s -X POST "http://127.0.0.1:$port/v1/completions" \
-H "Content-Type: application/json" \
-d "{
\"model\": \"frank-large\",
\"model\": \"frank-dragoon\",
\"prompt\": \"$prompt_text\",
\"max_tokens\": 128,
\"temperature\": 0,
@@ -220,7 +220,7 @@ run_single_test() {
}
echo "╔══════════════════════════════════════════════════════════╗"
echo "║ N-gram Mod Benchmark — frank-large ║"
echo "║ N-gram Mod Benchmark — frank-dragoon ║"
echo "║ STRIX ROCmFP4 · 64k context · two prompts ║"
echo "╚══════════════════════════════════════════════════════════╝"
echo ""
@@ -15,9 +15,9 @@ import time
import urllib.request
import urllib.error
SERVICE = "frank-large"
ENV_FILE = "/home/dank/serverAppConfigs/scripts/frank-large.env"
ALIAS = "frank-large"
SERVICE = "frank-dragoon"
ENV_FILE = "/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
ALIAS = "frank-dragoon"
DEFAULT_HOST = "localhost"
DEFAULT_PORT = 7770
DEFAULT_MAX_TOKENS = 16384
@@ -1,4 +1,4 @@
# Tunables for frank-large (mradermacher i1-Q4_K_M of
# Tunables for frank-dragoon (mradermacher i1-Q4_K_M of
# orcarouter/Qwen3.8-Flash-Next-Uncensored on ik_llama.cpp, Dankclaw,
# RTX 3090 CUDA + Strix Halo CPU path, since 2026-09-15).
#
@@ -65,7 +65,7 @@ MMPROJ_PATH=/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K
CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
# Alias + network.
ALIAS=frank-large
ALIAS=frank-dragoon
BIND_ADDR=0.0.0.0
LISTEN_PORT=7770
@@ -1,5 +1,5 @@
[Unit]
Description=ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 177B MoE (dankclaw, frank-large)
Description=ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 177B MoE (dankclaw, frank-dragoon)
After=network-online.target
Wants=network-online.target
@@ -12,8 +12,8 @@ Group=dank
# budget still needs room. Parity with the halogen quadlet's
# Ulimit=memlock=-1:-1.
LimitMEMLOCK=infinity
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
Restart=on-failure
RestartSec=10
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Serve mradermacher/Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M as frank-large
# Serve mradermacher/Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M as frank-dragoon
# via ik_llama.cpp on Dankclaw.
#
# Topology (ikawrakow's hybrid recipe; upstream issue #2402 for the PLE):
@@ -26,7 +26,7 @@
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
if [[ -f "$ENV_FILE" ]]; then
set -a
# shellcheck disable=SC1090
@@ -66,7 +66,7 @@ fi
# CPU-overridden weights mmapped (src/llama-load-tensors.cpp); the env file
# sets it. Refuse to run without it.
if [[ "${GGML_CUDA_NO_PINNED_WEIGHTS:-}" != "1" ]]; then
echo "frank-large: refusing to start — GGML_CUDA_NO_PINNED_WEIGHTS=1 is mandatory on this host" >&2
echo "frank-dragoon: refusing to start — GGML_CUDA_NO_PINNED_WEIGHTS=1 is mandatory on this host" >&2
exit 1
fi
@@ -112,7 +112,7 @@ fi
: "${SPEC_DRAFT_POLL_BATCH:=1}"
# Text only: the server must not try to autoload the vision tower. Vision
# stays on frank-large (Dankclaw), which is where the fleet routes images.
# stays on frank-dragoon (Dankclaw), which is where the fleet routes images.
NO_MMPROJ="${NO_MMPROJ:-1}"
export HIP_VISIBLE_DEVICES
+43
View File
@@ -0,0 +1,43 @@
#!/usr/bin/env bash
# apply-protected-cast.sh — RUNS ON DANKSERVER. Rewrite the recipe's
# PROTECTED-BEGIN/END section from Swift's OWN calibration grams: run
# derive-protected-experts.py over the calib dir, splice its cast lines into
# the marked section of the recipe in place, keep the pre-edit recipe in the
# campaign trash. Refuses to run twice (marker text says derived).
set -euo pipefail
CALIB=${1:-/home/dank/corpora/calib}
RECIPE=${2:-/home/dank/serverAppConfigs-swift15/scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe}
PY=/home/dank/venvs/radcorpus/bin/python
DERIVE=/home/dank/serverAppConfigs-swift15/scripts/derive-protected-experts.py
TS=$(date +%Y%m%d-%H%M%S)
grep -q "derive-protected-experts.py rewrote" "$RECIPE" && {
echo "recipe already carries derived cast lines; refusing to re-derive" >&2; exit 1; }
grep -q "PROTECTED-BEGIN" "$RECIPE" || { echo "no PROTECTED markers in $RECIPE" >&2; exit 1; }
"$PY" "$DERIVE" "$CALIB" > "/tmp/protected-cast-$TS.txt"
N=$(grep -c 'cast dtype=bf16' "/tmp/protected-cast-$TS.txt")
[ "$N" -ge 2 ] || { echo "derive produced $N cast lines; refusing" >&2; exit 1; }
echo "derived $N protected experts -> /tmp/protected-cast-$TS.txt"
cp -f "$RECIPE" "/home/dank/.AGENTTRASH/$TS-qwen4exp-swift15-w4dn8-i8-hc8m.recipe"
"$PY" - "$RECIPE" "/tmp/protected-cast-$TS.txt" <<'PYEOF'
import sys
recipe, castfile = sys.argv[1], sys.argv[2]
lines = open(recipe).read().splitlines()
cast = [l for l in open(castfile).read().splitlines() if 'cast dtype=bf16' in l]
out, insec = [], False
for l in lines:
if 'PROTECTED-BEGIN' in l:
insec = True
out.append(l)
out.extend(cast)
elif 'PROTECTED-END' in l:
insec = False
out.append(l)
out.append('# (derive-protected-experts.py rewrote this section from Swift\'s own grams)')
elif not insec:
out.append(l)
open(recipe, 'w').write('\n'.join(out) + '\n')
print(f'spliced {len(cast)} cast lines into {recipe}')
PYEOF
+2 -2
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""autoresearch-bench.py — deterministic greedy battery for frank-large.
"""autoresearch-bench.py — deterministic greedy battery for frank-dragoon.
Runs ON Dankclaw (no LAN jitter) against the candidate container on :7770.
Fixed embedded corpus, temperature 0, streaming so decode speed is measured
@@ -104,7 +104,7 @@ def stream_completion(url: str, model: str, prompt: str):
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--url", default="http://127.0.0.1:7770")
ap.add_argument("--model", default="frank-large")
ap.add_argument("--model", default="frank-dragoon")
args = ap.parse_args()
with urllib.request.urlopen(args.url.rstrip("/") + "/v1/models",
@@ -1,12 +1,12 @@
#!/usr/bin/env bash
# autoresearch-frank-primary.sh — RUNS ON DANKSERVER, driven over ssh by the
# dev-box autoresearch.sh. It is the canonical frank-primary benchmark entry
# autoresearch-frank-archon.sh — RUNS ON DANKSERVER, driven over ssh by the
# dev-box autoresearch.sh. It is the canonical frank-archon benchmark entry
# point for the autoresearch loop.
#
# Method (mandated by the Benchmarking Policy): the production service serves
# real users and shares the two R9700s, so a bench beside it starves the GPUs
# and corrupts results. This script therefore:
# 1. stops the production frank-primary unit (root, relink symlink: stop,
# 1. stops the production frank-archon unit (root, relink symlink: stop,
# NOT mask — mask refuses to replace a symlinked unit file),
# 2. boots a scratch radiance container named `arbench` on the SAME two
# cards with the candidate engine argv (the env file the loop edits),
@@ -18,7 +18,7 @@
# prefill and decode rates for that single request.
# 5. prints METRIC lines and tears the scratch down,
# 6. restores production (systemctl start), via a trap so a failed bench
# never leaves frank-primary down (the stale-probe-tail lesson).
# never leaves frank-archon down (the stale-probe-tail lesson).
#
# Determinism: the prompt is built from a fixed, sorted corpus of repo files
# with a fixed random seed, so prompt_n is stable across runs. Sampling is
@@ -32,10 +32,10 @@
# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
# differences: the container name (arbench, so prod's identity is untouched)
# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
# prefill is cold — the prod tier at kvcache/frank-primary is left intact.
# prefill is cold — the prod tier at kvcache/frank-archon is left intact.
set -uo pipefail
ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-primary.env}"
ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-archon.env}"
SCRATCH=arbench
PORT=7770
HEALTH="http://127.0.0.1:$PORT/health"
@@ -45,7 +45,7 @@ LOG=/home/dank/logs/arbench.log
restore() {
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
sudo systemctl start frank-primary >/dev/null 2>&1 || true
sudo systemctl start frank-archon >/dev/null 2>&1 || true
}
trap restore EXIT
@@ -57,13 +57,13 @@ fi
set -a; . "$ENV_FILE"; set +a
# --- stop production; the scratch needs both cards to itself ----------------
sudo systemctl stop frank-primary >/dev/null 2>&1 || true
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
FRESH_KV="$FRANK_PRIMARY_STATE_DIR/kvcache/$SCRATCH"
FRESH_KV="$FRANK_ARCHON_STATE_DIR/kvcache/$SCRATCH"
rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
mkdir -p "$FRANK_PRIMARY_STATE_DIR/tune"
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
# --- boot the scratch: the production run line, scratch name + fresh KV ------
podman run -d --name "$SCRATCH" \
@@ -72,30 +72,31 @@ podman run -d --name "$SCRATCH" \
--security-opt seccomp=unconfined \
--ulimit memlock=-1:-1 \
--device /dev/kfd --device /dev/dri \
-v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \
-v "$FRANK_PRIMARY_STATE_DIR":/data \
-v "$FRANK_PRIMARY_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
-v "$FRANK_PRIMARY_TEMPLATE":/templates/template.jinja:ro \
"$FRANK_PRIMARY_IMAGE" \
--model "/models/$FRANK_PRIMARY_MODEL_FILE" \
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-v "$FRANK_ARCHON_STATE_DIR":/data \
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
"$FRANK_ARCHON_IMAGE" \
--model "/models/$FRANK_ARCHON_MODEL_FILE" \
--host :: --port "$PORT" \
--served-model-name "$FRANK_PRIMARY_SERVED_NAME" \
--served-model-name "$FRANK_ARCHON_SERVED_NAME" \
--override-chat-template /templates/template.jinja \
--tp "$FRANK_PRIMARY_TP" --tp-wire "$FRANK_PRIMARY_TP_WIRE" \
--max-num-seqs "$FRANK_PRIMARY_MAX_SEQS" \
--max-model-len "$FRANK_PRIMARY_MAX_MODEL_LEN" \
--placement "$FRANK_PRIMARY_PLACEMENT" \
--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \
--gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \
--expert-vs-cache-ratio "$FRANK_PRIMARY_EXPERT_VS_CACHE" \
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \
--tp "$FRANK_ARCHON_TP" --tp-wire "$FRANK_ARCHON_TP_WIRE" \
--max-num-seqs "$FRANK_ARCHON_MAX_SEQS" \
--max-model-len "$FRANK_ARCHON_MAX_MODEL_LEN" \
--placement "$FRANK_ARCHON_PLACEMENT" \
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
--prefix-cache-dir "/data/kvcache/$SCRATCH" \
--prefix-cache-disk-mib "$FRANK_PRIMARY_PREFIX_DISK_MIB" \
--ngram-placement "$FRANK_PRIMARY_NGRAM_PLACEMENT" \
--num-speculative-tokens "$FRANK_PRIMARY_SPEC_TOKENS" \
--max-num-batched-tokens "$FRANK_PRIMARY_MAX_BATCHED_TOKENS" \
${FRANK_PRIMARY_EXTRA_ARGS:+$FRANK_PRIMARY_EXTRA_ARGS} \
--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
> "$LOG" 2>&1
if [ $? -ne 0 ]; then echo "scratch boot failed; see $LOG" >&2; exit 1; fi
@@ -145,7 +146,7 @@ while n < TGT:
n += len(seg) + 1
prompt = " ".join(parts)
json.dump(
{"model": "frank-primary", "prompt": prompt,
{"model": "frank-archon", "prompt": prompt,
"max_tokens": 256, "temperature": 0.0, "stream": False,
"stop": ["<|im_end|>"]},
open(sys.argv[1], "w"),
@@ -1,19 +1,19 @@
#!/usr/bin/env bash
# frank-primary-arms.sh — run a SEQUENCE of frank-primary bench arms and keep
# frank-archon-arms.sh — run a SEQUENCE of frank-archon bench arms and keep
# every arm's evidence. Runs on the operator box; each arm is one full window
# of ./autoresearch.sh (stop+mask -> boot the arm as the service -> measure ->
# restore production and verify), so the fleet is only ever down for one arm at
# a time and the production unit is back up between arms.
#
# An arm is the repo's own frank-primary.env plus an OVERLAY of engine knobs
# An arm is the repo's own frank-archon.env plus an OVERLAY of engine knobs
# appended after it (later assignment wins when the serve script sources it),
# staged through FRANK_PRIMARY_AR_ENV_FILE so the checkout is never mutated and
# staged through FRANK_ARCHON_AR_ENV_FILE so the checkout is never mutated and
# each arm gets its own fingerprint (the memo can never answer one arm with
# another arm's numbers).
#
# Usage: scripts/autoresearch/frank-primary-arms.sh [arm ...]
# scripts/autoresearch/frank-primary-arms.sh --list
# Output, under /tmp/frank-primary-arms-<UTC stamp>/ (override with ARMS_DIR):
# Usage: scripts/autoresearch/frank-archon-arms.sh [arm ...]
# scripts/autoresearch/frank-archon-arms.sh --list
# Output, under /tmp/frank-archon-arms-<UTC stamp>/ (override with ARMS_DIR):
#
# <arm>.out full autoresearch output (metrics + engine evidence)
# <arm>.env the exact staged env
@@ -26,8 +26,8 @@ set -uo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
REPO="$(cd -- "$SCRIPT_DIR/../.." && pwd)"
BASE_ENV="$REPO/scripts/frank-primary.env"
ARMS_DIR="${ARMS_DIR:-/tmp/frank-primary-arms-$(date -u +%Y%m%dT%H%M%SZ)}"
BASE_ENV="$REPO/scripts/frank-archon.env"
ARMS_DIR="${ARMS_DIR:-/tmp/frank-archon-arms-$(date -u +%Y%m%dT%H%M%SZ)}"
# --- arms ------------------------------------------------------------------
# baseline: the shipped shape E, measured with THIS harness so the comparison is
@@ -45,18 +45,18 @@ LADDER3='{"cudagraph_capture_sizes":[4,8,12],"max_cudagraph_capture_size":12,"in
# the engine as {temperature:0.7}.
GEN07='{"temperature":0.7}'
OVER_B=(
"FRANK_PRIMARY_SPEC='$SPEC3'"
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
"FRANK_ARCHON_SPEC='$SPEC3'"
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
)
# B2: identical to B with the default (7.60 GiB) overhead, i.e. precise memory
# OFF. It is the fallback if B dies at graph capture, and it prices the arena
# that precise memory buys (7,522 slots -> ~6,496).
OVER_B2=(
"FRANK_PRIMARY_SPEC='$SPEC3'"
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
'FRANK_PRIMARY_PRECISE_MEMORY='
"FRANK_ARCHON_SPEC='$SPEC3'"
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
'FRANK_ARCHON_PRECISE_MEMORY='
)
# C: the vendor's concurrency shape — SEQS 16 with the [4..32] ladder and util
# 0.95. This is also the direct test of the theory that pinning the capture
@@ -64,12 +64,12 @@ OVER_B2=(
# the constant 3, which is NOT in the default ladder; 4 is).
LADDER16='{"cudagraph_capture_sizes":[4,8,12,16,20,24,28,32],"max_cudagraph_capture_size":32,"inductor_compile_config":{"combo_kernels":false,"benchmark_combo_kernel":false}}'
OVER_C=(
"FRANK_PRIMARY_SPEC='$SPEC3'"
"FRANK_PRIMARY_COMPCONFIG='$LADDER16'"
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
'FRANK_PRIMARY_SEQS=16'
'FRANK_PRIMARY_MNBT=4096'
'FRANK_PRIMARY_UTIL=0.95'
"FRANK_ARCHON_SPEC='$SPEC3'"
"FRANK_ARCHON_COMPCONFIG='$LADDER16'"
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
'FRANK_ARCHON_SEQS=16'
'FRANK_ARCHON_MNBT=4096'
'FRANK_ARCHON_UTIL=0.95'
)
# D/E: the CONTEXT arms. Measured 2026-10-02 (arms run 20261002T233855Z): the
@@ -83,11 +83,11 @@ OVER_C=(
# D is arm B with EXACTLY ONE variable changed (the KV bytes), so its result
# says whether the 4k-depth collapse (B 23.09 vs baseline 33.99) was KV or the
# batch shape.
KV401K='FRANK_PRIMARY_KV_BYTES=4050000000'
KV401K='FRANK_ARCHON_KV_BYTES=4050000000'
OVER_D=(
"FRANK_PRIMARY_SPEC='$SPEC3'"
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
"FRANK_ARCHON_SPEC='$SPEC3'"
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
"$KV401K"
)
# E: D with C's max_num_batched_tokens (4096 — C is the only arm that did NOT
@@ -97,12 +97,12 @@ OVER_D=(
# only 0.32 GiB above the allowance while the CUDA-graph cost is an ANALYTIC
# estimate (C's was 1.47 GiB over 24 graphs), so it is measured, not assumed.
OVER_E=(
"FRANK_PRIMARY_SPEC='$SPEC3'"
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
"FRANK_ARCHON_SPEC='$SPEC3'"
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
"$KV401K"
'FRANK_PRIMARY_MNBT=4096'
'FRANK_PRIMARY_UTIL=0.99'
'FRANK_ARCHON_MNBT=4096'
'FRANK_ARCHON_UTIL=0.99'
)
arm_overlay() {
@@ -142,7 +142,7 @@ for arm in "${ARMS[@]}"; do
# holds JSON must still be one valid JSON object after the file is SOURCED
# (that is the step that silently eats unquoted braces).
if ! bash -c 'set -a; . "$1" >/dev/null; set +a
for k in FRANK_PRIMARY_SPEC FRANK_PRIMARY_COMPCONFIG FRANK_PRIMARY_GEN_CONFIG; do
for k in FRANK_ARCHON_SPEC FRANK_ARCHON_COMPCONFIG FRANK_ARCHON_GEN_CONFIG; do
v="${!k:-}"
[ -n "$v" ] || continue
case "$v" in
@@ -158,7 +158,7 @@ for arm in "${ARMS[@]}"; do
fi
echo "[arms] === $arm: opening window $(date -u +%H:%M:%S)Z ===" >&2
FRANK_PRIMARY_AR_ENV_FILE="$ARMS_DIR/$arm.env" \
FRANK_ARCHON_AR_ENV_FILE="$ARMS_DIR/$arm.env" \
"$REPO/autoresearch.sh" >"$ARMS_DIR/$arm.out" 2>&1
rc=$?
echo "$rc" >"$ARMS_DIR/$arm.rc"
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""frank-primary decode measurement — runs INSIDE the engine container.
"""frank-archon decode measurement — runs INSIDE the engine container.
The method is the fleet's established one (the 2026-09-29 / 2026-10-01
tuning campaigns, /home/dank/mtp_measure.py): SSE chunk counting is invalid
@@ -28,7 +28,7 @@ Workload (fixed, deterministic, offline):
Env:
AR_BASE endpoint (default http://127.0.0.1:7770)
AR_MODEL served name (default frank-primary)
AR_MODEL served name (default frank-archon)
AR_PROMPT_CEILING hard prompt ceiling in tokens (default 100000)
AR_TEMPS comma-separated sampling temperatures (default "0,0.7")
@@ -45,7 +45,7 @@ import urllib.error
import urllib.request
BASE = os.environ.get("AR_BASE", "http://127.0.0.1:7770")
MODEL = os.environ.get("AR_MODEL", "frank-primary")
MODEL = os.environ.get("AR_MODEL", "frank-archon")
PROMPT_CEILING = int(os.environ.get("AR_PROMPT_CEILING", "100000"))
DEPTHS = (115, 1056, 4096)
WARM_TOKENS = 64
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# frank-primary-window.sh — one full bench window for frank-primary, run ON
# frank-archon-window.sh — one full bench window for frank-archon, run ON
# DANKSERVER as the deploy user (invoked over ssh by ./autoresearch.sh).
#
# Discipline (AGENTS.md Benchmarking Policy): the production unit is
@@ -12,9 +12,9 @@
#
# Staging layout (created by the orchestrator, ephemeral — never a config
# source; the repo checkout is):
# /home/dank/.autoresearch-frank-primary/
# /home/dank/.autoresearch-frank-archon/
# serve.sh staged candidate serve script
# frank-primary.env staged candidate env (the candidate knobs)
# frank-archon.env staged candidate env (the candidate knobs)
# measure.py the in-container measure client
# hash candidate fingerprint
# run.json result (metrics + boot + kv + fingerprint)
@@ -27,16 +27,16 @@
# outcome; a failed restore forces exit 1).
set -uo pipefail
STAGE="${FRANK_PRIMARY_AR_STAGE:-/home/dank/.autoresearch-frank-primary}"
UNIT="${FRANK_PRIMARY_AR_UNIT:-frank-primary}"
CTR="${FRANK_PRIMARY_AR_CTR:-frank-primary-qwen}"
STAGE="${FRANK_ARCHON_AR_STAGE:-/home/dank/.autoresearch-frank-archon}"
UNIT="${FRANK_ARCHON_AR_UNIT:-frank-archon}"
CTR="${FRANK_ARCHON_AR_CTR:-frank-archon-qwen}"
HEALTH_URL="http://127.0.0.1:7770/health"
MODELS_URL="http://127.0.0.1:7770/v1/models"
BOOT_TIMEOUT="${FRANK_PRIMARY_AR_BOOT_TIMEOUT:-1200}"
RESTORE_TIMEOUT="${FRANK_PRIMARY_AR_RESTORE_TIMEOUT:-1200}"
MEASURE_TIMEOUT="${FRANK_PRIMARY_AR_MEASURE_TIMEOUT:-1800}"
SERVED="$(sed -n 's/^FRANK_PRIMARY_SERVED_NAME=//p' "$STAGE/frank-primary.env" 2>/dev/null | head -1)"
SERVED="${SERVED:-frank-primary}"
BOOT_TIMEOUT="${FRANK_ARCHON_AR_BOOT_TIMEOUT:-1200}"
RESTORE_TIMEOUT="${FRANK_ARCHON_AR_RESTORE_TIMEOUT:-1200}"
MEASURE_TIMEOUT="${FRANK_ARCHON_AR_MEASURE_TIMEOUT:-1800}"
SERVED="$(sed -n 's/^FRANK_ARCHON_SERVED_NAME=//p' "$STAGE/frank-archon.env" 2>/dev/null | head -1)"
SERVED="${SERVED:-frank-archon}"
RESTORED=0
@@ -72,7 +72,7 @@ capture_bench_failure() {
echo "=== staged serve.out (tail 60) ==="
tail -60 "$STAGE/serve.out" 2>/dev/null || true
echo "=== engine log (tail 60) ==="
tail -60 /home/dank/logs/frank_primary.log 2>/dev/null || true
tail -60 /home/dank/logs/frank_archon.log 2>/dev/null || true
} >"$STAGE/fail.log" 2>&1 || true
grep -E 'OutOfMemory|Tried to allocate|Engine core initialization failed|Error|error' \
"$STAGE/fail.log" 2>/dev/null | tail -25 >&2 || true
@@ -114,7 +114,7 @@ trap 'restore; rc=$?; [ "$rc" -ne 0 ] && exit 1' EXIT
main() {
local f
for f in serve.sh frank-primary.env measure.py hash; do
for f in serve.sh frank-archon.env measure.py hash; do
[ -f "$STAGE/$f" ] || { log "staged file missing: $STAGE/$f"; exit 1; }
done
mkdir -p "$STAGE/last"
@@ -133,7 +133,7 @@ main() {
t0="$(date +%s)"
log "starting bench instance from staged config (fingerprint $(tr -d '[:space:]' <"$STAGE/hash"))"
FRANK_PRIMARY_ENV_FILE="$STAGE/frank-primary.env" \
FRANK_ARCHON_ENV_FILE="$STAGE/frank-archon.env" \
setsid nohup bash "$STAGE/serve.sh" >"$STAGE/serve.out" 2>&1 </dev/null &
echo "$!" >"$STAGE/serve.pid"
+3 -3
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Decode throughput benchmark for frank-large.
# Decode throughput benchmark for frank-dragoon.
# Measures generation tok/s with 8K+ token prompts and 256 generated tokens.
set -euo pipefail
@@ -57,7 +57,7 @@ run_bench() {
# Warmup (small request)
curl -s -m 60 http://127.0.0.1:$PORT/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"frank-large","messages":[{"role":"user","content":"Hi"}],"max_tokens":16,"stream":false}' > /dev/null 2>&1 || true
-d '{"model":"frank-dragoon","messages":[{"role":"user","content":"Hi"}],"max_tokens":16,"stream":false}' > /dev/null 2>&1 || true
# Build 8K prompt
local prompt
@@ -75,7 +75,7 @@ print(json.dumps(' '.join(words)))
t0=$(date +%s%N)
resp=$(curl -s -m 600 http://127.0.0.1:$PORT/v1/chat/completions \
-H "Content-Type: application/json" \
-d "{\"model\":\"frank-large\",\"messages\":[{\"role\":\"user\",\"content\":$prompt}],\"max_tokens\":$N_GEN,\"stream\":false}" 2>&1)
-d "{\"model\":\"frank-dragoon\",\"messages\":[{\"role\":\"user\",\"content\":$prompt}],\"max_tokens\":$N_GEN,\"stream\":false}" 2>&1)
t1=$(date +%s%N)
if ! echo "$resp" | python3 -c "import sys,json; json.load(sys.stdin)" 2>/dev/null; then
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env python3
"""bench-probe.py — RUNS ON DANKSERVER. The campaign's decode probe: one
/completion request, a prompt assembled from calib.jsonl to ~TARGET tokens,
temperature 0, max_tokens 512. Reports the engine's own timings: decode rate,
prefill rate, draft accept. Run 3x; the median is the arm's number."""
import json, sys, urllib.request
TARGET = 43600 # tokens
BYTES_PER_TOK = 3.8
URL = "http://localhost:7770/v1/completions"
def build_prompt():
want = int(TARGET * BYTES_PER_TOK)
parts = []
n = 0
with open("/home/dank/corpora/out/calib.jsonl") as f:
for line in f:
d = json.loads(line)
p = d.get("prompt") or d.get("messages", [{}])[0].get("content", "")
parts.append(p)
n += len(p.encode())
if n >= want:
break
return "".join(parts)
def probe(salt):
prompt = f"[probe {salt}]\n" + build_prompt()
body = json.dumps({
"model": "frank-archon",
"prompt": prompt, "temperature": 0.0, "max_tokens": 512,
}).encode()
req = urllib.request.Request(URL, data=body,
headers={"Content-Type": "application/json"})
with urllib.request.urlopen(req, timeout=1800) as r:
d = json.load(r)
t = d.get("timings", {})
return {
"prompt_n": t.get("prompt_n"),
"prefill_tps": round(t.get("prompt_per_second", 0), 1),
"pred_n": t.get("predicted_n"),
"decode_tps": round(t.get("predicted_per_second", 0), 2),
"draft_n": t.get("draft_n"),
"accept": (round(t["draft_n_accepted"] / t["draft_n"], 4)
if t.get("draft_n") else None),
}
if __name__ == "__main__":
print(json.dumps(probe(sys.argv[1] if len(sys.argv) > 1 else "a")))
+2 -2
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
"""
HTTP-based prefill-throughput sweep for frank-large.
HTTP-based prefill-throughput sweep for frank-dragoon.
Starts the server with a given --threads-batch value, sends a 64K-prompt via
the chat API (max_tokens=0), measures prompt_tokens/wall_time.
@@ -39,7 +39,7 @@ def send_prefill_request():
with open(PROMPT_FILE, "r") as f:
prompt = f.read()
body = json.dumps({
"model": "frank-large",
"model": "frank-dragoon",
"messages": [{"role": "user", "content": prompt}],
"max_tokens": 1,
"stream": False
+2 -2
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Threads-batch sweep for frank-large on dankclaw
# Threads-batch sweep for frank-dragoon on dankclaw
# Benches prefill throughput at 65536 prompt tokens across threads-batch values.
set -uo pipefail
@@ -10,7 +10,7 @@ TB_LIST=(16 32 48 64 80 96 128)
ENV_FILE="/home/dank/GitPackages/ROCmFPX/build-qwen38/bin"
echo "========================================"
echo "THREADS-BATCH SWEEP — frank-large"
echo "THREADS-BATCH SWEEP — frank-dragoon"
echo "Model: $(basename "$MODEL") ($(du -h "$MODEL" | cut -f1))"
echo "Threads-batch values: ${TB_LIST[*]}"
echo "Prompt: 65536 tokens, n-gen: 0 (prefill-only)"
+4 -4
View File
@@ -1,14 +1,14 @@
#!/usr/bin/env bash
# Benchmark ubatch with llama-bench for frank-large.
# Benchmark ubatch with llama-bench for frank-dragoon.
#
# This is intentionally a ubatch-only benchmark. It uses one llama-bench
# process, an 8K prompt workload, and equal batch/ubatch values. No server
# process, HTTP requests, or log scraping are used.
#
# Run on Dankclaw after stopping frank-large:
# sudo systemctl stop frank-large
# Run on Dankclaw after stopping frank-dragoon:
# sudo systemctl stop frank-dragoon
# ./scripts/bench-ubatch.sh
# sudo systemctl start frank-large
# sudo systemctl start frank-dragoon
set -euo pipefail
: "${LLAMA_BENCH:=/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-bench}"
@@ -1,11 +1,11 @@
#!/usr/bin/env bash
# build-frank-primary-rad.sh — build frank-primary's .rad container from the
# build-frank-archon-rad.sh — build frank-archon's .rad container from the
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint, at NATIVE precision.
#
# bash scripts/build-frank-primary-rad.sh # convert
# bash scripts/build-frank-primary-rad.sh --plan-only # declare, no I/O
# bash scripts/build-frank-primary-rad.sh --restage # redo staging
# bash scripts/build-frank-primary-rad.sh --force # rebuild existing
# bash scripts/build-frank-archon-rad.sh # convert
# bash scripts/build-frank-archon-rad.sh --plan-only # declare, no I/O
# bash scripts/build-frank-archon-rad.sh --restage # redo staging
# bash scripts/build-frank-archon-rad.sh --force # rebuild existing
#
# WHY A STAGING DIR (the one non-obvious step): rad-convert selects its
# architecture plugin by the checkpoint's quantisation DESCRIPTOR —
@@ -55,8 +55,8 @@ OUT="$OUT_DIR/qwen4exp-fp8-native.rad"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck disable=SC1091
source "$SCRIPT_DIR/frank-primary.env"
IMAGE="$FRANK_PRIMARY_IMAGE"
source "$SCRIPT_DIR/frank-archon.env"
IMAGE="$FRANK_ARCHON_IMAGE"
plan_only=0 restage=0 force=0
for a in "$@"; do
@@ -69,11 +69,11 @@ for a in "$@"; do
done
if [ ! -f "$SRC/config.json" ]; then
echo "ERROR: raw checkpoint missing: $SRC (run scripts/download-frank-primary.sh --fetch)" >&2
echo "ERROR: raw checkpoint missing: $SRC (run scripts/download-frank-archon.sh --fetch)" >&2
exit 1
fi
echo "verifying raw checkpoint first..."
bash "$SCRIPT_DIR/download-frank-primary.sh"
bash "$SCRIPT_DIR/download-frank-archon.sh"
if [ -f "$OUT" ] && [ "$plan_only" -eq 0 ] && [ "$force" -eq 0 ]; then
echo "container already exists: $OUT"
@@ -83,7 +83,7 @@ fi
# ---- staging ----------------------------------------------------------------
if [ "$restage" -eq 1 ] && [ -e "$STAGE" ]; then
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-primary-stage"
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-archon-stage"
mkdir -p "$TRASH"
mv "$STAGE" "$TRASH/stage-$(date +%H%M%S)"
echo "old staging dir moved to $TRASH"
@@ -138,7 +138,7 @@ CONVERT_ARGS=(/models/stage-qwen4exp-fp8 -o /models/rad/qwen4exp-fp8-native.rad
# exactly and the plugin's one-element scale plane is produced.
CONVERT_ARGS+=(--quant "blk.*.ple_ngram.weight=rtn:codes=fp8_e4m3,block=*x*,scale=bf16,rule=fixed,scale_value=$PLE_SCALE")
[ "$force" -eq 1 ] && [ -f "$OUT" ] && {
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-primary-rad"
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-archon-rad"
mkdir -p "$TRASH"
mv "$OUT" "$TRASH/qwen4exp-fp8-native.rad.$(date +%H%M%S)"
echo "previous container moved to $TRASH"
@@ -158,8 +158,8 @@ echo "rad-info summary:"
podman run --rm --entrypoint rad-info -v /home/dank/models:/models:ro \
"$IMAGE" "/models/rad/qwen4exp-fp8-native.rad" | head -20
MIB=$(( $(stat -c %s "$OUT") / 1048576 ))
echo "container: $OUT (${MIB} MiB; env floor $FRANK_PRIMARY_RAD_MIN_MIB MiB)"
if [ "$MIB" -lt "$FRANK_PRIMARY_RAD_MIN_MIB" ]; then
echo "container: $OUT (${MIB} MiB; env floor $FRANK_ARCHON_RAD_MIN_MIB MiB)"
if [ "$MIB" -lt "$FRANK_ARCHON_RAD_MIN_MIB" ]; then
echo "ERROR: container below the env floor — the convert was truncated" >&2
exit 1
fi
+2 -2
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Build the gufo runtime that serves frank-large on Dankclaw.
# Build the gufo runtime that serves frank-dragoon on Dankclaw.
#
# bash scripts/build-gufo.sh # build + install the pinned tag
# bash scripts/build-gufo.sh --verify # only check the installed tree
@@ -93,4 +93,4 @@ cmake --install build/release
echo "== verify =="
"$PREFIX/bin/gufo" diagnose
echo "frank-large runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)"
echo "frank-dragoon runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)"
+4 -4
View File
@@ -1,11 +1,11 @@
#!/usr/bin/env bash
# Build the ROCmFPX runtime that serves frank-large on Dankclaw.
# Build the ROCmFPX runtime that serves frank-dragoon on Dankclaw.
#
# bash scripts/build-rocmfpx.sh # clone/fetch + build the pin
# bash scripts/build-rocmfpx.sh --verify # only check the installed tree
#
# Runtime: kingjones30/ROCmFPX — charlie12345/ROCmFPX plus the model-card
# patches, i.e. the fork that owns the ROCmFPx tensor formats. frank-large's
# patches, i.e. the fork that owns the ROCmFPx tensor formats. frank-dragoon's
# payload is ftype 115 (`Q8_0_ROCMFPX_AGENT`): 311 tensors of
# GGML_TYPE_Q8_0_ROCMFPX (103), a type that exists ONLY in this fork, so
# stock llama.cpp, gufo and every other loader on the host reject the file.
@@ -30,7 +30,7 @@
set -euo pipefail
ROCMFPX_REPO="${ROCMFPX_REPO:-https://github.com/kingjones30/ROCmFPX.git}"
# The commit frank-large was qualified with (measured 2026-10-02).
# The commit frank-dragoon was qualified with (measured 2026-10-02).
ROCMFPX_COMMIT="${ROCMFPX_COMMIT:-36e9acd40e10a87cd3c3ef8ec734668757dc8520}"
SRC_DIR="${ROCMFPX_SRC_DIR:-/home/dank/GitPackages/kingjones-ROCmFPX}"
BUILD_DIR="${ROCMFPX_BUILD_DIR:-$SRC_DIR/build}"
@@ -86,4 +86,4 @@ cmake --build "$BUILD_DIR" --target llama-server llama-quantize -j "$JOBS"
echo "== verify =="
"$BUILD_DIR/bin/llama-server" --version | head -1
echo "frank-large runtime ready: $BUILD_DIR/bin/llama-server ($ROCMFPX_COMMIT)"
echo "frank-dragoon runtime ready: $BUILD_DIR/bin/llama-server ($ROCMFPX_COMMIT)"
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env python3
"""calib-push.py — drive a calibration corpus through a radiance scratch
server's prefill so its tap writes layer Hessians (docs/MOE-W4.md).
Runs ON DANKSERVER next to the scratch server (the one autoresearch- /
serve-style scratch container booted with RADIANCE_CALIB_DIR set).
Reads the corpus JSONL produced by make-calib-corpus.py
({"prompt": "..."} a line), POSTs each prompt to /v1/completions with
max_tokens=1 (prefill-only cost, one token out), at N-way concurrency
matching the scratch server's --max-num-seqs budget.
Corpus rounds repeat until the engine prints the tap's completion line
("calibration wrote layer Hessians") in `podman logs <container>` — that
is the ground truth that RADIANCE_CALIB_TOKENS was reached. After that
line, further prompts are pure waste, so the driver exits.
Mechanics notes:
- requests carry no sampling params that matter (max_tokens=1, greedy).
- a per-request timeout of 30 min: at ~1k tok/s prefill a 12k-token
prompt is a few seconds; the timeout only catches a wedged engine.
- failures print and continue: the tap pools over tokens, a few lost
prompts do not bias the Hessians (the corpus is drawn round-robin over
source classes, so loss is class-neutral as long as failures are rare;
a systematic 4xx is loud in the log and must be fixed, not ignored).
Usage: calib-push.py [--port 7770] [--parallel 64] [--container arbench]
[--corpus /home/dank/corpora/out/calib.jsonl] [--max-rounds 4]
"""
import argparse
import concurrent.futures
import json
import subprocess
import sys
import time
import urllib.request
DONE_MARK = "calibration wrote"
def tap_done(container):
"""True if the engine logged the tap's completion line."""
try:
out = subprocess.run(
["podman", "logs", "--tail", "50", container],
capture_output=True, text=True, timeout=30,
)
except Exception:
return False
return DONE_MARK in (out.stdout + out.stderr)
def push_one(url, prompt):
body = json.dumps({
"model": "frank-archon",
"prompt": prompt,
"max_tokens": 1,
"temperature": 0,
"stream": False,
}).encode()
req = urllib.request.Request(
url, data=body, headers={"Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=1800) as r:
r.read()
return True
except Exception as e:
print(f"FAIL {str(e)[:120]}", flush=True)
return False
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--port", type=int, default=7770)
ap.add_argument("--parallel", type=int, default=64)
ap.add_argument("--container", default="arbench")
ap.add_argument("--corpus", default="/home/dank/corpora/out/calib.jsonl")
ap.add_argument("--max-rounds", type=int, default=4)
a = ap.parse_args()
url = f"http://127.0.0.1:{a.port}/v1/completions"
prompts = []
with open(a.corpus) as f:
for line in f:
line = line.strip()
if line:
prompts.append(json.loads(line)["prompt"])
print(f"corpus: {len(prompts)} prompts", flush=True)
for rnd in range(1, a.max_rounds + 1):
if tap_done(a.container):
break
t0 = time.time()
ok = 0
with concurrent.futures.ThreadPoolExecutor(a.parallel) as ex:
for good in ex.map(lambda p: push_one(url, p), prompts):
ok += 1 if good else 0
dt = time.time() - t0
print(f"round {rnd}: {ok}/{len(prompts)} ok in {dt:.0f}s "
f"({sum(len(p) for p in prompts)/4/dt:.0f} chars/s est)",
flush=True)
if tap_done(a.container):
break
else:
print("max rounds exhausted, tap not done", flush=True)
sys.exit(1)
print("TAP_DONE: calibration wrote layer Hessians", flush=True)
if __name__ == "__main__":
main()
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env bash
# convert-swift15-w4dn8.sh — RUNS ON DANKSERVER. The final recipe convert of
# the Swift 1.5 checkpoint: gate_up experts through GPTQ against the 10M-token
# calibration Grams, down projections block-fp8, protected experts bf16,
# trunk int8, PLE fixed-scale fp8 (scripts/recipes/
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe).
#
# The convert engine is 1.0.13 — the SAME pin that will serve the result, so
# the .rad crosses no version boundary. Only the Gram files cross versions
# (1.2.0 wrote them; the RADGRAM2 magic, gram.r%d.%s.bin naming and the
# calib_gram/calib_resolve/calib_stem symbols are byte-identical between the
# two libquant.so). rad-convert runs on the host backend (no GPU visible in
# the container) — production can stay up through this script.
#
# Prereqs (campaign order): calibration complete (calib-push printed TAP_DONE),
# the protected-expert cast lines rewritten into the recipe by
# derive-protected-experts.py, and the 335 GiB bf16 container deleted —
# the peak-disk arithmetic needs that space (checkpoint 335.3 + output ~132
# GiB on the volume; --reuse is NOT used, see the recipe header).
set -euo pipefail
PIN_1013=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
IMAGE="stilldeadcode/radiance@$PIN_1013"
CKPT=/home/dank/models/ukisai/Swift1.5-Qwen3.8-Flash-Next
RECIPE=/home/dank/serverAppConfigs-swift15/scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe
CALIB=/home/dank/corpora/calib
OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad
LOG=/home/dank/logs/swift15_convert.log
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert table;
# the plugin's one-encoding law dropped the block-fp8 down tier -- see the recipe header).
MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000}
# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a
# per-tensor stderr note, which is silently worse than failing here.
[ -d "$CKPT" ] || { echo "no checkpoint at $CKPT" >&2; exit 1; }
[ -f "$RECIPE" ] || { echo "no recipe at $RECIPE" >&2; exit 1; }
GRAMS=$(find "$CALIB" -maxdepth 1 -name 'gram.r*.bin' 2>/dev/null | wc -l)
[ "$GRAMS" -ge 48 ] || { echo "calib dir $CALIB holds $GRAMS gram files (<48)" >&2; exit 1; }
if [ -e "$OUT" ]; then echo "$OUT already exists; move it to trash first" >&2; exit 1; fi
if [ -e /home/dank/models/rad/qwen4exp-swift15-bf16.rad ]; then
echo "the bf16 container is still on disk; delete it first (peak disk)" >&2; exit 1
fi
# Gate 2: disk. Output ~132 GiB plus convert staging, 20 GiB of slack.
FREE_MIB=$(df --output=avail -m /home/dank/models | tail -1 | tr -d ' ')
[ "$FREE_MIB" -ge $((MIN_MIB + 20480)) ] || {
echo "only ${FREE_MIB} MiB free on /home/dank/models" >&2; exit 1; }
# The recipe names $CALIB; rad-convert expands it from the environment
# (MOE-W4.md:275). The recipe's cast section must name THIS checkpoint's
# protected experts — derive-protected-experts.py rewrote it.
echo "convert: $CKPT -> $OUT (gptq calib=$CALIB, $(date -Is))" | tee "$LOG"
podman run --rm --name swift15-convert \
-v /home/dank/models:/models \
-v /home/dank/serverAppConfigs-swift15:/campaign:ro \
-v "$CALIB:/data/calib:ro" \
-e CALIB=/data/calib \
--entrypoint /opt/radiance/bin/rad-convert \
"$IMAGE" "/models/${CKPT#/home/dank/models/}" \
--recipe /campaign/"${RECIPE#/home/dank/serverAppConfigs-swift15/}" \
-o /models/rad/"$(basename "$OUT")" 2>&1 | tee -a "$LOG"
rc=${PIPESTATUS[0]}
[ "$rc" -eq 0 ] || { echo "rad-convert rc=$rc; log: $LOG" >&2; exit "$rc"; }
# Gate 3: the artifact. rad-info's recipe echo is the receipt that GPTQ ran.
SIZE_MIB=$(( $(stat -c %s "$OUT") / 1048576 ))
[ "$SIZE_MIB" -ge "$MIN_MIB" ] || {
echo "$OUT is ${SIZE_MIB} MiB < floor ${MIN_MIB}" >&2; exit 1; }
podman run --rm --entrypoint /opt/radiance/bin/rad-info \
-v /home/dank/models:/models:ro "$IMAGE" /models/rad/"$(basename "$OUT")" \
2>&1 | tee -a "$LOG" | grep -c 'gptq' >/dev/null \
|| echo "WARNING: rad-info shows no gptq rule in the recorded recipe" >&2
echo "CONVERT_DONE ${SIZE_MIB} MiB -> $OUT"
+3 -3
View File
@@ -57,9 +57,9 @@ else
echo " none"
fi
section "frank-primary unit"
printf ' active: %s\n' "$(systemctl is-active frank-primary 2>/dev/null | head -1)"
printf ' enabled: %s\n' "$(systemctl is-enabled frank-primary 2>/dev/null | head -1)"
section "frank-archon unit"
printf ' active: %s\n' "$(systemctl is-active frank-archon 2>/dev/null | head -1)"
printf ' enabled: %s\n' "$(systemctl is-enabled frank-archon 2>/dev/null | head -1)"
section "glm serve tail"
if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qx "$GLM_CTR"; then
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""deep-probe.py — one chat completion at ~43.6k prompt tokens, temp 0, 512
generated, streamed; decode = completion_tokens / (total - ttft)."""
import json, time, urllib.request
TARGET_BYTES = int(43600 * 3.8)
def build():
parts, n = [], 0
with open("/home/dank/corpora/out/calib.jsonl") as f:
for line in f:
p = json.loads(line)["prompt"]
parts.append(p)
n += len(p.encode())
if n >= TARGET_BYTES:
break
return "".join(parts)
def main(salt):
body = json.dumps({
"model": "frank-archon",
"messages": [{"role": "user", "content": f"[probe {salt}] Continue and summarize this material in English, 400 words.\n\n" + build()}],
"temperature": 0.0, "max_tokens": 512,
"stream": True, "stream_options": {"include_usage": True},
}).encode()
req = urllib.request.Request("http://localhost:7770/v1/chat/completions",
data=body, headers={"Content-Type": "application/json"})
t0 = time.time(); ttft = None; ctok = 0; usage = {}
with urllib.request.urlopen(req, timeout=2400) as r:
for raw in r:
line = raw.decode().strip()
if not line.startswith("data:"):
continue
d = line[5:].strip()
if d == "[DONE]":
break
j = json.loads(d)
if j.get("usage"):
usage = j["usage"]
for ch in j.get("choices", []):
delta = (ch.get("delta") or {})
piece = delta.get("content") or delta.get("reasoning_content") or ""
if piece and ttft is None:
ttft = time.time() - t0
ctok += 1
total = time.time() - t0
gen = usage.get("completion_tokens") or ctok
dec = gen / (total - (ttft or 0)) if total > (ttft or 0) else 0
print(json.dumps({"salt": salt, "prompt_tokens": usage.get("prompt_tokens"),
"completion_tokens": gen, "ttft_s": round(ttft or 0, 2),
"total_s": round(total, 1), "decode_tps": round(dec, 2)}))
if __name__ == "__main__":
import sys
main(sys.argv[1] if len(sys.argv) > 1 else "x")
+23 -23
View File
@@ -61,9 +61,9 @@ SERVICE_ORDER = (
"sonarr",
"lidarr",
"readarr",
"frank-large",
"frank-secondary",
"frank-primary",
"frank-dragoon",
"frank-zealot",
"frank-archon",
)
DANKCLAW_SERVICES = {
@@ -93,8 +93,8 @@ DANKCLAW_SERVICES = {
"authentik-outpost-ldap",
"caddy",
"homepage",
"frank-large",
"frank-secondary",
"frank-dragoon",
"frank-zealot",
}
DANKSERVER_SERVICES = {
@@ -113,7 +113,7 @@ DANKSERVER_SERVICES = {
"sonarr",
"lidarr",
"readarr",
"frank-primary",
"frank-archon",
"node-exporter",
}
@@ -316,7 +316,7 @@ def services_for_path(path: str, target: str) -> set[str]:
if len(parts) >= 2 and parts[1] == "open-webui":
_add(services, "open-webui")
return services
# frank-large left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
# frank-dragoon left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
# build, root unit): its files live in scripts/ and map there.
if len(parts) >= 2 and parts[1] == "observability":
# The five observability containers became Quadlet units on
@@ -371,10 +371,10 @@ def services_for_path(path: str, target: str) -> set[str]:
if name == "node-exporter":
_add(services, "node-exporter")
return services
# frank-primary's quadlet-era directory left the tree on 2026-09-26
# frank-archon's quadlet-era directory left the tree on 2026-09-26
# (the Lucebox DS4.1 unit was retired for the llama.cpp root unit —
# the trio in scripts/ now maps it). Nothing lives under
# dankserver/frank-primary/ except dated trash, and trash plans
# dankserver/frank-archon/ except dated trash, and trash plans
# nothing (the retire rule above the service blocks).
return services
@@ -400,32 +400,32 @@ def services_for_path(path: str, target: str) -> set[str]:
# change to it redeploying alertmanager is the only path that
# reaches it on the host.
_add(services, "alertmanager")
if name in {"frank-secondary.env", "frank-secondary.service", "serve-frank-secondary.sh"}:
# llama.cpp frank-secondary (2026-09-16) keeps its unit, env and serve
# files here; each one selects frank-secondary only. The ik_llama.cpp
if name in {"frank-zealot.env", "frank-zealot.service", "serve-frank-zealot.sh"}:
# llama.cpp frank-zealot (2026-09-16) keeps its unit, env and serve
# files here; each one selects frank-zealot only. The ik_llama.cpp
# trio that lived here for two days is in scripts/.AGENTTRASH/,
# which is archive material and plans nothing.
_add(services, "frank-secondary")
if name in {"frank-large.env", "frank-large.service", "serve-frank-large.sh"}:
# CIRU v4.4.1 frank-large (2026-09-19) keeps its unit, env and
_add(services, "frank-zealot")
if name in {"frank-dragoon.env", "frank-dragoon.service", "serve-frank-dragoon.sh"}:
# CIRU v4.4.1 frank-dragoon (2026-09-19) keeps its unit, env and
# serve files here the same way: a root unit on a host-built
# runtime. The download script fetches the payload, which is
# host-side state, so it selects nothing.
_add(services, "frank-large")
_add(services, "frank-dragoon")
if name == "qwen-sharp-v22.4.0.jinja":
# The Qwen-Sharp chat template frank-large loads via
# The Qwen-Sharp chat template frank-dragoon loads via
# --chat-template-file. It lived under dankserver/frank-medium/ until
# the 2026-09-26 cutover retired that directory; editing it changes
# what the live frank-large endpoint renders, so it plans
# frank-large.
_add(services, "frank-large")
if name in {"frank-primary.env", "frank-primary.service", "serve-frank-primary.sh"}:
# llama.cpp frank-primary (2026-09-26, replacing the Lucebox
# what the live frank-dragoon endpoint renders, so it plans
# frank-dragoon.
_add(services, "frank-dragoon")
if name in {"frank-archon.env", "frank-archon.service", "serve-frank-archon.sh"}:
# llama.cpp frank-archon (2026-09-26, replacing the Lucebox
# DS4.1 quadlet) keeps its unit, env and serve files here the
# same way: a root unit on the unsloth b11030-mix prebuilt
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches
# the payload, which is host-side state, so it selects nothing.
_add(services, "frank-primary")
_add(services, "frank-archon")
# Deployment/test/benchmark scripts and other repo tooling do not
# restart a runtime service.
return services
+123 -116
View File
@@ -90,12 +90,12 @@ service_is_allowed() {
dankclaw:authentik-postgres|dankclaw:authentik-server|\
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\
dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
dankclaw:frank-large|dankclaw:frank-secondary|\
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
dankserver:frank-primary|dankserver:node-exporter|\
dankserver:frank-archon|dankserver:node-exporter|\
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
frankbook-pro:readarr)
@@ -192,7 +192,7 @@ add_pending_services() {
continue
fi
[[ -n "$recorded" && "$recorded" != "$REVISION" ]] || continue
if [[ "$TARGET" == dankserver && "$service" == frank-large ]]; then continue; fi
if [[ "$TARGET" == dankserver && "$service" == frank-dragoon ]]; then continue; fi
if ! changed="$(python3 scripts/deploy-plan.py --target "$TARGET" \
--repo "$REMOTE_DIR" --base "$recorded" --head "$REVISION")"; then
echo "WARN: unable to compare state for $service; retaining it as pending" >&2
@@ -322,6 +322,12 @@ deploy_homepage() {
if [[ -d shared/homepage/fonts ]]; then
run_privileged cp -r shared/homepage/fonts /srv/homepage/
fi
if [[ -d shared/homepage/img ]]; then
run_privileged cp -r shared/homepage/img /srv/homepage/
fi
if [[ -d shared/homepage/sfx ]]; then
run_privileged cp -r shared/homepage/sfx /srv/homepage/
fi
local deployed_at
deployed_at="$(git log -1 --format=%cI)"
printf 'window.DEPLOYED_AT = "%s";\n' "$deployed_at" > /tmp/deployed.js
@@ -1086,21 +1092,21 @@ deploy_anubis() {
wait_container_healthy anubis 60 || return 1
}
# frank-large is the halogen-flash-server CONTAINER (0.16.2, digest-pinned
# through scripts/frank-large.env) serving the Quat3rnion ABLITERATED
# frank-dragoon is the halogen-flash-server CONTAINER (0.16.2, digest-pinned
# through scripts/frank-dragoon.env) serving the Quat3rnion ABLITERATED
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint —
# qwen38-flash-next-v2-swift15-abliterated.hgn (66,687,678,464 B, ~62 GiB
# resident) + the REQUIRED n-gram lookup table + the vision tower —
# MTP spec decode (HALOGEN_SPEC_ADAPT=0), 2 slots x 200192, image input —
# on the Strix Halo iGPU. It stays a ROOT unit because the serve script
# drives `podman run` directly (the frank-primary pattern): relink_unit
# points /etc/systemd/system/frank-large.service at the repo file, boot
# drives `podman run` directly (the frank-archon pattern): relink_unit
# points /etc/systemd/system/frank-dragoon.service at the repo file, boot
# persistence is that unit's [Install], and LimitMEMLOCK=infinity is
# fleet parity. 2026-10-06 cutover from the halogen 27B-dense shape —
# rollback = revert the swap commit's trio; the 27B payload stays on
# disk (~/models/halogen-qwen3.8-27b).
#
# The gates mirror serve-frank-large.sh's predicates EXACTLY (the
# The gates mirror serve-frank-dragoon.sh's predicates EXACTLY (the
# 2026-10-01 crash-loop lesson): image digest in dank's podman storage,
# each .hgn payload (checkpoint, n-gram table, vision tower) at the exact
# byte count, the tokenizer directory holding every named file as a
@@ -1108,23 +1114,23 @@ deploy_anubis() {
# artifact would leave a restart-looping unit and a dark endpoint. The
# port gate is the same as every cutover since 2026-09-16: nothing
# starts until 7770 is released.
LLM_LARGE_IMAGE="${LLM_LARGE_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
LLM_LARGE_CKPT="${LLM_LARGE_CKPT:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-v2-swift15-abliterated.hgn}"
LLM_LARGE_CKPT_BYTES="${LLM_LARGE_CKPT_BYTES:-66687678464}"
LLM_LARGE_NGRAM="${LLM_LARGE_NGRAM:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-ngram.hgn}"
LLM_LARGE_NGRAM_BYTES="${LLM_LARGE_NGRAM_BYTES:-51200246144}"
LLM_LARGE_VISION="${LLM_LARGE_VISION:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-vision.hgn}"
LLM_LARGE_VISION_BYTES="${LLM_LARGE_VISION_BYTES:-897916416}"
LLM_LARGE_TOK_DIR="${LLM_LARGE_TOK_DIR:-/home/dank/models/halogen-flashnext-v2/tokenizer}"
LLM_LARGE_TOK_FILES="${LLM_LARGE_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
LLM_DRAGOON_CKPT="${LLM_DRAGOON_CKPT:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-v2-swift15-abliterated.hgn}"
LLM_DRAGOON_CKPT_BYTES="${LLM_DRAGOON_CKPT_BYTES:-66687678464}"
LLM_DRAGOON_NGRAM="${LLM_DRAGOON_NGRAM:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-ngram.hgn}"
LLM_DRAGOON_NGRAM_BYTES="${LLM_DRAGOON_NGRAM_BYTES:-51200246144}"
LLM_DRAGOON_VISION="${LLM_DRAGOON_VISION:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-vision.hgn}"
LLM_DRAGOON_VISION_BYTES="${LLM_DRAGOON_VISION_BYTES:-897916416}"
LLM_DRAGOON_TOK_DIR="${LLM_DRAGOON_TOK_DIR:-/home/dank/models/halogen-flashnext-v2/tokenizer}"
LLM_DRAGOON_TOK_FILES="${LLM_DRAGOON_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
retire_halogen_large_quadlet() {
retire_halogen_dragoon_quadlet() {
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
systemctl --user stop frank-large.service 2>/dev/null || true
if [[ -e "$quadlet_unit_dir/frank-large.container" ]]; then
systemctl --user stop frank-dragoon.service 2>/dev/null || true
if [[ -e "$quadlet_unit_dir/frank-dragoon.container" ]]; then
mkdir -p "$trash"
mv -- "$quadlet_unit_dir/frank-large.container" "$trash/frank-large.container"
echo "retired quadlet frank-large.container -> $trash"
mv -- "$quadlet_unit_dir/frank-dragoon.container" "$trash/frank-dragoon.container"
echo "retired quadlet frank-dragoon.container -> $trash"
fi
systemctl --user daemon-reload
}
@@ -1148,164 +1154,165 @@ wait_port_free() {
return 1
}
deploy_llm_large() {
deploy_llm_dragoon() {
# Refuse rather than restart-loop; every gate below has a twin in
# serve-frank-large.sh and they must agree (2026-10-01 lesson).
# serve-frank-dragoon.sh and they must agree (2026-10-01 lesson).
# Capture podman output before grepping: `podman images | grep -q`
# races under pipefail (SIGPIPE 141 becomes "no match", 2026-09-30).
if ! grep -qF "${LLM_LARGE_IMAGE#*@}" <<<"$(podman images --format '{{.RepoDigests}}' 2>/dev/null)"; then
echo "ERROR: halogen image absent: $LLM_LARGE_IMAGE" >&2
echo "frank-large: pull it: podman pull ${LLM_LARGE_IMAGE%@*}" >&2
if ! grep -qF "${LLM_DRAGOON_IMAGE#*@}" <<<"$(podman images --format '{{.RepoDigests}}' 2>/dev/null)"; then
echo "ERROR: halogen image absent: $LLM_DRAGOON_IMAGE" >&2
echo "frank-dragoon: pull it: podman pull ${LLM_DRAGOON_IMAGE%@*}" >&2
return 1
fi
if [[ ! -f "$LLM_LARGE_CKPT" ]]; then
echo "ERROR: halogen checkpoint absent at $LLM_LARGE_CKPT" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
if [[ ! -f "$LLM_DRAGOON_CKPT" ]]; then
echo "ERROR: halogen checkpoint absent at $LLM_DRAGOON_CKPT" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1
fi
if [[ "$(stat -c %s "$LLM_LARGE_CKPT")" != "$LLM_LARGE_CKPT_BYTES" ]]; then
echo "ERROR: halogen checkpoint is not $LLM_LARGE_CKPT_BYTES bytes (partial download?)" >&2
echo "frank-large: re-check: bash scripts/download-frank-large-flashnext.sh --verify-only" >&2
if [[ "$(stat -c %s "$LLM_DRAGOON_CKPT")" != "$LLM_DRAGOON_CKPT_BYTES" ]]; then
echo "ERROR: halogen checkpoint is not $LLM_DRAGOON_CKPT_BYTES bytes (partial download?)" >&2
echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2
return 1
fi
for gate in "$LLM_LARGE_NGRAM:$LLM_LARGE_NGRAM_BYTES:n-gram table" \
"$LLM_LARGE_VISION:$LLM_LARGE_VISION_BYTES:vision tower"; do
for gate in "$LLM_DRAGOON_NGRAM:$LLM_DRAGOON_NGRAM_BYTES:n-gram table" \
"$LLM_DRAGOON_VISION:$LLM_DRAGOON_VISION_BYTES:vision tower"; do
IFS=: read -r gpath gbytes gname <<<"$gate"
if [[ ! -f "$gpath" ]]; then
echo "ERROR: halogen $gname absent at $gpath" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1
fi
if [[ "$(stat -c %s "$gpath")" != "$gbytes" ]]; then
echo "ERROR: halogen $gname is not $gbytes bytes (partial download?)" >&2
echo "frank-large: re-check: bash scripts/download-frank-large-flashnext.sh --verify-only" >&2
echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2
return 1
fi
done
if [[ -L "$LLM_LARGE_TOK_DIR" || ! -d "$LLM_LARGE_TOK_DIR" ]]; then
echo "ERROR: tokenizer dir missing or a symlink at $LLM_LARGE_TOK_DIR" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
if [[ -L "$LLM_DRAGOON_TOK_DIR" || ! -d "$LLM_DRAGOON_TOK_DIR" ]]; then
echo "ERROR: tokenizer dir missing or a symlink at $LLM_DRAGOON_TOK_DIR" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1
fi
for tf in $LLM_LARGE_TOK_FILES; do
if [[ ! -f "$LLM_LARGE_TOK_DIR/$tf" || -L "$LLM_LARGE_TOK_DIR/$tf" ]]; then
echo "ERROR: tokenizer file missing or a symlink: $LLM_LARGE_TOK_DIR/$tf" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
for tf in $LLM_DRAGOON_TOK_FILES; do
if [[ ! -f "$LLM_DRAGOON_TOK_DIR/$tf" || -L "$LLM_DRAGOON_TOK_DIR/$tf" ]]; then
echo "ERROR: tokenizer file missing or a symlink: $LLM_DRAGOON_TOK_DIR/$tf" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1
fi
done
retire_halogen_large_quadlet
retire_halogen_dragoon_quadlet
# The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
# the container) before waiting on the port. The stray sweep is keyed
# on the FULL container name, which appears in the argv of a manual
# `podman run --name frank-large-halogen` / `podman wait` but NOT in
# `podman run --name frank-dragoon-halogen` / `podman wait` but NOT in
# this deployer's own command line (`bash deploy-remote.sh
# frank-large`) — so it cannot shoot itself in the foot (the fleet
# pkill law), and it can never match frank-secondary's llama-server.
run_privileged systemctl stop frank-large.service 2>/dev/null || true
pkill -f 'frank-large-halogen' 2>/dev/null || true
podman rm -f frank-large frank-large-halogen >/dev/null 2>&1 || true
# frank-dragoon`) — so it cannot shoot itself in the foot (the fleet
# pkill law), and it can never match frank-zealot's llama-server.
run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true
pkill -f 'frank-dragoon-halogen' 2>/dev/null || true
podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true
# Nothing may start until the predecessor has released the port
# (2026-09-16: "Deactivated successfully" was not "port free").
wait_port_free 7770 || return 1
relink_unit frank-large
relink_unit frank-dragoon
run_privileged systemctl daemon-reload
run_privileged systemctl enable frank-large.service
run_privileged systemctl restart frank-large.service
wait_for_health frank-large http://127.0.0.1:7770/health 1200
run_privileged systemctl enable frank-dragoon.service
run_privileged systemctl restart frank-dragoon.service
wait_for_health frank-dragoon http://127.0.0.1:7770/health 1200
}
# deploy_llm_secondary — frank-secondary is llama.cpp (master build, CUDA sm_86) on
# deploy_llm_zealot — frank-zealot is llama.cpp (master build, CUDA sm_86) on
# the RTX 3090 as a ROOT unit: the runtime is a native binary tree under
# ~/GitPackages/llama-master/build-cuda, so there is no container image and
# no generated unit to avoid. relink_unit points
# /etc/systemd/system/frank-secondary.service at the repo file and boot
# /etc/systemd/system/frank-zealot.service at the repo file and boot
# persistence is that unit's [Install] section.
#
# It is a SEPARATE endpoint on port 7771; frank-large keeps 7770. The 3090 is
# not shared: frank-large's engine runs on the gfx1151 iGPU.
LLM_SECONDARY_MODEL_FILE="${LLM_SECONDARY_MODEL_FILE:-/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf}"
LLM_SECONDARY_RUNTIME="${LLM_SECONDARY_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
# The projector IS the vision capability here: the GGUF carries no vision
# tensors, and the publisher ships mmproj-BF16.gguf in the same repo, so a
# deploy that starts without this file silently ships a text-only
# endpoint. Checked unconditionally, like frank-large's.
LLM_SECONDARY_MMPROJ="${LLM_SECONDARY_MMPROJ:-/home/dank/models/Swift15-UncIQ3XXS/mmproj-BF16.gguf}"
# It is a SEPARATE endpoint on port 7771; frank-dragoon keeps 7770. The 3090 is
# not shared: frank-dragoon's engine runs on the gfx1151 iGPU.
LLM_ZEALOT_MODEL_FILE="${LLM_ZEALOT_MODEL_FILE:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
LLM_ZEALOT_RUNTIME="${LLM_ZEALOT_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
# The projector IS the vision capability here: the model is natively
# multimodal and the Q6_K file carries no vision tensors — the publisher ships
# its own BF16 projector in the same repo, so a deploy that starts without
# this file silently ships a text-only endpoint. Checked unconditionally, like
# frank-dragoon's.
LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
deploy_llm_secondary() {
deploy_llm_zealot() {
# Refuse rather than restart-loop: a unit whose runtime or payload is
# missing never becomes healthy, and a green deploy would leave the
# endpoint dark with nothing in the log to say why.
if [[ ! -x "$LLM_SECONDARY_RUNTIME" ]]; then
echo "ERROR: llama.cpp runtime absent at $LLM_SECONDARY_RUNTIME" >&2
echo "frank-secondary: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
echo "frank-secondary: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
echo "frank-secondary: && cmake --build build-cuda --target llama-server) first." >&2
if [[ ! -x "$LLM_ZEALOT_RUNTIME" ]]; then
echo "ERROR: llama.cpp runtime absent at $LLM_ZEALOT_RUNTIME" >&2
echo "frank-zealot: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
echo "frank-zealot: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
echo "frank-zealot: && cmake --build build-cuda --target llama-server) first." >&2
return 1
fi
for f in "$LLM_SECONDARY_MODEL_FILE" "$LLM_SECONDARY_MMPROJ"; do
for f in "$LLM_ZEALOT_MODEL_FILE" "$LLM_ZEALOT_MMPROJ"; do
if [[ ! -f "$f" ]]; then
echo "ERROR: frank-secondary payload file absent at $f" >&2
echo "frank-secondary: run scripts/download-frank-secondary.sh first;" >&2
echo "frank-secondary: refusing to start a unit that cannot load." >&2
echo "ERROR: frank-zealot payload file absent at $f" >&2
echo "frank-zealot: run scripts/download-frank-zealot.sh first;" >&2
echo "frank-zealot: refusing to start a unit that cannot load." >&2
return 1
fi
done
relink_unit frank-secondary
relink_unit frank-zealot
run_privileged systemctl daemon-reload
run_privileged systemctl enable frank-secondary.service
run_privileged systemctl restart frank-secondary.service
wait_for_health frank-secondary http://127.0.0.1:7771/health 900 || return 1
# False-green hardening (frank-large's lesson): the port answering
run_privileged systemctl enable frank-zealot.service
run_privileged systemctl restart frank-zealot.service
wait_for_health frank-zealot http://127.0.0.1:7771/health 900 || return 1
# False-green hardening (frank-dragoon's lesson): the port answering
# proves nothing about WHO answers — the unit must be active with a
# live MainPID, or a stray SGLang holdover is masquerading as deployed.
local main_pid
main_pid="$(run_privileged systemctl show -p MainPID --value frank-secondary.service)"
main_pid="$(run_privileged systemctl show -p MainPID --value frank-zealot.service)"
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
echo "FAIL: :7771 answered but frank-secondary.service has no MainPID — a stray process holds the port" >&2
echo "FAIL: :7771 answered but frank-zealot.service has no MainPID — a stray process holds the port" >&2
return 1
fi
echo "frank-secondary: health answered by unit MainPID $main_pid"
echo "frank-zealot: health answered by unit MainPID $main_pid"
}
# deploy_llm_primary — frank-primary is the radiance engine (StillDeadcode,
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
# digest-pinned stilldeadcode/radiance image) serving the OFFICIAL
# Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision from the
# rad-convert container qwen4exp-fp8-native.rad, on BOTH R9700s (TP=2) as a
# ROOT unit whose serve script drives `podman run` directly (the engine is
# the image entrypoint; the image lives in dank's rootless podman storage).
# 2026-10-05 cutover from FreeToken (pre-cutover tag
# frank-primary-freetoken-pr587-20261005); before that, the tcclaviger vLLM
# fork (tag frank-primary-vllm-shapeG-20261003). Same port and alias; the
# frank-archon-freetoken-pr587-20261005); before that, the tcclaviger vLLM
# fork (tag frank-archon-vllm-shapeG-20261003). Same port and alias; the
# harness models.yml id is unchanged.
#
# Gates: the image must exist under its digest ref in dank's rootless
# storage (FRANK_PRIMARY_IMAGE), the .rad container file must exist at the
# env's size floor (FRANK_PRIMARY_MODEL_FILE >= FRANK_PRIMARY_RAD_MIN_MIB —
# storage (FRANK_ARCHON_IMAGE), the .rad container file must exist at the
# env's size floor (FRANK_ARCHON_MODEL_FILE >= FRANK_ARCHON_RAD_MIN_MIB —
# a truncated convert is a guaranteed long-timeout boot), and the unit env
# must be complete. Per-file sha256 verification is the fetch script's job
# at download time and rad-convert/rad-info's job at build time, never
# re-hashed here.
LLM_PRIMARY_CONTAINER=frank-primary
LLM_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe"
LLM_ARCHON_CONTAINER=frank-archon
LLM_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
deploy_llm_primary() {
deploy_llm_archon() {
# Refuse rather than restart-loop: a unit whose image or payload is
# missing never becomes healthy, and a green deploy would leave the
# endpoint dark with nothing in the log to say why.
local env_file="/home/dank/serverAppConfigs/scripts/frank-primary.env"
local env_file="/home/dank/serverAppConfigs/scripts/frank-archon.env"
local image model_dir model_file rad_min mib
if [[ ! -f "$env_file" ]]; then
echo "ERROR: frank-primary env file absent: $env_file" >&2
echo "ERROR: frank-archon env file absent: $env_file" >&2
return 1
fi
image="$(sed -n 's/^FRANK_PRIMARY_IMAGE=//p' "$env_file" | tail -1)"
model_dir="$(sed -n 's/^FRANK_PRIMARY_MODEL_DIR=//p' "$env_file" | tail -1)"
model_file="$(sed -n 's/^FRANK_PRIMARY_MODEL_FILE=//p' "$env_file" | tail -1)"
rad_min="$(sed -n 's/^FRANK_PRIMARY_RAD_MIN_MIB=//p' "$env_file" | tail -1)"
image="$(sed -n 's/^FRANK_ARCHON_IMAGE=//p' "$env_file" | tail -1)"
model_dir="$(sed -n 's/^FRANK_ARCHON_MODEL_DIR=//p' "$env_file" | tail -1)"
model_file="$(sed -n 's/^FRANK_ARCHON_MODEL_FILE=//p' "$env_file" | tail -1)"
rad_min="$(sed -n 's/^FRANK_ARCHON_RAD_MIN_MIB=//p' "$env_file" | tail -1)"
if [[ -z "$image" || -z "$model_dir" || -z "$model_file" || -z "$rad_min" ]]; then
echo "ERROR: frank-primary.env missing FRANK_PRIMARY_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
echo "ERROR: frank-archon.env missing FRANK_ARCHON_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
return 1
fi
# The image gate is the digest ref itself: a name@sha256 ref exists in
@@ -1313,18 +1320,18 @@ deploy_llm_primary() {
# pipefail/grep race (the 2026-09-30 lesson).
if ! podman image exists "$image"; then
echo "ERROR: runtime image absent from storage: $image" >&2
echo "frank-primary: podman pull docker://stilldeadcode/radiance:latest" >&2
echo "frank-archon: podman pull docker://stilldeadcode/radiance:latest" >&2
return 1
fi
if [[ ! -f "$model_dir/$model_file" ]]; then
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
echo "frank-primary: run scripts/build-frank-primary-rad.sh" >&2
echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
return 1
fi
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
if [[ "$mib" -lt "$rad_min" ]]; then
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
echo "frank-primary: rebuild it (scripts/build-frank-primary-rad.sh --force)" >&2
echo "frank-archon: rebuild it (scripts/build-frank-archon-rad.sh --force)" >&2
return 1
fi
@@ -1336,19 +1343,19 @@ deploy_llm_primary() {
# The legacy identities (the retired R9V container, the ad hoc
# experiment container qwen-fp8, the retired glm-probe) hold the same
# GPUs and die here too.
run_privileged systemctl stop frank-primary.service 2>/dev/null || true
run_privileged systemctl stop frank-archon.service 2>/dev/null || true
local ctr
for ctr in $LLM_PRIMARY_LEGACY_CONTAINERS "$LLM_PRIMARY_CONTAINER"; do
for ctr in $LLM_ARCHON_LEGACY_CONTAINERS "$LLM_ARCHON_CONTAINER"; do
podman rm -f "$ctr" >/dev/null 2>&1 || true
done
# Nothing starts until the port is released: the lucebox cutover
# (2026-09-26) and the frank-large cutover (2026-09-16) both failed
# (2026-09-26) and the frank-dragoon cutover (2026-09-16) both failed
# exactly here when the next engine bound 7770 a second too early.
wait_port_free 7770 || return 1
relink_unit frank-primary
relink_unit frank-archon
run_privileged systemctl daemon-reload
run_privileged systemctl enable frank-primary.service
run_privileged systemctl restart frank-primary.service
run_privileged systemctl enable frank-archon.service
run_privileged systemctl restart frank-archon.service
# Cold start: .rad load + PLE pin + expert host-pool upload (the
# publisher's compose healthcheck allows a 15-minute start_period).
# Parity with every predecessor on this port. The HTTP listener binds
@@ -1359,14 +1366,14 @@ deploy_llm_primary() {
local i ok=0
for ((i=1; i<=900; i++)); do
if curl -fsS http://127.0.0.1:7770/health 2>/dev/null >/dev/null; then
echo "frank-primary engine healthy after approximately $((i * 2))s"
echo "frank-archon engine healthy after approximately $((i * 2))s"
ok=1
break
fi
sleep 2
done
if [[ "$ok" != 1 ]]; then
echo "FAIL: frank-primary /health never answered 200 in 1800s" >&2
echo "FAIL: frank-archon /health never answered 200 in 1800s" >&2
return 1
fi
# False-green hardening (the 2026-09-26 lesson): a bare curl against the
@@ -1374,12 +1381,12 @@ deploy_llm_primary() {
# it while the unit is stopped. The deploy is only green if the unit
# itself is active with a live MainPID behind the health answer.
local main_pid
main_pid="$(run_privileged systemctl show -p MainPID --value frank-primary.service)"
main_pid="$(run_privileged systemctl show -p MainPID --value frank-archon.service)"
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
echo "FAIL: :7770 answered but frank-primary.service has no MainPID — a stray process holds the port" >&2
echo "FAIL: :7770 answered but frank-archon.service has no MainPID — a stray process holds the port" >&2
return 1
fi
echo "frank-primary: health answered by unit MainPID $main_pid"
echo "frank-archon: health answered by unit MainPID $main_pid"
}
wait_for_health() {
@@ -1566,9 +1573,9 @@ deploy_service() {
anubis) deploy_anubis ;;
seerr) deploy_seerr ;;
marreta) deploy_marreta ;;
frank-large) deploy_llm_large ;;
frank-secondary) deploy_llm_secondary ;;
frank-primary) deploy_llm_primary ;;
frank-dragoon) deploy_llm_dragoon ;;
frank-zealot) deploy_llm_zealot ;;
frank-archon) deploy_llm_archon ;;
hermes|hermes-dashboard|hermes-webui) deploy_hermes ;;
romm|romm-mariadb|romm-redis|mariadb|redis) deploy_romm_stack "$service" ;;
jellyfin) deploy_jellyfin ;;
+3 -3
View File
@@ -35,7 +35,7 @@ Options:
Examples:
./scripts/deploy.sh dankclaw
./scripts/deploy.sh dankclaw --service searxng
./scripts/deploy.sh dankserver --service frank-primary
./scripts/deploy.sh dankserver --service frank-archon
./scripts/deploy.sh dankclaw --dry-run
EOF
}
@@ -141,12 +141,12 @@ valid_service() {
dankclaw:alertmanager|\
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
dankclaw:frank-large|dankclaw:frank-secondary|\
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|\
dankserver:frank-primary|\
dankserver:frank-archon|\
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
frankbook-pro:readarr)
+117
View File
@@ -0,0 +1,117 @@
#!/usr/bin/env python3
"""derive-protected-experts.py — rewrite the recipe's protected-expert section
from a calibration recording. RUNS ON DANKSERVER (venv: numpy).
The law (docs/MOE-W4.md "Protected experts", restated in the recipe):
an expert whose share of its layer's down-input energy is at least 3% AND
at least three times its share of the routing is kept bf16; the count must
be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest
as two equal tables by parity) — an odd half gains its next most energetic
non-qualifying expert.
Data: the calibration dir's per-expert down Grams,
gram.r<rank>.blk.L.ffn_down_exps.<e>.bin — 32-byte RADGRAM2 header
(magic, k, rank, rows, domain) then k*k f32, named with the checkpoint's
GLOBAL expert numbering (arch/common/rad_block_moe_fp8.h: ge =
expert_base + e). rows (header) is the expert's routing count; the trace
of the Gram is its down-input energy. Rank halves: 0-255, 256-511
(contiguous expert_base split at TP=2).
Output: the cast-rule lines (recipe format) on stdout, a report on stderr.
Usage: derive-protected-experts.py <calib dir> [--min-share 0.03]
[--min-route-mult 3.0]
"""
import argparse
import glob
import os
import re
import struct
import sys
import numpy as np
NAME_RE = re.compile(r"^gram\.r(\d)\.blk\.(\d+)\.ffn_down_exps\.(\d+)\.bin$")
def read_gram(path):
"""(k, rank, rows, trace) from a RADGRAM2 file (full matrix on disk)."""
with open(path, "rb") as f:
hdr = f.read(32)
magic, k, rank, rows, domain = struct.unpack("<8sIIQI", hdr[:28])
if magic != b"RADGRAM2":
raise SystemExit(f"not a RADGRAM2 file: {path}")
n = k * k
a = np.frombuffer(f.read(n * 4), dtype=np.float32)
if a.size != n:
raise SystemExit(f"short gram file: {path}")
tr = float(a.reshape(k, k).diagonal().sum())
return k, rank, int(rows), tr
def main():
ap = argparse.ArgumentParser()
ap.add_argument("calib_dir")
ap.add_argument("--min-share", type=float, default=0.03)
ap.add_argument("--min-route-mult", type=float, default=3.0)
a = ap.parse_args()
# layer -> global expert -> (rank, rows, energy)
data = {}
for path in sorted(glob.glob(os.path.join(
a.calib_dir, "gram.r*.ffn_down_exps.*.bin"))):
m = NAME_RE.match(os.path.basename(path))
if not m:
continue
rank, layer, expert = int(m.group(1)), int(m.group(2)), int(m.group(3))
k, _, rows, tr = read_gram(path)
data.setdefault(layer, {})[expert] = (rank, rows, tr)
if not data:
raise SystemExit("no per-expert down grams found")
lines, report = [], []
total = 0
for layer in sorted(data):
experts = data[layer]
tot_e = sum(v[2] for v in experts.values())
tot_r = sum(v[1] for v in experts.values())
if tot_e <= 0 or tot_r <= 0:
report.append(f"blk.{layer}: empty recording, skipped")
continue
qual, pool = [], []
for e, (rank, rows, tr) in experts.items():
es, rs = tr / tot_e, rows / max(tot_r, 1)
rec = (e, rank, es, rs)
if es >= a.min_share and es >= a.min_route_mult * rs:
qual.append(rec)
elif es > 0:
pool.append(rec)
chosen = {r[0]: (r[2], r[3]) for r in qual}
for half in (0, 1): # rank halves by the contiguous expert_base split
n_half = sum(1 for e in chosen if e // 256 == half)
if n_half % 2:
pool.sort(key=lambda r: -r[2])
for cand in pool:
if cand[1] == half and cand[0] not in chosen:
chosen[cand[0]] = (cand[2], cand[3])
report.append(
f"blk.{layer}: half {half} odd -> added e{cand[0]} "
f"(energy {cand[2]*100:.2f}%)")
break
for e in sorted(chosen):
es, rs = chosen[e]
lines.append(f"blk.{layer}.ffn_*_exps.{e}.weight"
f" cast dtype=bf16")
report.append(f"blk.{layer} e{e:3d} (rank {e // 256}): "
f"energy {es*100:5.2f}%, routing {rs*100:5.3f}%, "
f"ratio {es/max(rs,1e-12):6.1f}")
total += len(chosen)
print("\n".join(lines))
print("\n".join(report), file=sys.stderr)
print(f"total protected: {total}", file=sys.stderr)
if __name__ == "__main__":
main()
+113
View File
@@ -0,0 +1,113 @@
#!/usr/bin/env bash
# download-frank-archon-swift15.sh — payload fetch/verify for the planned
# frank-archon swap to the Swift 1.5 finetune: ukisai/Swift1.5-Qwen3.8-Flash-Next
# (BF16 source, 149 files, 360,037,355,038 bytes = 335.3 GiB: 131 safetensors
# shards + weight-map index + tokenizer + vision preprocessing configs + chat
# template + licenses). Key set verified identical to the official
# Qwen/Qwen3.8-Flash-Next base (0 added / 0 missing), so the qwen4exp recipes
# apply verbatim; the checkpoint ships BF16 with NO quantization_config.
# Repo is UNGATED — no token needed (never put one on the command line).
#
# bash scripts/download-frank-archon-swift15.sh --fetch resume/complete
# bash scripts/download-frank-archon-swift15.sh verify only
#
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
# listing/resume downloader, curl+jq only): per-file size checks against
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
# Verify (default) is the AGENTS.md law in three parts: every listed file
# present with the listed size, the shard count matches the weight-map
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
# only the public API.
set -euo pipefail
REPO="ukisai/Swift1.5-Qwen3.8-Flash-Next"
DEST="/home/dank/models/ukisai/Swift1.5-Qwen3.8-Flash-Next"
EXPECTED_SHARDS=131
mode="verify"
[[ "${1:-}" == "--fetch" ]] && mode="fetch"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [[ "$mode" == "fetch" ]]; then
# HFGET_JOBS=4 is hfget's default (a few streams saturate the link).
bash "$SCRIPT_DIR/hfget.sh" "$REPO" "$DEST" main
fi
echo "verifying local payload: $DEST"
fail=0
INDEX="$DEST/model.safetensors.index.json"
if [[ ! -f "$INDEX" ]]; then
echo " BAD weight-map index missing: $INDEX" >&2
exit 1
fi
python3 - "$REPO" "$DEST" <<'PY' || fail=1
import hashlib, json, os, sys, urllib.request
repo, dest = sys.argv[1], sys.argv[2]
def listing():
"""The tree API, following the Link cursor exactly like hfget.sh."""
out, api, pages = [], (
f"https://huggingface.co/api/models/{repo}/tree/main"
"?recursive=true&expand=true"), 0
while api:
with urllib.request.urlopen(api) as r:
out += json.load(r)
api = None
link = r.headers.get("Link", "")
for part in link.split(","):
if 'rel="next"' in part:
api = part[part.find("<") + 1:part.find(">")]
pages += 1
if pages > 200:
raise SystemExit("listing did not terminate")
return out
files = [e for e in listing() if e.get("type") == "file"]
sizes = {e["path"]: e.get("size", 0) for e in files}
lfs = {e["path"]: e["lfs"]["oid"] for e in files if e.get("lfs")}
bad = 0
for path, want in sorted(sizes.items()):
p = os.path.join(dest, path)
if not os.path.isfile(p):
print(f" BAD {path}: missing")
bad += 1
elif os.path.getsize(p) != want:
print(f" BAD {path}: {os.path.getsize(p)} bytes, listing says {want}")
bad += 1
print(f" {'OK' if bad == 0 else 'FAIL'}: {len(sizes) - bad}/{len(sizes)} files present at listed size")
if bad:
raise SystemExit(1)
# Shard set against the weight-map index.
wm = json.load(open(os.path.join(dest, "model.safetensors.index.json")))["weight_map"]
shards = sorted(set(wm.values()))
print(f" index maps {len(wm)} tensors across {len(shards)} shards")
if len(shards) != 131:
print(f" BAD shard count {len(shards)}, want 131")
raise SystemExit(1)
# Content: every LFS-tracked file hashed against the Hub oid. The safetensors
# shards dominate the bytes; tokenizer/config files ride the same check when
# the Hub tracks them in LFS (they are plain blobs here, already size-checked).
todo = [p for p in shards if p in lfs]
for i, path in enumerate(todo):
h = hashlib.sha256()
with open(os.path.join(dest, path), "rb") as f:
for chunk in iter(lambda: f.read(1 << 24), b""):
h.update(chunk)
if h.hexdigest() != lfs[path]:
print(f" BAD {path}: sha256 mismatch (want {lfs[path]})")
bad += 1
if (i + 1) % 16 == 0:
print(f" ...hashed {i + 1}/{len(todo)}")
print(f" {'OK' if bad == 0 else 'FAIL'}: {len(todo) - bad}/{len(todo)} shards sha256-match the Hub")
if bad:
raise SystemExit(1)
PY
[[ "$fail" -eq 0 ]] || exit 1
echo "payload verified: 131 shards + index + tokenizer + configs, sizes and sha256 checked"
@@ -1,13 +1,13 @@
#!/usr/bin/env bash
# download-frank-primary.sh — payload fetch/verify for frank-primary:
# download-frank-archon.sh — payload fetch/verify for frank-archon:
# the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint (131 safetensors
# shards + weight-map index + tokenizer + vision preprocessing configs,
# ~172.8 GB). 2026-10-05: replaces the orcarouter uncensored payload of the
# FreeToken era (that script lived here; its repo was gated and needed
# HF_TOKEN — this one is UNGATED and needs no token at all).
#
# bash scripts/download-frank-primary.sh --fetch resume/complete
# bash scripts/download-frank-primary.sh verify only
# bash scripts/download-frank-archon.sh --fetch resume/complete
# bash scripts/download-frank-archon.sh verify only
#
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
# listing/resume downloader, curl+jq only): per-file size checks against
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# download-frank-large-flashnext.sh — fetch frank-large's flash-server v2
# download-frank-dragoon-flashnext.sh — fetch frank-dragoon's flash-server v2
# payload: the Quat3rnion abliterated Swift1.5 checkpoint of peonist's v2
# .hgn, the REQUIRED lookup table and the optional vision tower from
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
@@ -29,7 +29,7 @@
# fetched fresh rather than symlinked, per the no-symlink law.
set -euo pipefail
DEST="${FRANK_LARGE_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
ABLIT="https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated/resolve/main"
PEONIST="https://huggingface.co/peonist-ai/halogen-qwen3.8-flash-next/resolve"
NGRAM_REV="5cc17cea1a10b502b4a5db41d8d2b1a0f22dba26"
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# download-frank-large-halogen.sh — fetch frank-large's halogen payload:
# download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload:
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet
# secret law, none would ever touch argv).
@@ -10,7 +10,7 @@
# existing payload without downloading.
set -euo pipefail
DEST="${FRANK_LARGE_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
BASE="https://huggingface.co/peonist-ai/halogen-qwen3.8-27b/resolve/main"
FILES=(qwen3.8-27b-p1w4d-d2.hgn
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Download kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF for
# frank-large (Dankclaw iGPU). Public repo — no auth header, and never a token
# frank-dragoon (Dankclaw iGPU). Public repo — no auth header, and never a token
# on the command line (it is world-readable in /proc/<pid>/cmdline).
#
# The payload is the repo's Q6 AGENT rung:
@@ -10,7 +10,7 @@
# names for MTP serving ("Take Q6 AGENT (114) if you want protected
# heads"). Like every ROCmFPX file it loads on nothing but the fork.
#
# The other two files frank-large needs are NOT re-downloaded:
# The other two files frank-dragoon needs are NOT re-downloaded:
# * mtp-Qwen3.8-27B-Q4_0.gguf — the uncensored repo redistributes the
# ALIGNED repo's draft head byte for byte (identical Hub LFS oid
# 051a1764…), so it is hardlinked out of the aligned payload dir when that
@@ -25,7 +25,7 @@
# byte count and the Hub LFS sha256. Idempotent: complete files are skipped.
#
# Usage:
# scripts/download-frank-large-kingjones-uncensored.sh [DEST_DIR] [--verify-only]
# scripts/download-frank-dragoon-kingjones-uncensored.sh [DEST_DIR] [--verify-only]
set -euo pipefail
REPO="kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF"
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Download kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF for frank-large
# Download kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF for frank-dragoon
# (Dankclaw iGPU). Public repo — no auth header, and never a token on the
# command line (it is world-readable in /proc/<pid>/cmdline).
#
@@ -24,7 +24,7 @@
# come from the Hub LFS oids (api/models/<repo>/tree/main).
#
# Usage:
# scripts/download-frank-large-kingjones.sh [DEST_DIR] [--verify-only]
# scripts/download-frank-dragoon-kingjones.sh [DEST_DIR] [--verify-only]
set -euo pipefail
REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF"
@@ -1,9 +1,9 @@
#!/usr/bin/env bash
# Download and assemble jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca (the frank-large
# Download and assemble jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca (the frank-dragoon
# model) on Dankclaw.
#
# bash scripts/download-frank-large-orca.sh
# bash scripts/download-frank-large-orca.sh --verify-only
# bash scripts/download-frank-dragoon-orca.sh
# bash scripts/download-frank-dragoon-orca.sh --verify-only
#
# The package is 136 GB of Hub-sized transport parts (HF caps new uploads at
# 50 GB, so the 79 GB main GGUF, the 52 GB PLE payload and the 4 GB MTP head
@@ -105,4 +105,4 @@ fi
echo "verifying assembled files against SHA256SUMS"
(cd "$MODEL_DIR" && sha256sum -c SHA256SUMS)
(cd "$MODEL_DIR/vision" && sha256sum -c SHA256SUMS)
echo "frank-large payload complete: $MODEL_DIR"
echo "frank-dragoon payload complete: $MODEL_DIR"
@@ -1,9 +1,9 @@
#!/usr/bin/env bash
# Download the Q4_K_XL frank-large payload on Dankclaw:
# Download the Q4_K_XL frank-dragoon payload on Dankclaw:
# unsloth/Qwen3.8-27B-GGUF -> Qwen3.8-27B-UD-Q4_K_XL.gguf
#
# bash scripts/download-frank-large-q4.sh
# bash scripts/download-frank-large-q4.sh --verify-only
# bash scripts/download-frank-dragoon-q4.sh
# bash scripts/download-frank-dragoon-q4.sh --verify-only
#
# Integrity: the file is checked against its declared byte count and sha256
# (the LFS oid from the Hub tree API, 2026-10-01). NO AUTH HEADER IS USED:
@@ -21,12 +21,12 @@
# x 2 (K,V) x 2 B), so 4 sessions x 163840 ctx = 640,000 tokens = 39.06 GiB.
#
# The speculative drafter is UNCHANGED and still fetched by its own script
# (scripts/download-frank-large.sh, z-lab's Hub repo). That name must not
# appear here at all — the frank-large seam greps THIS file for it to prove
# (scripts/download-frank-dragoon.sh, z-lab's Hub repo). That name must not
# appear here at all — the frank-dragoon seam greps THIS file for it to prove
# the drafter was not folded into the payload fetch.
#
# NOTE ON MTP: this payload has no baked-in MTP head (the Hub repo ships one
# separately as MTP/mtp-Qwen3.8-27B-Q4_0.gguf). frank-large does not need it —
# separately as MTP/mtp-Qwen3.8-27B-Q4_0.gguf). frank-dragoon does not need it —
# spec decode here is the block-diffusion drafter, not MTP.
set -euo pipefail
@@ -1,10 +1,10 @@
#!/usr/bin/env bash
# Download the Swift-1.5 frank-large candidate on Dankclaw:
# Download the Swift-1.5 frank-dragoon candidate on Dankclaw:
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
#
# bash scripts/download-frank-large-swift.sh
# bash scripts/download-frank-large-swift.sh --verify-only
# bash scripts/download-frank-dragoon-swift.sh
# bash scripts/download-frank-dragoon-swift.sh --verify-only
#
# Integrity: the file is checked against its declared byte count and sha256
# (the LFS oid from the Hub, 2026-09-28). NO AUTH HEADER IS USED: the repo is
@@ -15,9 +15,9 @@
# Layout after a successful run (MODEL_DIR):
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf 31457990272
#
# This is the CANDIDATE payload, not the deployed one. frank-large still
# serves scripts/download-frank-large.sh's output; do not repoint
# frank-large.env at MODEL_DIR until the gufo load test passes.
# This is the CANDIDATE payload, not the deployed one. frank-dragoon still
# serves scripts/download-frank-dragoon.sh's output; do not repoint
# frank-dragoon.env at MODEL_DIR until the gufo load test passes.
set -euo pipefail
MODEL_DIR="/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL"
@@ -1,9 +1,9 @@
#!/usr/bin/env bash
# Download unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (the
# frank-large model) on Dankclaw.
# frank-dragoon model) on Dankclaw.
#
# bash scripts/download-frank-large-unsloth.sh
# bash scripts/download-frank-large-unsloth.sh --verify-only
# bash scripts/download-frank-dragoon-unsloth.sh
# bash scripts/download-frank-dragoon-unsloth.sh --verify-only
#
# The UD-IQ4_XS GGUF ships as three llama.cpp split parts (HF caps uploads at
# 50 GB). The parts ARE the canonical payload: the serve script points -m at
@@ -112,4 +112,4 @@ if [[ "$mode" == "download" ]]; then
[[ -e "$MODEL_DIR/vision/mmproj-F16.gguf" ]] || mv "$MODEL_DIR/mmproj-F16.gguf" "$MODEL_DIR/vision/"
fi
echo "frank-large payload complete: $MODEL_DIR"
echo "frank-dragoon payload complete: $MODEL_DIR"
@@ -1,10 +1,10 @@
#!/usr/bin/env bash
# Download the frank-large payload on Dankclaw:
# Download the frank-dragoon payload on Dankclaw:
# - unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (the 27B dense main GGUF)
# - z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M (the DFlash2 drafter)
#
# bash scripts/download-frank-large.sh
# bash scripts/download-frank-large.sh --verify-only
# bash scripts/download-frank-dragoon.sh
# bash scripts/download-frank-dragoon.sh --verify-only
#
# Integrity: every file is checked against its declared byte count and sha256
# (the values below are the LFS oids from the Hub, 2026-09-21). An HF token
@@ -86,4 +86,4 @@ if [[ "$mode" == "download" ]]; then
done
fi
echo "frank-large payload complete: $MODEL_DIR"
echo "frank-dragoon payload complete: $MODEL_DIR"
-88
View File
@@ -1,88 +0,0 @@
#!/usr/bin/env bash
# Download the frank-secondary payload for llama.cpp on the RTX 3090:
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF:RCO-IQ3_XXS
# **with the MTP head in the file** (the -mtp variant), plus the
# PUBLISHER's OWN vision projector (mmproj-BF16.gguf, same repo). Public
# repo — no auth header, and never a token on the command line (it is
# world-readable in /proc/<pid>/cmdline).
#
# Fresh downloads, no resume (`curl -C -` once produced a 213 MB oversized
# file on a 31 GiB payload); -f/--fail-with-body so an HTTP error is not a
# success, then byte count and sha256. This repo is on Xet storage, so
# the Hub tree/blob APIs expose no LFS oid: the sha256 values below were
# computed locally after the first verified fetch (byte counts match the
# Hub's declared sizes). Idempotent: complete files are skipped. The
# projector is stored under a self-documenting dest name; the URL name
# and sha stay pinned in the table below.
#
# 2026-10-06 (Frank, evening): the payload moved to the UNCELERSED
# Swift-1.5-Qwen3.8-27B at RCO-IQ3_XXS (1.56 GiB lighter than the
# IQ3_S-mtp it replaced; the lighter quant funds the MTP draft ctx plus
# the second 131,072 slot). Rollback payload stays on disk:
# /home/dank/models/Swift15-IQ3S/ (the IQ3_S-mtp that was live until this
# swap); earlier payloads Ornith-Q8/ and MiMo-Q6K/ as well. Their
# re-fetch recipes are in git history (predecessors of this script).
#
# Usage:
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only]
set -euo pipefail
DEST=""
VERIFY_ONLY=0
for arg in "$@"; do
case "$arg" in
--verify-only) VERIFY_ONLY=1 ;;
*) DEST="$arg" ;;
esac
done
DEST="${DEST:-/home/dank/models/Swift15-UncIQ3XXS}"
# repo | hub-name | dest-name | bytes | sha256 (computed locally; Xet repo)
FILES=(
"ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF|RCO/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf|Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf|10442827392|e13d3e7400b6c7f980ebd30c0edd6be43eb1865b50be8562a9db042706c7d803"
"ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF|mmproj-BF16.gguf|mmproj-BF16.gguf|931145888|19acd7fcf4ee504328566eddf0f8e603ef853a205b4830209d2a4908213abdea"
)
mkdir -p "$DEST"
status=0
for entry in "${FILES[@]}"; do
IFS='|' read -r repo hubname destname size sha <<<"$entry"
url="https://huggingface.co/${repo}/resolve/main/${hubname}"
f="$DEST/$destname"
have=0
if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
if sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
have=1
fi
fi
if (( have )); then
echo "OK $destname"
continue
fi
if (( VERIFY_ONLY )); then
echo "MISSING $destname"
status=1
continue
fi
echo "FETCHING $destname (from $repo)"
tmp="$f.part"
curl -fsSL --fail-with-body -o "$tmp" "$url"
mv "$tmp" "$f"
if [[ "$(stat -c%s "$f")" != "$size" ]]; then
echo "SIZE FAIL $destname ($(stat -c%s "$f") != $size)"
status=1
continue
fi
if ! sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
echo "SHA FAIL $destname"
status=1
continue
fi
echo "VERIFIED $destname"
done
if (( status == 0 )); then
echo "DOWNLOAD_DONE $DEST"
else
echo "DOWNLOAD_FAILED $DEST" >&2
fi
exit "$status"
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env bash
# Download the frank-zealot payload for llama.cpp on the RTX 3090:
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the
# PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively
# multimodal, so this file is the endpoint's eyes). Public repo — no auth
# header, and never a token on the command line (it is world-readable in
# /proc/<pid>/cmdline).
#
# Fresh downloads, no resume (`curl -C -` once produced a 213 MB oversized
# file on a 31 GiB payload); -f/--fail-with-body so an HTTP error is not a
# success, then byte count and sha256. The Hub exposes no LFS oid for this
# repo (Xet storage), so the sha256 values below were computed locally after
# the first verified fetch — byte counts match the Hub's declared sizes.
# A sha of "-" means "first fetch: size-verified, sha printed for pinning".
# Idempotent: complete files are skipped.
#
# 2026-10-07 (Frank): the payload moved to Qwen3.5-9B-Uncensored Q6_K at
# 2 x 200,704 (qwen35 hybrid: 32 layers, 8 full-attention + 24 Gated
# DeltaNet, no MTP tensors in this file). Rollback payloads stay on disk:
# /home/dank/models/Gemma4-26B-A4B/ (Q4_K_M, the 2026-10-07 era being
# replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/.
#
# Usage:
# scripts/download-frank-zealot.sh [DEST_DIR] [--verify-only]
set -euo pipefail
DEST=""
VERIFY_ONLY=0
for arg in "$@"; do
case "$arg" in
--verify-only) VERIFY_ONLY=1 ;;
*) DEST="$arg" ;;
esac
done
DEST="${DEST:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K}"
# repo | hub-name | dest-name | bytes | sha256 (computed locally; Xet repo)
FILES=(
"HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive|Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf|Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf|7359259008|c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986"
"HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive|mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf|mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf|921704448|05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934"
)
status=0
for entry in "${FILES[@]}"; do
IFS='|' read -r repo hubname destname size sha <<<"$entry"
url="https://huggingface.co/${repo}/resolve/main/${hubname}"
f="$DEST/$destname"
have=0
if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
if [[ "$sha" == "-" ]]; then
have=1
elif sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
have=1
fi
fi
if (( have )); then
echo "OK $destname"
continue
fi
if (( VERIFY_ONLY )); then
echo "MISSING $destname"
status=1
continue
fi
echo "FETCHING $destname (from $repo)"
tmp="$f.part"
curl -fsSL --fail-with-body -o "$tmp" "$url"
mv "$tmp" "$f"
if [[ "$(stat -c%s "$f")" != "$size" ]]; then
echo "SIZE FAIL $destname ($(stat -c%s "$f") != $size)"
status=1
continue
fi
if [[ "$sha" == "-" ]]; then
echo "VERIFIED $destname (size only) sha256=$(sha256sum "$f" | cut -d' ' -f1)"
continue
fi
if ! sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
echo "SHA FAIL $destname"
status=1
continue
fi
echo "VERIFIED $destname"
done
if (( status == 0 )); then
echo "DOWNLOAD_DONE $DEST"
else
echo "DOWNLOAD_FAILED $DEST" >&2
fi
exit "$status"
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env bash
# fleet-probes-swift15.sh — RUNS ON DANKSERVER. The cutover contract probes.
set -uo pipefail
B=http://localhost:7770/v1
echo "== vision (solid red 512px, expect 'Red')"
python3 - <<'PY'
import base64, json, urllib.request, struct, zlib
def png(color):
w = h = 512
raw = b"".join(b"\x00" + bytes(color) * w for _ in range(h))
def chunk(t, d):
c = t + d
return struct.pack(">I", len(d)) + c + struct.pack(">I", zlib.crc32(c))
return (b"\x89PNG\r\n\x1a\n"
+ chunk(b"IHDR", struct.pack(">IIBBBBB", w, h, 8, 2, 0, 0, 0))
+ chunk(b"IDAT", zlib.compress(raw)) + chunk(b"IEND", b""))
b64 = base64.b64encode(png((255, 0, 0))).decode()
body = json.dumps({"model": "frank-archon", "max_tokens": 256, "temperature": 0,
"messages": [{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": "data:image/png;base64," + b64}},
{"type": "text", "text": "What color is this image? Answer with one word."}]}]}).encode()
r = urllib.request.Request("http://localhost:7770/v1" + "/chat/completions",
data=body, headers={"Content-Type": "application/json"})
d = json.load(urllib.request.urlopen(r, timeout=300))
m = d["choices"][0]["message"]
ans = (m.get("content") or m.get("reasoning_content") or "")
print("VISION ANSWER:", ans[:80].replace("\n", " "))
PY
echo "== tools (expect tool_calls)"
curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' -d '{
"model":"frank-archon","temperature":0,"max_tokens":128,
"messages":[{"role":"user","content":"What is 23 times 47? Use the calculator tool."}],
"tools":[{"type":"function","function":{"name":"calculator","description":"evaluate arithmetic",
"parameters":{"type":"object","properties":{"expr":{"type":"string"}},"required":["expr"]}}}]}' \
| python3 -c 'import json,sys; d=json.load(sys.stdin); m=d["choices"][0]["message"]; print("TOOL_CALLS:", m.get("tool_calls"))'
echo "== reasoning_effort fold (xhigh/medium/low/high all 200 on froggeric v22.5)"
for lvl in xhigh medium low high; do
code=$(curl -s -o /dev/null -w "%{http_code}" -m 120 $B/chat/completions \
-H 'Content-Type: application/json' \
-d "{\"model\":\"frank-archon\",\"max_tokens\":8,\"reasoning_effort\":\"$lvl\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}")
echo " reasoning_effort=$lvl -> HTTP $code"
done
echo "== reasoning_content channel"
curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' \
-d '{"model":"frank-archon","max_tokens":64,"messages":[{"role":"user","content":"Is 97 prime? Think briefly."}]}' \
| python3 -c 'import json,sys; m=json.load(sys.stdin)["choices"][0]["message"]; print("has reasoning_content:", bool(m.get("reasoning_content")), "| content:", (m.get("content") or "")[:60].replace(chr(10)," "))'
echo "== response_format json_object"
curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' \
-d '{"model":"frank-archon","max_tokens":64,"response_format":{"type":"json_object"},"messages":[{"role":"user","content":"Return JSON {\"ok\": true}"}]}' \
| python3 -c 'import json,sys; d=json.load(sys.stdin); print("JSON MODE:", (d["choices"][0]["message"].get("content") or "")[:60])'
echo "PROBES_DONE"
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Measure per-block / per-expert expert-routing activity on live frank-primary.
"""Measure per-block / per-expert expert-routing activity on live frank-archon.
Method: AMD IBS op sampling (data addresses, `perf record -e ibs_op// --data`)
on the llama-server process while concurrent generations of mixed content run.
+286
View File
@@ -0,0 +1,286 @@
# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving
# ukisai/Swift1.5-Qwen3.8-Flash-Next converted to GPTQ w4 (MODEL_FILE below,
# built by scripts/convert-swift15-w4dn8.sh) on BOTH R9700s (TP=2),
# dankserver :7770.
#
# SHAPE (2026-10-13): THREE decode slots x 200,000 tokens, fp8 (quantized) KV,
# with the per-card budget spent on resident experts: 25,600 MiB of slab +
# 5,120 MiB of pool = 30,720 MiB, inside the 30,969 MiB ceiling the engine's own
# budget table allows (card 32,624 MiB, settle floor 209 MiB). The pool is sized
# to the contract and nothing more (3 x 200,000 needs 5,063 MiB at fp8; 5,120
# predicts 608,027 tokens, 1.3 % slack), because every MiB taken from the slab
# is prefill and TTFT (2026-10-11 A/B: slab -14.8 % cost prefill -29 %, TTFT
# +25 % at the full window) while KV beyond the contract buys nothing.
# Consequence, deliberate: a FOURTH full-height conversation does not fit this
# pair, and the engine refuses it — it never truncates. See the KV block for the
# arithmetic and the measured dtype law. The fp8-native payload this file was
# written for is the rollback target and keeps its own trio under the
# fp8-native runbook.
#
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
# tag and rebuildable). Before that: tcclaviger vLLM fork (tag
# frank-archon-vllm-shapeG-20261003).
#
# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
# gfx1201, drives `podman run` directly from the digest-pinned
# stilldeadcode/radiance image (287 MiB; host needs only the amdgpu driver),
# and its qwen4exp plugin reads the modelopt FP8 encoding NATIVELY
# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
# The 4-bit-codebook publisher shape (qwen3.8-next-flash-fp8-iq4r-moe.rad,
# --host-pool-mib 12288) remains the documented fallback if the native
# container proves unbootable.
#
# PAYLOAD: /home/dank/models/rad/qwen4exp-fp8-native.rad — a rad-convert
# NO-RECIPE pure copy of the official 131-shard checkpoint (every weight
# keeps its block-FP8 encoding; ~161 GiB). The convert runs against a
# staging dir whose config.json drops `quantization_config`: the descriptor
# string is only the plugin SELECTION key (spec §2.4 — exact match, no
# fallback), and no shipped plugin claims (qwen4exp, "fp8"); the shipped
# qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
# containers carry. Per-weight encodings are data, not the descriptor —
# removing the key changes nothing about what the weights hold. Built by
# scripts/build-frank-archon-rad.sh (staging + convert + rad-info).
#
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
# then engine.serve()) — during load the port refuses connections, so a
# bare curl /health gate is honest. /health == /ping: 200 JSON while
# serving, 503 only while stopping (core/server/admin.cpp:385).
# - Reasoning: --reasoning-format defaults to auto, emitting thinking in
# `reasoning_content` (config.cpp). Tool calls: the qwen3.8 dialect, from
# the chat format. Both need no parser flags.
# - Sampling: the OpenAI endpoint accepts temperature, top_p, top_k, min_p,
# typical_p, presence_penalty, frequency_penalty, repetition_penalty,
# repeat_last_n, chat_template_kwargs, reasoning_effort,
# enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
# models.yml's extraBody set is fully inside it. Unknown params are
# refused (400), which is why nothing unverified rides the wire.
# - MTP: the official checkpoint carries the mtp.* head (3,101 tensors,
# mtp_num_hidden_layers=1) and the plugin probes draft depth 3 — the
# publisher's throughput peak for this model. Stated explicitly with
# --num-speculative-tokens 3.
# - The fused TP all-reduce is declared over max-num-seqs x
# (1 + --num-speculative-tokens) rows; 2 x 3 = 6 rows, under the 25-row
# exactness bound of the wht6 wire.
# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
# 25558e07fc60d20328f1346f96f3c6e00845eb763f01fa, tag 1.2.0, commit
# e95d397) was benched 2026-10-06 on this exact shape (fp8-native rad,
# vram pair, spec 2, seqs 3, 43.6k greedy probe): decode 29.21 t/s and
# draft acceptance 0.7123 vs the 1.0.13 control's 30.08 and 0.7549.
# Acceptance on the greedy harness is deterministic (the control
# reproduced the recorded 0.755 exactly), so the drop is a real numerics
# change — the 1.2.0 MoE weight-rotation change (92b8aaf) is the named
# suspect; decode loss tracks the acceptance ratio (0.943 vs 0.945).
# The vram-pair translation itself is exact (1.2.0 boot ledger took
# vram_weights 25.46 GiB / vram_kv 3.46 GiB per card byte-identical to
# the 1.0.13 ledger at EVC 0.82). Re-test 1.2.0+ when the rotation change
# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of
# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this
# file already uses the pair, which BOTH pins accept (rollback symmetry).
FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
# The .rad container FILE (mounted under /models) and the state dir for the
# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
# .rad files; the raw HF download and its staging dir live beside it and
# are never mounted.
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
FRANK_ARCHON_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad
FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
# at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the
# cache must ride a host mount — serve script mounts it, see the runbook's
# "Tuning levers").
# Container identity is the SERVICE name (Frank's call, 2026-10-05): one
# name across systemctl and podman ps. The runtime changes; the name does
# not. The legacy list retires every predecessor identity that could hold
# the GPUs or the port.
FRANK_ARCHON_CONTAINER=frank-archon
FRANK_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
# Served model id — Protoss name (wire rename 2026-10-09). The container,
# unit, scripts and this file keep the frank-archon name above; only the
# alias on the wire moved. Radiance validates the requested model name, so
# every client (harness, bench scripts, probes) MUST send this value.
FRANK_ARCHON_SERVED_NAME=frank-archon
FRANK_ARCHON_PORT=7770
# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
# repo's root chat_template.jinja, vendored here; the checkpoint's own
# template is baked into the .rad by rad-convert and this overrides it).
# Radiance derives the reasoning/tool-call reply format from it.
FRANK_ARCHON_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
# --- engine shape (the boot ledger arbitrates every number below) ---------
# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
# x8/x8 on root ports) made collectives legal on this host; radiance's TP
# wire is libr4d's own, not RCCL.
FRANK_ARCHON_TP=2
# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
# engine default.
FRANK_ARCHON_TP_WIRE=wht6
# Decode slots: THREE, so the served window is a CAPACITY CONTRACT — three
# concurrent conversations of --max-model-len fit the KV pool (see the KV
# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
# tcclaviger vLLM fork's torch.compile shape specialization on
# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
# (30.9-33.1 t/s, inside the +-3% noise floor) — slot count buys concurrency,
# not speed, so a slot costs KV, never decode.
# Cost of each extra slot: 110 MiB of per-slot recurrent state inside the KV
# pool, and the fused TP all-reduce is declared over seqs x (1 + spec) rows —
# 3 x 3 = 9 here, under the 25-row bound at which the wht6 wire stays EXACT
# (single-sequence decode and any step of <= 25 rows is bit-exact; only busier
# steps are lossy).
# The pool is sized to this contract and nothing more (see the KV block):
# 3 x 200,000 at fp8 = 5,063 MiB, pool 5,120, and the freed 1,792 MiB went into
# the resident expert slab. The trade is explicit and it is Frank's call:
# a FOURTH full-height conversation no longer fits (5,120 MiB of pool backs
# ~608k tokens; 4 x 200,000 needs 6,601), so a fourth 200k request is refused
# as too large to fit — it is NOT squeezed, paged, or truncated. Third-party
# traffic that wants four full-height sessions is a --vram-kv-mib change, not
# a code change.
FRANK_ARCHON_MAX_SEQS=3
# Served window. Native is 262,144; 200,000 is the publisher's production
# value (compose) and what models.yml's contextWindow must quote — a longer
# prompt is refused by the engine, never truncated. The engine treats
# `kv group tokens >= max-model-len` as a HARD limit ("ONE request longer
# than N tokens is accepted by the API and then refused as too large to ever
# fit — this is a hard limit, not a backing ratio"), so the pool is sized
# for seqs x maxlen, not for one sequence.
FRANK_ARCHON_MAX_MODEL_LEN=200000
FRANK_ARCHON_PLACEMENT=expert_tiered
# Pinned host pool for the expert rows that do not fit VRAM. vram_weights and
# this pool serve the per-rank resident set TOGETHER: at the fp8-KV pair
# (26071/3543) the ledger showed the VRAM slab filled to 259 KiB of slack,
# i.e. the pair was exactly sized. Cutting vram-weights by 3.8 GiB to pay for
# the bf16 KV pool SPILLS ~3.8 GiB/rank here, so the pool moves with it —
# under-sizing it is a boot failure (36864 refused the boot in the fp8 era).
# The boot ledger's coverage line remains the referee, not this arithmetic.
FRANK_ARCHON_HOST_POOL_MIB=44928
# VRAM left unclaimed per card (publisher default 96 MiB).
FRANK_ARCHON_GPU_HEADROOM_MIB=96
# VRAM split of claimed memory per card, MiB (0 = derive from the card).
# THE CEILING, read off this engine's own VRAM-budget table (boot 2026-10-13,
# pair 24000/5120): card 32,624 MiB; budgets claimed 29,120; `3.42 GiB is
# neither budgeted nor accounted — the driver's context, the activation arena
# and --gpu-headroom-mib come out of it; anything past those is throughput on
# the floor`; and `card 0 settled at 2.01 GiB free against 209.19 MiB (the
# 96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)`.
# So the reserve above the ENGINE's floor is 2,058 - 209 = 1,849 MiB and the
# ceiling is 29,120 + 1,849 = 30,969 MiB. The 2026-10-11 bound of 29,120
# ("31,856 - 1,710 outside both budgets - keep 1 GiB free") was MY reserve, not
# the engine's; the pair below spends the difference and lands 458 MiB above
# the engine's floor. `weights + kv <= 30,969` is the gate.
# THE PRICE, and it is paid twice: on this engine idle KV VRAM is ALSO the
# expert cache's spillover (`mover: 2275 slots can be lent out of the KV
# cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <= 256 tokens`),
# so a bigger KV pool steals weight residency from the slab AND from the
# lendable cache. Priors on this card/model: stealing 2.14 GiB of weights for
# a bigger pool (MAX_MODEL_LEN 262144) cost 2.5% decode; the EVC ladder's
# other direction (0.82 -> 0.84, +2.5 GiB weights) cost 8.9%. The 2026-10-11
# exact-KV A/B measured the slab's own price (22208 vs the fp8-era 26071):
# prefill -29 %, TTFT at the full window +25 %, decode inside the noise band on
# 2 of 3 passes, draft acceptance UP 0.661 -> 0.740.
# 2026-10-13 (late): THREE slots, so the pool is cut to the contract (5,120)
# and every spare MiB goes to the slab: 22208 -> 24000 (the third slot's
# surplus KV), then 24000 -> 25600 (the card's settling slack). The direction
# the A/B data points: prefill is slab-limited, KV beyond the contract is not.
# The margin is thin by design (608,027 predicted tokens against the 600,000
# contract), so the boot ledger's `backs N sequences all at 200000 tokens at
# once` line is the referee for THIS pair and the concurrent probe is its
# regression test.
# VERDICT on the 25,600 arm (measured 2026-10-13, scripts/probe-frank-archon-kv.py):
# the threshold claim is FALSIFIED — the slab is a SLOPE at this end. A-cold
# 43.5k single-stream: 22,208 -> 3,397, 24,000 -> 3,290, 25,600 -> 3,584 t/s;
# the 3-way 197k arm's aggregate prefill 3,051 -> 3,589 t/s (wall
# 193.7 -> 164.5 s). So +1,600 MiB of slab = +8.9 % single-stream and
# +17.6 % concurrent, both outside the +-3 % noise floor. The 22,208/24,000
# pair landed inside noise of EACH OTHER, which is what made it read as a
# plateau; two points are not a curve.
# The cliff that remains is between 25,600 (3,584) and 26,071 (4,379, the
# 2026-10-11 fp8 control at seqs 3 / pool 3,543) — +22 % for +471 MiB. At THREE
# slots that is only reachable by breaking the ceiling (26,112 + 5,120 = 31,232
# vs 30,969, ~214 MiB above the engine's floor); at TWO slots it is free:
# pool 3,072 + slab 27,897 = exactly the ceiling, slab above the cliff.
FRANK_ARCHON_VRAM_WEIGHTS_MIB=25600
FRANK_ARCHON_VRAM_KV_MIB=5120
# KV cache QUANTIZED (fp8). This engine's flag takes ONLY bf16|fp8 — there is
# ONE kv_dtype for the whole cache, so "exact K, quantized V" (llama.cpp's
# --cache-type-k/--cache-type-v pair, which is what frank-zealot runs) is
# UNSERVABLE here: the rung below bf16 is fp8 on both halves.
# Measured per-rank cost, read out of the boot ledger's group table (fp8 boot
# 2026-10-08, bf16 boot 2026-10-11):
# kv_attn 13 layers, 4-token blocks, page 26.00 KiB (fp8) -> 52.00 KiB
# kv_qsa_bkey 13 layers, 4-token blocks, page 3.25 KiB at BOTH dtypes —
# the QSA block key is an index, not attention state, and does
# NOT follow --kv-cache-dtype
# => 7.3125 KiB/token fp8, 13.8125 KiB/token bf16: bf16 costs 1.89x fp8,
# not 2x, precisely because the bkey plane is dtype-fixed.
# The dtype-INDEPENDENT terms also live in the pool: kv_gdn_state 54 MiB x 2
# copies per sequence (+ conv/tail/ple ~= 110 MiB a sequence) and the prefix
# checkpoint region 8 x 55.99 = 448 MiB. So:
# pool_mib >= tokens x 7.3125 KiB + 110 x seqs + 448
# 3 x 200,000 at fp8 needs 4,285 + 330 + 448 = 5,063 MiB; the pool is 5,120,
# and it predicts (5120 - 330 - 448) x 1024 / 7.3125 = 608,027 tokens: the
# three-slot contract with 1.3 % slack, every MiB above 5,063 having been
# handed to the expert slab instead. The ledger line is the referee ("backs N
# sequences all at 200000 tokens at once") — a pair this tight is read off the
# boot, never assumed.
# At bf16 the same three slots cost 8,093 + 778 = 8,871 MiB, which would cut
# the slab to 20,249 MiB (-9 %); at FOUR slots bf16 costs 10,791 + 888 =
# 11,679 MiB and the slab falls to 17,441 (-33 %). fp8 keeps 22,208 at both
# slot counts. The 2026-10-11 bf16 shape (6,912 / seqs 2) backed 462,752
# tokens = 2 x 200,000; the historical fp8 pool of 3,543 gave 386,904 (1.93
# sessions), and the four-slot fp8 boot of 2026-10-13 19:21 backed 843,276.
# Secondary saving: the prefix-cache entry shrinks with the attention plane
# (one entry 110.50 KiB at bf16 -> 58.50 KiB at fp8, measured on the two boot
# ledgers' `kv tier transfer` lines), so the host tier below parks twice as
# many conversations as it did at bf16 — no re-size needed to park three slots.
FRANK_ARCHON_KV_DTYPE=fp8
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
# flag states it so the ledger records the intent).
FRANK_ARCHON_SPEC_TOKENS=2
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
# Prefix cache tiers (publisher pattern). The host tier is sized in ENTRIES,
# and one entry tracks the cache dtype (58.50 KiB at fp8, 117 KiB at bf16, per
# the ledger's `kv tier transfer` line): at fp8, 8192 MiB parks ~5.8 sessions
# of 200k (at bf16 the same tier parked ~2.9), so four live slots plus a full
# parked set fits with the tier as it stands — at RAM the host has (104 GiB
# available with the service loaded).
FRANK_ARCHON_PREFIX_HOST_MIB=8192
FRANK_ARCHON_PREFIX_DISK_MIB=65536
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
FRANK_ARCHON_NGRAM_PLACEMENT=ram
# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
# pages_limit (the current kernel exposes NO pages_max; the cap function
# takes the min over the knobs that exist). Below the plan the load dies
# mid-pin with an allocator error instead of this named refusal.
# THE CAP IS A RUNTIME KNOB — the kernel default is half of RAM, so every
# raise dies with a reboot. Measured the hard way: the 2026-10-05 manual raise
# to 52,428,800 pages (200 GiB) was gone after the 2026-10-12 23:17 POWER
# FAILURE, the cap fell back to 125,988 MiB, and the preflight refused the
# start 6,919 times in ~20 h while Restart=always re-ran it. The gate in
# scripts/ttm-preflight.sh now RAISES the knob itself before refusing (the
# root unit is privileged; the knob is a 0644 root module parameter), so the
# recovery is a boot, not a visit to the machine.
FRANK_ARCHON_TTM_MIN_MIB=150000
FRANK_ARCHON_TTM_TARGET_MIB=204800
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
# multi-flag values as one string; the serve script expands it unquoted on
# purpose.
FRANK_ARCHON_EXTRA_ARGS=
# The .rad's minimum size in MiB (deployer gate; a truncated convert is a
# guaranteed long-timeout boot instead of a fast failure). The converted
# container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
# plan-only estimate 172.76 GiB covers the data planes but not the metadata
# tail) - floor sits just under the payload.
FRANK_ARCHON_RAD_MIN_MIB=110000
@@ -1,8 +1,8 @@
# frank-primary.service — radiance (StillDeadcode, Apache-2.0) endpoint
# frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint
# (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the
# rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2,
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag
# frank-primary-freetoken-pr587-20261005 is the pre-cutover state; the
# frank-archon-freetoken-pr587-20261005 is the pre-cutover state; the
# retired unit/env/serve trio and Dockerfile are in that tag).
#
# Same port, same served alias, same container identity as the unit name;
@@ -19,7 +19,7 @@
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
# engine still runs out of dank's rootless podman storage.
[Unit]
Description=frank-primary — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
Description=frank-archon — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
After=network-online.target
Wants=network-online.target
# Never let the start-rate limiter give up after a crash loop (a failed LLM
@@ -33,14 +33,21 @@ Group=dank
# RLIMIT_MEMLOCK smaller than the pinned set fails the load mid-start.
# PID1 grants infinity here — parity with the fleet's other LLM units.
LimitMEMLOCK=infinity
Environment="FRANK_PRIMARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-primary.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-primary.sh
Environment="FRANK_ARCHON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-archon.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-archon.sh
# The `+` prefix runs this as root even though the service is User=dank — the
# ttm pages cap is a root-owned knob and the pinned-RAM ledger (~147 GiB) is
# unusable below it. After a reboot the knob is back at its default (half of
# RAM), so the raise must be part of the start, not a note in a runbook: the
# 2026-10-12 power failure reset it and the endpoint refused to boot 6,919
# times. The serve script re-verifies the cap after this hook.
ExecStartPre=+/home/dank/serverAppConfigs/scripts/ttm-preflight.sh --raise
# systemd stop: wait alone cannot stop the container child; stop it
# explicitly. No graceful drain contract is claimed (the OpenAI front end
# holds no queue to drain — in-flight decodes are lost, which a stop
# during a bench window accepts; radiance takes SIGTERM on the engine
# process and podman reaps the container).
ExecStop=/usr/bin/podman stop -t 120 frank-primary
ExecStop=/usr/bin/podman stop -t 120 frank-archon
# Boot cost: .rad load (weights + PLE upload) + expert host-pool pin. The
# publisher's compose healthcheck allows a 15-minute start_period; the
# vLLM-era budget (1800 s) carried forward.
@@ -1,11 +1,11 @@
# frank-large.env — tunables for the frank-large endpoint on Dankclaw's
# frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
# Strix Halo iGPU (gfx1151), host port 7770.
#
# RUNTIME (2026-10-06, Frank): halogen-flash-server 0.16.2 — the
# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
# digest-pinned below) serving the Quat3rnion ABLITERATED Swift1.5 finetune
# of peonist's Qwen3.8-Flash-Next v2 checkpoint, as a podman container under
# a root unit, driven by scripts/serve-frank-large.sh. This CUTS OVER the
# a root unit, driven by scripts/serve-frank-dragoon.sh. This CUTS OVER the
# 2026-10-05 halogen 27B-dense shape (censored base, DFlash2, 31.7 t/s):
# rollback = revert the swap commit's trio; the 27B payload stays on disk at
# /home/dank/models/halogen-qwen3.8-27b/.
@@ -22,15 +22,25 @@
# tool calls verified, and the native 262,144 context.
#
# SHAPE — the knobs that are NOT free, with the arithmetic:
# HALOGEN_CTX=200192 is the PER-SLOT window and the per-conversation hard
# HALOGEN_CTX=262144 is the PER-SLOT window and the per-conversation hard
# ceiling — a prompt over it is a 400 naming the limit, never a silent
# truncation. Harness compaction fires at 70% = 140,134, and 140,134 +
# 57,344 (harness maxTokens) = 197,478 <= 200,192 keeps prompt + budget
# inside the slot. models.yml contextWindow must quote 200192 or the
# harness discovers the wall as a 400.
# HALOGEN_KV_POOL_POSITIONS=400384 = KV_SLOTS(2) x CTX — the explicit
# truncation. RAISED from 200192 on 2026-10-07: the harness clamps
# max_tokens to contextWindow − est − 4096, its chars/4 est UNDER-counts
# the rendered prompt by ~3 % on code/agent traffic, and compaction only
# fires at 85 % — so with window == engine ceiling, every request over
# ~146k est (65-85 % of the window) was a guaranteed 400. Measured
# 2026-10-07: est 152,938, server 157,388, request 43,158 vs room 42,804
# — over by 354, margin 4,096 < the 4,450 under-count.
# Harness contract (models.yml): contextWindow 217088, maxTokens 45056.
# Compaction fires at 85 % = 184,524 est ≈ 190k measured (Frank's 180k
# target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 %
# est-error ≈ 219k <= 262144 — the 45k gap between harness window and
# engine ceiling absorbs the estimate error with ~7x margin (robust to a
# 10 % under-count). Compaction rides on frank-zealot: input ≈
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot.
# HALOGEN_KV_POOL_POSITIONS=524288 = KV_SLOTS(2) x CTX — the explicit
# form of the engine's default, so the seam test can pin the product.
# KV costs ~28 KiB/position (FLAGS.md) => 400,384 x 28 KiB = 10.7 GiB.
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
# drafter is NOT slot-locked on this engine (batching and speculation
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
@@ -39,11 +49,11 @@
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
# mmap'd page cache beside the checkpoint, not a pinned charge — the
# card's own quickstart arithmetic does not bill it):
# weights ~62 GiB resident + KV 10.7 + arena (HALOGEN_MAX_TOK=8192 =>
# weights ~62 GiB resident + KV 14.0 + arena (HALOGEN_MAX_TOK=8192 =>
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2
# = ~82.7 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
# honest; baseline co-tenancy (frank-secondary, ComfyUI, Caddy/observ)
# ~10-15. ~18 GiB margin. The boot ledger is the referee — read it
# = ~86 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
# honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ)
# ~10-15. ~15 GiB margin. The boot ledger is the referee — read it
# after deploy and re-size if the engine prints a clamp.
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
# turns the MTP head off for whole requests after low acceptance and
@@ -57,8 +67,8 @@
# an absent temperature is no longer greedy. models.yml extraBody
# carries the same values — consistent, not load-bearing.
# * HALOGEN_MAX_TOK=8192 is only the DEFAULT for requests that omit
# max_tokens; the harness always sends 57344. Requests over
# HALOGEN_MAX_TOKENS_CAP (65536) are a 400 naming the cap. Cap and
# max_tokens; the harness sends at most its maxTokens 45056. Requests
# over HALOGEN_MAX_TOKENS_CAP (65536) are a 400 naming the cap. Cap and
# queue timeout are COUPLED (FLAGS.md table): 65536 at ~10 t/s serial
# worst case needs > 6550 s; 7200 honors it. Do not move one alone.
# * max_tokens INCLUDES thinking; a reasoning reply over a small budget
@@ -81,48 +91,51 @@
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
# is what was qualified). 0.16.2 = the version the model card names as its
# quickstart pin; RepoDigest captured at pull on 2026-10-06.
FRANK_LARGE_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
# Container identity: kept from the 27B era (unit ExecStop names it).
FRANK_LARGE_CONTAINER=frank-large-halogen
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
# Old quadlet-era container identity; the serve script clears stale names.
FRANK_LARGE_LEGACY_CONTAINERS=frank-large
FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
# default); the engine port (8730) is never published.
FRANK_LARGE_PUBLISH=7770:8731
FRANK_DRAGOON_PUBLISH=7770:8731
FRANK_LARGE_MODEL_DIR=/home/dank/models/halogen-flashnext-v2
FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-flashnext-v2
# The checkpoint: Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated
# qwen38-flash-next-v2-swift15-abliterated.hgn — compacted r2 of the card's
# append-only original (byte-identical live tensors, padding dropped:
# 66,687,678,464 B vs 89,775,868,032 B). LFS sha256 verified at fetch.
FRANK_LARGE_CHECKPOINT=qwen38-flash-next-v2-swift15-abliterated.hgn
FRANK_LARGE_CHECKPOINT_BYTES=66687678464
FRANK_LARGE_CHECKPOINT_SHA256=9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037
FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-swift15-abliterated.hgn
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678464
FRANK_DRAGOON_CHECKPOINT_SHA256=9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037
# The n-gram lookup table: REQUIRED (the engine reads it beside the
# checkpoint; the card's quickstart names it explicitly). From
# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
FRANK_LARGE_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
FRANK_LARGE_NGRAM_BYTES=51200246144
FRANK_LARGE_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
FRANK_DRAGOON_NGRAM_BYTES=51200246144
FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
# The vision tower: OPTIONAL on the engine, ENABLED here (image input).
# From peonist-ai/halogen-qwen3.8-flash-next @ e053f488.
FRANK_LARGE_VISION_TOWER=qwen38-flash-next-vision.hgn
FRANK_LARGE_VISION_BYTES=897916416
FRANK_LARGE_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
# The tokenizer is a FLAT DIRECTORY, $FRANK_LARGE_MODEL_DIR/tokenizer,
FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
FRANK_DRAGOON_VISION_BYTES=897916416
FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
# The tokenizer is a FLAT DIRECTORY, $FRANK_DRAGOON_MODEL_DIR/tokenizer,
# mounted whole at /tokenizer. The two repos ship the same one; fetched
# fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to
# the 27B-era copy; the rest are plain git blobs — size-checked only). The
# serve gate and the deployer refuse a missing file or ANY symlink: an HF
# cache snapshot symlinks each file into blobs/, and those links dangle
# inside the container mount.
FRANK_LARGE_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
# stays at the image default, which is what was qualified) ------------------
HALOGEN_MODEL_ID=frank-large
HALOGEN_CTX=200192
HALOGEN_KV_POOL_POSITIONS=400384
# Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
# scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
# model_ids for the :7770 backend MUST name this value.
HALOGEN_MODEL_ID=frank-dragoon
HALOGEN_CTX=262144
HALOGEN_KV_POOL_POSITIONS=524288
HALOGEN_KV_SLOTS=2
HALOGEN_HOST_RESERVE_GIB=20
# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
@@ -1,5 +1,5 @@
[Unit]
Description=halogen-flash-server container (Quat3rnion abliterated Swift1.5 of Qwen3.8-Flash-Next v2, MTP spec decode, 2 slots x 200192, vision ON), root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
Description=halogen-flash-server container (Quat3rnion abliterated Swift1.5 of Qwen3.8-Flash-Next v2, MTP spec decode, 2 slots x 200192, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
After=network-online.target
Wants=network-online.target
@@ -11,11 +11,11 @@ Group=dank
# RLIMIT smaller than the pinned set cannot fail the load. Parity with the
# other LLM units on this fleet.
LimitMEMLOCK=infinity
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
# systemd stop alone cannot stop the container child; stop it explicitly
# (frank-primary pattern). The engine drains on SIGTERM.
ExecStop=/usr/bin/podman stop -t 60 frank-large-halogen
# (frank-archon pattern). The engine drains on SIGTERM.
ExecStop=/usr/bin/podman stop -t 60 frank-dragoon-halogen
TimeoutStartSec=1200
TimeoutStopSec=120
# always, not on-failure: an engine that exits 0 after an internal fault
@@ -23,7 +23,7 @@ TimeoutStopSec=120
Restart=always
RestartSec=10
# The engine pins ~62 GiB of v2 weights and keeps a multi-GiB working set
# on a box that also hosts frank-secondary, ComfyUI and the Caddy/observability
# on a box that also hosts frank-zealot, ComfyUI and the Caddy/observability
# stack; make it a late OOM candidate, not the first.
OOMScoreAdjust=-500
-142
View File
@@ -1,142 +0,0 @@
# frank-primary.env — radiance (StillDeadcode, Apache-2.0) serving the
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at NATIVE precision
# (modelopt block-FP8 throughout, no requantisation) on BOTH R9700s (TP=2),
# dankserver :7770.
#
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
# pinned host RAM; git tag frank-primary-freetoken-pr587-20261005 is the
# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
# tag and rebuildable). Before that: tcclaviger vLLM fork (tag
# frank-primary-vllm-shapeG-20261003).
#
# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
# gfx1201, drives `podman run` directly from the digest-pinned
# stilldeadcode/radiance image (287 MiB; host needs only the amdgpu driver),
# and its qwen4exp plugin reads the modelopt FP8 encoding NATIVELY
# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
# The 4-bit-codebook publisher shape (qwen3.8-next-flash-fp8-iq4r-moe.rad,
# --host-pool-mib 12288) remains the documented fallback if the native
# container proves unbootable.
#
# PAYLOAD: /home/dank/models/rad/qwen4exp-fp8-native.rad — a rad-convert
# NO-RECIPE pure copy of the official 131-shard checkpoint (every weight
# keeps its block-FP8 encoding; ~161 GiB). The convert runs against a
# staging dir whose config.json drops `quantization_config`: the descriptor
# string is only the plugin SELECTION key (spec §2.4 — exact match, no
# fallback), and no shipped plugin claims (qwen4exp, "fp8"); the shipped
# qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
# containers carry. Per-weight encodings are data, not the descriptor —
# removing the key changes nothing about what the weights hold. Built by
# scripts/build-frank-primary-rad.sh (staging + convert + rad-info).
#
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
# then engine.serve()) — during load the port refuses connections, so a
# bare curl /health gate is honest. /health == /ping: 200 JSON while
# serving, 503 only while stopping (core/server/admin.cpp:385).
# - Reasoning: --reasoning-format defaults to auto, emitting thinking in
# `reasoning_content` (config.cpp). Tool calls: the qwen3.8 dialect, from
# the chat format. Both need no parser flags.
# - Sampling: the OpenAI endpoint accepts temperature, top_p, top_k, min_p,
# typical_p, presence_penalty, frequency_penalty, repetition_penalty,
# repeat_last_n, chat_template_kwargs, reasoning_effort,
# enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
# models.yml's extraBody set is fully inside it. Unknown params are
# refused (400), which is why nothing unverified rides the wire.
# - MTP: the official checkpoint carries the mtp.* head (3,101 tensors,
# mtp_num_hidden_layers=1) and the plugin probes draft depth 3 — the
# publisher's throughput peak for this model. Stated explicitly with
# --num-speculative-tokens 3.
# - The fused TP all-reduce is declared over max-num-seqs x
# (1 + --num-speculative-tokens) rows; 2 x 4 = 8 rows fits either wire.
FRANK_PRIMARY_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
# The .rad container FILE (mounted under /models) and the state dir for the
# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
# .rad files; the raw HF download and its staging dir live beside it and
# are never mounted.
FRANK_PRIMARY_MODEL_DIR=/home/dank/models/rad
FRANK_PRIMARY_MODEL_FILE=qwen4exp-fp8-native.rad
FRANK_PRIMARY_STATE_DIR=/home/dank/radiance-state
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
# at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the
# cache must ride a host mount — serve script mounts it, see the runbook's
# "Tuning levers").
# Container identity is the SERVICE name (Frank's call, 2026-10-05): one
# name across systemctl and podman ps. The runtime changes; the name does
# not. The legacy list retires every predecessor identity that could hold
# the GPUs or the port.
FRANK_PRIMARY_CONTAINER=frank-primary
FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe"
FRANK_PRIMARY_SERVED_NAME=frank-primary
FRANK_PRIMARY_PORT=7770
# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
# repo's root chat_template.jinja, vendored here; the checkpoint's own
# template is baked into the .rad by rad-convert and this overrides it).
# Radiance derives the reasoning/tool-call reply format from it.
FRANK_PRIMARY_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
# --- engine shape (first-boot conservative; the boot ledger arbitrates) --
# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
# x8/x8 on root ports) made collectives legal on this host; radiance's TP
# wire is libr4d's own, not RCCL.
FRANK_PRIMARY_TP=2
# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
# engine default.
FRANK_PRIMARY_TP_WIRE=wht6
# Decode slots. 2 is Frank's first-boot value ("use parallelism = 2 to
# start"); raise only after a real-traffic look at slot occupancy and the
# KV pool's headroom line. The recurrent-state cost scales with it (54
# MiB/copy/card, 2 copies with speculation).
FRANK_PRIMARY_MAX_SEQS=3
# Served window. Native is 262,144; 200,000 is the publisher's production
# value (compose) and what models.yml's contextWindow must quote — a longer
# prompt is refused by the engine, never truncated.
FRANK_PRIMARY_MAX_MODEL_LEN=200000
FRANK_PRIMARY_PLACEMENT=expert_tiered
# Pinned host pool for the expert rows that do not fit VRAM. Native FP8
# experts are 112.8 GiB total (56.4 GiB per rank slice) vs the 4-bit
# publisher shape's ~29 GiB — this pool is the cost of native precision.
# The boot ledger prints the pool's coverage ("the pool backs N% of the
# worst case ... M full-length sessions"); resize from that line, not from
# arithmetic.
FRANK_PRIMARY_HOST_POOL_MIB=40960
# VRAM left unclaimed per card (publisher default 96 MiB).
FRANK_PRIMARY_GPU_HEADROOM_MIB=96
# Weights share of claimed VRAM (publisher production value; higher = more
# prefix-cache hit rate, less KV headroom).
FRANK_PRIMARY_EXPERT_VS_CACHE=0.82
# KV cache in FP8 (7.32 KiB/token at this model's 12 full-attention layers,
# 2 KV heads, 256 head_dim, K+V).
FRANK_PRIMARY_KV_DTYPE=fp8
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
# flag states it so the ledger records the intent).
FRANK_PRIMARY_SPEC_TOKENS=2
FRANK_PRIMARY_MAX_BATCHED_TOKENS=2048
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
FRANK_PRIMARY_PREFIX_HOST_MIB=4096
FRANK_PRIMARY_PREFIX_DISK_MIB=65536
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
FRANK_PRIMARY_NGRAM_PLACEMENT=ram
# TTM preflight: the pinned ledger (2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
# prefix host) is ~132 GiB. ttm's pages_limit/pages_max cap the GPU-mappable
# pinned pages; the current cap (32,252,917 pages = ~123 GiB) is BELOW the
# plan, so the serve script refuses to start until both are raised. This is
# the floor it enforces (in MiB), with ~8 GiB of margin.
FRANK_PRIMARY_TTM_MIN_MIB=140000
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
# multi-flag values as one string; the serve script expands it unquoted on
# purpose.
FRANK_PRIMARY_EXTRA_ARGS=
# The .rad's minimum size in MiB (deployer gate; a truncated convert is a
# guaranteed long-timeout boot instead of a fast failure). The converted
# container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
# plan-only estimate 172.76 GiB covers the data planes but not the metadata
# tail) - floor sits just under the payload.
FRANK_PRIMARY_RAD_MIN_MIB=177000
-98
View File
@@ -1,98 +0,0 @@
# frank-secondary — llama.cpp (master, CUDA sm_86) serving
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF:RCO-IQ3_XXS
# **with the MTP head in the file** on the RTX 3090, :7771.
#
# 2026-10-06 (Frank, evening): the payload moved to ajgazin's UNCELERSED
# Swift-1.5-Qwen3.8-27B at RCO-IQ3_XXS (10,442,827,392 B) — 1.56 GiB
# lighter than the IQ3_S-mtp it replaces — with MTP BACK ON (draft-mtp,
# depth 3), TWO 131,072 slots, and ubatch 512. The uncensored finetune
# changes behavior by design (publisher: 23/100 refusals vs Swift 1.5's
# 98/100, KL 0.0884 from Swift 1.5) — Frank's order. The RCO tier is the
# same per-tensor ALLOCATION recipe as the IQ3_S file (histogram: iq3_s
# 97 / bf16 96 / iq3_xxs 83 / iq2_s 72 / iq4_xs 33 / iq2_xs 32 / iq2_xxs
# 30 / q2_k 28 / q4_k 19 / q6_k 8 / iq1_s 4 / iq1_m 4 — all stock types,
# stock llama.cpp loads it); publisher KLD: Mean 0.002631, 90 % 0.0421
# (kld_results.csv). RCO files are "more experimental" per the README —
# the quality floor is the IQ3_S rung, not a promise.
#
# The architecture is unchanged (same model family): qwen35 hybrid —
# block_count 65 (64 real layers + the blk.64 MTP head),
# full_attention_interval 4 => SIXTEEN full-attention layers, 24 Q heads
# / 4 KV heads, k/v length 256, native window 262144, dense,
# nextn_predict_layers=1. The MTP head ships in every main GGUF of this
# repo (README); --spec-type draft-mtp loads it against the main model's
# own weights.
#
# THE KV CONTRACT STAYS q8_0 K + q8_0 V (Frank, 2026-10-05): 34 KiB/token
# (16 attn layers x 4 heads x 256 x 1.0625 B, K and V).
#
# Why TWO slots + MTP fits now (it did not under IQ3_S):
# card 24,576 MiB. Weights 9,963 MiB (10,442,827,392 B; IQ3_S was
# 11,561). Projector ~1,290 MiB (the PUBLISHER's OWN mmproj-BF16.gguf,
# 931,145,888 B, at the measured ~1.45x card law — no substitution).
# Compute buffers at ubatch 512 ~1.3 GiB. CUDA ctx ~500 MiB. DeltaNet
# state ~150 MiB/slot x 2. MTP draft ctx back: ~0.4 GiB. Non-KV ~13.9
# GiB. Two q8_0 slots = 8,704 MiB. Predicted free ~2.6-3.2 GiB — back
# over the old 2-3 GiB floor that the IQ3_S two-slot boot violated
# (measured there: 1,397 MiB free). MEASURE nvidia-smi free at boot
# (quote the free field) and let the runbook carry the number.
#
# Payload history (rollback trail): /home/dank/models/Swift15-IQ3S/ (the
# IQ3_S-mtp that was live until this swap), Ornith-Q8/, MiMo-Q6K/ all
# stay on disk; one MODEL_FILE + shape edit back each.
LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server
MODEL_FILE=/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf
# Vision: the publisher SHIPS the projector — mmproj-BF16.gguf
# (931,145,888 B) from the same repo, fetched by
# scripts/download-frank-secondary.sh. No substitution needed for this
# payload (the unsloth base-tower proof of 2026-10-05 remains the
# fallback argument for the IQ3_S rollback payload).
# Empty string = text-only on purpose.
MMPROJ_FILE=/home/dank/models/Swift15-UncIQ3XXS/mmproj-BF16.gguf
# Fleet endpoint: same port and alias as every frank-secondary before it, so the
# harness models.yml id and the homepage probe do not move.
ALIAS=frank-secondary
BIND_ADDR=0.0.0.0
LISTEN_PORT=7771
# TWO conversations at Frank's 112k cap (2026-10-06 evening): --ctx-size is
# the POOL llama.cpp divides across slots; 229376 / 2 = 114,688 per
# conversation. Lowering from 128k (131,072) frees ~1 GiB of KV, which
# funds the ubatch 512 -> 1024 increase (prefill throughput). The
# per-slot window is the product contract, so models.yml follows to
# 114,688.
CTX_SIZE=229376
PARALLEL=2
# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves);
# the 1024 rung was the first A/B back once the per-slot context dropped
# to 112k (2026-10-06 evening), and 2048 is the second (Frank, same night)
# — the rung measured +4.6 % prefill for 616 MiB on the 3090 (2026-10-05
# boot pair), affordable inside the 1,751 MiB the 112k cut freed. The pair
# is a measurement, not a law (loud failure at graph capture if it does
# not fit).
BATCH_SIZE=4096
UBATCH_SIZE=2048
# The KV contract for this swap (Frank, 2026-10-05): q8_0 K + q8_0 V —
# llama.cpp's FP8 class. The flags stay explicit so the choice is visible in
# the argv either way, and --flash-attn stays pinned on in the serve script:
# it is required for quantized V (the build carries
# GGML_CUDA_FA_ALL_QUANTS=ON since 2026-09-17, so q8_0 KV is a real path,
# not a silent f16 fallback).
CACHE_TYPE_K=q8_0
CACHE_TYPE_V=q8_0
# Speculative decoding is ON (Frank, 2026-10-06 evening): draft-mtp at
# depth 3 against the head IN the file (the -mtp variant ships the
# blk.64.nextn.* tensors). ngram-mod stays skipped — on a qwen35 hybrid
# it disables recurrent-state rollback, so a rejected draft would
# corrupt the DeltaNet state; MTP needs no such bypass (its draft ctx
# re-derives state from the target's own weights). The flags move as a
# set: the serve script emits BOTH only when SPEC_TYPE names a drafter.
SPEC_TYPE=draft-mtp
SPEC_DRAFT_N_MAX=3
NGL=99
-19
View File
@@ -1,19 +0,0 @@
[Unit]
Description=llama.cpp Swift-1.5-Qwen3.8-27B Uncensored RCO IQ3_XXS GGUF (qwen35 hybrid, 2x131072 slots, q8_0 K+V, draft-mtp depth 3, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary)
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=dank
Group=dank
# CUDA pins its staging buffers, so the process needs the memlock budget even
# though the weights themselves are copied into VRAM.
LimitMEMLOCK=infinity
Environment="FRANK_SECONDARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-secondary.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-secondary.sh
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
+62
View File
@@ -0,0 +1,62 @@
# frank-zealot — llama.cpp (master, CUDA sm_86) serving
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090,
# :7771.
#
# 2026-10-07 (Frank): swap from Gemma4-26B-A4B QAT Q4_K_M (one 200,704 slot)
# to Qwen3.5-9B Q6_K at TWO 200,704-token conversations, EXACT (f16) KV,
# MTP disabled (SPEC_TYPE=none), log verbosity 4.
#
# Architecture — read from the GGUF's own header (HTTP range read, not the
# model card): arch qwen35, 32 layers, dense 9B, full_attention_interval 4
# ⇒ 8 full-attention layers (3,7,11,...,31) + 24 Gated DeltaNet linear layers,
# 16 attention heads / 4 KV heads × 256 dim, ssm conv_kernel 4 / state_size
# 128 / group_count 16 / time_step_rank 32 / inner_size 4096, native 262,144,
# vocab 248,320, file_type 18 (Q6_K). Tensor-type histogram: Q6_K×250 +
# F32×177 — standard types only, so the stock master build loads it (no
# custom-quant lock-in like the ROCmFPX era). NO nextn/mtp tensors exist, so
# in-file MTP is impossible here; SPEC_TYPE=none is forced by the payload.
#
# KV rate law: 8 full layers × 4 KV heads × 256 dim = 8,192 elements per
# token per cache. f16 K + f16 V = 32,768 B/token = 32 KiB/token, so the
# 401,408-token pool is 12.25 GiB. The 24 DeltaNet layers cost a fixed
# recurrent state per SLOT (not per token) — that is why two 200k slots fit
# where the 26B MoE fit only one.
#
# Feasibility comes from the predecessor's MEASURED boots, not a formula:
# MiMo-V2.6-Distill-Qwen-9B (identical qwen35 geometry, 7,793,710,624 B) at
# 2×200,704 f16 K + f16 V measured 21,276 used / 2,849 free at ubatch 1024
# and 21,846 / 2,279 at ubatch 2048. This file is 415 MiB lighter in weights,
# its BF16 mmproj is 3 MiB heavier ⇒ projected ~21,430 used / ~2,700 free.
# The verbosity-4 boot ledger is the referee.
#
# Ubatch 2048 is the measured CUDA optimum (measured on the MiMo predecessor
# at this exact arch + shape: +4.6 % prefill at 8.9k depth, decode flat, cost
# 616 MiB). 1024 is the fallback rung if the boot lands under the 2 GiB floor.
LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server
MODEL_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf
MMPROJ_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf
# Served model id — the harness/family name (Protoss rename completed to the
# wire 2026-10-09: harness ids + homepage cards 2026-10-08, /v1/models aliases
# this day). The SERVICE, unit, scripts and this env file keep the
# frank-zealot name; only the alias on the wire changed. Consumers that
# key on the served id: OpenWebUI (model row, access_grant, api_configs
# model_ids) and the harness models.yml.
ALIAS=frank-zealot
BIND_ADDR=0.0.0.0
LISTEN_PORT=7771
# CTX_SIZE is the POOL llama.cpp divides across slots: 401,408 / 2 = 200,704
# per conversation. MUST stay divisible by PARALLEL (the seam enforces it) and
# the harness contextWindow quotes the PER-SLOT number — a longer prompt is
# answered with a 400, never truncated.
CTX_SIZE=401408
PARALLEL=2
BATCH_SIZE=4096
UBATCH_SIZE=2048
CACHE_TYPE_K=f16
CACHE_TYPE_V=f16
SPEC_TYPE=none
NGL=99
# Log verbosity threshold (llama.cpp -lv). 4 = debug: KV layout, buffer sizes
# and graph capture land in the journal, which is what the VRAM-fit forensics
# read. Levels above 4 flood the journal.
VERBOSITY=4
+19
View File
@@ -0,0 +1,19 @@
[Unit]
Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-zealot)
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=dank
Group=dank
# CUDA pins its staging buffers, so the process needs the memlock budget even
# though the weights themselves are copied into VRAM.
LimitMEMLOCK=infinity
Environment="FRANK_ZEALOT_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-zealot.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env bash
# kld-gate-swift15.sh — RUNS ON DANKSERVER. Score the quantized candidate
# against the BF16 reference and apply the pre-approved gate:
# mean KL <= 0.03 AND top-1 agreement >= 95%.
# Same engine (1.0.13) and same boot shape as the reference RECORD
# (run-swift15-bf16.sh kld): disk-tier, no speculation, deterministic, fp8 KV.
# --kld-ref reads its tokens from the reference (no --kld-corpus here — the
# (run-swift15-bf16.sh kld): 1.0.13, tp 2, disk-tier, fp8 KV, no speculation,
# deterministic, max-model-len 16384, vram-kv 3072, 4 slots. Two deviations
# are forced by the w4 file: MNBT 2048 (the w4 kernels stream from the file
# tier ONLY at MNBT<=2048 — at the recording's 32768 the engine demands a
# widened form: "DID NOT FIT ... stored form is not the container's"), and
# pool 40960 (the recording's 12 GiB pool predates the w4 tables; at 12288 the
# gate_up slab cannot be cached). Chunk size is softmax-invariant, so the
# MNBT delta is float-order noise riding on the KL floor.
set -uo pipefail
PIN=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
C=swift15gate
LOG=/home/dank/logs/swift15_kldgate.log
KLD_REF_DIR=/home/dank/corpora/kld-ref
KLD_OUT=/home/dank/radiance-state/kld-gate.json
[ -d "$KLD_REF_DIR" ] || { echo "missing reference: $KLD_REF_DIR"; exit 2; }
: > "$KLD_OUT"
cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \
/home/dank/models/tpl.jinja 2>/dev/null || true
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
podman rm -f "$C" >/dev/null 2>&1 || true
podman run --rm --name "$C" \
--network host --init \
--security-opt seccomp=unconfined \
--ulimit memlock=-1:-1 \
--device /dev/kfd --device /dev/dri \
-v /home/dank/models:/models:ro \
-v /home/dank/radiance-state:/data \
-v "$KLD_REF_DIR:/data/kld-ref:ro" \
"stilldeadcode/radiance@$PIN" \
--model /models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad \
--tp 2 --placement expert_tiered --weights-disk-tier \
--host-pool-mib 40960 --deterministic --gpu-headroom-mib 768 \
--num-speculative-tokens 0 \
--kv-cache-dtype fp8 --max-model-len 16384 \
--ngram-placement ram \
--override-chat-template /models/tpl.jinja \
--kld-ref /data/kld-ref --kld-out /data/kld-gate.json \
--vram-kv-mib 3072 \
--max-num-seqs 4 --max-num-batched-tokens 2048 \
2>&1 | tee "$LOG"
rc=${PIPESTATUS[0]}
[ "$rc" -eq 0 ] || { echo "KLD GATE BOOT FAILED rc=$rc; log: $LOG"; exit "$rc"; }
python3 - "$KLD_OUT" "$LOG" <<'PY'
import json, sys, re
out, log = sys.argv[1], sys.argv[2]
def num(pat):
m = re.findall(pat, open(log, errors="replace").read(), re.I)
return float(m[-1]) if m else None
kl = top1 = None
try:
d = json.load(open(out))
def find(obj, keys):
if isinstance(obj, dict):
for k, v in obj.items():
if any(s in k.lower() for s in keys) and isinstance(v, (int, float)):
return float(v)
for v in obj.values():
r = find(v, keys)
if r is not None: return r
return None
kl = find(d, ("mean_kl", "mean kl", "kl_mean", "mean"))
top1 = find(d, ("top1", "top_1", "top-1", "agreement"))
if top1 is not None and top1 <= 1.0:
top1 *= 100.0
except Exception as e:
print("report JSON unusable (%s); falling back to log" % e)
if kl is None:
kl = num(r"mean[_ ]kl[^0-9]*([0-9]*\.?[0-9]+(?:e[-+][0-9]+)?)")
if top1 is None:
top1 = num(r"top[-_ ]?1[^0-9]*([0-9]*\.?[0-9]+)")
if kl is None or top1 is None:
print("KLD GATE PARSE FAILED: mean_kl=%s top1=%s — inspect %s / %s" % (kl, top1, out, log))
sys.exit(2)
print("KLD GATE: mean KL = %.4f (max 0.03) | top-1 = %.2f%% (min 95.0)" % (kl, top1))
ok = (kl <= 0.03) and (top1 >= 95.0)
print("KLD GATE " + ("PASS" if ok else "FAIL"))
sys.exit(0 if ok else 1)
PY
gate=$?
echo "kld-gate complete rc=$gate; log: $LOG report: $KLD_OUT"
exit "$gate"
+40
View File
@@ -0,0 +1,40 @@
#!/bin/bash
# kld-then-calib.sh — RUNS ON DANKSERVER. Campaign chain: wait for the KLD
# record to finish, verify it, then boot the 1.2.0 calibration server and
# push the 10M-token corpus. One GPU, so the phases serialize; prod is
# restored by each run script's trap.
set -u
KLOG=/home/dank/logs/swift15-kld-driver.log
CLOG=/home/dank/logs/swift15-calib-driver.log
S=/home/dank/serverAppConfigs-swift15/scripts
# Phase 1: KLD record (SKIP_KLD=1 when the record is already complete —
# kld.json's rows count is the receipt; a mid-run script swap can kill the
# wrapper with rc=127 after the engine exited clean).
if [ "${SKIP_KLD:-0}" != 1 ]; then
while ! grep -q "exited rc=" "$KLOG" 2>/dev/null; do sleep 60; done
RC=$(grep -o "exited rc=[0-9]*" "$KLOG" | tail -1)
if [ "$RC" != "exited rc=0" ]; then echo "KLD FAILED ($RC); calib NOT started"; exit 1; fi
fi
N=$(find /home/dank/corpora/kld-ref -type f | wc -l)
SZ=$(du -sm /home/dank/corpora/kld-ref | cut -f1)
echo "KLD OK files=$N size_mib=$SZ"
[ "$N" -ge 5 ] && [ "$SZ" -ge 1000 ] || { echo "kld-ref looks thin; calib NOT started"; exit 1; }
grep -q '"rows": 50235' /home/dank/corpora/kld-ref/kld.json || {
echo "kld.json rows != 50235; calib NOT started"; exit 1; }
# Phase 2: calibration serve (1.2.0) + corpus push.
nohup bash "$S/run-swift15-bf16.sh" calib > "$CLOG" 2>&1 &
CP=$!
for _ in $(seq 1 160); do
sleep 15
curl -sf localhost:7770/health >/dev/null 2>&1 && break
done
if ! curl -sf localhost:7770/health >/dev/null 2>&1; then
echo "CALIB BOOT FAILED; see /home/dank/logs/swift15_bf16.log"; kill "$CP" 2>/dev/null; exit 1
fi
echo "CALIB UP, pushing corpus"
python3 "$S/calib-push.py" --port 7770 --parallel 16 --container swift15ref --max-rounds 6
PUSH=$?
kill "$CP" 2>/dev/null
sleep 10
echo "CALIB_CHAIN_DONE push=$PUSH grams=$(find /home/dank/corpora/calib -name 'gram.r*.bin' 2>/dev/null | wc -l)"
+317
View File
@@ -0,0 +1,317 @@
#!/usr/bin/env python3
"""Build the two Swift1.5 campaign corpora (calibration + KLD).
calib.jsonl — 10M tokens of agent-shaped RAW text (45% code, 30% prose,
25% tool-call JSON), one {"prompt": text} per line, pushed to
/v1/completions with max_tokens=1 during the calibration run (the Hessian
tap is of the prompt tokens; docs/MOE-W4.md step 2). Code and prose mix
the repo checkout with HF dataset parquets (Frank's call 2026-10-06:
local material is only ~1.6M unique tokens; MOE-W4 wants 10M):
nampdn-ai/tiny-codes (code) and Salesforce/wikitext wikitext-103-raw
(prose). Tool calls are synthesized from commands harvested from the
repo's own scripts plus a fleet-ops set, with argument-shaped variants.
kld.jsonl — 1.8M tokens, one {"prompt", "score_from", "source"} per
line; each prompt is a multi-turn conversation rendered through the
Froggeric v22.5 template with a >= --kld-min-prompt (8192) token target,
and the last 256 tokens (the tail of the assistant answer) are the
scored span: score_from = n - 256. At 8k-token prompts the 1.8M corpus
is ~220 prompts = ~56k scored positions ~= 25 GB of record, matching
the published Flash-Next KLD scale (83k positions, MOE-W4.md). The KLD
reference run and every candidate run consume the SAME file (TOOLS.md
--kld-record / --kld-ref). The KLD corpus is LOCAL material only, so the
file-level disjointness rule (fnv1a64(path) % 5 == 0 -> KLD pool) is
untouched by the HF additions.
Deps: tokenizers, jinja2, pyarrow (venv /home/dank/venvs/radcorpus).
Usage: make-calib-corpus.py --repo ROOT --tokenizer-json PATH
--template PATH --out-dir DIR [--hf-code-parquet F,F]
[--hf-prose-parquet F,F] [--calib-tokens 10000000]
[--kld-tokens 1800000]
"""
import argparse
import json
import os
import random
import re
from jinja2 import Environment, FileSystemLoader
from tokenizers import Tokenizer
CODE_EXT = (".sh", ".py", ".jinja", ".recipe", ".env", ".example",
".yaml", ".yml", ".bats", ".service", ".container", ".toml")
JSONL_EXTS = (".jsonl", ".json")
DOC_DIR = "serverAppConfigs-docs"
FLEET_CMDS = [
"systemctl status frank-archon --no-pager",
"journalctl -u frank-archon -n 100 --no-pager",
"podman ps --format '{{.Names}} {{.Status}}'",
"podman logs --tail 50 frank-archon",
"curl -s http://192.168.0.235:7770/health",
"curl -s http://192.168.0.235:7770/metrics | grep radiance_",
"df -h /home/dank | tail -1",
"free -g",
"sudo systemctl restart frank-archon",
"cd /home/dank/serverAppConfigs && git pull --ff-only",
"./scripts/deploy.sh dankserver --service frank-archon",
"./tests/run.sh 2>&1 | tail -5",
"jq . /home/dank/models/rad/index.json | head -40",
"python3 -c \"import json; print(json.load(open('config.json'))['num_hidden_layers'])\"",
"ip neigh show 192.168.0.235",
"amdgpu_top --device-info 2>/dev/null | head -20",
"grep -E 'MemAvailable' /proc/meminfo",
"sha256sum model-00001-of-00131.safetensors",
"git log --oneline -10",
"awk '{s+=$1} END {print s}' shard_sizes.txt",
]
TOOL_NAMES = ["bash", "read", "grep", "web_search", "edit", "write"]
WORKDIRS = ["/home/dank", "/home/dank/serverAppConfigs", "/home/dank/models",
"/etc/systemd/system", "/var/log"]
DESCS = ["check service state", "tail the engine ledger", "list containers",
"query health", "read metrics", "check disk", "check memory",
"restart the endpoint", "sync the checkout", "run deployer",
"run the test seams", "inspect the index", "probe the peer"]
def fnv(s: str) -> int:
h = 0xcbf29ce484222325
for b in s.encode():
h = ((h ^ b) * 0x100000001b3) & 0xFFFFFFFFFFFFFFFF
return h
def harvest_cmds(repo: str, rng: random.Random) -> list[str]:
cmds = list(FLEET_CMDS)
pat = re.compile(r"^\s*(?:sudo\s+)?([a-z][\w./-]*(?:\s+\S+){1,6})\s*$")
for root, dirs, files in os.walk(repo):
dirs[:] = [d for d in dirs if not d.startswith(".")]
for f in files:
if not f.endswith(".sh"):
continue
try:
for line in open(os.path.join(root, f), encoding="utf-8",
errors="replace"):
m = pat.match(line.rstrip())
if m and len(m.group(1)) > 12:
cmds.append("ssh dank@192.168.0.235 '"
+ m.group(1)[:90] + "'")
except OSError:
pass
rng.shuffle(cmds)
return cmds[:4000]
def tool_call(rng: random.Random, cmds: list[str]) -> str:
"""One agent turn: 2-4 tool calls with argument-shaped JSON."""
parts = []
for _ in range(rng.randint(2, 4)):
args: dict = {"command": rng.choice(cmds)}
if rng.random() < 0.4:
args["cwd"] = rng.choice(WORKDIRS)
if rng.random() < 0.3:
args["timeout"] = rng.choice([10, 30, 60, 300, 600])
if rng.random() < 0.5:
args["description"] = rng.choice(DESCS)
parts.append('<tool_call>{"name": "%s", "arguments": %s}<|im_end|>'
% (rng.choice(TOOL_NAMES),
json.dumps(args, ensure_ascii=False)))
return "\n".join(parts)
def load_parquet_texts(path: str, kind: str, rng: random.Random,
cap: int) -> list[str]:
import pyarrow.parquet as pq
d = pq.read_table(path).to_pydict()
n = len(next(iter(d.values())))
idx = range(n) if n <= cap else rng.sample(range(n), cap)
out = []
for i in idx:
if kind == "code":
if "content" in d:
frag = d["content"][i]
else:
frag = ("// %s\n### Instruction:\n%s\n### Response:\n%s"
% (d.get("file_name", [""])[i], d["prompt"][i],
d["code"][i] or d["response"][i]))
else:
frag = d["text"][i]
if frag and frag.strip():
out.append(frag[:8000])
rng.shuffle(out)
return out[:cap]
def main() -> None:
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--repo", required=True)
ap.add_argument("--tokenizer-json", required=True)
ap.add_argument("--template", required=True)
ap.add_argument("--out-dir", required=True)
ap.add_argument("--calib-tokens", type=int, default=10_000_000)
ap.add_argument("--kld-tokens", type=int, default=1_800_000)
ap.add_argument("--chunk-tokens", type=int, default=8192)
ap.add_argument("--score-tail", type=int, default=256)
ap.add_argument("--kld-min-prompt", type=int, default=8192)
ap.add_argument("--hf-code-parquet", default="")
ap.add_argument("--hf-prose-parquet", default="")
ap.add_argument("--seed", type=int, default=20261006)
a = ap.parse_args()
rng = random.Random(a.seed)
tokenizer = Tokenizer.from_file(a.tokenizer_json)
env = Environment(loader=FileSystemLoader(os.path.dirname(a.template)),
trim_blocks=True, lstrip_blocks=True)
env.filters["tojson"] = lambda v: json.dumps(v, ensure_ascii=False)
env.filters["raise_exception"] = lambda m: (_ for _ in ()).throw(
ValueError(m))
tname = os.path.basename(a.template)
code, prose, jsonl = [], [], []
for root, dirs, files in os.walk(a.repo):
dirs[:] = [d for d in dirs if not d.startswith(".")]
for f in sorted(files):
p = os.path.relpath(os.path.join(root, f), a.repo)
if DOC_DIR in p and f.endswith(".md"):
prose.append(p)
elif p.startswith(DOC_DIR):
pass
elif f.endswith(CODE_EXT):
code.append(p)
elif f.endswith(JSONL_EXTS) and f != "package-lock.json":
jsonl.append(p)
# file-level split: bucket 0 -> KLD pool, 1..4 -> calibration pool
pools: dict[str, tuple[list, list]] = {}
for cls, lst in (("code", code), ("prose", prose), ("tools", jsonl)):
pools[cls] = ([p for p in lst if fnv(p) % 5 == 0],
[p for p in lst if fnv(p) % 5 != 0])
def text_of(p: str) -> str:
path = os.path.join(a.repo, p)
raw = open(path, encoding="utf-8", errors="replace").read()
if p.endswith(JSONL_EXTS):
lines = [json.loads(l)["content"] for l in raw.splitlines()
if l.strip() and "content" in l]
raw = "\n".join(lines)
toks = tokenizer.encode(raw, add_special_tokens=False).ids
return tokenizer.decode(toks[:2400], skip_special_tokens=True)
cmds = harvest_cmds(a.repo, rng)
# fragment pools per class, corpus-split first
frags: dict[str, dict[str, list[str]]] = {}
for cls in pools:
kld_files, cal_files = pools[cls]
fr = {}
src = {"kld": kld_files, "cal": cal_files}
if cls == "tools":
fr["kld"] = [tool_call(rng, cmds) for _ in range(600)]
fr["cal"] = [tool_call(rng, cmds) for _ in range(40000)]
else:
fr["kld"] = [text_of(p) for p in src["kld"]]
fr["cal"] = [text_of(p) for p in src["cal"]]
frags[cls] = fr
# per-class fragment caps: prose rows are short (~66 tok) so need many;
# codeparrot files are long (~2.6k tok) so a small cap suffices
hfcap = {"code": 40000, "prose": 60000, "tools": 0}
for cls, spec in (("code", a.hf_code_parquet),
("prose", a.hf_prose_parquet)):
for path in [p for p in spec.split(",") if p]:
frags[cls]["cal"].extend(
load_parquet_texts(path, cls, rng, cap=hfcap[cls]))
for cls in frags:
rng.shuffle(frags[cls]["cal"])
def toks(s: str) -> int:
return len(tokenizer.encode(s, add_special_tokens=False).ids)
def emit_calib() -> tuple[int, list[dict]]:
targets = {"code": .45, "prose": .30, "tools": .25}
rows: list[dict] = []
for cls, frac in targets.items():
want = int(a.calib_tokens * frac)
pool = frags[cls]["cal"]
if not pool:
continue
got, i, buf, nbuf = 0, 0, [], 0
while got < want:
frag = pool[i % len(pool)]
i += 1
n = toks(frag)
if nbuf + n > a.chunk_tokens and buf:
rows.append({"prompt": "\n".join(buf), "cls": cls})
got += nbuf
buf, nbuf = [], 0
if got >= want:
break
buf.append(frag)
nbuf += n
if i > len(pool) * 40:
break # pool exhausted with repeats; stop
if buf and got < want:
rows.append({"prompt": "\n".join(buf), "cls": cls})
got += nbuf
print(f"calib {cls}: {got} tokens, {i} fragments")
rng.shuffle(rows)
return len(rows), [{k: v for k, v in r.items() if k == "prompt"}
for r in rows]
def emit_kld() -> tuple[int, list[dict]]:
rows, total = [], 0
classes = ["code", "prose", "tools"]
i = {c: 0 for c in classes}
guard = 0
while total < a.kld_tokens and guard < 5000:
guard += 1
msgs, srcs = [], []
for t in range(rng.choice([2, 3, 4])):
cls = classes[(len(rows) + t) % 3]
pool = frags[cls]["kld"]
if not pool:
continue
frag = pool[i[cls] % len(pool)]
i[cls] += 2
msgs.append({"role": "user",
"content": "Summarize and explain this "
f"engineering material:\n{frag}"})
msgs.append({"role": "assistant",
"content": frag[:4000]})
srcs.append(cls)
rendered = env.get_template(tname).render(
messages=msgs, add_generation_prompt=False)
n = toks(rendered)
if n < a.kld_min_prompt:
continue
rows.append({"prompt": rendered,
"score_from": n - a.score_tail,
"source": "+".join(sorted(set(srcs)))})
total += n
return len(rows), rows
os.makedirs(a.out_dir, exist_ok=True)
nc, rows = emit_calib()
with open(os.path.join(a.out_dir, "calib.jsonl"), "w") as f:
for r in rows:
f.write(json.dumps(r) + "\n")
nk, krows = emit_kld()
with open(os.path.join(a.out_dir, "kld.jsonl"), "w") as f:
for r in krows:
f.write(json.dumps(r) + "\n")
scored = sum(len(tokenizer.encode(r["prompt"], add_special_tokens=False).ids)
- r["score_from"] for r in krows)
manifest = {
"seed": a.seed, "template": tname,
"tokenizer": os.path.basename(a.tokenizer_json),
"calib_prompts": nc, "kld_prompts": nk,
"kld_scored_positions": scored,
"kld_score_tail": a.score_tail,
"disjoint_file_pools": "local KLD material only; HF code feeds calib",
"calib_mix": {"code": .45, "prose": .30, "tools": .25},
"hf_code": a.hf_code_parquet, "hf_prose": a.hf_prose_parquet,
}
with open(os.path.join(a.out_dir, "manifest.json"), "w") as f:
json.dump(manifest, f, indent=1)
print(json.dumps(manifest, indent=1))
if __name__ == "__main__":
main()
@@ -2,7 +2,7 @@
#
# STATUS: WIP — first of three patches in the "compile the GLM trunk" changelist. The
# set does NOT yet boot the engine (see 03 for the next blocker), so it is NOT part of
# the deployed frank-primary shape. Nothing in scripts/ mounts these files.
# the deployed frank-archon shape. Nothing in scripts/ mounts these files.
#
# WHY THIS IS THE RIGHT LEVER (measured 2026-09-30, not inferred):
#
+1 -1
View File
@@ -1,6 +1,6 @@
# gpu_worker-kv-cap.patch — fork-local fix for the expert-offload KV cap.
#
# STATUS: NOT part of the deployed frank-primary shape. It exists because the
# STATUS: NOT part of the deployed frank-archon shape. It exists because the
# fork's plan under-models this model's KV, and it is the enabler for a future
# MTP attempt (the production GLM shape runs without MTP and boots fine on the
# plan's own reserve).
+201
View File
@@ -0,0 +1,201 @@
#!/usr/bin/env python3
"""probe-frank-archon-kv.py — verify the seqs x max-model-len KV contract on a
frank-archon (radiance) service. Run it ON dankserver: it talks to
127.0.0.1:7770 and builds its prompts from the repo checkout there.
No scratch container, no mask: the service under test is the service.
ssh dankserver 'python3 /home/dank/serverAppConfigs/scripts/probe-frank-archon-kv.py'
Four probes, in increasing cost:
A decode at depth — one ~51k-token chat completion, 512 tokens, greedy.
Reports the engine's own `timings` (prefill, decode, draft acceptance).
The bench law applies: >=128 generated tokens, greedy, real text.
B the capacity contract — as many CONCURRENT chats of ~197k prompt tokens as
the deployed `FRANK_ARCHON_MAX_SEQS` (read from the repo env file on this
host), each carrying a unique marker at its head. All must answer their own
marker. This is what the endpoint owes: N full-height conversations
resident at once, exact retrieval in every one of them.
C prefix-cache restore — probe A's cold prompt again, so the disk tier has
to give the blocks back (cache_n > 0, TTFT collapses).
D concurrency decode — `seqs` CONCURRENT 512-token generations at ~25 % of
the window. This is the cost a slot count imposes on DECODE, which no
prefill number shows: the fused-step schedule is unfair, so every stream is
printed plus the aggregate. Measured 2026-10-13 at 3 x 200,000 fp8:
single-stream 99.0 t/s (43.5k) and 93.3 (197k) — decode is nearly
window-free — against 3-stream 113.9/43.3/10.9 t/s = 18.4 t/s aggregate.
Concurrency, not the slab, is what decode pays for slots. The 2026-10-13
shape hunt missed this for a whole day because leg B generates 24 tokens.
PROBE LAW (learned the hard way, twice): radiance routes the thinking block to
`reasoning_content`, so a chat request whose budget is spent inside
`<think>` returns EMPTY `content` with `finish_reason: stop`. A probe that
reads `content` alone scores 0/2 on a healthy server. Every request here sets
chat_template_kwargs.enable_thinking=false and reads both fields.
A/B method for a shape change (no scratch container): write a variant env file
on the host, point a transient systemd drop-in at it
(`Environment=FRANK_ARCHON_ENV_FILE=...` — drop-ins are read AFTER the unit's
own Environment= lines, so they win), restart, probe, delete the drop-in,
daemon-reload, restart. `systemctl mask` is not available: the unit file in
/etc/systemd/system is a relink symlink into the checkout.
"""
import json
import glob
import time
import urllib.request
import concurrent.futures
BASE = "http://127.0.0.1:7770"
MODEL = "frank-archon"
REPO = "/home/dank/serverAppConfigs"
GLOBS = [f"{REPO}/scripts/*.sh", f"{REPO}/serverAppConfigs-docs/**/*.md"]
def post(payload, timeout=1800):
req = urllib.request.Request(
BASE + "/v1/chat/completions",
data=json.dumps(payload).encode(),
headers={"Content-Type": "application/json"},
)
t0 = time.time()
with urllib.request.urlopen(req, timeout=timeout) as r:
body = json.loads(r.read().decode())
hdr = dict(r.headers.items())
return body, time.time() - t0, hdr
def get(path, timeout=60):
with urllib.request.urlopen(BASE + path, timeout=timeout) as r:
return json.loads(r.read().decode())
def corpus(min_chars):
files = []
for g in GLOBS:
files.extend(sorted(glob.glob(g, recursive=True)))
out, total = [], 0
for f in files:
try:
t = open(f, encoding="utf-8", errors="replace").read()
except OSError:
continue
out.append(t)
total += len(t)
if total > min_chars:
break
return "".join(text for text in out)
def prompt(text, chars, salt, fact):
"""Real repo text, salted so the prefix cache cannot answer, marker planted
near the head, question at the tail."""
head = f"\n[SALT {salt}] confidential marker: {fact}\n"
body = text + text
p = (head + body * max(1, chars // len(body)))[:chars]
return p + "\n\nQuestion: what is the confidential marker in the " \
"SALT block near the top of this document? Answer with the marker " \
"alone, no other words.\n"
def chat(text, chars, salt, fact, max_tokens):
return {"model": MODEL,
"messages": [{"role": "user",
"content": prompt(text, chars, salt, fact)}],
"max_tokens": max_tokens, "temperature": 0.0,
"chat_template_kwargs": {"enable_thinking": False}}
def show(tag, j, wall, expect=None):
tim, u = j.get("timings") or {}, j.get("usage", {})
ch = (j.get("choices") or [{}])[0]
msg = ch.get("message") or {}
txt = (msg.get("content") or msg.get("reasoning_content") or "").strip()
acc = (tim["draft_n_accepted"] / tim["draft_n"]) if tim.get("draft_n") else None
line = (f"[{tag}] prompt={u.get('prompt_tokens')} gen={u.get('completion_tokens')} "
f"wall={wall:.1f}s prefill={tim.get('prompt_per_second', 0):.0f} t/s "
f"decode={tim.get('predicted_per_second', 0):.1f} t/s "
f"draft={tim.get('draft_n')}/{tim.get('draft_n_accepted')} "
f"accept={acc if acc is None else round(acc, 4)} "
f"cache_n={tim.get('cache_n')} finish={ch.get('finish_reason')}")
if expect:
hit = expect.split("-")[0] in txt.lower()
line += f" expected={expect} answered={txt[:80]!r} HIT={hit}"
print(line)
def deployed_shape():
"""The contract is the deployed env, not a constant: slot count and window
come from the checkout on this host, so the probe cannot drift from the
service it is testing (the 2026-10-13 shape is 4 x 200,000; before that
2 x 200,000, and the fp8 era 3 x 200,000)."""
shape = {}
with open(f"{REPO}/scripts/frank-archon.env") as fh:
for line in fh:
line = line.strip()
if line.startswith("FRANK_ARCHON_") and "=" in line:
key, _, val = line.partition("=")
shape[key] = val.strip().strip('"')
return (int(shape["FRANK_ARCHON_MAX_SEQS"]),
int(shape["FRANK_ARCHON_MAX_MODEL_LEN"]),
shape["FRANK_ARCHON_KV_DTYPE"])
def main():
seqs, maxlen, dtype = deployed_shape()
print(f"shape: {seqs} x {maxlen:,} tokens, kv {dtype}")
print("health:", json.dumps(get("/health")))
print("models:", [m.get("id") for m in get("/v1/models").get("data", [])])
deep = corpus(160_000)
wide = corpus(1_100_000)
# 3.28 chars/token is the measured ratio of this corpus through the
# froggeric template (646,000 chars -> 196,919 tokens at the 2026-10-11
# deploy); 98.5 % of the window leaves the response room under the ceiling.
chars = int(0.985 * maxlen * 3.28)
# Each marker's FIRST hyphen-separated word is unique, because show()
# scores the hit on that word — a shared prefix would score every slot HIT
# on any answer that says the word.
words = ["cobalt", "quartz", "basalt", "ember", "willow", "cobweb", "flint"]
facts = {f"B{i+1}": f"{words[i % len(words)]}-flag-{(i + 1) * 137}"
for i in range(seqs)}
bodies = {s: chat(wide, chars, s, f, 24) for s, f in facts.items()}
print(f"[B] {len(bodies['B1']['messages'][0]['content']):,} chars per "
f"request, x{seqs} concurrent")
# A + C first (cheap), B last (the expensive set).
for tag, salt in (("A-cold", "A1-9137"), ):
j, wall, _ = post(chat(deep, 140_000, salt, "zephyr-lantern-7", 512))
show(tag, j, wall, "zephyr-lantern-7")
t0 = time.time()
with concurrent.futures.ThreadPoolExecutor(max_workers=seqs) as ex:
futs = {s: ex.submit(post, b) for s, b in bodies.items()}
for s, fut in futs.items():
try:
j, wall, _ = fut.result()
show(f"B-{s}", j, wall, facts[s])
except Exception as e: # a preemption/timeout lands here
print(f"[B-{s}] FAILED after {time.time() - t0:.1f}s: {e!r}")
print(f"[B] total {time.time() - t0:.1f}s")
# D — concurrency decode: seqs streams, 512 tokens each, ~25 % depth.
dchars = int(0.25 * maxlen * 3.28)
dbodies = [chat(deep, dchars, f"D{i + 1}-9137", f"flint-marker-{i}", 512)
for i in range(seqs)]
print(f"[D] {seqs} concurrent streams x 512 tokens, "
f"{dchars:,} chars each")
t0 = time.time()
with concurrent.futures.ThreadPoolExecutor(max_workers=seqs) as ex:
futs = [ex.submit(post, b) for b in dbodies]
for i, fut in enumerate(futs, 1):
try:
j, wall, _ = fut.result()
show(f"D-s{i}", j, wall)
except Exception as e: # a preemption/timeout lands here
print(f"[D-s{i}] FAILED after {time.time() - t0:.1f}s: {e!r}")
tot = time.time() - t0
print(f"[D] {seqs} x 512 tokens in {tot:.1f}s = "
f"{seqs * 512 / tot:.1f} t/s aggregate decode")
j, wall, _ = post(chat(deep, 140_000, "A1-9137", "zephyr-lantern-7", 512))
show("C-prefix-hit", j, wall, "zephyr-lantern-7")
if __name__ == "__main__":
main()
@@ -0,0 +1,105 @@
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
# base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 +
# gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it.
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
#
# Why down is the calibrated half: the calibration tap rides the down projection, so its GPTQ
# rounding propagates against real down-input Hessians; gate_up shares the same w4 table
# uncalibrated. The published w5dn64 comparison (mean KL 0.0789 -> 0.0717, top-1 92.85% ->
# 93.34%, decode +4%, prefill -19%) is the 5-bit upper bound; this tier trades part of that
# quality back for prefill.
#
# CALIB=DIR rad-convert <Swift1.5-Qwen3.8-Flash-Next checkpoint> \
# --recipe scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe -o qwen4exp-swift15-w4dn8-i8-hc8m.rad
#
# DIR is the calibration directory the bf16 container wrote (RADIANCE_CALIB_DIR, 10M tokens of
# traffic-shaped corpus). Everything not named here is served as the checkpoint holds it (bf16:
# embeddings, the vision tower, the QSA indexer); the first rule that matches a weight decides it.
# The bf16 container is NOT passed with --reuse: the peak-disk arithmetic does not fit (checkpoint +
# container + container + output > purge-freed space), and --reuse only saves a re-read.
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's
# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
blk.1.ffn_*_exps.271.weight cast dtype=bf16
blk.1.ffn_*_exps.355.weight cast dtype=bf16
blk.2.ffn_*_exps.279.weight cast dtype=bf16
blk.2.ffn_*_exps.413.weight cast dtype=bf16
blk.34.ffn_*_exps.146.weight cast dtype=bf16
blk.34.ffn_*_exps.153.weight cast dtype=bf16
blk.34.ffn_*_exps.407.weight cast dtype=bf16
blk.34.ffn_*_exps.453.weight cast dtype=bf16
blk.37.ffn_*_exps.288.weight cast dtype=bf16
blk.37.ffn_*_exps.377.weight cast dtype=bf16
blk.38.ffn_*_exps.406.weight cast dtype=bf16
blk.38.ffn_*_exps.420.weight cast dtype=bf16
blk.42.ffn_*_exps.37.weight cast dtype=bf16
blk.42.ffn_*_exps.90.weight cast dtype=bf16
blk.42.ffn_*_exps.302.weight cast dtype=bf16
blk.42.ffn_*_exps.388.weight cast dtype=bf16
blk.44.ffn_*_exps.315.weight cast dtype=bf16
blk.44.ffn_*_exps.350.weight cast dtype=bf16
blk.46.ffn_*_exps.290.weight cast dtype=bf16
blk.46.ffn_*_exps.419.weight cast dtype=bf16
blk.47.ffn_*_exps.119.weight cast dtype=bf16
blk.47.ffn_*_exps.143.weight cast dtype=bf16
blk.47.ffn_*_exps.399.weight cast dtype=bf16
blk.47.ffn_*_exps.445.weight cast dtype=bf16
# PROTECTED-END
# (derive-protected-experts.py rewrote this section from Swift's own grams)
# Every other routed expert, trunk and MTP layer alike: u4 codes into the w4nl table, rotated by
# the 128-wide Hadamard, an E4M3 scale a group of 64 under the fixed 2^-13 searched for least
# error as stored, chosen by GPTQ against $CALIB and swept three times by coordinate descent.
# The calibration Grams are down-input Hessians, so down is the calibrated half of the table;
# gate_up has no recording of its own and quantises uncalibrated (said on stderr, as the base
# recipe says). The MTP layer has no recording either. Expert mass: gate_up 150.00 GiB -> ~37.6,
# down 75.00 GiB -> ~18.8: ~56 GiB of experts, ~28 GiB a rank at TP=2 -- well inside the pool.
blk.*.ffn_*_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB
# Every connection's two mixing matrices -- the 96 trunk connections, the final mixer and the MTP
# head's three -- as E4M3 rows with an f32 scale a group of each row. A 128x128 block cannot cover
# them: lowrank is 320, 2.5 blocks. Down is [320, 10240] and takes a scale every 128 columns; up is
# [10240, 320] and takes four scales a row, 80 columns each.
*_hc_down.weight rtn codes=fp8_e4m3 group=128 scale=f32
*_hc_up.weight rtn codes=fp8_e4m3 group=80 scale=f32
# The MTP head's fc_hidden and fc_embedding, the same rows. They only ever change the drafts.
mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32
mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32
# The trunk's block linears as int8, a scale per 128 columns of a row searched for least error.
# The QSA indexer's projection is not one of them: its top-k is a hard choice between blocks, and
# keeping it as the checkpoint holds it costs 20 MB.
blk.*.attn_qg.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.attn_k.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.attn_v.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.attn_output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ssm_inz.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ffn_gate_up_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ffn_down_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
# The lm_head, int8 with a bf16 scale a row per 128 columns (logits_gemm_i8): at E4M3 it costs
# ~0.6 points of top-1 on its own.
output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
# The n-gram table at one fixed E4M3 scale; qwen4exp-w4.recipe says where the value comes from.
# The value sits on a three-octave plateau (2^-13..2^-10, rel_l2 0.02662) with 1.7x headroom over
# the base table's absmax; the prep phase verifies Swift's PLE absmax lands inside the plateau
# before this rule is trusted.
blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4
# The MTP head's 2-bit draft head. INERT for this checkpoint: the HF weight map has no
# mtp.draft_head.weight (same as the base), kept for shape parity with the family.
mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16
+116
View File
@@ -0,0 +1,116 @@
#!/usr/bin/env bash
# run-swift15-bf16.sh — RUNS ON DANKSERVER. Boot the no-recipe bf16 Swift1.5
# container for the two offline campaign phases. The engine owns the GPUs, so
# the production unit must be stopped first (this script does it and restores
# on exit, autoresearch pattern: stop, never mask — relinked unit).
#
# kld --kld-record the reference logits from the bf16 model
# (runs the corpus and EXITS; engine writes DIR and leaves)
# calib serve with the calibration tap; a separate driver
# (scripts/calib-push.py) pushes the corpus; the engine writes the
# Hessians at RADIANCE_CALIB_TOKENS and the tap switches itself off.
# Stop this script after calib-push prints TAP_DONE.
#
# Shape = the vendor's bf16 calibration lines (GUIDE.md §4.3:528 + MOE-W4.md
# "calibrate the checkpoint as it ships"): expert_tiered + weights-disk-tier,
# no speculation, deterministic. Large prefill steps because a disk-tier step
# costs the same at 1 token as at 32K. The container declared --max-ctx 4096;
# --max-model-len 16384 exceeds it on purpose (the KLD prompts run to 12,007
# tokens, calib to 8,242) — the boot ledger arbitrates the QSA buffers; if
# declare refuses, re-convert with a wider --max-ctx.
set -uo pipefail
MODE="${1:?usage: run-swift15-bf16.sh kld|calib}"
PIN_1013=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
PIN_120=sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa
RAD=/home/dank/models/rad/qwen4exp-swift15-bf16.rad
C=swift15ref
LOG=/home/dank/logs/swift15_bf16.log
KLD_REF_DIR=/home/dank/corpora/kld-ref
CALIB_DIR=/home/dank/corpora/calib
KLD_CORPUS=/home/dank/corpora/out/kld.jsonl
restore() { sudo systemctl start frank-archon >/dev/null 2>&1 || true; }
trap restore EXIT
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
podman rm -f "$C" >/dev/null 2>&1 || true
ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad
--tp 2 --placement expert_tiered --weights-disk-tier
--host-pool-mib 12288 --deterministic --gpu-headroom-mib 768
--num-speculative-tokens 0
--kv-cache-dtype fp8 --max-model-len 16384
--ngram-placement ram
--override-chat-template /models/tpl.jinja)
case "$MODE" in
kld)
# 1.0.13: the engine that will serve the candidate — the reference is
# recorded on the same engine that scores it. The weights slab DERIVES as
# card - headroom - kv, and the disk tier refuses under 9.57 GiB (4.21 of
# layer buffers + 5.36 fixed), so kv is explicit SMALL: recording is
# prefill-only, self-batched at MNBT 32768 = two 16k prompts at a time;
# 4 slots x 16k fp8 (29 KiB/pos, 1.7 GiB) + headroom fits 3072.
IMAGE="stilldeadcode/radiance@$PIN_1013"
mkdir -p "$KLD_REF_DIR"
ARGS+=(--kld-record /data/kld-ref --kld-corpus /data/kld.jsonl
--vram-kv-mib 3072
--max-num-seqs 4 --max-num-batched-tokens 32768)
;;
calib)
# 1.2.0: the calibration tap (RADIANCE_CALIB_DIR + RADIANCE_CALIB_TOKENS)
# lives in the 1.2.0 qwen4exp plugin; 1.0.13 has only the DIR half with an
# unparameterisable threshold. Grams are bf16 activations — engine-invariant.
# Same KV ceiling as kld: the disk-tier weights floor (9.57 GiB) plus
# ~18 GiB of fixed per-rank claims cap the pool near 3.5 GiB on a 32 GiB
# card — pool minus the 2.17 GiB fixed holds ~237k fp8 tokens a rank, so
# the vendor's big-step calibration shape is affordable: MOE-W4.md:244
# measures 640 tok/s at a 32k step against 930 at 55k, and the doc's own
# run drives MNBT 65536. 8 slots x 16k = 131k resident tokens fits the
# pool; the step batches 65k. KLD's observed rate confirmed step size,
# not slot count, is the throughput lever.
IMAGE="stilldeadcode/radiance@$PIN_120"
mkdir -p "$CALIB_DIR"
ARGS+=(--port 7770 --vram-kv-mib 3072
--max-num-seqs 8 --max-num-batched-tokens 65536
--prefix-cache-dir /data/calib-kvcache)
;;
probe)
IMAGE="stilldeadcode/radiance@$PIN_1013"
ARGS+=(--port 7770 --max-num-seqs 8 --max-num-batched-tokens 4096)
;;
*) echo "unknown mode: $MODE" >&2; exit 2 ;;
esac
# The prod template copy, readable from the container: mount the models tree
# ro plus the state dir rw (KLD record + prefix tier land there).
cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \
/home/dank/models/tpl.jinja 2>/dev/null || true
KLD_MOUNT=()
if [ "$MODE" = kld ]; then
# Nested under the /data mount, so these must come after it (line order
# in podman run below): the corpus read-only, the record dir writable.
KLD_MOUNT=(-v "$KLD_CORPUS:/data/kld.jsonl:ro"
-v "$KLD_REF_DIR:/data/kld-ref")
fi
EXTRA_ENV=()
if [ "$MODE" = calib ]; then
EXTRA_ENV=(-v "$CALIB_DIR:/data/calib"
-e RADIANCE_CALIB_DIR=/data/calib
-e RADIANCE_CALIB_TOKENS=10000000)
fi
podman run --rm --name "$C" \
--network host --init \
--security-opt seccomp=unconfined \
--ulimit memlock=-1:-1 \
--device /dev/kfd --device /dev/dri \
-v /home/dank/models:/models:ro \
-v /home/dank/radiance-state:/data \
"${KLD_MOUNT[@]}" \
"${EXTRA_ENV[@]}" \
"$IMAGE" "${ARGS[@]}" 2>&1 | tee "$LOG"
rc=${PIPESTATUS[0]}
echo "swift15-bf16 $MODE exited rc=$rc; log: $LOG"
exit "$rc"
+196
View File
@@ -0,0 +1,196 @@
#!/usr/bin/env bash
# serve-frank-archon.sh — frank-archon endpoint: radiance (StillDeadcode,
# Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at
# NATIVE block-FP8 precision from a rad-convert container
# (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2,
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587,
# one card; git tag frank-archon-freetoken-pr587-20261005 is the
# pre-cutover state).
#
# The run line below IS the engine argv: the image's entrypoint is the
# engine and its tools sit beside it on PATH. Unlike every predecessor
# there is no per-boot VRAM planner to second-guess: the boot ledger in the
# log states every budget before anything is allocated (weights vs KV
# split, the expert host pool's coverage of the worst case, the PLE pin,
# the P2P answer per pair) — read it after every resize.
# Deployed shape (2026-10-13): THREE decode slots x 200,000 tokens, fp8
# (quantized) KV in a 5,120 MiB pool (contract 3 x 200,000 + 1.3 %) and a
# 25,600 MiB resident expert slab — 30,720 MiB of the 30,969 MiB the engine's
# own budget table makes claimable, spent on the slab because prefill is
# slab-limited. 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache host
# 8 GiB + disk 64 GiB, PLE pinned in RAM.
#
# The root unit (scripts/frank-archon.service, User=dank,
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
# the 47.7 GiB PLE table + 8 GiB prefix host tier — ~147 GiB of pinned
# pages, which is ALSO why this script refuses to start when ttm's
# pages_limit/pages_max cap is under FRANK_ARCHON_TTM_MIN_MIB (see below).
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_ARCHON_ENV_FILE:-$SCRIPT_DIR/frank-archon.env}"
if [ ! -f "$ENV_FILE" ]; then
echo "ERROR: frank-archon env file absent: $ENV_FILE" >&2
exit 1
fi
set -a
# shellcheck disable=SC1090
source "$ENV_FILE"
set +a
: "${FRANK_ARCHON_IMAGE:?FRANK_ARCHON_IMAGE must name the radiance image}"
: "${FRANK_ARCHON_MODEL_DIR:?FRANK_ARCHON_MODEL_DIR must name the .rad dir}"
: "${FRANK_ARCHON_MODEL_FILE:?FRANK_ARCHON_MODEL_FILE must name the .rad file}"
: "${FRANK_ARCHON_STATE_DIR:?FRANK_ARCHON_STATE_DIR must name the /data state dir}"
: "${FRANK_ARCHON_CONTAINER:?FRANK_ARCHON_CONTAINER must name the container}"
: "${FRANK_ARCHON_SERVED_NAME:?FRANK_ARCHON_SERVED_NAME must name the alias}"
: "${FRANK_ARCHON_PORT:?FRANK_ARCHON_PORT must be the port}"
: "${FRANK_ARCHON_TEMPLATE:?FRANK_ARCHON_TEMPLATE must name the chat template}"
: "${FRANK_ARCHON_TP:?FRANK_ARCHON_TP must be set}"
: "${FRANK_ARCHON_TP_WIRE:?FRANK_ARCHON_TP_WIRE must be set}"
: "${FRANK_ARCHON_MAX_SEQS:?FRANK_ARCHON_MAX_SEQS must be set}"
: "${FRANK_ARCHON_MAX_MODEL_LEN:?FRANK_ARCHON_MAX_MODEL_LEN must be set}"
: "${FRANK_ARCHON_PLACEMENT:?FRANK_ARCHON_PLACEMENT must be set}"
: "${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
: "${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
: "${FRANK_ARCHON_VRAM_WEIGHTS_MIB:?FRANK_ARCHON_VRAM_WEIGHTS_MIB must be set}"
: "${FRANK_ARCHON_VRAM_KV_MIB:?FRANK_ARCHON_VRAM_KV_MIB must be set}"
: "${FRANK_ARCHON_KV_DTYPE:?FRANK_ARCHON_KV_DTYPE must be set}"
: "${FRANK_ARCHON_SPEC_TOKENS:?FRANK_ARCHON_SPEC_TOKENS must be set}"
: "${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
: "${FRANK_ARCHON_PREFIX_HOST_MIB:?FRANK_ARCHON_PREFIX_HOST_MIB must be set}"
: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
: "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
: "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
: "${FRANK_ARCHON_TTM_TARGET_MIB:?FRANK_ARCHON_TTM_TARGET_MIB must be set}"
: "${FRANK_ARCHON_RAD_MIN_MIB:?FRANK_ARCHON_RAD_MIN_MIB must be set}"
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
# in storage only if THAT manifest was pulled — a retagged look-alike or a
# newer latest does not satisfy it. `podman image exists` takes a
# name@digest ref in one command — no pipefail/grep race (the 2026-09-30
# lesson). Runs as dank: the unit's process uses dank's ROOTLESS podman
# storage (root unit, User=dank — the fleet's frank-archon shape).
if ! podman image exists "$FRANK_ARCHON_IMAGE"; then
echo "ERROR: radiance image absent: $FRANK_ARCHON_IMAGE" >&2
echo "frank-archon: podman pull docker://stilldeadcode/radiance:latest (as dank)" >&2
exit 1
fi
# The payload gate: the .rad container file, sized. A truncated convert
# would otherwise buy a guaranteed long-timeout boot instead of this fast
# failure. The staging dir / raw checkpoint are NOT this script's concern
# (scripts/build-frank-archon-rad.sh built the file; its rad-info output
# is the record).
RAD="$FRANK_ARCHON_MODEL_DIR/$FRANK_ARCHON_MODEL_FILE"
if [ ! -f "$RAD" ]; then
echo "ERROR: .rad container missing: $RAD" >&2
echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
exit 1
fi
RAD_MIB=$(( $(stat -c %s "$RAD") / 1048576 ))
if [ "$RAD_MIB" -lt "$FRANK_ARCHON_RAD_MIN_MIB" ]; then
echo "ERROR: .rad container too small: $RAD is ${RAD_MIB} MiB, want >= $FRANK_ARCHON_RAD_MIN_MIB" >&2
echo "frank-archon: the convert was truncated — rebuild it" >&2
exit 1
fi
if [ ! -f "$FRANK_ARCHON_TEMPLATE" ]; then
echo "ERROR: chat template missing: $FRANK_ARCHON_TEMPLATE" >&2
exit 1
fi
mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
# The engine's create is NOT recursive: with only the state dir mounted, its
# "cannot create /data/kvcache/frank-archon: No such file or directory"
# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
# the full path on the host so the tier opens on first boot.
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
# rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
# the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
# $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
# the cache must ride a host mount or it dies with the --rm container
# that wrote it. The dir is pre-created for the same reason as kvcache
# (the engine's create is not recursive).
if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
exit 1
fi
# TTM preflight: the pinned ledger (2 x FRANK_ARCHON_HOST_POOL_MIB + the
# 47.7 GiB PLE table + the prefix host tier, ~147 GiB) rides TTM's shared
# pages, capped by ttm's pages_limit (the current kernel exposes NO
# pages_max; the cap function takes the min over the knobs that exist).
# Below FRANK_ARCHON_TTM_MIN_MIB the load dies mid-pin with an allocator
# error, so the gate refuses first — and RAISES the knob before refusing:
# the cap is a runtime knob whose default is half of RAM, so a power failure
# resets it (measured 2026-10-12 23:17 on this host: 200 GiB -> 125,988 MiB,
# and Restart=always re-ran the refusal 6,919 times over ~20 h).
# FRANK_ARCHON_TTM_TARGET_MIB is the recorded host ceiling, not the floor.
# shellcheck source=/dev/null
source "$SCRIPT_DIR/ttm-preflight.sh"
ttm_ensure_cap || exit 1
# A crashed/killed unit can leave the container name registered (systemd
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
# identities (the FreeToken-era frank-archon-qwen, the retired R9V
# container and the ad hoc experiment containers all hold this GPU); all
# idempotent.
for CTR in $FRANK_ARCHON_LEGACY_CONTAINERS "$FRANK_ARCHON_CONTAINER"; do
podman rm -f "$CTR" >/dev/null 2>&1 || true
done
LOG=/home/dank/logs/frank_archon.log
mkdir -p "$(dirname "$LOG")"
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
# --network host (the server binds the host's addresses directly, no NAT),
# --init (the engine releases the cards on SIGTERM), seccomp=unconfined
# (Docker/podman's default profile refuses io_uring, which the n-gram
# table's reads go through, and the NUMA binding calls the ROCm runtime
# makes for host memory), memlock unbounded (the pinned ledger — legal
# because the root unit grants LimitMEMLOCK=infinity to this dank process),
# /dev/kfd + /dev/dri (render/video groups are moot for a root container).
# The template rides a single-file mount; the .rad dir is ro; /data holds
# the prefix-cache disk tier.
podman run -d --name "$FRANK_ARCHON_CONTAINER" \
--network host \
--init \
--security-opt seccomp=unconfined \
--ulimit memlock=-1:-1 \
--device /dev/kfd --device /dev/dri \
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-v "$FRANK_ARCHON_STATE_DIR":/data \
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
"$FRANK_ARCHON_IMAGE" \
--model "/models/$FRANK_ARCHON_MODEL_FILE" \
--host :: --port "$FRANK_ARCHON_PORT" \
--served-model-name "$FRANK_ARCHON_SERVED_NAME" \
--override-chat-template /templates/template.jinja \
--tp "$FRANK_ARCHON_TP" --tp-wire "$FRANK_ARCHON_TP_WIRE" \
--max-num-seqs "$FRANK_ARCHON_MAX_SEQS" \
--max-model-len "$FRANK_ARCHON_MAX_MODEL_LEN" \
--placement "$FRANK_ARCHON_PLACEMENT" \
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
--prefix-cache-dir /data/kvcache/frank-archon \
--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
> "$LOG" 2>&1
# From here on, this script's lifetime IS the container's: wait returns the
# container's exit code, so a crash is a nonzero ExecStart and the unit's
# Restart=always is real. (systemd stop goes through ExecStop in the unit —
# podman wait alone cannot stop the container child.)
exec podman wait "$FRANK_ARCHON_CONTAINER"
@@ -1,13 +1,13 @@
#!/usr/bin/env bash
# serve-frank-large.sh — frank-large endpoint: halogen-flash-server 0.16.2
# serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.16.2
# (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server,
# digest-pinned in frank-large.env) serving the Quat3rnion abliterated
# digest-pinned in frank-dragoon.env) serving the Quat3rnion abliterated
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint on
# Dankclaw's Strix Halo iGPU, host :7770. 2026-10-06 cutover from the
# halogen 27B-dense shape — revert the swap commit's trio to roll back
# (the 27B payload stays on disk).
#
# The engine is driven by `podman run` DIRECTLY (the fleet's frank-primary
# The engine is driven by `podman run` DIRECTLY (the fleet's frank-archon
# pattern): the image starts engine + OpenAI frontend in one container.
# The frontend listens on 8731 inside (HALOGEN_API_PORT default) and is
# published as host 7770; the engine's own token port (8730, no auth)
@@ -26,13 +26,13 @@
# Why containers instead of a bare binary: the payload is .hgn and only
# the flash-server line reads it; it ships as a container; the root unit
# keeps LimitMEMLOCK=infinity for parity and to outlive the login session,
# exactly like frank-primary's.
# exactly like frank-archon's.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
if [ ! -f "$ENV_FILE" ]; then
echo "frank-large: env file not found: $ENV_FILE" >&2
echo "frank-dragoon: env file not found: $ENV_FILE" >&2
exit 1
fi
set -a
@@ -40,18 +40,18 @@ set -a
source "$ENV_FILE"
set +a
: "${FRANK_LARGE_IMAGE:?FRANK_LARGE_IMAGE must name the digest-pinned image}"
: "${FRANK_LARGE_CONTAINER:?FRANK_LARGE_CONTAINER must name the container}"
: "${FRANK_LARGE_PUBLISH:?FRANK_LARGE_PUBLISH must be host:container}"
: "${FRANK_LARGE_MODEL_DIR:?FRANK_LARGE_MODEL_DIR must name the payload dir}"
: "${FRANK_LARGE_CHECKPOINT:?FRANK_LARGE_CHECKPOINT must name the .hgn inside it}"
: "${FRANK_LARGE_CHECKPOINT_BYTES:?FRANK_LARGE_CHECKPOINT_BYTES must pin the size}"
: "${FRANK_LARGE_NGRAM_TABLE:?FRANK_LARGE_NGRAM_TABLE must name the lookup table}"
: "${FRANK_LARGE_NGRAM_BYTES:?FRANK_LARGE_NGRAM_BYTES must pin the table size}"
: "${FRANK_LARGE_VISION_TOWER:?FRANK_LARGE_VISION_TOWER must name the tower}"
: "${FRANK_LARGE_VISION_BYTES:?FRANK_LARGE_VISION_BYTES must pin the tower size}"
: "${FRANK_LARGE_TOKENIZER_FILES:?FRANK_LARGE_TOKENIZER_FILES must list the tokenizer dir contents}"
: "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-large)}"
: "${FRANK_DRAGOON_IMAGE:?FRANK_DRAGOON_IMAGE must name the digest-pinned image}"
: "${FRANK_DRAGOON_CONTAINER:?FRANK_DRAGOON_CONTAINER must name the container}"
: "${FRANK_DRAGOON_PUBLISH:?FRANK_DRAGOON_PUBLISH must be host:container}"
: "${FRANK_DRAGOON_MODEL_DIR:?FRANK_DRAGOON_MODEL_DIR must name the payload dir}"
: "${FRANK_DRAGOON_CHECKPOINT:?FRANK_DRAGOON_CHECKPOINT must name the .hgn inside it}"
: "${FRANK_DRAGOON_CHECKPOINT_BYTES:?FRANK_DRAGOON_CHECKPOINT_BYTES must pin the size}"
: "${FRANK_DRAGOON_NGRAM_TABLE:?FRANK_DRAGOON_NGRAM_TABLE must name the lookup table}"
: "${FRANK_DRAGOON_NGRAM_BYTES:?FRANK_DRAGOON_NGRAM_BYTES must pin the table size}"
: "${FRANK_DRAGOON_VISION_TOWER:?FRANK_DRAGOON_VISION_TOWER must name the tower}"
: "${FRANK_DRAGOON_VISION_BYTES:?FRANK_DRAGOON_VISION_BYTES must pin the tower size}"
: "${FRANK_DRAGOON_TOKENIZER_FILES:?FRANK_DRAGOON_TOKENIZER_FILES must list the tokenizer dir contents}"
: "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-dragoon)}"
: "${HALOGEN_CTX:?HALOGEN_CTX must be set (per-slot window)}"
: "${HALOGEN_KV_POOL_POSITIONS:?HALOGEN_KV_POOL_POSITIONS must be set}"
: "${HALOGEN_KV_SLOTS:?HALOGEN_KV_SLOTS must be set}"
@@ -62,9 +62,9 @@ set +a
# podman dies SIGPIPE 141, pipefail turns the 141 into "no match" — the
# 2026-09-30 deploy refusal).
DIGESTS="$(podman images --format '{{.RepoDigests}}')"
if ! grep -qF "${FRANK_LARGE_IMAGE#*@}" <<<"$DIGESTS"; then
echo "frank-large: image $FRANK_LARGE_IMAGE not in podman storage;" >&2
echo " pull it: podman pull ${FRANK_LARGE_IMAGE%@*}" >&2
if ! grep -qF "${FRANK_DRAGOON_IMAGE#*@}" <<<"$DIGESTS"; then
echo "frank-dragoon: image $FRANK_DRAGOON_IMAGE not in podman storage;" >&2
echo " pull it: podman pull ${FRANK_DRAGOON_IMAGE%@*}" >&2
exit 1
fi
@@ -74,45 +74,45 @@ fi
ck() {
# ck <env-name> <path> <expected-bytes>
if [ ! -f "$2" ]; then
echo "frank-large: $1 $2 missing;" >&2
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
echo "frank-dragoon: $1 $2 missing;" >&2
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
exit 1
fi
local sz
sz="$(stat -c %s "$2")"
if [ "$sz" != "$3" ]; then
echo "frank-large: $1 is $sz bytes, expected $3 (partial download?)" >&2
echo "frank-dragoon: $1 is $sz bytes, expected $3 (partial download?)" >&2
exit 1
fi
}
ck checkpoint "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_CHECKPOINT" "$FRANK_LARGE_CHECKPOINT_BYTES"
ck n-gram-table "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_NGRAM_TABLE" "$FRANK_LARGE_NGRAM_BYTES"
ck vision-tower "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_VISION_TOWER" "$FRANK_LARGE_VISION_BYTES"
ck checkpoint "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_CHECKPOINT" "$FRANK_DRAGOON_CHECKPOINT_BYTES"
ck n-gram-table "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_NGRAM_TABLE" "$FRANK_DRAGOON_NGRAM_BYTES"
ck vision-tower "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_VISION_TOWER" "$FRANK_DRAGOON_VISION_BYTES"
# The payload ships the tokenizer as a FLAT DIRECTORY. A HuggingFace cache
# snapshot symlinks each file into a sibling blobs/, and those symlinks
# dangle inside the container mount — so the directory must be real and
# every named file must be a regular file, not a link.
TOK_DIR="$FRANK_LARGE_MODEL_DIR/tokenizer"
TOK_DIR="$FRANK_DRAGOON_MODEL_DIR/tokenizer"
if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then
echo "frank-large: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
echo "frank-dragoon: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
exit 1
fi
for TF in $FRANK_LARGE_TOKENIZER_FILES; do
for TF in $FRANK_DRAGOON_TOKENIZER_FILES; do
if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then
echo "frank-large: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
echo "frank-dragoon: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
exit 1
fi
done
# GPU access is mandatory but the node paths are variables so the seam can
# drive the gate on a box without an AMD GPU.
DEV_KFD="${FRANK_LARGE_DEV_KFD:-/dev/kfd}"
DEV_DRI="${FRANK_LARGE_DEV_DRI:-/dev/dri}"
DEV_KFD="${FRANK_DRAGOON_DEV_KFD:-/dev/kfd}"
DEV_DRI="${FRANK_DRAGOON_DEV_DRI:-/dev/dri}"
if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
echo "frank-large: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
echo "frank-dragoon: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
exit 1
fi
# Host-RAM preflight — the engine's pin gate, checked BEFORE podman run.
@@ -125,15 +125,15 @@ fi
# self-resolved in ~2 min when the dying 27B engine freed RAM. Check here and
# WAIT for the RAM to clear, so a transient dip boots once instead of
# crash-looping; if it never clears, refuse loudly.
MEMINFO="${FRANK_LARGE_MEMINFO:-/proc/meminfo}"
PIN_KB=$(( FRANK_LARGE_CHECKPOINT_BYTES / 1024 ))
MEMINFO="${FRANK_DRAGOON_MEMINFO:-/proc/meminfo}"
PIN_KB=$(( FRANK_DRAGOON_CHECKPOINT_BYTES / 1024 ))
NEED_KB=$(( PIN_KB + 16 * 1048576 ))
ram_avail_kb() { awk '/^MemAvailable:/{print $2; exit}' "$MEMINFO"; }
MEMAVAIL_KB="$(ram_avail_kb)" || MEMAVAIL_KB=""
if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
WAIT_S="${FRANK_LARGE_RAM_WAIT:-600}"
WAIT_S="${FRANK_DRAGOON_RAM_WAIT:-600}"
GIB() { awk -v kb="$1" 'BEGIN{printf "%.2f", kb/1048576}'; }
echo "frank-large: host RAM below the pin gate — MemAvailable $(GIB "$MEMAVAIL_KB") GiB < needed $(GIB "$NEED_KB") GiB ($(GIB "$PIN_KB") GiB checkpoint + 16.00 GiB floor); waiting up to ${WAIT_S}s for host RAM to clear" >&2
echo "frank-dragoon: host RAM below the pin gate — MemAvailable $(GIB "$MEMAVAIL_KB") GiB < needed $(GIB "$NEED_KB") GiB ($(GIB "$PIN_KB") GiB checkpoint + 16.00 GiB floor); waiting up to ${WAIT_S}s for host RAM to clear" >&2
deadline=$(( $(date +%s) + WAIT_S ))
while [ "$(date +%s)" -lt "$deadline" ]; do
sleep 5
@@ -143,16 +143,16 @@ if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
fi
done
if [ -z "$MEMAVAIL_KB" ] || [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
echo "frank-large: MemAvailable $(GIB "${MEMAVAIL_KB:-0}") GiB after ${WAIT_S}s; refusing to start (the engine would refuse the pin)" >&2
echo "frank-dragoon: MemAvailable $(GIB "${MEMAVAIL_KB:-0}") GiB after ${WAIT_S}s; refusing to start (the engine would refuse the pin)" >&2
exit 1
fi
echo "frank-large: MemAvailable cleared to $(GIB "$MEMAVAIL_KB") GiB; starting" >&2
echo "frank-dragoon: MemAvailable cleared to $(GIB "$MEMAVAIL_KB") GiB; starting" >&2
fi
for CTR in ${FRANK_LARGE_LEGACY_CONTAINERS:-} "$FRANK_LARGE_CONTAINER"; do
for CTR in ${FRANK_DRAGOON_LEGACY_CONTAINERS:-} "$FRANK_DRAGOON_CONTAINER"; do
podman rm -f "$CTR" >/dev/null 2>&1 || true
done
LOG="${FRANK_LARGE_LOG:-/home/dank/logs/frank_large.log}"
LOG="${FRANK_DRAGOON_LOG:-/home/dank/logs/frank_dragoon.log}"
mkdir -p "$(dirname "$LOG")"
# --ipc=host for the udev-granted supplementary group, seccomp unconfined
# for the ROCm ioctl surface, and unbounded memlock (the unit's
@@ -163,17 +163,17 @@ mkdir -p "$(dirname "$LOG")"
# and both sibling serve scripts pass no nofile either. No comment lines
# inside the backslash chain — a mid-chain # terminates the continuation
# (the 2026-10-05 exit-127 crash loop on the sibling serve script).
podman run --name "$FRANK_LARGE_CONTAINER" \
--publish "$FRANK_LARGE_PUBLISH" \
podman run --name "$FRANK_DRAGOON_CONTAINER" \
--publish "$FRANK_DRAGOON_PUBLISH" \
--ipc=host \
--ulimit memlock=-1:-1 \
--device "$DEV_KFD" --device "$DEV_DRI" \
--group-add keep-groups \
--security-opt seccomp=unconfined \
-e HALOGEN_MODEL_ID="$HALOGEN_MODEL_ID" \
-e HALOGEN_CHECKPOINT="/models/$FRANK_LARGE_CHECKPOINT" \
-e HALOGEN_NGRAM_TABLE="/models/$FRANK_LARGE_NGRAM_TABLE" \
-e HALOGEN_VISION_TOWER="/models/$FRANK_LARGE_VISION_TOWER" \
-e HALOGEN_CHECKPOINT="/models/$FRANK_DRAGOON_CHECKPOINT" \
-e HALOGEN_NGRAM_TABLE="/models/$FRANK_DRAGOON_NGRAM_TABLE" \
-e HALOGEN_VISION_TOWER="/models/$FRANK_DRAGOON_VISION_TOWER" \
-e HALOGEN_TOKENIZER=/tokenizer \
-e HALOGEN_CTX="$HALOGEN_CTX" \
-e HALOGEN_KV_POOL_POSITIONS="$HALOGEN_KV_POOL_POSITIONS" \
@@ -187,9 +187,9 @@ podman run --name "$FRANK_LARGE_CONTAINER" \
-e HALOGEN_TOP_P="${HALOGEN_TOP_P:-0.95}" \
-e HALOGEN_TOP_K="${HALOGEN_TOP_K:-20}" \
${HALOGEN_WEIGHTS_LOCK:+-e HALOGEN_WEIGHTS_LOCK="$HALOGEN_WEIGHTS_LOCK"} \
-v "$FRANK_LARGE_MODEL_DIR:/models:ro" \
-v "$FRANK_LARGE_MODEL_DIR/tokenizer:/tokenizer:ro" \
"$FRANK_LARGE_IMAGE" \
-v "$FRANK_DRAGOON_MODEL_DIR:/models:ro" \
-v "$FRANK_DRAGOON_MODEL_DIR/tokenizer:/tokenizer:ro" \
"$FRANK_DRAGOON_IMAGE" \
> "$LOG" 2>&1
# FOREGROUND `podman run`: the engine's stdout/stderr are this script's
-204
View File
@@ -1,204 +0,0 @@
#!/usr/bin/env bash
# serve-frank-primary.sh — frank-primary endpoint: radiance (StillDeadcode,
# Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at
# NATIVE block-FP8 precision from a rad-convert container
# (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2,
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587,
# one card; git tag frank-primary-freetoken-pr587-20261005 is the
# pre-cutover state).
#
# The run line below IS the engine argv: the image's entrypoint is the
# engine and its tools sit beside it on PATH. Unlike every predecessor
# there is no per-boot VRAM planner to second-guess: the boot ledger in the
# log states every budget before anything is allocated (weights vs KV
# split, the expert host pool's coverage of the worst case, the PLE pin,
# the P2P answer per pair) — read it after every resize. First-boot shape:
# 2 decode slots (Frank's call), 40 GiB/rank pinned expert pool, fp8 KV,
# MTP depth 3, prefix cache host 4 GiB + disk 64 GiB, PLE pinned in RAM.
#
# The root unit (scripts/frank-primary.service, User=dank,
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 40 GiB expert pools +
# the 47.7 GiB PLE table + 4 GiB prefix host tier — ~132 GiB of pinned
# pages, which is ALSO why this script refuses to start when ttm's
# pages_limit/pages_max cap is under FRANK_PRIMARY_TTM_MIN_MIB (see below).
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_PRIMARY_ENV_FILE:-$SCRIPT_DIR/frank-primary.env}"
if [ ! -f "$ENV_FILE" ]; then
echo "ERROR: frank-primary env file absent: $ENV_FILE" >&2
exit 1
fi
set -a
# shellcheck disable=SC1090
source "$ENV_FILE"
set +a
: "${FRANK_PRIMARY_IMAGE:?FRANK_PRIMARY_IMAGE must name the radiance image}"
: "${FRANK_PRIMARY_MODEL_DIR:?FRANK_PRIMARY_MODEL_DIR must name the .rad dir}"
: "${FRANK_PRIMARY_MODEL_FILE:?FRANK_PRIMARY_MODEL_FILE must name the .rad file}"
: "${FRANK_PRIMARY_STATE_DIR:?FRANK_PRIMARY_STATE_DIR must name the /data state dir}"
: "${FRANK_PRIMARY_CONTAINER:?FRANK_PRIMARY_CONTAINER must name the container}"
: "${FRANK_PRIMARY_SERVED_NAME:?FRANK_PRIMARY_SERVED_NAME must name the alias}"
: "${FRANK_PRIMARY_PORT:?FRANK_PRIMARY_PORT must be the port}"
: "${FRANK_PRIMARY_TEMPLATE:?FRANK_PRIMARY_TEMPLATE must name the chat template}"
: "${FRANK_PRIMARY_TP:?FRANK_PRIMARY_TP must be set}"
: "${FRANK_PRIMARY_TP_WIRE:?FRANK_PRIMARY_TP_WIRE must be set}"
: "${FRANK_PRIMARY_MAX_SEQS:?FRANK_PRIMARY_MAX_SEQS must be set}"
: "${FRANK_PRIMARY_MAX_MODEL_LEN:?FRANK_PRIMARY_MAX_MODEL_LEN must be set}"
: "${FRANK_PRIMARY_PLACEMENT:?FRANK_PRIMARY_PLACEMENT must be set}"
: "${FRANK_PRIMARY_HOST_POOL_MIB:?FRANK_PRIMARY_HOST_POOL_MIB must be set}"
: "${FRANK_PRIMARY_GPU_HEADROOM_MIB:?FRANK_PRIMARY_GPU_HEADROOM_MIB must be set}"
: "${FRANK_PRIMARY_EXPERT_VS_CACHE:?FRANK_PRIMARY_EXPERT_VS_CACHE must be set}"
: "${FRANK_PRIMARY_KV_DTYPE:?FRANK_PRIMARY_KV_DTYPE must be set}"
: "${FRANK_PRIMARY_SPEC_TOKENS:?FRANK_PRIMARY_SPEC_TOKENS must be set}"
: "${FRANK_PRIMARY_MAX_BATCHED_TOKENS:?FRANK_PRIMARY_MAX_BATCHED_TOKENS must be set}"
: "${FRANK_PRIMARY_PREFIX_HOST_MIB:?FRANK_PRIMARY_PREFIX_HOST_MIB must be set}"
: "${FRANK_PRIMARY_PREFIX_DISK_MIB:?FRANK_PRIMARY_PREFIX_DISK_MIB must be set}"
: "${FRANK_PRIMARY_NGRAM_PLACEMENT:?FRANK_PRIMARY_NGRAM_PLACEMENT must be set}"
: "${FRANK_PRIMARY_TTM_MIN_MIB:?FRANK_PRIMARY_TTM_MIN_MIB must be set}"
: "${FRANK_PRIMARY_RAD_MIN_MIB:?FRANK_PRIMARY_RAD_MIN_MIB must be set}"
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
# in storage only if THAT manifest was pulled — a retagged look-alike or a
# newer latest does not satisfy it. `podman image exists` takes a
# name@digest ref in one command — no pipefail/grep race (the 2026-09-30
# lesson). Runs as dank: the unit's process uses dank's ROOTLESS podman
# storage (root unit, User=dank — the fleet's frank-primary shape).
if ! podman image exists "$FRANK_PRIMARY_IMAGE"; then
echo "ERROR: radiance image absent: $FRANK_PRIMARY_IMAGE" >&2
echo "frank-primary: podman pull docker://stilldeadcode/radiance:latest (as dank)" >&2
exit 1
fi
# The payload gate: the .rad container file, sized. A truncated convert
# would otherwise buy a guaranteed long-timeout boot instead of this fast
# failure. The staging dir / raw checkpoint are NOT this script's concern
# (scripts/build-frank-primary-rad.sh built the file; its rad-info output
# is the record).
RAD="$FRANK_PRIMARY_MODEL_DIR/$FRANK_PRIMARY_MODEL_FILE"
if [ ! -f "$RAD" ]; then
echo "ERROR: .rad container missing: $RAD" >&2
echo "frank-primary: run scripts/build-frank-primary-rad.sh" >&2
exit 1
fi
RAD_MIB=$(( $(stat -c %s "$RAD") / 1048576 ))
if [ "$RAD_MIB" -lt "$FRANK_PRIMARY_RAD_MIN_MIB" ]; then
echo "ERROR: .rad container too small: $RAD is ${RAD_MIB} MiB, want >= $FRANK_PRIMARY_RAD_MIN_MIB" >&2
echo "frank-primary: the convert was truncated — rebuild it" >&2
exit 1
fi
if [ ! -f "$FRANK_PRIMARY_TEMPLATE" ]; then
echo "ERROR: chat template missing: $FRANK_PRIMARY_TEMPLATE" >&2
exit 1
fi
mkdir -p "$FRANK_PRIMARY_STATE_DIR/kvcache/frank-primary"
# The engine's create is NOT recursive: with only the state dir mounted, its
# "cannot create /data/kvcache/frank-primary: No such file or directory"
# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
# the full path on the host so the tier opens on first boot.
mkdir -p "$FRANK_PRIMARY_STATE_DIR/tune"
# rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
# the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
# $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
# the cache must ride a host mount or it dies with the --rm container
# that wrote it. The dir is pre-created for the same reason as kvcache
# (the engine's create is not recursive).
if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
exit 1
fi
# TTM preflight: the pinned ledger (~2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
# prefix host = ~132 GiB) rides TTM's shared pages, capped by ttm's
# pages_limit (the current kernel exposes NO pages_max; the cap function
# takes the min over the knobs that exist). Below FRANK_PRIMARY_TTM_MIN_MIB
# the load dies mid-pin with an allocator error instead of this named
# refusal — raise the knob first (runtime sysfs write; kernel cmdline
# fallback + one reboot only if the file is read-only), see the runbook.
# RAISED 2026-10-05 pre-cutover to 52,428,800 pages = 200 GiB.
ttm_cap_mib() {
# min over the knobs THIS kernel exposes — the current kernel has
# pages_limit only (no pages_max); a knob that is absent is skipped,
# not a zero (a zero would refuse a healthy cap).
local v cur=""
for p in /sys/module/ttm/parameters/pages_limit /sys/module/ttm/parameters/pages_max; do
[ -r "$p" ] || continue
v="$(cat "$p" 2>/dev/null)" || continue
[ -n "$v" ] || continue
v=$(( v * 4 / 1024 ))
if [ "$v" -lt "${cur:-999999999}" ]; then cur=$v; fi
done
echo "${cur:-0}"
}
TTM_CAP="$(ttm_cap_mib)"
if [ "$TTM_CAP" -lt "$FRANK_PRIMARY_TTM_MIN_MIB" ]; then
echo "ERROR: ttm pages cap is ${TTM_CAP} MiB, the pinned ledger needs >= $FRANK_PRIMARY_TTM_MIN_MIB" >&2
echo "frank-primary: raise /sys/module/ttm/parameters/pages_limit (and pages_max if the kernel exposes one) to >= $FRANK_PRIMARY_TTM_MIN_MIB in 4 KiB pages" >&2
exit 1
fi
# A crashed/killed unit can leave the container name registered (systemd
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
# identities (the FreeToken-era frank-primary-qwen, the retired R9V
# container and the ad hoc experiment containers all hold this GPU); all
# idempotent.
for CTR in $FRANK_PRIMARY_LEGACY_CONTAINERS "$FRANK_PRIMARY_CONTAINER"; do
podman rm -f "$CTR" >/dev/null 2>&1 || true
done
LOG=/home/dank/logs/frank_primary.log
mkdir -p "$(dirname "$LOG")"
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
# --network host (the server binds the host's addresses directly, no NAT),
# --init (the engine releases the cards on SIGTERM), seccomp=unconfined
# (Docker/podman's default profile refuses io_uring, which the n-gram
# table's reads go through, and the NUMA binding calls the ROCm runtime
# makes for host memory), memlock unbounded (the pinned ledger — legal
# because the root unit grants LimitMEMLOCK=infinity to this dank process),
# /dev/kfd + /dev/dri (render/video groups are moot for a root container).
# The template rides a single-file mount; the .rad dir is ro; /data holds
# the prefix-cache disk tier.
podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
--network host \
--init \
--security-opt seccomp=unconfined \
--ulimit memlock=-1:-1 \
--device /dev/kfd --device /dev/dri \
-v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \
-v "$FRANK_PRIMARY_STATE_DIR":/data \
-v "$FRANK_PRIMARY_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
-v "$FRANK_PRIMARY_TEMPLATE":/templates/template.jinja:ro \
"$FRANK_PRIMARY_IMAGE" \
--model "/models/$FRANK_PRIMARY_MODEL_FILE" \
--host :: --port "$FRANK_PRIMARY_PORT" \
--served-model-name "$FRANK_PRIMARY_SERVED_NAME" \
--override-chat-template /templates/template.jinja \
--tp "$FRANK_PRIMARY_TP" --tp-wire "$FRANK_PRIMARY_TP_WIRE" \
--max-num-seqs "$FRANK_PRIMARY_MAX_SEQS" \
--max-model-len "$FRANK_PRIMARY_MAX_MODEL_LEN" \
--placement "$FRANK_PRIMARY_PLACEMENT" \
--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \
--gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \
--expert-vs-cache-ratio "$FRANK_PRIMARY_EXPERT_VS_CACHE" \
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \
--prefix-cache-dir /data/kvcache/frank-primary \
--prefix-cache-disk-mib "$FRANK_PRIMARY_PREFIX_DISK_MIB" \
--ngram-placement "$FRANK_PRIMARY_NGRAM_PLACEMENT" \
--num-speculative-tokens "$FRANK_PRIMARY_SPEC_TOKENS" \
--max-num-batched-tokens "$FRANK_PRIMARY_MAX_BATCHED_TOKENS" \
${FRANK_PRIMARY_EXTRA_ARGS:+$FRANK_PRIMARY_EXTRA_ARGS} \
> "$LOG" 2>&1
# From here on, this script's lifetime IS the container's: wait returns the
# container's exit code, so a crash is a nonzero ExecStart and the unit's
# Restart=always is real. (systemd stop goes through ExecStop in the unit —
# podman wait alone cannot stop the container child.)
exec podman wait "$FRANK_PRIMARY_CONTAINER"
-120
View File
@@ -1,120 +0,0 @@
#!/usr/bin/env bash
# Serve ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
# (RCO-IQ3_XXS, **MTP head in the file**) as frank-secondary on Dankclaw's
# RTX 3090 via llama.cpp (master build, CUDA sm_86).
#
# 2026-10-06 (Frank): TWO 131,072 conversations (262144 / 2) at the 128k
# cap, ubatch 512, and MTP BACK ON (draft-mtp, depth 3 — the payload is
# 1.56 GiB lighter than the IQ3_S-mtp it replaced, which is what funds
# the draft ctx plus the second slot; ngram stays skipped — on a qwen35
# hybrid it disables recurrent-state rollback). The q8_0 KV contract
# (FP8 class) is unchanged; the per-tensor flags stay explicit either
# way.
#
# Architecture: qwen35 hybrid — 64 real layers, 16 full-attention + 48
# Gated DeltaNet (full_attention_interval 4), native 262144 window, plus
# the blk.64 MTP head (nextn_predict_layers=1). KV lives only in the 16
# attention layers (34 KiB/token at q8_0); the DeltaNet state is per-slot
# fixed size.
# Vision comes from the projector file (the GGUF carries no vision
# tensors; the publisher SHIPS mmproj-BF16.gguf in the same repo), so
# --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is the
# documented text-only opt-out.
#
# Sampling: llama.cpp takes it per request (the fleet harness sends its
# recipe from models.yml); greedy is the harness default.
#
# Tunables come from frank-secondary.env unless FRANK_SECONDARY_ENV_FILE overrides.
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_SECONDARY_ENV_FILE:-$SCRIPT_DIR/frank-secondary.env}"
if [[ -f "$ENV_FILE" ]]; then
set -a
# shellcheck disable=SC1090
source "$ENV_FILE"
set +a
fi
# --- defaults (used when the env file does not set them) -------------------
: "${LLAMA_BIN:=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
: "${MODEL_FILE:=/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf}"
: "${ALIAS:=frank-secondary}"
: "${BIND_ADDR:=0.0.0.0}"
: "${LISTEN_PORT:=7771}"
# ctx-size is the POOL llama.cpp divides across slots: 262144 / 2 gives
# TWO 131,072 conversations — Frank's 128k cap each. 2026-10-06 (Frank):
# drop MTP, drop ubatch to 512, take two slots; the fit is tight by his
# explicit order (frank-secondary.env carries the arithmetic and the floor
# violation).
: "${CTX_SIZE:=262144}"
: "${PARALLEL:=2}"
# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
# Frank's 2026-10-06 tight-fit order ships 4096/512 (claws back ~0.3 GiB
# toward the second slot); ubatch 1024 is the first A/B back if a future
# boot lands with headroom (a miss fails loud at graph capture). The
# logical batch is VRAM- and throughput-neutral at a fixed ubatch.
: "${BATCH_SIZE:=4096}"
: "${UBATCH_SIZE:=512}"
# KV at FP8 class = q8_0 on llama.cpp (1.0625 B/elem => 8.5 effective bits;
# there is no true fp8 KV dtype). Explicit either way so the contract is
# visible in the argv.
: "${CACHE_TYPE_K:=q8_0}"
: "${CACHE_TYPE_V:=q8_0}"
# Speculative decoding is ON by default (Frank, 2026-10-06 evening):
# draft-mtp against the head that is IN the file. SPEC_TYPE none strips
# BOTH speculative flags (flags move as a set) — a no-spec baseline must
# not carry half a spec pair.
: "${SPEC_TYPE:=draft-mtp}"
: "${SPEC_DRAFT_N_MAX:=3}"
: "${NGL:=99}"
# Vision projector. No colon on purpose: an explicitly empty value means
# "text-only, on purpose" and must NOT be replaced by the default path.
: "${MMPROJ_FILE=/home/dank/models/Swift15-UncIQ3XXS/mmproj-BF16.gguf}"
# Fail loudly rather than half-loading: a missing file is a download that
# has not finished, and Restart=always would turn it into a crash loop that
# reads like a runtime fault.
for path in "$LLAMA_BIN" "$MODEL_FILE"; do
[[ -e "$path" ]] || { echo "frank-secondary: missing $path" >&2; exit 1; }
done
EXTRA_ARGS=()
if [[ -n "$MMPROJ_FILE" ]]; then
[[ -e "$MMPROJ_FILE" ]] || { echo "frank-secondary: missing $MMPROJ_FILE" >&2; exit 1; }
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
fi
# Speculative flags move as a set: they reach the argv ONLY when a drafter
# is named. SPEC_TYPE=none (the shipped contract) strips both — a no-spec
# baseline must not carry half a spec pair.
if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
fi
# The UVM device node is created lazily by nvidia-modprobe on the first CUDA
# open after boot; make sure it exists before the server needs it.
nvidia-modprobe -u -c=0 2>/dev/null || true
# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
# the q8_0 KV contract in frank-secondary.env is a real path — without this
# flag or that build option the engine silently falls back and the
# contract becomes a lie.
ARGS=(
--model "$MODEL_FILE"
--alias "$ALIAS"
--host "$BIND_ADDR"
--port "$LISTEN_PORT"
--ctx-size "$CTX_SIZE"
--parallel "$PARALLEL"
--batch-size "$BATCH_SIZE"
--ubatch-size "$UBATCH_SIZE"
--n-gpu-layers "$NGL"
--flash-attn on
--cache-type-k "$CACHE_TYPE_K"
--cache-type-v "$CACHE_TYPE_V"
--jinja
--reasoning auto
--reasoning-format deepseek
)
exec "$LLAMA_BIN" "${ARGS[@]}" ${EXTRA_ARGS[@]+"${EXTRA_ARGS[@]}"}
+132
View File
@@ -0,0 +1,132 @@
#!/usr/bin/env bash
# Serve HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K) as
# frank-zealot on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
#
# 2026-10-07 (Frank): Qwen3.5-9B swap at TWO 200,704-token conversations
# (--ctx-size 401408 --parallel 2) with EXACT f16 KV — the 9B rung frees the
# room the 26B MoE Gemma4 shape lacked, so quantized V retires. MTP stays off:
# this GGUF carries no MTP tensors at all (verified from its tensor-name
# list), so there is no in-file head to draft with. Verbosity 4 ships so the
# VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in the journal.
#
# Architecture: qwen35 — dense 9B, 32 layers with full_attention_interval 4
# (8 full-attention layers at 3,7,...,31 + 24 Gated DeltaNet linear layers),
# 16 heads / 4 KV heads × 256 dim, native 262,144, vocab 248,320. The KV pool
# scales with the 8 full layers: 32 KiB/token at f16 K + f16 V (12.25 GiB at
# the 401,408 pool); the DeltaNet layers cost a fixed recurrent state per
# slot. Feasibility is priced from the MiMo-V2.6-Distill-Qwen-9B boots (same
# geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279
# at ubatch 2048 — the boot ledger, not arithmetic, is the referee.
# Vision comes from the publisher's own BF16 projector (the model is natively
# multimodal), so --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is
# the documented text-only opt-out.
#
# Sampling: llama.cpp takes it per request (the fleet harness sends the
# publisher recipe from models.yml: thinking temp 0.6 / top_p 0.95 / top_k 20).
#
# Tunables come from frank-zealot.env unless FRANK_ZEALOT_ENV_FILE overrides.
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_ZEALOT_ENV_FILE:-$SCRIPT_DIR/frank-zealot.env}"
if [[ -f "$ENV_FILE" ]]; then
set -a
# shellcheck disable=SC1090
source "$ENV_FILE"
set +a
fi
# --- defaults (used when the env file does not set them) -------------------
: "${LLAMA_BIN:=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
: "${MODEL_FILE:=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
: "${ALIAS:=frank-zealot}"
: "${BIND_ADDR:=0.0.0.0}"
: "${LISTEN_PORT:=7771}"
# ctx-size is the POOL llama.cpp divides across slots: 401,408 / 2 = 200,704
# per conversation, inside the model's trained 262,144. The harness
# contextWindow MUST quote the PER-SLOT number — a longer prompt is answered
# with a 400, never truncated.
: "${CTX_SIZE:=401408}"
: "${PARALLEL:=2}"
# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
# 2048 is the measured CUDA optimum (MiMo era, same arch/shape: +4.6 % prefill
# at 8.9k depth, decode flat, 616 MiB). The logical batch is VRAM- and
# throughput-neutral at a fixed ubatch.
: "${BATCH_SIZE:=4096}"
: "${UBATCH_SIZE:=2048}"
# Exact (unquantized) KV: f16 K + f16 V, restoring the MiMo-era contract that
# the 26B Gemma4 shape could not afford. 32 KiB/token at this geometry
# (8 full-attention layers × 4 KV heads × 256 dim); 12.25 GiB at the 401,408
# pool. Explicit either way so the contract is visible in the argv, and
# --flash-attn stays required for any quantized-V fallback (q8_0 = llama.cpp's
# FP8 class; no true fp8 KV dtype exists in this build).
: "${CACHE_TYPE_K:=f16}"
: "${CACHE_TYPE_V:=f16}"
# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors, so
# there is no in-file head (the law: MTP is a property of the file, not of the
# flags). SPEC_TYPE=none strips BOTH speculative flags (flags move as a set);
# naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4
# assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35.
: "${SPEC_TYPE:=none}"
: "${SPEC_DRAFT_N_MAX:=3}"
: "${SPEC_DRAFT_FILE:=}"
: "${NGL:=99}"
# Log verbosity threshold (llama.cpp -lv). 3 is the engine default (info);
# 4 adds debug detail (KV layout, buffer sizes, graph capture) which is what
# the VRAM-fit forensics need. Levels above 4 flood the journal.
: "${VERBOSITY:=3}"
# Vision projector. No colon on purpose: an explicitly empty value means
# "text-only, on purpose" and must NOT be replaced by the default path.
: "${MMPROJ_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
# Fail loudly rather than half-loading: a missing file is a download that
# has not finished, and Restart=always would turn it into a crash loop that
# reads like a runtime fault.
for path in "$LLAMA_BIN" "$MODEL_FILE"; do
[[ -e "$path" ]] || { echo "frank-zealot: missing $path" >&2; exit 1; }
done
EXTRA_ARGS=()
if [[ -n "$MMPROJ_FILE" ]]; then
[[ -e "$MMPROJ_FILE" ]] || { echo "frank-zealot: missing $MMPROJ_FILE" >&2; exit 1; }
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
fi
# Speculative flags move as a set: they reach the argv ONLY when a drafter
# is named. SPEC_TYPE=none (the shipped contract) strips both — a no-spec
# baseline must not carry half a spec pair.
if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
if [[ -n "$SPEC_DRAFT_FILE" ]]; then
[[ -e "$SPEC_DRAFT_FILE" ]] || { echo "frank-zealot: missing $SPEC_DRAFT_FILE" >&2; exit 1; }
EXTRA_ARGS+=(--spec-draft-model "$SPEC_DRAFT_FILE")
fi
fi
# The UVM device node is created lazily by nvidia-modprobe on the first CUDA
# open after boot; make sure it exists before the server needs it.
nvidia-modprobe -u -c=0 2>/dev/null || true
# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
# the q8_0 KV contract in frank-zealot.env is a real path — without this
# flag or that build option the engine silently falls back and the
# contract becomes a lie.
ARGS=(
--model "$MODEL_FILE"
--alias "$ALIAS"
--host "$BIND_ADDR"
--port "$LISTEN_PORT"
--ctx-size "$CTX_SIZE"
--parallel "$PARALLEL"
--batch-size "$BATCH_SIZE"
--ubatch-size "$UBATCH_SIZE"
--n-gpu-layers "$NGL"
--flash-attn on
--cache-type-k "$CACHE_TYPE_K"
--cache-type-v "$CACHE_TYPE_V"
--log-verbosity "$VERBOSITY"
--jinja
--reasoning auto
--reasoning-format deepseek
)
exec "$LLAMA_BIN" "${ARGS[@]}" ${EXTRA_ARGS[@]+"${EXTRA_ARGS[@]}"}
+11 -11
View File
@@ -145,12 +145,12 @@ for path in "${CHANGED[@]}"; do
add_test tests/homepage_caddy.bats
add_test tests/deployment_scoping.bats
;;
# frank-large is a rootless quadlet again since 2026-09-16 (the
# frank-dragoon is a rootless quadlet again since 2026-09-16 (the
# halogen-flash-server restore). Its unit carries the fleet's GPU and
# podman posture, so it gets its own seam rather than the generic
# fallback below.
dankclaw/frank-large/*)
add_test tests/frank_large_unit.bats
dankclaw/frank-dragoon/*)
add_test tests/frank_dragoon_unit.bats
add_test tests/deployment_scoping.bats
add_test tests/docs_completeness.bats
;;
@@ -187,29 +187,29 @@ for path in "${CHANGED[@]}"; do
# as additions, with "no focused-test mapping".
.AGENTTRASH/*|*/.AGENTTRASH/*)
;;
# frank-primary is llama.cpp (the unsloth b11030-mix prebuilt) on the
# frank-archon is llama.cpp (the unsloth b11030-mix prebuilt) on the
# two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
# Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
# live in scripts/, which is why this case has to precede the generic
# scripts/*.service and scripts/* fallbacks below.
scripts/frank-primary.*|scripts/serve-frank-primary.sh)
add_test tests/frank_primary_unit.bats
scripts/frank-archon.*|scripts/serve-frank-archon.sh)
add_test tests/frank_archon_unit.bats
add_test tests/llm_units.bats
add_test tests/deployment_scoping.bats
add_test tests/scripts_syntax.bats
;;
# frank-large's chat template lives in scripts/ (moved there on
# frank-dragoon's chat template lives in scripts/ (moved there on
# 2026-09-26 when its old home was retired): a template edit changes
# what the live endpoint renders, so it selects the large unit seam.
scripts/qwen-sharp-v22.4.0.jinja)
add_test tests/frank_large_unit.bats
add_test tests/frank_dragoon_unit.bats
;;
# frank-secondary is llama.cpp on the 3090 as a ROOT unit: a native CUDA
# frank-zealot is llama.cpp on the 3090 as a ROOT unit: a native CUDA
# binary, so no generated quadlet is involved. Its env, unit and serve
# script all live in scripts/, which is why this case has to precede
# the generic scripts/*.service and scripts/* fallbacks below.
scripts/frank-secondary.*|scripts/serve-frank-secondary.sh)
add_test tests/frank_secondary_unit.bats
scripts/frank-zealot.*|scripts/serve-frank-zealot.sh)
add_test tests/frank_zealot_unit.bats
add_test tests/llm_units.bats
add_test tests/deployment_scoping.bats
add_test tests/scripts_syntax.bats
+104
View File
@@ -0,0 +1,104 @@
#!/usr/bin/env bash
# ttm-preflight.sh — the TTM pages-cap gate for engines that PIN host RAM.
#
# Sourced, never executed. An engine that pins tens of GiB of host RAM (the
# radiance expert pools + the PLE n-gram table) has its pinned pages charged
# against the kernel's ttm page cap: /sys/module/ttm/parameters/pages_limit
# (and pages_max on kernels that expose one). The cap function below takes the
# MIN over the knobs the running kernel actually exposes — an absent knob is
# skipped, never read as zero, because a zero would refuse a healthy cap.
#
# THE LAW THIS FILE EXISTS FOR: the cap is a RUNTIME knob. The kernel default is
# half of RAM, so every raise is lost on reboot. On 2026-10-12 23:17 a power
# failure on dankserver reset it 200 GiB -> 125,988 MiB, and frank-archon's
# preflight refused the start — 6,919 crash-restarts over ~20 hours, with
# `Restart=always` cheerfully re-running a refusal. A manual sysfs write is
# therefore not an acceptable repair procedure: the gate raises the knob itself
# (the units that need it run as root, and the knob is a 0644 root-owned module
# parameter), re-reads, and only then refuses.
#
# Contract (set by the sourcing script's env file):
# FRANK_ARCHON_TTM_MIN_MIB floor the pinned ledger needs; refuse below it
# FRANK_ARCHON_TTM_TARGET_MIB value to raise the cap to when it is below the
# floor (the recorded host ceiling — writing the
# floor itself would leave the ledger no slack)
# FRANK_ARCHON_TTM_SYS_DIR knob directory; defaults to the real sysfs
# path, a test points it at a temp dir.
# Pages are 4 KiB: pages = MiB x 256.
: "${FRANK_ARCHON_TTM_SYS_DIR:=/sys/module/ttm/parameters}"
# Prints the effective cap in MiB, 0 when no knob is readable.
ttm_cap_mib() {
local p v cur=""
for p in "$FRANK_ARCHON_TTM_SYS_DIR/pages_limit" \
"$FRANK_ARCHON_TTM_SYS_DIR/pages_max"; do
[ -r "$p" ] || continue
v="$(cat "$p" 2>/dev/null)" || continue
[ -n "$v" ] || continue
v=$(( v * 4 / 1024 ))
if [ "$v" -lt "${cur:-999999999}" ]; then cur=$v; fi
done
echo "${cur:-0}"
}
# Writes $1 MiB (as 4 KiB pages) into every knob present. Failures are
# swallowed: the caller re-reads the cap, and a write the kernel refused shows
# up there (ttm refuses a limit below the pages it has already pinned).
#
# PRIVILEGE, measured the hard way: the knob is root-owned 0644 and the service
# process runs as User=dank, so `-w` is FALSE there and a plain echo fails. The
# canonical privileged path is the unit's ExecStartPre with the `+` prefix
# (systemd runs it as root even under User=); `sudo -n` is the fallback for a
# human running the gate from a shell (passwordless sudo on both hosts).
ttm_raise_cap() {
local pages=$(( $1 * 256 )) p
for p in "$FRANK_ARCHON_TTM_SYS_DIR/pages_limit" \
"$FRANK_ARCHON_TTM_SYS_DIR/pages_max"; do
[ -e "$p" ] || continue
if [ -w "$p" ]; then
echo "$pages" > "$p" 2>/dev/null || true
elif command -v sudo >/dev/null 2>&1; then
sudo -n sh -c "echo $pages > '$p'" >/dev/null 2>&1 || true
fi
done
}
# Returns 0 when the cap clears FRANK_ARCHON_TTM_MIN_MIB, raising it first if
# needed; returns 1 with a named refusal when it cannot be made to.
ttm_ensure_cap() {
local cap
cap="$(ttm_cap_mib)"
if [ "$cap" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
echo "frank-archon: ttm pages cap is ${cap} MiB, below the pinned-ledger" \
"floor $FRANK_ARCHON_TTM_MIN_MIB — raising the knob to" \
"$FRANK_ARCHON_TTM_TARGET_MIB MiB ($(( FRANK_ARCHON_TTM_TARGET_MIB * 256 )) pages)" >&2
ttm_raise_cap "$FRANK_ARCHON_TTM_TARGET_MIB"
cap="$(ttm_cap_mib)"
fi
if [ "$cap" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
echo "ERROR: ttm pages cap is ${cap} MiB, the pinned ledger needs >= $FRANK_ARCHON_TTM_MIN_MIB" >&2
echo "frank-archon: raise $FRANK_ARCHON_TTM_SYS_DIR/pages_limit (and pages_max if the kernel exposes one) to >= $(( FRANK_ARCHON_TTM_MIN_MIB * 256 )) in 4 KiB pages; a read-only knob needs the module option (options ttm pages_limit=N) and a reboot" >&2
return 1
fi
echo "frank-archon: ttm pages cap ${cap} MiB clears the pinned-ledger floor $FRANK_ARCHON_TTM_MIN_MIB" >&2
}
# Executed (not sourced) it is the privileged pre-start hook: systemd runs this
# as root from the unit's `ExecStartPre=+…`, where the knob write is legal and
# the `Restart=always` loop can never outrun it.
if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
set -euo pipefail
case "${1:-}" in
--raise)
TTM_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=/dev/null
source "${FRANK_ARCHON_ENV_FILE:-$TTM_DIR/frank-archon.env}"
ttm_ensure_cap
;;
*)
echo "usage: ${0##*/} --raise (raise the ttm pages cap, then verify)" >&2
exit 2
;;
esac
fi
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env python3
"""Streaming absmax audit of the Swift1.5 PLE n-gram table (BF16).
The PLE fp8 encoding uses ONE fixed scale (1.9931793212890625e-04, the
official checkpoint's plateau value). fp8_e4m3 saturates at 448, so the
fixed scale is legal iff absmax <= 448 * SCALE. This walks every
ngram_embedding.shard_*.weight tensor in the checkpoint (mmap, no copy),
computes the absmax from the BF16 bit pattern (clear the sign bit, max
the remaining uint16 — BF16 magnitudes are monotone in their bits), and
reports the headroom. Exit 0 only when headroom >= 1.5x.
Run from any cwd EXCEPT /tmp on dankserver (/tmp/inspect.py shadows the
stdlib). Usage: verify-swift15-ple.py <ckpt-dir>
"""
import json
import mmap
import os
import struct
import sys
SCALE = 1.9931793212890625e-04 # bf16 0x3951, the official plateau scale
FP8_E4M3_MAX = 448.0
import numpy as np
gmax = 0.0
gfile = ""
over_cap = 0
total = 0
ckpt = sys.argv[1]
index = json.load(open(os.path.join(ckpt, "model.safetensors.index.json")))
shards: dict[str, list[str]] = {}
for k, f in index["weight_map"].items():
if "ngram_embedding.shard_" in k and k.endswith(".weight"):
shards.setdefault(f, []).append(k)
files = len(shards)
tensors = sum(len(v) for v in shards.values())
print(f"n-gram shards: {files} files, {tensors} tensors")
import math
cap = FP8_E4M3_MAX * SCALE
# largest bf16 <= cap: truncate the float32 bits (cap > 0); bf16
# magnitudes are monotone in their bits, so |w| > cap <=> bits > cap_bits
cap_bits = int((np.float32(cap).view(np.uint32) >> 16) & 0xFFFF)
for fname, keys in sorted(shards.items()):
path = os.path.join(ckpt, fname)
with open(path, "rb") as fh:
hlen = struct.unpack("<Q", fh.read(8))[0]
header = json.loads(fh.read(hlen))
base = 8 + hlen
mm = mmap.mmap(fh.fileno(), 0, prot=mmap.PROT_READ)
for k in keys:
meta = header[k]
off, end = meta["data_offsets"]
a = np.frombuffer(mm, dtype=np.uint16, count=(end - off) // 2,
offset=base + off)
v = a & 0x7FFF
m = int(v.max())
over_cap += int((v > np.uint16(cap_bits)).sum())
total += v.size
del a
if m > gmax:
gmax = m
gfile = k
mm.close()
# rebuild the BF16 value: the 16 bits are the HIGH half of the float32
absmax = np.frombuffer(b"\x00\x00" + np.uint16(gmax).tobytes(),
dtype=np.float32)[0]
print(f"absmax = {absmax!r} (tensor: {gfile})")
print(f"2^log2 = 2^{math.log2(absmax):.3f}")
print(f"fp8 cap = 448 * {SCALE:.8g} = {cap:.8g}")
print(f"headroom = {cap / absmax:.3f}x")
ok = cap / absmax >= 1.5
print("VERDICT:", "PASS (>= 1.5x headroom over the fixed scale)" if ok
else "FAIL — fixed scale does not cover the Swift PLE")
print(f"values > cap = {over_cap} of {total} ({over_cap / total:.3e})")
nxt_bits = int((np.float32(absmax / FP8_E4M3_MAX).view(np.uint32)
>> 16) & 0xFFFF) + 1
nxt_scale = np.frombuffer(b"\x00\x00" + np.uint16(nxt_bits).tobytes(),
dtype=np.float32)[0]
print(f"next-bf16 scale covering absmax: {nxt_scale:.8g} "
f"(0x{nxt_bits:04x}, headroom {FP8_E4M3_MAX * nxt_scale / absmax:.4f}x)")
sys.exit(0 if ok else 1)
+19 -7
View File
@@ -13,6 +13,18 @@ registry; single source of truth — there is no copy on the servers) and
`~/.omp/agent/config.yml` (roles, compaction, global samplers). Nothing in
this file is repo-managed; edits are harness-side only.
**Model ids are the StarCraft unit names (official 2026-10-08, on the wire
since 2026-10-09):** `frank-archon` (dankserver :7770, radiance),
`frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw
:7771, the 3090 compaction hub) — formerly `frank-archon`, `frank-dragoon`,
`frank-zealot`. The SERVICES keep the old names (units, scripts, env
files, runbook pages); pages written in the old era name the service,
the harness names the model. The wire now matches: each endpoint's
`/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI
picker/grants, the homepage docs and the raw API all use one id per model
(see [[OpenWebUI]] for the grant re-key and the [[frank-archon-radiance-swift15-w4dn8]]
note on radiance echoing the requested name).
## Compaction chain arithmetic (verify before ANY window change)
When a fleet endpoint's context grows past its harness-declared window, OMP
@@ -33,14 +45,14 @@ The arithmetic (the coupled contract every endpoint's maxTokens comment
encodes):
1. **Source trigger** = `thresholdPercent × source contextWindow`
(0.70 × 200,704 = 140,492 for frank-primary on the current shape)
(0.70 × 200,704 = 140,492 for frank-archon on the current shape)
2. **Output budget** = capped at the target model's `maxTokens`
(32,768 for frank-secondary; 36,864 frank-primary; 57,344 frank-large — the
(32,768 for frank-zealot; 36,864 frank-archon; 57,344 frank-dragoon — the
largest, because halogen's `HALOGEN_MAX_TOKENS_CAP` must stay ≥ it)
3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the target's per-slot
window. Worked pairs from the live registry: 140,492 + 36,864 = 177,356 ≤
200,704 (frank-primary); 140,490 + 57,344 = 197,834 ≤ 200,700
(frank-large); 140,492 + 32,768 = 173,260 ≤ 200,704 (frank-secondary).
200,704 (frank-archon); 140,490 + 57,344 = 197,834 ≤ 200,700
(frank-dragoon); 140,492 + 32,768 = 173,260 ≤ 200,704 (frank-zealot).
Procedure (recompute from the LIVE files every time — windows rotate with
every swap; the numbers above are the 2026-10-05 state):
@@ -100,19 +112,19 @@ Use **wire names**, not OMP camelCase. Two traps:
`qwenPreserveThinking` is auto-derived in `buildOpenAICompat` from the
*auto-detected* `thinkingFormat` + local baseUrl, **before**
`applyCompatOverrides`. Setting `compat.thinkingFormat: qwen` in
models.yml does **not** enable it. Custom-named models (`frank-large`)
models.yml does **not** enable it. Custom-named models (`frank-dragoon`)
match no Qwen id pattern → use `extraBody: {preserve_thinking: true}`.
2. **Never put `chat_template_kwargs` in `extraBody`** where a compat policy
also sets it — `Object.assign` is shallow and replaces the whole object,
silently dropping the `reasoning_effort` kwarg the compat policy set.
Top-level `preserve_thinking` is honoured by halogen. (frank-primary's
Top-level `preserve_thinking` is honoured by halogen. (frank-archon's
FreeToken entry currently carries a `chat_template_kwargs` extraBody with
a comment asking to verify-and-prune — the exact trap to check at that
endpoint's next swap.)
**Gotcha: silent greedy decode.** halogen has **no server-side sampler
defaults** — `temperature` absent or `0` = greedy. If OMP sends nothing, the
endpoint decodes greedy while looking configured. This is why frank-large's
endpoint decodes greedy while looking configured. This is why frank-dragoon's
entry carries the recipe on the wire and a comment saying "do not drop it".
**Reasoning-effort folding.** Qwen3.8's template accepts only
@@ -23,7 +23,7 @@ marker. So `marker != HEAD` means "not dispatched via deploy.sh", NOT
**Why it matters:** a plain `deploy.sh` run re-selects and restarts a service
whose marker lags. On llama.cpp/R9V endpoints that is a ~16–19 min reload of
a 100+ GiB payload — and frank-primary (dankserver:7770) is also the
a 100+ GiB payload — and frank-archon (dankserver:7770) is also the
harness's own model backend, so a blind deploy is a self-inflicted outage for
possibly zero behavioral change.
@@ -62,8 +62,8 @@ service on the other host.
|---|---|---|
| Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
| `frank-large` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
| `frank-secondary` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
| `frank-dragoon` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
| `frank-zealot` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
| `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. |
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
@@ -95,7 +95,7 @@ host-side `workflows/` dir and the repo copy.
## Recommended additions
Quantized preferred, sized for the 8060S iGPU (GTT-backed; the RTX 3090 is
frank-secondary's). All repos verified 2026-10-05. None are on disk yet.
frank-zealot's). All repos verified 2026-10-05. None are on disk yet.
| # | What | Repo | Grab | Size | Why |
|---|---|---|---|---|---|
@@ -13,7 +13,7 @@ timestamp: 2026-08-26
# Dankclaw Stack
Dankclaw (192.168.0.143) runs the AI, search, development, and observability
stack. Its LLM inference endpoint `frank-large` (`:7770`) is a ROCmFPX
stack. Its LLM inference endpoint `frank-dragoon` (`:7770`) is a ROCmFPX
llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on
the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`,
FLM v1.0.1) serves sight+sound models. The completion endpoint
@@ -30,7 +30,7 @@ graph LR
S --> V[Gluetun VPN]
O --> L[llama.cpp dankserver :7770]
O --> H[Hermes]
O --> E[frank-large :7770]
O --> E[frank-dragoon :7770]
```
## Services
@@ -48,14 +48,14 @@ graph LR
| Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] |
| Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — |
| Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] |
| frank-large (host service) | — | `7770` | [[deployment]] |
| frank-dragoon (host service) | — | `7770` | [[deployment]] |
| frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] |
| frank-medium (host service, Dankserver) | — | `7770` | [[frank-medium-rocmfpx]] |
The LLM rows are bare-metal systemd services, not containers. This
repo owns their scripts, units, and knowledge base (see
[LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03):
`frank-large` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
`frank-dragoon` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
on the card's pinned `kingjones30/ROCmFPX` llama.cpp build;
`frank-npu` runs FLM v1.0.1 on the XDNA2 NPU;
`frank-medium` runs charlie12345/ROCmFPX llama.cpp serving
+3 -3
View File
@@ -47,7 +47,7 @@ is not durable. The tracked, sanitized configuration baseline is
`0.0.0.0:8080`, so the Signal HTTP API is reachable from the whole LAN.
- The data volume persists skills, memory, and session history.
- Hermes has two named local providers: `dankclaw-large` reaches
`http://192.168.0.143:7770/v1` and exposes `frank-large`;
`http://192.168.0.143:7770/v1` and exposes `frank-dragoon`;
`dankserver-small` reaches `http://192.168.0.235:7770/v1` and exposes
`frank-medium` (renamed from `frank-small` on 2026-09-15).
- The live `config.yaml` contains secrets and user state and is not committed;
@@ -64,7 +64,7 @@ is not durable. The tracked, sanitized configuration baseline is
## Gotchas
- Hermes is under active development; pin the image tag for reproducibility.
- Model choice drives quality; the local `frank-large` (Dankclaw, 7770) is the intended backend.
- Model choice drives quality; the local `frank-dragoon` (Dankclaw, 7770) is the intended backend.
- Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth.
## Browser automation (Chrome DevTools Protocol daemon)
@@ -189,7 +189,7 @@ What actually breaks, and the fix:
auxiliary:
title_generation:
provider: dankserver-small # names track the live config; diff first
model: frank-secondary
model: frank-zealot
api_mode: codex_responses # real per-task key
extra_body:
reasoning:
+95 -17
View File
@@ -24,29 +24,45 @@ https://github.com/open-webui/open-webui
| Upstream latest | 0.11.4 (2026-09-21) — security/access-control advisory; `:main` is the rolling image, not a stable pin. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
| Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
| Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
| Backends | Three fleet endpoints: `frank-large` (host.containers.internal:7770), `frank-medium` (192.168.0.235:7770), `frank-secondary` (host.containers.internal:7771) — see [The LLM backends](#the-llm-backends) |
| Backends | Picker exposes exactly one model — `frank-zealot` (host.containers.internal:7771). The `frank-dragoon` and `frank-archon` connections remain configured in the DB, disabled (2026-10-07) — see [The LLM backends](#the-llm-backends) |
| Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) |
| Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` |
| Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` |
## The LLM backends
OpenWebUI serves the three fleet LLM endpoints. The model picker is built from
the `model` table in `webui.db` (one row per model), NOT from live connection
discovery — so the picker shows exactly the models seeded there.
The picker is built from the `model` table in `webui.db` (one row per model)
plus the `model_ids` of each **enabled** OpenAI connection, NOT from live
connection discovery — so the picker shows exactly what is seeded there.
As of 2026-10-07 the picker exposes **exactly one model: `frank-zealot`**.
| Picker model | Connection URL | Runtime | Host |
|--------------|----------------|---------|------|
| `frank-large` | `http://host.containers.internal:7770/v1` | unsloth llama.cpp (Strix Halo iGPU, 200k ctx) | Dankclaw |
| `frank-medium` | `http://192.168.0.235:7770/v1` | vllm-radiance TP=2 (FP8, DFlash2, 262k ctx) | Dankserver |
| `frank-secondary` | `http://host.containers.internal:7771/v1` | llama.cpp (RTX 3090, Gemma4-12B, 262k ctx) | Dankclaw |
| Connection (`openai.api_configs` idx) | URL | Enabled | `model_ids` | Runtime | Host |
|---|---|---|---|---|---|
| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | halogen flash-server (Strix Halo iGPU) | Dankclaw |
| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | radiance TP=2 (2× R9700) | Dankserver |
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | llama.cpp CUDA (RTX 3090, 2×200,704) | Dankclaw |
The `model` table carries exactly one decoration row, `frank-zealot`
(context length 200,704 = the per-slot window; description: Qwen3.5-9B
Q6_K + BF16 vision mmproj).
A third gate sits *after* the table, and it is the one that bites family
accounts (2026-10-07): a model row is only visible to a non-admin user when
its **access permission** grants them. A seeded row defaults to admin-only,
so the picker renders "No results found" for every regular user even though
the connection is enabled, `model_ids` is correct, and the endpoint answers
`/v1/models` — and `GET /api/v1/models` returns `[]` for that user while an
admin token returns the model. Fix: set the row's `access_control` to public
(the `access` column on the `model` row) via the admin UI (Admin Panel →
Models → visibility) or the DB. The verification battery below must therefore
run **as a non-admin user**, not only as an admin.
Two gotchas make the wiring non-obvious:
- **llama.cpp `/v1/models` is Ollama-shaped.** llama.cpp returns
`{"models":[{"name":...}]}`; OpenWebUI's `extract_data()` only reads the
OpenAI `{"data":[{"id":...}]}` shape, so connection discovery silently misses
the llama.cpp endpoints (frank-large, frank-secondary). The fix is to set
the llama.cpp endpoints (frank-dragoon, frank-zealot). The fix is to set
`model_ids` in each connection's `api_config` — when non-empty, OpenWebUI
builds the model list directly and never fetches `/v1/models`. frank-medium
(vLLM) serves the OpenAI shape and would discover fine, but uses `model_ids`
@@ -62,7 +78,9 @@ The Ollama connection is disabled (`ollama.enable=false`).
> All three endpoints are bare-metal systemd services managed by this
> repository. Model swaps, context sizing, and spec tuning happen there. See
> the [LLM knowledge base](../LLM/index.md).
> the [LLM knowledge base](../LLM/index.md). Disabling a connection here only
> hides it from this picker — the endpoint itself keeps serving every other
> client (the harness, Hermes, API consumers).
## Container introspection (DB and backend python)
@@ -96,16 +114,33 @@ The Ollama connection is disabled (`ollama.enable=false`).
1. `GET /api/v1/models/list?limit=100` → returns EXACTLY the intended
models. Response key is `items`, not `data`.
2. `POST /openai/chat/completions` per model with a trivial prompt ("Reply
2. `POST /api/chat/completions` per model with a trivial prompt ("Reply
with exactly one word: pong") → each returns content `pong`. Reasoning
models need `max_tokens>=100` — the thinking block eats the budget.
models need a generous budget: frank-zealot (Qwen3.5 template) returns
empty content + `finish_reason: "length"` at `max_tokens 200` (the
thinking block eats it) and answers `pong` at 1500.
3. A 500 "Server Connection Error" → `podman logs open-webui` for
`ClientConnectorError`: a network-reachability problem (host LAN IP vs
`host.containers.internal`), not a model problem.
**Durability:** the model table + connections are DB-only (not git-managed);
a fresh data volume loses them. For reproducibility, add an idempotent
first-start seed script or document the setup in the service doc.
a fresh data volume loses them — including the public access grant above.
For reproducibility, add an idempotent first-start seed script or document
the setup in the service doc.
## API access (auth-gated)
Every `/api/*` endpoint requires an authenticated session (SSO login JWT)
or a per-user API key (`auth.enable_api_keys=true`; keys are minted per
account under Profile → API Keys). Anonymous web visitors get the `pending`
role (`ui.default_user_role=pending`), which cannot chat. Verified
2026-10-07 from loopback `:3005` and over the public vhost:
`GET /api/v1/models/list`, `GET /api/v1/models` and
`POST /api/chat/completions` all return **401** with no key (also with a
bogus bearer token); only `/api/config`, `/api/version` and `/health`
answer publicly, and none of them expose the model list. An external
program needs an account + API key — there is no keyless access.
## Web search
@@ -149,6 +184,18 @@ browser-driven and only touches the two exempt vhosts (`ai.` and `auth.`).
into the existing local account with the same email — the pre-SSO admin
(frank8465@protonmail.com) keeps its data. `ENABLE_OAUTH_SIGNUP=true` lets
other `openwebui_users` create accounts on first login.
- **First login mints role `pending`.** `ui.default_user_role=pending` (config
DB, verified 2026-10-09), so a freshly-provisioned `openwebui_users` account
completes SSO and then cannot chat (picker empty) until its role is flipped:
`UPDATE "user" SET role='user' WHERE email='<user>@franksdank.cloud';` —
run with host `python3` over ssh against
`/home/dank/Containers/open-webui/data/webui.db` (no `sqlite3` CLI on
Dankclaw; table is `user`, singular; no `active` column). No per-user model
grant is needed: the picker model `frank-zealot` carries the global
`access_grant` row `(model, frank-zealot, user, *, read)`, so any
`role='user'` account sees it. Flip on FIRST-LOGIN SIGHTING — the row does
not exist until the account signs in (2026-10-09: five provisioned accounts
had no row).
- **IdP reachability:** the container is on its own bridge (10.89.3.0/24) and
cannot reach `authentik-server:9000` by name. It reaches the IdP over the
public URL `https://auth.franksdank.cloud` — Caddy publishes `0.0.0.0:443`
@@ -218,18 +265,49 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
- The image tracks upstream `main`; pin a digest in the quadlet unit if reproducible rollback matters more than tracking. (The old "CPU-only host" framing is stale: this APU runs the gfx1151 ROCm build — see [[ComfyUI]].)
- The default admin account is created on first login; secure it immediately.
- If the backend is loading (3–8 minute model load), OpenWebUI shows connection errors until the model is ready.
- The on-host LLM services (frank-large `:7770`, frank-secondary `:7771`) are reached via `host.containers.internal`, NOT the host LAN IP: the container's netavark bridge cannot hairpin to its own host's non-published ports via `192.168.0.143` (connection refused), and the bridge gateway (`10.89.3.1`) lives in the rootless netns while llama-server runs in the host netns, so it refuses too. `host.containers.internal` (169.254.1.2) resolves to the host on the bridge and works — verified end-to-end for both endpoints. The Dankserver backend (frank-medium) is a different host, reached via its LAN IP `192.168.0.235:7770`.
- The on-host LLM services (Dankclaw `:7770`, `:7771`) are reached via `host.containers.internal`, NOT the host LAN IP: the container's netavark bridge cannot hairpin to its own host's non-published ports via `192.168.0.143` (connection refused), and the bridge gateway (`10.89.3.1`) lives in the rootless netns while llama-server runs in the host netns, so it refuses too. `host.containers.internal` (169.254.1.2) resolves to the host on the bridge and works — verified end-to-end for both endpoints. The Dankserver backend is a different host, reached via its LAN IP `192.168.0.235:7770`.
## History
- **2026-08-22 — slirp4netns → pinned bridge.** The unit ran on `Network=slirp4netns`. The slirp4netns helper process died (no OOM, no kill entry, no crash log found), and the container kept running with only `lo` in its netns: published port 3005 accepted TCP and reset every request, so chat served empty responses for ~2 days. Nothing logged an error. Moving to a kernel bridge removes the per-container helper entirely, and the healthcheck's interface check makes the zombie state visible if it ever returns.
- **2026-09-21 — model picker rebuilt for the 3-model fleet.** The picker carried 39 stale rows (37 dead Ollama models + 2 `frank-default-*`) and was missing `frank-medium`. Rebuilt the `model` table to exactly `frank-large`/`frank-medium`/`frank-secondary` and set `model_ids` on each connection (llama.cpp's Ollama-shaped `/v1/models` is invisible to OpenWebUI's discovery). Pointed the two on-host endpoints at `host.containers.internal` — the bridge refuses the host LAN IP for non-published ports. All three verified end-to-end via `/openai/chat/completions`.
- **2026-09-21 — model picker rebuilt for the 3-model fleet.** The picker carried 39 stale rows (37 dead Ollama models + 2 `frank-default-*`) and was missing `frank-medium`. Rebuilt the `model` table to exactly `frank-dragoon`/`frank-medium`/`frank-zealot` and set `model_ids` on each connection (llama.cpp's Ollama-shaped `/v1/models` is invisible to OpenWebUI's discovery). Pointed the two on-host endpoints at `host.containers.internal` — the bridge refuses the host LAN IP for non-published ports. All three verified end-to-end via `/openai/chat/completions`.
- **2026-09-24 — SSO via authentik.** OIDC authorization-code flow wired per the [SSO (authentik)](#sso-authentik) section: application/provider/entitlements created in authentik, client secret in the host env file, roles from the `roles` scope claim. E2E coverage in `tests/e2e/specs/18-openwebui-sso.spec.ts`.
- **2026-09-24 — flipped to SSO-only.** The OIDC round-trip verified in a real browser (login view renders no password form, only "Continue with authentik", and the flow lands on a signed-in session), so the local username/password path was retired: `ENABLE_LOGIN_FORM=false` + `ENABLE_PASSWORD_AUTH=false` in the quadlet and `ui.enable_login_form` set to `false` in the config DB (the DB wins over env, so both were changed together). `qa-e2e-admin` was demoted out of the `authentik Admins` group (it keeps `openwebui_users` / `homepage-admins` for app access).
- **2026-10-07 — picker narrowed to `frank-zealot` only.** Per Frank: the only model accessible in OpenWebUI is `frank-zealot`. Rebuilt the `model` table to a single `frank-zealot` row (the old rows `frank-dragoon`/`frank-archon`/`frank-small` are deleted), set `openai.api_configs` indices 0 (frank-dragoon) and 1 (frank-archon) to `enable: false`, and retargeted index 2's `model_ids` from the stale `frank-small` label to `frank-zealot` — llama.cpp serves by alias, so the old label addressed a model name the server does not know. Pre-change snapshot: `/home/dank/Containers/open-webui/data/webui.db.bak-20261007` (+ `-wal`/`-shm`). Verified: `/api/v1/models/list` returns exactly `["frank-zealot"]`, chat completion answers `pong` (needs `max_tokens≥1500` — the Qwen3.5 thinking block consumes a 200-token budget), and anonymous API probes are all 401 (see [API access](#api-access-auth-gated)).
- **2026-10-09 — re-keyed to the Protoss ids (wire rename).** The endpoints
began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three
id-keyed objects had to move in the same window as the alias flip:
`model.id`+`name` (`frank-zealot` → `frank-zealot`, `params`
`{"context":{"length":200704}}` and the `meta.description` kept),
`access_grant` (`resource_type='model'`, `resource_id`, principal
`user`/`*`, permission `read`), and `openai.api_configs` per-backend
`model_ids` (index 2 = `:7771` is the enabled family backend; indices 0/1
= `:7770` and Dankserver `:7770` stay `enable: false`). Snapshot first:
`/home/dank/Containers/open-webui/data/webui.db.bak-20261009`. Apply from
inside the container (`podman exec -i open-webui python3 -`, sqlite3; the
image has no sqlite3 binary) then `systemctl --user restart open-webui.service`
(it is a rootless quadlet) to drop the cached model list. Verify in a real
browser as the family QA account: the welcome screen selects
`frank-zealot` and answers. **A stale id fails silently** — the picker
simply lists nothing, because OpenWebUI intersects the backend's
`/v1/models` with `model_ids`.
Law: **the model id is a foreign key in three tables.** Any future alias
change on an endpoint OpenWebUI fronts must ship with this re-key, in the
same deploy.
- **2026-10-09 — composer is `contenteditable`, not `<textarea>`.** A
Playwright probe that waits for `textarea` after SSO times out on the
welcome screen: OpenWebUI 0.6 renders the composer as a contenteditable
div. The stable anchor is the model button (`getByRole("button", {
name: /selected model/i })`) and the welcome text "How can I help you
today?"; that snapshot is also the cheapest proof the picker is populated.
## See Also
- [[Dankclaw Stack]]

Some files were not shown because too many files have changed in this diff Show More