frank-dragoon's health gate never checked WHO answered :7770 — the
false-green the fleet learned about on 2026-09-26 (a stray probe holding
the port satisfies a bare curl while the unit is stopped). The other two
handlers carried that check as inlined copies; all three now call
assert_unit_owns_port, and dragoon gets the protection it was missing.
The seam follows: the endpoints' text-grep deployer tests are deleted,
because tests/llm_deploy_gates.bats EXECUTES the same handlers end to end
(stub systemctl/podman/sudo/ss/curl) and asserts refusals, restarts, unit
links, the pending marker, and the MainPID hardening. GATE_PREFIX is
exported at each handler for the same reason the serve scripts export it.
deploy_llm_dragoon/zealot/arkon now call require_image / require_bytes /
require_min_mib / require_file / require_dir_files, so the deployer and the
serve scripts refuse the same missing payload with the same words — the
2026-10-01 crash-loop law, enforced by construction instead of by duplicated
shell. require_exec joins the library for the native-runtime gate (a
compiled tree that lost its +x refuses like a missing one). relink_unit
takes SYSTEMD_SYSTEM_DIR so a handler can be driven end to end in a sandbox.
The library's own refusals now carry their status: a bash function that
ends inside an if returns the CONDITION's status, so a gate whose last
statement is its refusal printed the message and returned SUCCESS — the
new seam caught it, and the gate tests pin it.
New seam tests/llm_deploy_gates.bash + tests/stubs/: deploy-remote.sh's LLM
handlers EXECUTED against stub systemctl/podman/sudo/ss/curl, asserting
exit status, refusal text, no restart, and the pending state marker.
The image/.rad/template gates are require_image/require_min_mib/require_file,
and the gates an operator can only reach on the host (GPU nodes, the ttm pages
cap, the run line) are now drivable: FRANK_ARCHON_DEV_KFD, FRANK_ARCHON_DEV_DRI,
FRANK_ARCHON_TTM_SYSFS and FRANK_ARCHON_LOG default to the production paths.
Seven new seam tests EXECUTE the serve script against fixtures (stub podman,
stub nodes, stub sysfs, temp payload) and assert exit status, the refusal
words and the engine argv. The old text-grep gate test is deleted: it pinned
the words of the source, not the behaviour.
The three -e gates (runtime, GGUF, mmproj) are require_file calls: the refusal
now names the artifact class, the exact path and the script that produces it.
The seam asserts that observable triple instead of the old sentence order.
The hand-written image/ck/tokenizer gates are require_image, require_bytes and
require_dir_files from scripts/lib/llm-gates.sh. The serve seam is unchanged
and still green (its podman stub now models `podman image exists`, the
primitive the gate uses). Behaviour preserved: every refusal names the
artifact, the endpoint and the fetch script.
The serve scripts and the deployer each hand-wrote the same missing/truncated
artifact gates; the 2026-10-01 crash-loop was those twins disagreeing
mid-cutover. scripts/lib/llm-gates.sh is the single place the refusal lives:
require_file / require_bytes / require_min_mib / require_dir_files /
require_image, each returning 1 and naming the endpoint, the artifact and the
remedy. New seam tests/llm_gates.bats pins the contract at the arguments-in /
status-and-stderr-out boundary.
Runbook section `2026-10-11 — unquantized KV at 2 x 200,000`: the dtype law
(only bf16|fp8, no K/V split; the QSA block key is dtype-fixed, which is why
bf16 costs 1.89x and not 2x), the pool formula that reproduces both live
boots within 0.1%, the seqs-3-to-2 provenance (the tcclaviger fork's
torch.compile constraint, not radiance's), the boot ledger, and the A/B.
The A/B, fp8 control arm through a transient FRANK_ARCHON_ENV_FILE drop-in:
prefill -29% (4,379 -> 3,126 cold), TTFT at the full window +25% (the 2 x 197k
pair: 110.5 s -> 138.1 s), decode inside the +-3% band on two of three passes,
draft acceptance UP 0.661 -> 0.740. The price is prefill, not decode. Open
call for Frank: keep exact KV, or take the cheap fp8 2 x 200k fit (pool 3,648).
Also corrects two stale index entries (w4dn8 said STAGED, fp8-native said
ACTIVE), adds the worklog entry, and commits the verification tool as
scripts/probe-frank-archon-kv.py — it carries the probe law that cost two
false negatives: radiance answers in reasoning_content, so a chat probe with a
small budget reads as empty content + finish_reason stop on a healthy server.
Tests 368/368.
The KV pool at the fp8 shape backed 386,904 tokens — 1.93 sessions of the
served 200,000 window, so a second full-height conversation preempted to the
host/disk prefix tiers. Moving the cache to bf16 (the engine's only
unquantized rung; `--kv-cache-dtype` takes bf16|fp8 and has no per-tensor
K/V split) doubles what a token costs: 7.3125 -> 14.625 KiB per rank
(kv_attn 26.00 -> 52.00 KiB per 4-token page + kv_qsa_bkey 3.25 -> 6.50).
Pool 3543 -> 6912 MiB (437k tokens, >= 2 x 200,000), and the weights slab
pays for all of it: 26071 -> 22208 MiB, the pair pinned to the measured
card ceiling 29,120 MiB (card 31,856 - 1,710 consumed outside both budgets
- 1,024 free). The resident set that leaves VRAM spills into the pinned host
pool, so 40960 -> 44928 MiB, and the prefix host tier doubles with the entry
size (58.50 -> 117 KiB): 4096 -> 8192 MiB. Pinned ledger ~147 GiB, ttm floor
140000 -> 150000 (cap 200 GiB).
--max-num-seqs 3 -> 2: two slots x the window is the capacity contract, and
the "seqs 2 cannot boot" law is the retired tcclaviger fork's torch.compile
query_start_loc constraint, not radiance's (radiance booted seqs 2 in the
fp8-native era; the 2026-10-06 sweep had 2 and 3 tied inside +-3%). Bonus:
the fused all-reduce is now declared over 2 x 3 = 6 rows, under the 25-row
bound at which the wht6 wire is bit-exact.
Cost, paid twice on this engine: idle KV VRAM is also the expert cache's
spillover (lends 2.69 GiB + 740 MiB of arena), so the pool steals weight
residency from the slab AND the lendable cache. Priors: 2.14 GiB of stolen
weights cost 2.5% decode (MAX_MODEL_LEN 262144), the EVC ladder's other
direction cost 8.9%. Prediction -3..-8% vs the fp8 control (136 t/s at 44.5k,
acceptance 0.7246); the deploy boot ledger and live probes are the referee.
Seams: MAX_SEQS/HOST_POOL/KV_DTYPE/PREFIX_HOST literals follow the shape, and
two new invariants guard it — the KV pool must back seqs x max-model-len at
the declared dtype (an engine hard limit: an over-long request is accepted
then refused as too large to ever fit; the formula reproduces the fp8 boot's
386,904 to within 0.1%), and the VRAM pair must stay inside the ceiling.
The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.
Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.
- AGENTS.md Model Config: the Protoss names are now the ids on the wire
(per-endpoint knob named); verify a rename on /v1/models, never on a chat
response — radiance echoes the requested name
- OMP Harness.md: harness id == served id; one id per model everywhere
- Homepage.md: the docs-page "api id" annotations and the switchover promise
are retired
- OpenWebUI.md: the id-keyed trio (model row, access_grant, api_configs
model_ids) is a foreign-key set — re-key it in the same deploy; plus the
contenteditable-composer probe note
- frank-secondary/frank-large/frank-primary runbooks: served ids
frank-zealot/frank-dragoon/frank-archon, and the bench.conf drop-in that
made every frank-primary deploy since 2026-10-08 04:09 restart the staging
worktree behind a green health gate
The 2026-10-08 rename moved the harness model ids and the homepage cards;
the served aliases on the wire stayed frank-primary/frank-large/
frank-secondary. This closes the gap: the alias each endpoint announces is
now the family name, while the services, units, scripts, env files and
runbooks keep the old names (container identity stays stable).
- scripts/frank-secondary.env ALIAS, scripts/frank-large.env
HALOGEN_MODEL_ID, scripts/frank-primary.env FRANK_PRIMARY_SERVED_NAME
(+ the two serve-script defaults/gate messages)
- seams re-pinned: frank_secondary/frank_large/frank_primary unit bats
- fleet harnesses retargeted: bench-probe, deep-probe, calib-push,
fleet-probes-swift15, autoresearch(+measure), bench-threads-batch-http,
bench-decode
- shared/homepage/docs.html: api-id annotations and the "switchover will
rename the ids" footnote removed — the page's names are now the ids;
opencode/Pi/curl examples send frank-zealot
- dankclaw/hermes/config.yaml.example: frank-large -> frank-dragoon and the
live 262144 context (Hermes routes by served id; see Hermes.md)
Consumers with live state re-keyed out-of-band in the same window:
OpenWebUI (model row, access_grant, openai.api_configs model_ids) and the
Hermes config.yaml.
The panel title carries a right-aligned 'AI guide ->' link to
/docs#models (the running-models section), at the point of use.
Bare mode recolors the link; layout is flex baseline, doctrine-minimal.
Seam: homepage_config.bats 6/6; headless smoke census + screenshot.
- Icons: Copyparty -> improvised amber folder, Marreta -> crumbling
wall (bricks + red crack), Calibre-Web -> book, OpenWebUI -> official
logo.png from openwebui.com, Seerr -> official seerr.dev
os_logo_filled.svg.
- Tiles with an open link are clickable whole (.panel.clickable: hover
lift + glow; inner link clicks pass through).
- Any DOWN panel (app tile or model card) gets a red gradient wash and a
slow pulsing glow (.panel.down, .mi-card:has(.badge.down)).
- docs / admin / account-settings buttons moved from the footer to the
header (nav.top-links); footer keeps the deployment stamp.
- Zealot compaction advice promoted from the Use paragraph to a bordered
glowing model.compaction callout (.mi-comp); component seam pinned to
the new field.
- Every app card carries hex-clipped art: 19 amber-tinted brand marks
from the Simple Icons CDN plus 19 improvised monogram SVGs, wired via
a per-service "art" field; homepage_config.bats pins art presence for
all 32 apps.
- Model cards render zealot -> dragoon -> archon (registry order pinned
in the seam).
- Dragoon and zealot share one common uncensored legal warning (adult
content, harmful/hateful/illegal descriptions, sensitive offensive
cybersecurity topics, severe legal risk, "don't ruin this server for
everybody"), hover label on card and docs.
- Archon gets a bordered red permission callout: ask Frank before use.
- SearXNG obfuscation-not-anonymity note moved to a card-level hover
warning (.svc-warn) and a docs hover note.
- Gitea: self-service signup flow; no need to ask Frank.
- Footer docs/admin/account-settings become glowing bordered buttons;
the admin button is shown only to homepage-admins members via a new
Caddy /whoami endpoint echoing the outpost-injected X-Authentik-*
headers (spoof-proof; the /admin* 403 stays the real gate). e2e pins
both sides.
- Docs naming swept: Protoss names primary, bracketed api ids, explicit
note that a future switchover renames the ids themselves.
The Protoss model cards render as .mi-name inside one full-width panel;
the panel-name census now counts both render styles, keeping services.json
the single source of truth.
The model cards become the StarCraft trio: frank-large -> frank-dragoon,
frank-primary -> frank-archon, frank-secondary -> frank-zealot (family
names only; endpoint/API ids unchanged). The three cards fuse into one
Model Information component: per-unit Protoss accent colours and hex-
clipped unit art, 15px type, clickable StarCraft unit-voice quotations
(original .ogg files from the fandom wiki, served from /sfx/), supervision
notes (dragoon/archon safe for autonomous use, zealot tight supervision),
compaction-model recommendation for frank-zealot, and uncensored-model
legal notes rendered as hover tooltips instead of wall text. docs.html
carries the same renames plus api-id mapping.
App-card updates: Seerr, Calibre-Web, RomM, Jellyfin set to WIP: DOWN
with Open links removed (new wip "down" badge state); Copyparty notes it
serves all other media apps content including WIP apps; Gitea and Plex
noted as needing logins separate from SSO; SearXNG noted as obfuscated,
not anonymous (upstream API keys tie to Frank).
Plumbing: Caddy @data allowlist grew /sfx/*; deploy_homepage copies sfx/;
homepage_config.bats pins the new probe names, the component fields,
art/voice-file presence, and the wip-down schema.
Every fleet model card (and the #models docs section) now states, in
plain family-facing terms, what the model is equivalent to:
frank-large ~ Claude Sonnet 5 / OpenAI-Luna, frank-primary ~ Claude
Opus 4.8 / OpenAI-Terra (both Summer 2026 frontier), frank-secondary ~
last year (2025) frontier. Rendered as a new details.equiv line
(desc-equiv, styled in the head color, bare-mode clean); pinned in
homepage_config.bats.
The harness clamps max_tokens to min(45056, contextWindow - est - 4096),
its chars/4 est under-counts the rendered prompt by ~3 % (measured:
152,938 -> 157,388 and 158,906 -> 165,301), and compaction fires only at
85 % -- so with window == engine ceiling 200192, every request in the
est ~146k-184k zone was a guaranteed 400. HALOGEN_CTX=262144 (native) +
KV pool 524288; harness window 217088 = ceiling - 45k absorbs the
estimate error (worst-case request ~219k <= 262144). models.yml updated
out-of-repo: contextWindow 217088, maxTokens 45056.
The @data path allowlist on the post-auth listener did not include /img/*,
so docs screenshots fell through to the index.html catch-all and the
browser got 200 text/html for every PNG (docs images broken live
2026-10-08). Pinned by the homepage_caddy seam.
- style.css: green phosphor -> amber CRT-AMBIOS palette; docs styles for
screenshots, code blocks, key warning (full + bare mode)
- index/admin/docs: VT323 from the Google Fonts CDN
- docs.html: rewrite #chat/#models for the live fleet (flash-server v2,
radiance native FP8, Qwen3.5-9B Q6_K); new #api-access with OpenWebUI
key walkthrough + Opencode and Pi configs; three OpenWebUI screenshots
- services.json: AI card details match the current fleet
- deploy-remote.sh: ship shared/homepage/img/ to /srv/homepage/
- docs: Homepage.md theme wave + OpenWebUI.md picker access-control law
rad_arch_declare rejects the split tier (down block-fp8 + gate_up w4): the
qwen4exp_fp8 kernel serves one encoding per expert table (w4nl64a8h) with only
whole bf16 experts beside it. Recipe now sends gate_up AND down through the
GPTQ w4 rule (down calibrated by the Grams, gate_up uncalibrated), drops the
down-fp8 rule, and lowers the size floor 126000->108000 MiB for the ~19 GiB
lighter container. Convert script: checkpoint lives under models/ukisai/, and
the in-container path must keep the subdir (basename stripped it).
OpenWebUI model picker narrowed to exactly frank-secondary (connections
frank-large/frank-primary disabled in openai.api_configs, model table rebuilt,
stale frank-small model_id retargeted to the served alias). Documented the
auth-gated API surface (anonymous probes all 401; keys per account).
Authentik roster gains artem (pk 16) in the human-user template.
kassie (pk 14) and alex (pk 15, deleted pk 7 then recreated) now share the
ridley shape: <user>@franksdank.cloud + Non-Admin Users + jellyfin_users +
openwebui_users. Audit before destroy: the old alex row carried no MFA
device, token, entitlement, or proxy session - checked across the MFA
device tables, authentik_core_token, and providers_proxy_proxysession
(the fleet's column names differ from the ORM names: events.user is
denormalized JSON, certificatekeypair has no user_id). Events survive the
delete, so the audit trail keeps its shape.
Co-Authored-By: omp <omp@franksdank.cloud>
Deploy verified: nvidia-smi 21,436 MiB used / 2,689 free at ubatch 2048 -
the MiMo-priced projection (~21,430/~2,700) landed within 11 MiB, and the
verbosity-4 ledger confirms the KV law to the MiB (CUDA0 KV buffer
12,544.00 MiB = K(f16) 6,272 + V(f16) 6,272 = 32 KiB/token over the
401,408 pool), compute buffers 1,136 MiB CUDA0 + 848 MiB host, mmproj
879 + 248 MiB. Playwright gate: 33 passed.
Two lessons recorded: the 203/EXEC boot loop my shebang-less rewrite
caused (journal signature + the head -1 check), and that llama.cpp's
"n_ctx_seq < n_ctx_train" line is informational when the context is BELOW
the trained window - the Gemma4 drafter warning was the opposite case.
The unit's Description named the Swift-1.5 era (and said 2x131072/q8_0/
draft-mtp); it now names the Q6_K shape. Cosmetic, so it is NOT deployed
as its own restart: the next frank-secondary deploy relinks it.
Co-Authored-By: omp <omp@franksdank.cloud>
Capacity trade: the 9B dense-hybrid (qwen35: 32 layers = 8 full-attention
+ 24 Gated DeltaNet, 4 KV heads x 256 dim, native 262,144, MTP-less,
natively multimodal) costs ~7 GiB of weights where the 26B MoE cost
~16.6 GiB. The freed room buys back the second conversation slot and
exact KV: --ctx-size 401408 --parallel 2 (two 200,704 windows), f16 K +
f16 V, MTP off (the file carries no nextn/mtp tensors - flag-only enable
is impossible), vision via the publisher's BF16 mmproj, ubatch 1024
(the CUDA decode-optimal value), verbosity 4, publisher sampler recipe
(0.6/0.95/20/0) in models.yml.
Hosts with the MiMo-V2.6-Distill Q6_K boots at this geometry: 21,276 MiB
used / 2,849 free at ubatch 1024, 40,960 tokens of fp16 KV at ubatch
2048. Compaction stays routed here; window math is unchanged (maxTokens
16,384 keeps the worst-case harness request inside the slot).
Gemma4 payload (Q4_K_M + mmproj + MTP assistant) stays on disk as the
rollback; its runbook is retired in place, keeping the attention_k_eq_v
pair-billing law.
Co-Authored-By: omp <omp@franksdank.cloud>
Verbosity 4 paid for itself: the engine's own llama_kv_cache lines give the
exact accounting — the cache bills the K+V pair TWICE under attention_k_eq_v
(6.125 B/elem at f16 K + q8_0 V), reproducing 3,001.25 MiB (5 full layers at
n_head_kv 2) + 306.25 MiB (25 SWA rings at n_head_kv 8, window 1024) to the
byte. The earlier "bill K-only" note was wrong; full layers have 2 KV heads,
not 8.
Deployed boot: 21,868 used / 2,257 free — the f16-K raise (+2,161 MiB KV) is
funded by the ubatch 2048 -> 1024 cut (~616 MiB of compute buffer) plus
variance, so exact K came essentially free. Measured decode 139.9 t/s temp 0;
vision probe returns Red at prompt_tokens 151.
Payload: HauhauCS/Gemma4-26B-A4B-QAT-Uncensored Q4_K_M (16,796,015,520 B)
+ the publisher's BF16 mmproj, /home/dank/models/Gemma4-26B-A4B/.
Shape: ONE 200,704-token slot (--parallel 1; 2x200,704 does not fit the
24 GiB card), f16 K + q8_0 V under flash attention, batch 4096 / ubatch
1024, --log-verbosity 4, MTP OFF (the gemma4-assistant file is on disk,
not loaded: its draft KV is built against the target's full window,
measured +1.9 GiB -> 23,806/319 free, no margin beside the f16-K raise).
Measured boots (nvidia-smi): q8_0 K + q8_0 V / ubatch 2048 / MTP off =
21,890 used / 2,235 free; with the MTP assistant = 23,806 / 319.
Engine facts recorded: gemma4 hybrid attention (25 sliding-window + 5
full-attention layers) so KV scales with 5 layers only, and
attention_k_eq_v is honoured by build 5fc4f3c (pool bills K-only; the
measured q8_0 pool = 2.05 GiB = the K-only figure).
Serve script gained VERBOSITY (+ --log-verbosity in the argv), the
SPEC_DRAFT_FILE drafter path, and defaults/lore retargeted to Gemma4.
deploy-remote.sh payload gates follow the env. The frank-secondary bats
seam is rewritten for the new contract (argv-based) and fixes a
pre-existing vacuity: serve_argv regenerates the test env, so
mutations made before it were silently undone - tests that mutate now
exec through run_stage. frank-secondary is the fleet compaction hub
(200,704 is the only window that fits a worst-case compaction input).
The KLD engine exited clean with the record complete (kld.json rows:
50235, 24 GiB, 5 files), but the wrapper died rc=127: a mid-run scp of
the calib-shape edit shifted bash's lazy-read offset — the fleet's own
script-swap law, biting on the same class as the autoresearch collision.
Resume path: SKIP_KLD=1 verifies the receipt (rows + size + files) and
goes straight to calib. Never scp a script a running process owns.
Execution facts, blocker trail (plugin-tap probe law, weights-slab
formula, KLD_MOUNT), engine verdict, and the post-chain order. Linked
from the docs index per the completeness seam; the cutover runbook
replaces it.
MNBT 65536 / seqs 8: MOE-W4.md:244 measures 640 tok/s at a 32k step
vs 930 at 55k — the KLD record's own rate confirmed step size drives
disk-tier prefill. 131k resident tokens fits the 3072 pool (~237k
tokens/rank at fp8). Cuts the 10M-token calibration from ~4.3 h to
~3 h.
apply-protected-cast.sh splices derive-protected-experts.py output into
the recipe's PROTECTED-BEGIN/END section (refuses to re-derive, trashes
the pre-edit recipe). kld-gate-swift15.sh boots the candidate on the
serving pin in --kld-ref mode (pool fully resident, no disk tier) and
enforces mean KL<=0.03 / top-1>=95% from the JSON report.
One-GPU serialization: the chain waits for the kld driver log to close
rc=0, sanity-checks kld-ref, boots the calibration server, pushes the
corpus at parallel 16 (4 engine slots; queue absorbs the rest), then
kills the server so its trap restores production.
kld died: 'cannot read --kld-corpus /data/kld.jsonl' — KLD_MOUNT was
referenced but never defined, so the corpus/ref-dir mounts silently
expanded to nothing. Add them (nested order: after the /data mount).
feat(convert-swift15-w4dn8): final recipe convert on 1.0.13
Convert with the SERVING pin so the .rad crosses no version boundary;
only the Gram files cross (RADGRAM2 magic, gram.r%d.%s.bin naming and
calib_gram/calib_resolve/calib_stem are byte-identical between the
1.0.13 and 1.2.0 libquant). Host-backend run: prod can stay up.
Gates: grams present (>=48), bf16 container deleted, disk floor,
rad-info gptq receipt, size floor 126000 MiB.
The weights slab derives as card - headroom - kv, and the disk tier
refuses under 9.57 GiB (4.21 layer buffers + 5.36 fixed). 4608 starved
it (8.68 budget, DID NOT FIT). 3072 leaves 10.2 for weights; 4x16k fp8
slots fit the pool, recording is prefill-bound so slot count only
queues pushes.
kld stays on 1.0.13 (record on the scoring engine) with --vram-kv-mib
4608 + 16 slots: the ratio-derived pool came up SHORT by 217 MiB at
32x16k. calib moves to 1.2.0: the RADIANCE_CALIB_TOKENS tap is a 1.2.0
qwen4exp plugin feature (1.0.13 plugin has the DIR half only), and
Grams are bf16 activations - engine-invariant. --vram-kv-mib 9984
covers 64x16k fp8 slots.
Engine v1.2.0 (Codeberg e95d397, Docker Hub tag 1.2.0, index digest
665775ed...) removes --expert-vs-cache-ratio. The explicit
--vram-weights-mib 26071 / --vram-kv-mib 3543 pair transcribes the live
1.0.13 boot ledger at ratio 0.82 (weights 25.46 GiB, KV 3.46 GiB per
card); bracket arms scale KV linearly in (1-r): 0.90 -> 1968, 0.95 -> 984.
Serve script, autoresearch harness, and bats gates follow; the bats
shape asserts are realigned to the deployed seqs=3/spec=2 sweep result
(95923d3, df6b5bb) they failed to track.
- download-frank-primary-swift15.sh: ukisai/Swift1.5-Qwen3.8-Flash-Next
(BF16, 131 shards + index + configs, 360037355038 B = 335.3 GiB, ungated)
via the vendored hfget.sh; verify = listed sizes + 131-shard weight-map
count + every shard's sha256 against the Hub LFS oid. Pattern cloned
from scripts/download-frank-primary.sh.
- qwen4exp-swift15-w4dn8-i8-hc8m.recipe: routed experts with DOWN at
block-fp8 (rtn codes=fp8_e4m3 block=128x128 scale=bf16, the served
FP8-native tile form; moe_gemm_fp8a8 reads it; minicpm5-2b is the
shipped bf16->fp8a8 precedent) and gate_up at w4nl64 GPTQ cd=3;
hc/MTP-fc fp8, trunk i8, lm_head i8, PLE fixed scale, draft_head u2
(inert). Protected-expert cast lines carry the BASE checkpoint's names
as a warm start; the derivation step rewrites them from Swift's own
calibration recording before the recipe is ever run (law documented
in the file).
23,120 used / 1,005 free at boot, 763 settled after a vision probe
(encoder buffers resident). Rung cost 746 MiB, not the transferable
616 prediction. Prefill baseline 1,209 t/s @ 3,760 recorded; decode
unchanged as the law says. 2048 = ceiling for this shape.
Frank's order the same night. The 1024->2048 rung measured +4.6 %
prefill for 616 MiB on the 3090 (2026-10-05 boot pair); the 112k cut
left 1,751 MiB free, so it rides with ~1.1 GiB predicted margin.
Decode is ubatch-invariant; batch 4096 >= 2048 keeps the seam happy.
Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:
- Service trio: scripts/{frank-small.env,frank-small.service,
serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
frank-secondary-*.md (9 files); the FreeToken runbook stays
freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
3090, and the Dankserver service that became frank-medium) are
preserved verbatim — they describe what the service was called
at the time.
The 2026-10-06 cutover crash-looped frank-large 8 boots (03:09:39-03:11:42):
the engine's pin gate refused to start while host MemAvailable was ~74.7 GiB,
3.34 GiB short of the 78.11 GiB it needs (62.11 GiB checkpoint + 16 GiB floor).
It self-resolved in ~2 min when the dying 27B engine freed RAM, but left the
host with only ~1.8 GiB MemAvailable - a restart would crash-loop again.
serve-frank-large.sh now checks MemAvailable against the pin gate BEFORE
podman run and WAITS up to FRANK_LARGE_RAM_WAIT (default 600 s) for the RAM
to clear, so a transient dip boots once instead of crash-looping; if it never
clears it refuses loudly. The 16 GiB floor is the engine's (not
HALOGEN_HOST_RESERVE_GIB=20, a separate overall-budget knob).
frank_large_unit.bats: FRANK_LARGE_MEMINFO seam + a refuse test. Runbook +
log: the pin-gate law, the incident, and the fix.
Frank
