frank-large moves from the kingjones30/ROCmFPX llama.cpp build to
peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b
.hgn 115.55 GiB + quality overlay). Upstream measures this engine at
roughly 4x the prefill throughput of the llama.cpp family on the same
gfx1151 silicon, and prefill dominates any prompt with real context.
Shape: a rootless Quadlet container (dankclaw/frank-large/quadlet/),
engine on container loopback :8730 (never published), OpenAI front-end
:8731 published as host 7770, HALOGEN_MODEL_ID=frank-large so no client
renamed anything. Context 262144, KV pool 262144 positions (~7.2 GiB,
conservative for a shared 121 GiB host), 4 slots, host reserve 24,
vision tower on, weights pinned, all other flags at vendor defaults.
Weights are fetched by HALOGEN_DOWNLOAD on first start into a writable
/home/dank/models/halogen.
The llama.cpp era (unit, env, serve script) is staged in .AGENTTRASH/
rather than deleted; the STRIX payload and the ROCmFPX build stay on
disk as the rollback target. deploy_llm_large becomes a user-quadlet
handler: it retires the old ROOT unit (moving its checkout symlink into
the home trash, never rm) and gates the first start on the download
rather than reporting a failure for a fetch that is working exactly as
designed.
frank-npu is retired in the same change. The kernel cmdline moves from
iommu=pt to amd_iommu=off for the 13-16% of prefill upstream measured,
and that takes the XDNA2 NPU with it: without PASID/SVA the driver
cannot open /dev/accel/accel0 (err=-19). Its five dependent routes are
repointed (Hermes vision -> frank-large, Hermes title generation ->
frank-small, harness vision -> frank-large, tiny -> frank-small, task ->
frank-large:medium) and its provider blocks are removed from both the
harness config and Hermes's live config. Audio input leaves the fleet
with no replacement; image input survives on halogen's vision tower.
Docs: new halogen-flash-server runbook, new IOMMU decision record, and
frank-npu rewritten as a retirement record. AGENTS.md topology, runtime
sections, benchmarking policy (frank-large is a user unit now) and the
harness model-config section are corrected.
FrankBook-Pro becomes the only *arr/qBittorrent deployment. Every fleet
container (qB + five arrs) now shares gluetun's netns, so all egress —
torrent, indexer APIs, metadata lookups — exits the WireGuard tunnel by
construction (AllowedIPs=0.0.0.0/0); the arrs' own ports publish on the
gluetun unit, none on the arr units themselves.
- dankserver/compose.yaml: remove the six arr services (the Aug 20 compose
copy — never healthy, superseded; config dirs trashed on the host)
- dankclaw/Caddyfile + shared/homepage/services.json: arr vhosts and
probes repoint 192.168.0.235 -> 192.168.0.112 (FrankBook-Pro)
- frankbook-pro units: five arrs join gluetun netns (Wants/After/Network);
gluetun publishes 9696/8989/7878/8686/8787 alongside 8080/6881
- tests/frankbook_quadlet.bats: new VPN egress seam — every arr+qB unit
must share gluetun's netns and publish no port; gluetun must publish the
whole fleet's ports
- deploy-plan.py: gluetun changes carry the whole netns family (a gluetun
recreation kills all six joiners); dankserver compose set loses the arrs
- AGENTS.md: deployment is complete only when unit tests AND the Playwright
E2E suite both pass
Boot tag flips from qwen3.8-distilled-heretic:9b to nuextract3:4b (verified
live: per-request routing already serves every registered tag; the boot tag
only decides which model is resident at start, so the first request after a
swap pays the load cost). FLM_CTX_LEN=-1 drops the global --ctx-len override
so each pack loads with its own trained context (nuextract3 32768, gemma4
131072).
frank-default-large -> frank-large (Dankclaw :7770), frank-default-small ->
frank-small (Dankserver :7770), frank-default-sightsound -> frank-npu
(Dankclaw :7771). Units, env files, serve scripts, deploy-plan mappings,
deploy-remote dispatch (stop/disable old units + relink + enable new), and the
test seams move together. Retired frank-default-npu.* artifacts moved to
scripts/.AGENTTRASH/ (tracked-in-history; ignored on disk).
Served aliases become frank-large / frank-small via --alias; the NPU endpoint
serves the whole FLM registry, so its rename changes no served tags.
- tautulli.container: HealthCmd switches to the repo's proven curl pattern
(jellyfin unit) — quadlet's systemd-style value parsing mangles embedded
double quotes; PublishPort=8181:8181 restored (a mis-aimed line edit
replaced it; the PublishPort seam test caught the regression)
- deploy_plex readiness gate: 36x5s -> 120x5s; the 2026-09 cutover showed
the first container boot of a years-old library runs ~10 min while PMS
pre-binds 32400 and empty-replies; 180s failed the gate twice with a
healthy, still-booting service
Evidence from the failed first cutover: podman top shows container root
maps to dank (1000) under UserNS=host rootless (subuid range 100000:65536
cannot express 997), so PUID=997 fell back to root and PMS crash-looped on
write access to its 997:984-owned data root. Fleet model instead: PUID=0/
PGID=0 (container root = dank, same as jellyfin/copyparty data roots) and
deploy_plex chowns /var/lib/plexmediaserver to 1000:1000 inside the
first-cutover block. Zero bytes moved; rollback regains one chown -R
997:984 step, documented in Plex.md. Seam test pins PUID=0/PGID=0 and the
absence of any userns key.
- deploy_plex(): install unit, stop+mask native plexmediaserver.service on
first cutover (mask, never disable), gate on /identity for 180s
- deploy_tautulli(): install unit, stop+disable snap on first cutover,
seed config.ini with PMS_Web_Host=plex only if absent
- allow-lists (deploy.sh valid_service, deploy-remote.sh service_is_allowed),
planner routing (SERVICE_ORDER, DANKSERVER_SERVICES, quadlet_services),
test-changed cutover case, quadlet seam enumeration + new
userns/PUID=997/PGID=984 ownership-contract test
The frankbook handler ran port verification even for services the
credential gate skipped, so a parked qbittorrent failed the deploy and
aborted the five arrs. Gate now lives Mac-side (test -s gluetun/env
inside the VM) before restart+verify.
The qB first-boot seed wrote a plaintext WebUI\Password, which qB >= 4.2
ignores (PBKDF2 @ByteArray only), and omitted HostHeaderValidation=false,
which would 401 the LAN forwarder path. Seed now generates the PBKDF2
hash in the VM (python3 hashlib) and sets ServerDomains=* +
HostHeaderValidation=false. Arr categories (lidarr/radarr/tv-sonarr)
pre-created with save_path=/Volumes/COMPLETE/Finished Torrents so any
grab lands in the review dir regardless of wiring details.
Seven units installed INSIDE the Mac VM (/home/core/.config/containers/
systemd) by a new deploy_frankbook_stack handler: gluetun (ProtonVPN
WireGuard, env-file credentials, publishes qbit's 8080/6881), qbittorrent
(Network=gluetun.container, verbatim /Volumes data roots, first-boot
config seed with fresh machine-local WebUI password and the Finished
Torrents category), and prowlarr/sonarr/radarr/lidarr/readarr (grab-only,
config migrated from dankserver at provisioning, readarr pinned to
amd64-1.0.980). deploy.sh routes the new target through the ssh-config
alias with REMOTE_SHELL=/usr/local/bin/bash (macOS bash 3.2 lacks
associative arrays); the planner learns FRANKBOOK_SERVICES and the
gluetun->qbittorrent netns pairing; new seam test pins the contract.
Two live-verified loader failures on Dankserver (2026-09-01), each a
clean exit at load (no swap-thrash, 37 GB fits RAM):
1. ornith-ai/Ornith-1.5-35B-A3B-FP8 (compressed-tensors PER-CHANNEL fp8):
ValueError: Missing MoE expert source layers: {'gate_up': [0, 1, ...]}
— nonstandard expert tensor naming, provider finds no expert sources.
2. lamianlbe/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-fp8
(compressed-tensors BLOCK-fp8 128x128, standard per-expert naming):
RuntimeError: The size of tensor a (2048) must match the size of
tensor b (...) in _dequant_fp8_weight — the dequant path multiplies
weight * weight_scale directly and cannot expand compressed-tensors
block scales.
Empirical matrix for FreeToken v0.1.2 on qwen3_5_moe (RTX 3090):
modelopt block-fp8 (quant_method 'fp8' + weight_scale_inv) -> LOADS
compressed-tensors per-channel FP8 -> fails
compressed-tensors block-fp8 128x128 -> fails
batsclamp/Huihui-...-abliterated-FP8 is the proven uncensored checkpoint;
restoring it until a modelopt/NVFP4 requant of a wanted model exists or
FreeToken learns compressed-tensors.
Ornith-1.5-35B-A3B-FP8 failed to load on FreeToken v0.1.2: its
compressed-tensors checkpoint uses a nonstandard expert tensor layout
(FreeToken: 'Missing MoE expert source layers: gate_up'). Retarget to
the HauhauCS Aggressive FP8 finetune of the SAME Qwen3.6-35B-A3B arch
Huihui used, which carries compressed-tensors BLOCK-fp8 (128x128) with
the standard per-expert naming (mlp.experts.N.gate_proj/up_proj/down_proj
+ .weight_scale) that FreeToken's MoE loader reads.
- MODEL_DIR -> /home/dank/models/hauhaucs-qwen36-35b-a3b-aggressive-fp8
- use the checkpoint's own chat_template.jinja (drop the Sharp override;
forcing a template the finetune wasn't trained on risks format drift)
- pool plan unchanged (same ~35 GB FP8 mass / qwen3_5_moe arch)
rename() of a directory requires write on the directory itself; mariadb/
redis data roots are owned by the containers' mapped subuids (100998),
so the deploy user cannot move them. Use run_privileged mv (numeric
ownership preserved = rootless mapping contract) and skip the chown pass
for romm via the new keep mode — chowning mapped subuid data to the
deploy uid breaks the in-container mysql/redis users. Collisions with
rescued anonymous-volume stubs now merge one level deep instead of
stranding real data under /opt. Reverse the podman rm order so the
dependent romm container is removed before its backends.
dankserver's ~/.config/containers/systemd did not exist (no quadlet units
had ever been installed there); the dankserver media handlers cp'd into it
without mkdir -p, failing before touching any container.
- MODEL_DIR -> /home/dank/models/ornith-1.5-35b-a3b-fp8 (37 GB,
compressed-tensors per-channel FP8, same qwen3_5_moe arch/mass as
Huihui so the pool plan carries unchanged)
- CHAT_TEMPLATE_FILE -> scripts/chat-template.jinja (Qwen-Sharp, shared
with frank-default-large); the checkpoint's stock template
mis-serializes reasoning per its own model card
- env/serve/unit headers updated; loader-risk + BF16-does-not-fit note
mv of a data root onto a non-empty directory fails (Directory not empty).
The romm/copyparty cutover creates Containers/<svc>/ before migrating:
the seed env file lands there first, and the romm anonymous-volume rescue
extracts into it before the /opt move. Merge the old root's entries in
without overwriting existing state (mv -n), warn and leave residue if a
collision kept the new side, and keep the plain rename when the
destination is absent or empty.
The first deploy after a unit-file edit restarted with the stale generated
unit (generator had not seen the new Volume= paths), crash-looping hermes on
statfs ENOENT for the just-renamed data root. Reload between cp and restart.
Cutover of the six surviving dankserver services from compose-era bare
containers to boot-persistent Quadlet units (WantedBy=default.target; podman
5.x v1 generator honors it at user-manager start).
Units are faithful to live inspect ground truth (image pins = running tags,
not compose inventory):
- romm trio shares one data root: /home/dank/Containers/romm mounted whole at
/romm; mariadb datadir and redis data dir live inside it. Aliases mariadb/
redis preserved on dankserver_appnet via Network=...:alias=. Secrets stay in
the host-local env file (EnvironmentFile + ${VAR} expansion in the generated
ExecStart, gitea precedent). copyparty passes its admin account via
Exec=-a ${COPAY_ACCOUNT}:${COPAY_PASSWORD} (Exec= not Command=: the v1
generator hard-fails on Command=); entropy stays rw only there.
- jellyfin keeps every creation-time env as Environment= lines (zero drift);
Device=/dev/dri + GroupAdd=keep-groups preserve hardware transcoding.
- calibre-web pins the exact running image tag (id 986277563e76 =
0.6.27-ls397), pasta network preserved.
Deployer: first-cutover handlers retire the old container, rescue unshadowed
anonymous-volume content (podman volume export + tar --skip-old-files), mv
data roots on the same fs, then start units in dependency order with health
gates. Re-deploys restart only the selected unit. deploy-plan.py maps compose
keys mariadb/redis to the new unit names; allowlists updated. New seam test
tests/dankserver_quadlet.bats pins boot-persistence + v1-generator contract;
test-changed.sh maps the new dirs to it. Full suite green (209 tests).
Adds a digest-pinned headless-shell Chrome DevTools Protocol daemon as a
boot-persistent quadlet unit (chrome-devtools-protocol.container). The
entrypoint override drops the image's socat front so the CDP endpoint binds
loopback-only on 127.0.0.1:9222 in the host network namespace; only the two
host-network consumers (Signal gateway, webui server) can drive it.
hermes-webui flips Network=dankclaw_default -> Network=host so its agent
process shares the daemon's namespace (0.0.0.0:8787 bind is LAN-facing as
before; no exposure change). deploy_hermes installs the new unit and starts
it if inactive without restarting in-flight browser sessions.
The first revision guarded the kfd line with SUBSYSTEM=="misc", which never
matches (misc-class devices carry no subsystem for udev matching), so the rule
was read but never fired and /dev/kfd stayed 0660 root:render. Name-only match
applies 0666 on creation and on trigger.
Also fixes test-changed.sh to exit 0 when changed paths map to zero tests
(udev rules are mapped-but-untestable) instead of invoking bats with an empty
selection.
The comfyui container is spawned by a systemd user unit whose spawn context
carries no supplementary groups, so render-group access to /dev/kfd and
renderD128 was structurally unreachable (keep-groups preserved an empty set;
named group_add 'render' fails name lookup inside the userns). World-mode on
the compute nodes is the only spawn-context-independent grant; display cards
keep their modes. Apply: install to /etc/udev/rules.d, udevadm control
--reload-rules, then trigger by subsystem to re-apply modes to live nodes.
Also maps dankclaw/udev/* in the focused-test selector (not bats-testable).
Env file is the source of truth (unit ExecStart runs the serve script which
sources it); serve-script default and the deployer health-gate URL flipped in
lockstep so the post-restart health gate polls the new port. Docs, fleet table,
and both index bullets updated to match.
Prior docs claimed unknown tags silently serve llama3.2:1b. Live probes on the
live endpoint disprove it: /v1/chat/completions answers HTTP 200 with a flat
{"error":"Failed to load <tag> model!"} body and no choices; /v1/completions
answers a nested OpenAI-shaped server_error (code 500) under HTTP 200. Valid
tags route per-request to the named model (response echoes the requested tag);
no restart needed to switch between registry tags. Corrected in the runbook,
the AGENTS.md runtime section, the serve-script header comment, and the
historical FastFlowLM incident note.
The trio ran as bare compose containers and died silently at the Aug 26
reboot — nothing restarts compose containers at boot. Each now has a
boot-persistent unit (WantedBy=default.target; the v1 quadlet generator on
this host cannot enable generated units). The deploy handler retires the
compose-era containers only while the units are not yet active, then
(re)starts each unit.
Units run the image under a localhost/hermes-agent:latest alias tag (the
quadlet create path on this host cannot resolve registry-prefixed refs from
local storage); the deploy handler tags first. The gateway's :8090 publish
is dropped — nothing consumes it; the public vhost now routes to the
dashboard's login-gated UI on 9119.
The first deploy attempt pulled c3ed3a6 on the target but the remote runner
rejected the explicit --service selection because service_is_allowed() keeps
its own per-target allowlist; add the new LLM service to it.
New Dankclaw service frank-default-sightsound: FLM 1.0.1 serving the
Gemma4-E4B-IT-NPU2 pack (vision+audio+reasoning, 131072 ctx) on the XDNA2
NPU at port 52626. Model choice rationale: gemma4-it is the only NPU2 model
family with BOTH image and audio towers; the uncensored candidate qwythos:9b
is text-only, so sight+sound wins over uncensored here.
Deployer wiring: planner maps env/unit/serve files to the new service name;
deploy-remote gains deploy_llm_sightsound (relink + daemon-reload + enable +
restart + health gate on /v1/version — FLM has no /health route). Unlike the
retired frank-default-npu relink-only path, this unit is boot-enabled.
- same qwen3_5_moe arch and native block-fp8 format (e4m3, 128x128
weight_block_size, un-fused per-expert layout) as the official
Qwen FP8 checkpoint: identical FreeToken loader path, pool numbers
unchanged (2048 MoE slots / 453005 KV tokens / 4 requests)
- checkpoint downloaded to /home/dank/models/huihui-qwen36-35b-a3b-abliterated-fp8
(26 shards, ~35G); official checkpoint kept for rollback
- ships froggeric fixed Qwen3.6 chat template, read from checkpoint
- also refresh both .service Descriptions (large still said STRIX_LEAN)
- runtime: charlie build-qwen38 -> kingjones30/ROCmFPX (card recipe
build, LLAMA_BUILD_WEBUI=OFF since npm/HF-bucket provisioning fails;
all 35 of our serving flags verified present in the tree)
- model: agentionai FAST imatrix -> kingjones777 STRIX single GGUF
(on-host copy sha256 == HF LFS oid c6770d74...49fa verified)
- args unchanged from our validated profile (401408x2, b4096/ub512,
t16/tb16, f16 KV, fa on, fit off); mmap stays default per card
(PLE streams via page cache; --no-mmap = silent cgroup OOM)
fa on aborted graph build: build_attn_qsa called ggml_fill on the
F16-converted kq mask (GGML_ASSERT F32 at ggml.c:5371). Patch mirrors the
MLA sparse path in llm_graph_context::build_attn (F32 base mask for the
top-k surgery, F16 cast only at the attention call, gated on
cparams.flash_attn). Vendored at scripts/patches/qwen4exp-qsa-f32-mask.patch
(sha256 f2f4a486...); the build-qwen38 binary on Dankclaw is rebuilt with
it and validated live at CTX_SIZE=32768 (health 200, real completion).
- Main model (v2 joined PLE layout, sha256 1e4160ca…) loads and serves on
build-qwen38 only with --flash-attn off: the QSA indexer path calls
ggml_fill on the F16-cast kq mask and aborts graph build (GGML_ASSERT
F32 at ggml.c:5371). Verified live: model loaded, health 200, real
completion.
- MTP draft head (agentionai FAST, 2.28 GiB) does not load on this binary:
qwen4exp loader requires full per-layer blk.<il>.hc_attn_norm.weight,
head carries only blk.48. Third confirmation (EasiiX Q8_0 + agentionai
FAST vs kingjones STRIX_LEAN previously, now own-matched pair). Spec
decoding runs ngram-mod only; head retained for a future compatible build.
- FLASH_ATTN=off and SPEC_TYPE=ngram-mod wired through env + serve script;
runbook and AGENTS.md updated with the verified constraints.
Model: agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF (87.06 GiB,
Q4_0_ROCMFP4_FAST imatrix-calibrated, per-head PLE table inside the GGUF —
fully VRAM-resident). Draft: agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST
(2.28 GiB, matched ROCmFP4-FAST quant, ~66% acceptance at n-max 3).
Vision tower (f16) loaded via mmproj/.
Runtime: charlie12345/ROCmFPX build-qwen38 (HIP, ROCm 7.2.4, gfx1151) —
the only binary with the ROCmFP4 FAST types. Replaces the jcbtc CIRU v1.1
deployment (ciru-ai runtime).
Profile: card sampling table (temp 0.7, top_p 0.8, top_k 20, min_p 0.0,
presence 1.5) mixed with the validated build-qwen38 serving profile:
262144 ctx x 1 slot, b 4096 / ub 4096, t 16 / tb 16, f16 KV, fa on,
cont-batching, prompt cache + 8 GiB cache-ram + idle slots, 16 ctx
checkpoints (no --checkpoint-min-step — unsupported on this binary),
--fit off (auto-fit segfaults on qwen4exp hyper-connection config),
--spec-type draft-mtp,ngram-mod with n-max 3 (adaptive p-split measured
worse per author card), p-min 0.75. Qwen-Sharp chat template kept;
reasoning stays in DeepSeek format.
The 2026-08-28 failure of this draft head on the kingjones STRIX_LEAN
model was a cross-vendor pairing problem: the head carries
blk.48.hc_attn_norm.weight (verified in-GGUF, same shape as the working
CIRU draft). Docs: new runbook agentionai-fast-imatrix, index updates.
Runtime: ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4 llama.cpp at tag v1.1, built
with the vendor build-linux-amd.sh flag set plus GGML_CUDA_FA +
GGML_CUDA_FA_ALL_QUANTS (the Q4_1_MOE_FORCE_J path is live in the hipified
mmq kernels). Model: jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 (Q4_1 IU4 experts,
MTP Q8_0 draft, CIRUPLE1 PLE sidecar) with checksums verified at download.
Vendor production profile verbatim on first deployment: 262144 ctx x 1 slot,
b 2048 / ub 512, t 8 / tb 8, f16 KV, fa on, cont-batching, prompt cache +
8 GiB cache-ram + idle slots, 32 ctx checkpoints (min step 8192), MTP depth
3 with p-split 0.10, PLE pager env knobs (Q41_MOE_FORCE_J=32,
PLE_WORKERS=16, PLE_STRICT_SHA=0, HIPBLASLT=1). Qwen-Sharp chat template
kept deliberately; reasoning stays in DeepSeek format.
Frank