New ACTIVE runbook (setup/doctor PASS=34/FAIL=0, UVA transport verified
in serving workers, wire-verified alias/reasoning/metrics), index +
vault entries flipped: llama.cpp b11030 endpoint RETIRED 2026-09-27 to
rollback target. Topology diagram updated.
The 2026-09-26 same-day cutover that ended the Lucebox DS4.1 detour:
Dankserver's :7770 endpoint is now unsloth llama.cpp (the b11030-mix
prebuilt, ROCm gfx120X, bundled runtime) serving
unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 — 175.30 GiB across six sha256-
verified split parts — on both Radeon AI PRO R9700s, as a ROOT unit (the
frank-large shape: no container, no generated unit).
The deployed contract:
- Placement law: layer split -ts 1,1 with -ngl 999; the routed experts
and the 51B-param n-gram (PLE) table are COMPUTED in host RAM via
-ot exps=CPU,ngram_embeddings=CPU (the x4-pair measurement: streaming
over PCIe loses at 3.7 t/s, computing in RAM does not care).
per_layer_token_embd.weight=CPU is required by the shared MTP head's
borrow path (a GPU-resident borrow crashes the draft load).
mmap stays on; --cache-ram 16384.
- Spec chain draft-mtp,ngram-mod: the shared Q8_0 MTP head
(mtp-...-shared-Q8_0.gguf, 2,786,568,256 B) at depth 2, with the
draftless ngram-mod matcher chained behind it — the only legal way to
layer two spec implementations in one process.
- Vision explicit: --mmproj names the BF16 projector (this build never
autoloads; a server without it comes up healthy and text-only).
- Reasoning: --reasoning-format deepseek → reasoning_content; the
Qwen3.8 template's {xhigh, medium, low} reasoning_effort gate folded
by reasoningEffortMap in models.yml.
- One stream, the model's full 262,144 context; greedy server default
with Qwen-recommended sampling riding compat.extraBody.
The deployer's deploy_llm_primary is rewritten for the root unit: it
gates every artifact at its declared byte count (stat -c %s, never wc -c
— which read the whole 377 GiB lucebox file for minutes on 2026-09-26),
retires the lucebox user quadlet to dated home trash, waits out port
7770, relinks/enables/restarts the root unit and waits up to 1800 s for
/health. The quadlet-era tree under dankserver/frank-primary/ is
archived (dankserver/.AGENTTRASH/20260926-llamacpp-cutover/ on this
box; git history everywhere else) and its planner mapping replaced by
the scripts/ trio.
Observability flips back to the llamacpp namespace: a frank-primary
Prometheus job at 192.168.0.235:7770 (cross-host LAN — the netavark
published-port quirk that forces host.containers.internal for the two
Dankclaw endpoints is same-host only), the LlmEndpointDown runbook
text, and the dashboard description for three llama.cpp endpoints.
Docs: the new runbook (frank-primary-llamacpp-qwen38-flash-next-q8.md),
RETIRED banners on the lucebox and radiance runbooks and their index
entries, the AGENTS.md fleet section rewritten for the root-unit shape,
and the homepage frank-medium card renamed to frank-primary with the
Q8_0 Flash-Next spec (docs.html #models and the modality note updated:
images on frank-primary and frank-small, audio on frank-small only).
models.yml: the dankserver provider is "llamacpp (dankserver)" serving
frank-primary, contextWindow 262144, maxTokens 65536 (compaction-gap
law: 183,501 + 65,536 <= 262,144), input [text, image].
Suite: 346 green (frank_primary_unit.bats rewritten for the root unit —
16 tests pinning the placement law, the shared-head requirement, the
explicit projector, HIP env hygiene (BIND_HOST, never bare HOST), and
the deployer's byte-count gates; the quadlet-era planner and
changed-tests mappings rewritten for the trio; the observability scrape
test asserts three llamacpp jobs again).
Rollback: the lucebox image (localhost/lucebox-rocm:gfx1201) and the
378 GiB payload stay on the host; the Qwen payload stays fetched, so a
re-swap in either direction is a redeploy, not a refetch.
Replaces frank-medium (vllm-radiance TP=2) on :7770. The 377.2 GiB
43-layer MoE (256 routed experts, top-6) is mmapped: dense work and the
MLA KV cache on the R9700, routed experts computed on CPU from the
page cache (246 GiB RAM). The unit bypasses the image entrypoint
(Qwen/DFlash-only flag set) and execs luce_server with the DS4 launch
shape; --ds4-fused-decode's monolithic-load fallback IS the serving
mode here. deploy_llm_primary gates on image + model byte count,
stops/masks the radiance unit, waits for :7770, installs and starts.
Runbook: frank-primary-lucebox-ds4.md.
Knowledge-base follow-up to a45f396. The runbook, both index one-liners, and
the AGENTS.md topology row all still declared FP8 active; flipped to
just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq with unquantized bf16 KV (pool
486,464 tokens, full 262,144 window), and recorded the causal nuance: fp8 KV
was on both arms, so the FP8-vs-MXFP4 gap was weight precision, not KV - clean
KV is a quality hypothesis for W4A8, not a throughput one. No verdict yet.
The quadlet's "TEXT ONLY ... 400 here" comment was wrong: vision works on the
MXFP4 checkpoint (Qwen3_5ForConditionalGeneration carries the vision tower
through the Gemma4 merge path, unquantized) - verified by naming a solid red
48x48 PNG "Red". Comment corrected; harness models.yml input stays [text,image].
Docs-only; no re-deploy (the endpoint already serves vision).
New runbook frank-large-unsloth-qwen38-flash-next.md (runtime, payload,
deployed config, the explicit -md, the 114 GiB GTT memory model, client
contract, rollback). frank-large-ciru-orca.md marked RETIRED (rollback
target, payload and runtime intact on disk). AGENTS.md topology row,
frank-large section, harness note, runbook index, and the GPU-budget
lesson (GTT aperture ~114 GiB, not the stale 64 GiB rocm-smi figure —
the CIRU Orca held 92.6 GiB on the same device). LLM index header,
mermaid, active list, and the vault root index updated. Work-log entry.
The radiance TP=2 cutover changed the metric namespace frank-medium exports
(llamacpp:* -> vllm:*) and left two scrape targets addressing host processes
by an address a container cannot reach.
- shared/prometheus/rules/llm-fleet.yml: two namespaces side by side. Added
LlmRequestsWaiting, LlmKvCacheSaturated (vllm-only gauges llama.cpp never
had), the job:llm_vllm_prefix_cache_reuse:rate5m and
job:llm_spec_accept_length:rate5m recording rules, and re-cut
LlmPrefixCacheCollapse / LlmSpecDecodeOff to OR both families with
per-family traffic floors so an idle endpoint cannot fire them. Every
expression validated against live Prometheus.
- shared/prometheus/prometheus.yml: frank-small -> host.containers.internal:7771
and frank-large scraped at host.containers.internal:7770. Both are HOST
processes on Dankclaw and netavark DNATs published container ports only, so
the LAN-IP targets were red by construction; frank-large's CIRU llama.cpp
build does export the llamacpp:* family, contra the old comment.
- tests/observability_quadlet.bats: the two seams now encode the new contract
(per-endpoint reachability by process type; every named series must be on the
measured list for its engine; engine-independent rules must name both).
- homepage docs.html/services.json: frank-medium sees pictures, frank-large is
CIRU Orca with a single seat (was halogen w4b, two seats, 200k).
- docs: radiance runbook is ACTIVE (deployed flags, three-arm battery, verified
capabilities, rollback, replica-era sections labelled); llama.cpp runbook
carries the SUPERSEDED banner; Dual-R9700-Cooperation states the measured
verdict; Observability records the namespace split and the DNAT lesson;
AGENTS.md topology/frank-medium/Model Config/Benchmarking Policy/runbook
index flipped; both indexes updated; work-log entry added.
Also repaired agent-pass damage: prose prepended above frontmatter openers,
duplicated runbook bodies, a literal tool-call token pasted into a runbook, and
an index destroyed by diff artifacts (restored from HEAD, flips re-applied).
Vault link check: 128 pages, 0 unresolved. ./tests/run.sh: 325 pass.
Reanalysis after the 2026-09-20 swap to an ASUS ProArt B850-Creator WiFi
NEO: both R9700s now sit on CPU GPP root ports (x8/x8 Gen5 bifurcation),
every bridge reads AtomicOpsCap Routing+, and both endpoints read
AtomicOpsCtl ReqEn+ (CAP_EXP+28.w = 0x1440) — the exact precondition
Dual-R9700-Cooperation's test plan named. TP=2 is therefore unblocked,
not proven: no collective has been dispatched since the swap,
GGML_HIP_RCCL is still OFF, and the 272 t/s "tensor" number on record
measured a butterfly fallback.
- Dual-R9700-Cooperation: short answer rewritten, re-measured 2026-09-20
topology table, rebuild-hypothesis door marked moot, probe plan is now
standing work.
- Dankserver System Specs: Motherboard section; GPU BDFs 0b:00.0 ->
07:00.0, iGPU 10:00.0 -> 11:00.0, kernel 7.0.0-31, SMT on, and the new
eno2 (192.168.0.247) second 5GbE port holding the low-metric default
route while eno1 keeps the fleet's static .235.
- frank-medium-llamacpp-dflash2: rationale and hardware row split past
from present; dated work-log sections stay as history.
- AGENTS.md + both indexes: no longer assert a wall that no longer exists.
Docs only; frank-medium stayed active and healthy throughout (verified
/health). ./tests/run.sh 331/331.
The vLLM detour (a2b39b1, same day) is retired: the Q4_K_M llama.cpp setup
is the requested shape and the vLLM KV-quant options on sm_86 all measured
dead (fp8 hardware-impossible, int8 page-size unification, turboquant
mm-prefix). Root unit restored, vLLM quadlet/Dockerfile/fetch script moved
to .AGENTTRASH, runbook listed as same-day-reverted. Rollback target:
frank-small-vllm-gemma4.
Runtime swap on the RTX 3090 (port 7771):
- vLLM gemma-x86_64-cu129 dev build, digest-pinned, audio extra pip-installed
into localhost/vllm/vllm-openai:frank-small (Dockerfile, no COPY lines)
- google/gemma-4-12B-it-qat-w4a16-ct (10.26 GiB int4) + Google's bf16 MTP
assistant (845 MB) — the w4a16-ct assistant cannot draft in vLLM (drafter
path skips Marlin; acceptance 1.00), bf16 measures 2.80-3.56
- root systemd unit -> rootless Quadlet (dankclaw/frank-small/quadlet/);
manual CUDA device+lib mounts (no nvidia-container-toolkit on host)
- bf16 KV (fp8 hardware-dead on sm_86), 289,647-token pool, 2 concurrent
full-context sessions, CUDA graphs on
- native text+image+audio: the fleet's multimodal endpoint
- measured: pp@67k 1293 t/s, pp@213k 608 t/s, tg@213k ~13.6 t/s,
tg@85k ~17.6 t/s
- host driver 610.57.04 (Ubuntu 26.04); sm_86 is correct, no arch override
- planner/scoping: frank-small now selected by dankclaw/frank-small/ +
env/serve/download trio; old unit retired to scripts/.AGENTTRASH/
- docs: new runbook frank-small-vllm-gemma4.md; AGENTS.md, LLM index,
top docs index updated; old runbook moved to retired
AGENTS.md topology, Active Services, the CUDA-quant-KV lesson (f16 is no
longer why), and the runbook index. New frank-small-gemma4-12b runbook:
payload digests, the SWA/q8 arithmetic behind --fit's 262k window, the
MTP-sidecar + ngram-mod chain, the always-accepted reasoning_effort, and
the fit-probe error that looks like a fault at boot. The Qwen runbook and
both indexes carry the retirement; work log updated.
New page [[frank-medium-batch-ubatch]]: the sweep's method (one invocation per
cell, a VRAM sampler per cell, 64k prefill measured AT 64k depth), the full
batch x ubatch matrix, the depth-0 control, the deployed-shape VRAM table, and
the limitations — including the one that cost me two runs, that amdgpu does not
return ~26 GiB at once, so a reload after a kill fails with "unable to allocate
ROCm0 buffer" and reads exactly like a configuration that does not fit.
AGENTS.md: the batch lesson now carries the measured curve (the peak at 512, the
10 % loss at 2048, the irrelevant logical batch, the invariant decode) instead
of the "~3.3 vs ~1.7 GiB" figure, which was measured at the bench's shape; at
the deployed shape the asymmetry is 2.41 against 1.30. Fleet row and the
frank-medium section now say 35:30 and batch 6144x512.
Runbook: deployed flags, the split history with the ubatch-512 table, the fault
note, and the verified VRAM (6.44 / 6.90 idle, 6.23 / 6.64 under two concurrent
requests).
Dual-R9700-Cooperation: the live capability table (card 0 ReqEn+ against card 1
ReqEn-, x1 effective link on card 1) and an honest label on the M.2_2 riser
premise — it is ASUS's spec sheet, not a measurement, and CPU-attached does not
imply Routing+ on this board (00:08.3 is CPU-side and Routing-).
log.md: the round trip recorded, newest first.
Asked whether Vulkan on both devices unlocks the split. It does not: the
per-tensor design aborts byte-identically, because the gate is not the driver
mix but `--override-tensor` — `has_tensor_overrides()` is a required term of
the scheduler's `pipeline_parallel` flag (llama-context.cpp:433, the only place
in the tree that reads it), and that flag sets `n_copies` in
`ggml_backend_sched_new`. With `n_copies == 1` a pre-allocated weight on a
non-primary device has no backend and the graph aborts.
Chasing it exposed a harness artifact that invalidated the earlier conclusion:
llama-bench splits a device LIST with `/` and SWEEP values with `,`, so
`-dev CUDA0,Vulkan1` was two single-device runs all along (and `-ts 0.85,0.15`
two sweep values, i.e. device 0 takes everything — which is where the bogus OOM
came from). With the correct syntax two-device splits work in both flavours:
pure Vulkan -dev Vulkan1/Vulkan0 -ts 0.85/0.15 -> 27.55 tok/s @64k (12.8 GiB on the 3090)
cross-vendor -dev CUDA0/Vulkan1 -ts 0.15/0.85 -> 27.23 tok/s @64k, pp512 268.5 (15.0 GiB)
iGPU alone 25.77 tok/s @64k, pp512 231.8
+7 % decode, +16 % prefill — real but not a regime change, and it costs the
3090 ~15 GiB, which this fleet spends on its only vision endpoint. The design's
premise still fails: whole layers carry their experts (1.62 GiB each), the 12
attention layers are spread every 4th, and the 32.8 GiB PLE must stay on the
main device, so KV cannot be consolidated on the 3090.
Also records the evening's incident (a dead rootless podman pause process took
all container management down; `podman system migrate` panics in 5.7.0; reboot
repaired it) and the netavark host-address lesson from the hermes 502.
The design we sketched (trunk + KV on the 3090, MoE experts on the iGPU) is not
expressible in mainline llama.cpp. Per-tensor cross-backend placement aborts in
the scheduler -- "pre-allocated tensor (blk.0.ffn_down_exps.weight) in a buffer
(Vulkan1) that cannot run the operation (NONE)" -- and both per-layer
cross-backend forms fail at load. CUDA + Vulkan cannot serve one model; the CPU
is the only fallback side that works, which is exactly why -ot exps=CPU runs.
What the same build does instead is the surprise: iGPU-only (Vulkan/RADV, PLE in
page cache) gives 32.64 tok/s shallow, 25.77 tok/s at 64k depth and 456.7 tok/s
prefill, with the 3090 idle at 0 %. That matches halogen's deployed ~25 tok/s --
but commits 116 of the host's 121 GiB against halogen's 95.4 held with 14.7
spare, so it is not deployable beside the co-tenants. Throughput parity, memory
bankruptcy: the fleet's two-endpoint split stands, now on evidence rather than
preference.
New page LLM/x4-pair-experiment.md carries the tensor inventory, the results
table, the two scheduler limits, reproduction commands, and the traps found on
the way. Both index pages also stop calling halogen retired -- the ik swap's
stale claims were still sitting in them.
The runbook for the new deployment, plus the sweep of everything that still
described the vllm-radiance replica pair: the vault index and its ACTIVE
marker, the superseded banner on the radiance page, the pointer from the
ROCmFPX page, the homepage card, the Grafana dashboard description and the
observability page's scrape table.
The new page carries the two facts a future agent needs and the flags do not
say: llama.cpp's layer split is what lets BOTH cards serve ONE model without
any collective (vLLM's PP handoff is an RCCL send/recv, which is the primitive
this host cannot dispatch), and the rocBLAS Tensile-path trap that makes a
healthy-looking build fail on the first request.
New page: LLM/Dual-R9700-Cooperation.md — registered in the vault index, with
the measured evidence rather than the theory.
**The answer.** Card #2 sits behind the B650 chipset, whose bridges advertise
`AtomicOpsCap: Routing-` (read-only), so ROCr refuses to fill the hostcall
buffer RCCL's kernels declare and no RCCL op can dispatch on it. Two paths fix
it: an M.2→PCIe x4 riser into the free, CPU-attached M.2_2 slot (~$30; per
ASUS's spec page M.2_1 and M.2_2 are CPU-attached, PCIEX16_2 is chipset in x1
mode, and there is no x8/x8 bifurcation on this board), or a board whose CPU x16
bifurcates to x8/x8. Bandwidth is not the constraint: ≈10 MB per token-step ≈
1.4 ms at Gen4 x4.
**New measured results.** A bare 2-rank torch.distributed run: rank 0 (CPU-lane
card) passes broadcast/all_reduce/barrier; rank 1 (chipset card) fails all
three, and `send`/`recv` fails too — which kills pipeline parallelism as an
escape, since vLLM's PP handoff is exactly that primitive.
**Correction.** I previously said no software path exists. That overshot. One is
open and unverified: recompiling RCCL so its kernels stop declaring a hostcall
buffer (`-mprintf-kind=buffered` is the claim). Counter-evidence recorded: RCCL's
device source contains no live printf/assert/__assertfail, so the trigger is
unexplained, and the claim traces to search summaries, not a primary source. The
page carries the two-command test that would settle it. Also corrected: 0.9.3 is
radiance's version, not vLLM's — the base is vLLM 0.27.1 / torch 2.11.0 / ROCm
7.14, and its shipped RCCL exposes no `ncclDevComm*`, so torch's symmetric-memory
path cannot run.
**Rule added to the benchmarking policy**: it covers ANY GPU experiment, not just
benchmarks — including a bare 2-rank probe. Running one against the live service
on 2026-09-16 preceded the host going hard down (no ARP, no ICMP, power cycle
required); a hung AQL dispatch wedging the driver is the plausible mechanism.
The policy also now says how to tell off from wedged: `ip neigh` from a peer.
Retiring dankclaw/compose.yaml and adding alert rules invalidated claims across
the repo, including in the rule file agents read first. Clean cutover includes
the prose.
**Tests that had gone hollow.** Four assertions grepped dankclaw/compose.yaml
for a service name. With the file gone they passed by *absence* — a gate that
can no longer fail. Each now asserts the real invariant ("no dankclaw compose
file competes with the quadlet units") and keeps a live check for a re-added
entry, so it still fails when one appears: verified by recreating a compose file
with a comfyui service and watching three tests go red, then removing it.
**AGENTS.md, which was wrong about ComfyUI.** It said ComfyUI runs CPU-only
("no ROCm wheels for gfx1151, CUDA images wrong"). That was true in the
compose era and is false now: the restored unit runs with /dev/kfd + /dev/dri and
must NOT pass --cpu. An agent trusting the rule file would have re-broken it.
Also corrected: the observability paragraph ("RUNNING BUT UNCONFIGURED, no
datasource, no rules" — all three have been configured since 2026-09-16), the
Active runbook row that still named halogen for frank-large, halogen's absence
from the Retired table, the benchmark policy naming halogen's harness, and the
seam list.
**Unit and page claims.** prometheus.container said "no alerting rules anywhere
in this stack"; node-exporter.container said Dankserver runs from its compose
stack; Prometheus.md documented three jobs as working (they scrape HTML: they
can never report up) and said no dashboard existed; index.md and the three
authentik service pages pointed at the retired file; Authentik.md's upgrade
instruction told you to bump an image tag in it.
One bad edit of my own on the way (a stale line number clobbered a See Also
entry in Prometheus.md) — caught by reading back, fixed, and the vault link
checker now proves the cross-references hold.
The radiance MXFP4 stack cannot run TP=2 on this host. RCCL's collective
kernels carry a hidden hostcall-buffer parameter, and ROCr refuses to
dispatch them on a device without PCIe atomics: the AQL dispatch fails with
hipErrorIllegalState ("the operation cannot be performed in the present
state"), which is what every two-rank collective hit. The second R9700 sits
behind the chipset bridges 05:00.0/06:09.0, which advertise
`AtomicOpsCap: Routing-` — a read-only capability, so no software can enable
atomic routing on that path and the driver's
pci_enable_atomic_ops_to_root() fails for that card. The first card, on CPU
root port 00:01.1, has atomics enabled.
Ruled out: kernel version (7.0.0-28/-30/-31 identical, after a full upgrade),
iommu=pt, P2P hardware (45.3 GB/s between the cards), the shipped librccl (a
from-source rebuild reproduces the failure at the same call path), vLLM and
pynccl (a bare two-rank torch.distributed allreduce fails the same way), all
NCCL transport knobs, and single-rank tests (NCCL short-circuits at
world_size=1, so one-card success proves nothing).
Deployment: three rootless Quadlets — frank-medium-a (:7771, card 1),
frank-medium-b (:7772, card 2), both TP=1 at 114688 context (262144 and
131072 do not fit one card's KV) with vLLM-profiled KV — and
frank-medium-proxy (:7770), a round-robin that preserves the fleet contract
of one endpoint and one served model name. frank-medium.container stays as
the TP=2 target for a host whose second card is atomic-capable; the deploy
handler does not install it and trashes any stale copy.
Measured through the proxy (vllm bench serve, random dataset, --ignore-eos):
conc 1 42.8 tok/s, conc 4 147.6 tok/s, conc 8 230.8 tok/s, 32k prefill at
~2.8k tok/s per card, TPOT p50 23.6-28.8 ms.
Also: healthcheck reads PORT so one probe serves all three units, the hermes
example and the harness models.yml drop to the real 114688 context, and the
runbook records the full evidence chain.
- Runbook with the measured envelope (prefill ~192 t/s at the VNNI ceiling;
decode ~3.8-4.3 t/s gated by 100 backend splits/token, GPU 95% idle), the
root-cause evidence chain, tuning levers, and co-tenancy traps
- AGENTS.md: fleet topology row + Active Services section for the ik
deployment; halogen demoted to rollback; bench policy unit type fixed
- LLM + root indexes relinked (frank-large-ik-uncensored ACTIVE,
halogen-flash-server retired)
New runbook page serverAppConfigs-docs/LLM/frank-medium-rocmfpx.md records
the hardware swap, the self-contained gfx1201 build, the measured layer-vs-
tensor decision (llama-bench multi-device syntax gotcha included), the
tensor-mode draft abort, served-path numbers, and the post-cleanup host ROCm
posture. AGENTS.md, README, docs indexes, Dankserver System Specs,
deployment/Hermes/OpenWebUI/Plex/Stack pages follow the rename;
freetoken-frank-small is marked RETIRED and kingjones-q8agent SUPERSEDED.
Work log entry added.
Raised /swap.img from 8 GiB to 64 GiB on Dankclaw. Host-level state with no
repo file, so it was invisible to the repo until now: fstab line 12 drives the
generated swap.img.swap unit, ext4, fallocated hole-free, swappiness 60,
zswap off. 748 GiB still free on /.
The reusable part is the ordering. The old file held 4 GiB of live pages and
MemAvailable read 73 GiB while only ~5 GiB was genuinely free -- the pinned
model weights are counted as reclaimable page cache, so the headline lies by
the size of the model. Swapping off before the replacement is online would have
pushed those pages into RAM that does not exist. Creating and enabling the new
swap first let them migrate instead: swapoff took 65 s and the host never
wobbled.
New page serverAppConfigs-docs/Dankclaw/Swap Configuration.md, linked from
the index, plus a pointer in the AGENTS.md memory-model section so nobody
reasons about headroom without checking swapon.
Old 8 GiB file staged at dankclaw:~/.AGENTTRASH/20260912-swapfile-8g/
pending approval to purge.
Rewrites the [[Observability]] umbrella page and the five component pages for
what is actually deployed: rootless Quadlet units instead of compose services,
the loopback-only publish posture (node-exporter's LAN binding explained), the
loki healthcheck living in its image because the image has no shell, and the
Home=/root trap that killed the first grafana start.
Adds the note that matters most to a reader: the stack is installed, healthy
and UNCONFIGURED. Grafana reports zero datasources and zero dashboards with an
empty provisioning directory; Prometheus has no rule files and no Alertmanager;
Loki is configured but has no reader; node-exporter runs its default
collectors. Alloy is the only configured piece. Metrics and logs are collected
and retained, and nothing is visualized or alerted on.
AGENTS.md's boot-persistence gap note is replaced by the resolution.
frank-large moves from the kingjones30/ROCmFPX llama.cpp build to
peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b
.hgn 115.55 GiB + quality overlay). Upstream measures this engine at
roughly 4x the prefill throughput of the llama.cpp family on the same
gfx1151 silicon, and prefill dominates any prompt with real context.
Shape: a rootless Quadlet container (dankclaw/frank-large/quadlet/),
engine on container loopback :8730 (never published), OpenAI front-end
:8731 published as host 7770, HALOGEN_MODEL_ID=frank-large so no client
renamed anything. Context 262144, KV pool 262144 positions (~7.2 GiB,
conservative for a shared 121 GiB host), 4 slots, host reserve 24,
vision tower on, weights pinned, all other flags at vendor defaults.
Weights are fetched by HALOGEN_DOWNLOAD on first start into a writable
/home/dank/models/halogen.
The llama.cpp era (unit, env, serve script) is staged in .AGENTTRASH/
rather than deleted; the STRIX payload and the ROCmFPX build stay on
disk as the rollback target. deploy_llm_large becomes a user-quadlet
handler: it retires the old ROOT unit (moving its checkout symlink into
the home trash, never rm) and gates the first start on the download
rather than reporting a failure for a fetch that is working exactly as
designed.
frank-npu is retired in the same change. The kernel cmdline moves from
iommu=pt to amd_iommu=off for the 13-16% of prefill upstream measured,
and that takes the XDNA2 NPU with it: without PASID/SVA the driver
cannot open /dev/accel/accel0 (err=-19). Its five dependent routes are
repointed (Hermes vision -> frank-large, Hermes title generation ->
frank-small, harness vision -> frank-large, tiny -> frank-small, task ->
frank-large:medium) and its provider blocks are removed from both the
harness config and Hermes's live config. Audio input leaves the fleet
with no replacement; image input survives on halogen's vision tower.
Docs: new halogen-flash-server runbook, new IOMMU decision record, and
frank-npu rewritten as a retirement record. AGENTS.md topology, runtime
sections, benchmarking policy (frank-large is a user unit now) and the
harness model-config section are corrected.
Mechanical identity rename across the knowledge base, AGENTS.md, README, the
hermes config example, and homepage services.json. No behavior change; served
aliases on the LLM endpoints change with the deploy of commit 69f87c8.
New page Architecture/Container Network Namespaces.md: which local services are
addressable from which Podman network namespace on Dankclaw (rootless podman +
netavark). Verified live 2026-09-02: gateway/webui/CDP run in the host netns and
dial loopback; hermes-dashboard is the only Hermes container on a bridge network
(dankclaw_default, 10.89.1.0/24 gw 10.89.1.1) and can dial published ports only —
unpublished host-netns listeners (signal-cli :8080, webui :8787) are refused from
that namespace even though both bind 0.0.0.0.
Corrected while verifying the above:
- Hermes.md: data root is /home/dank/Containers/hermes/data (not the retired
/home/dank/hermes-data); gateway dials the Signal daemon on loopback; the
daemon itself is a host user unit (signal-cli-daemon.service) bound to
0.0.0.0:8080, so the Signal HTTP API is LAN-reachable.
- Hermes-Dashboard.md: it is a Quadlet unit (Network=dankclaw_default), not a
compose service; data root + /opt/data anonymous volume corrected.
- Hermes-WebUI.md: binds 0.0.0.0:8787 in the host netns (verified: unauthenticated
GET /health from another LAN machine returned 200) — the SSH tunnel is an access
path, not a containment boundary; HealthCmd and data-path corrections.
- Network Architecture.md: no hermes vhost exists in either the tracked or the live
Caddyfile (hermes.franksdank.cloud resolves but routes nowhere); SearXNG exits the
tunnel via the shared named network searxng.network, not service:gluetun.
Docs only — deploy is a no-op for this change.
Document the Mac *arr/qBittorrent stack (VM placement, gluetun credential
gate, socat LaunchAgents, PBKDF2 qB seed) and log the day: dankserver
recovery + full quadlet cutover of the six media services, the frankbook
skip-propagation/PBKDF2 handler fix, the 7-service frankbook deploy, and
the jellyfin GPU-parity finding.
Env file is the source of truth (unit ExecStart runs the serve script which
sources it); serve-script default and the deployer health-gate URL flipped in
lockstep so the post-restart health gate polls the new port. Docs, fleet table,
and both index bullets updated to match.
frank-default-sightsound (FLM v1.0.1, XDNA2 NPU, port 52626) is live on
Dankclaw. Documents the first-deploy crash-loop root cause: iommu=off on the
kernel cmdline breaks the XDNA2 SVA bind (amdxdna_drm_open ret -19), so every
/dev/accel/accel0 open fails with ENODEV; fixed with iommu=pt + reboot.
Whole-registry single-port serving with per-request routing by exact FLM tag;
unknown tags fall back to llama3.2:1b (no 404). Health = GET /v1/version.
Vision + audio towers proven live (image_url color probe; audio control-pair
token-count probe). Updates AGENTS.md fleet table + runtime section, both
index pages, and the change log.
Model: agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF (87.06 GiB,
Q4_0_ROCMFP4_FAST imatrix-calibrated, per-head PLE table inside the GGUF —
fully VRAM-resident). Draft: agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST
(2.28 GiB, matched ROCmFP4-FAST quant, ~66% acceptance at n-max 3).
Vision tower (f16) loaded via mmproj/.
Runtime: charlie12345/ROCmFPX build-qwen38 (HIP, ROCm 7.2.4, gfx1151) —
the only binary with the ROCmFP4 FAST types. Replaces the jcbtc CIRU v1.1
deployment (ciru-ai runtime).
Profile: card sampling table (temp 0.7, top_p 0.8, top_k 20, min_p 0.0,
presence 1.5) mixed with the validated build-qwen38 serving profile:
262144 ctx x 1 slot, b 4096 / ub 4096, t 16 / tb 16, f16 KV, fa on,
cont-batching, prompt cache + 8 GiB cache-ram + idle slots, 16 ctx
checkpoints (no --checkpoint-min-step — unsupported on this binary),
--fit off (auto-fit segfaults on qwen4exp hyper-connection config),
--spec-type draft-mtp,ngram-mod with n-max 3 (adaptive p-split measured
worse per author card), p-min 0.75. Qwen-Sharp chat template kept;
reasoning stays in DeepSeek format.
The 2026-08-28 failure of this draft head on the kingjones STRIX_LEAN
model was a cross-vendor pairing problem: the head carries
blk.48.hc_attn_norm.weight (verified in-GGUF, same shape as the working
CIRU draft). Docs: new runbook agentionai-fast-imatrix, index updates.
Live-verified 2026-08-29: plexmediaserver.service is an enabled native
systemd unit on Dankserver (not a quadlet; updates via apt, not deploy.sh),
and tautulli runs as a snap with no auth wall on 0.0.0.0:8181. The hunt list
named both; neither had a page. Index linked per the docs-completeness seam.
Three compounding deploy-handler bugs found by live deployment:
- podman image exists/pull used a mixed-case ghcr.io ref that podman's
reference parser rejects before any network I/O (repository name must be
lowercase); all refs now lowercase — ghcr resolves case-insensitively.
- the unit runs the image under a localhost/ alias tag: this host's podman
5.7 quadlet path fails container create with 'image not known' for
registry-prefixed refs even when present in storage (verified live).
- the health probe was shell-form and the distroless ko-build image has no
shell; HealthCmd is now a JSON exec-form array, which passes through the
quadlet generator's argv assembly verbatim.
Tests updated to assert the alias-tag reality plus the pull guard.
All in-repo path references (AGENTS.md, README, old scripts, report artifacts,
test code paths) rewritten in the same commit; focused-test map extended for
.gitignore, old/scripts/*, dankclaw/comfyui/*, and report/* per this repo's
own convention ('add one before committing'). Obsidian wikilinks are
title-based and unaffected.
Frank
