docs(frank-tts): renderD128 is the NVIDIA node — pass /dev/dri, not a guessed number

The trial's quadlet line said AddDevice=/dev/dri/renderD128. Resolved the nodes by
driver instead of assuming: on this hybrid host renderD128 -> nvidia and
renderD129 -> amdgpu, so the guess pointed the engine at the wrong card. Also
records the gate-1 command that works (vulkaninfo --text; --batch prints help and
exits 1) and the probe script path.

AGENTS.md

@@ -47,7 +47,7 @@ per attempt and no separate swap-history file.
   The managed skill library is distilled there one page per skill-cluster:
   `LLM/index.md` → *Cross-Cutting References*, plus `Operations/index.md`,
   `Agent/index.md`, and the Infrastructure/Dankclaw/FrankBook-Pro service
-  pages (`serverAppConfigs-docs/log.md` carries the wave plan and results)
+  pages. There is no `log.md`: the change diary was destroyed 2026-10-19
 - `tests/` — bats suite + Playwright e2e
 
 ## Git Workflow — trunk-based (fleet law, aligned 2026-10-18)
@@ -318,35 +318,36 @@ the next), so never report request #1.
 deletes the unit file from `/etc/systemd/system/`, not just the symlink.
 `mask` symlinks to `/dev/null` and is safe/reversible.
 
-## Documentation Policy (fleet law, 2026-10-18)
+## Documentation Policy (fleet law, 2026-10-19)
 
-**One fact, one owner.** A page owns exactly one thing — an endpoint, a
-service, or a technique — and every other file links to it with `[[Page]]`.
-Restating a shape, a digest, a port list, or a measured number in a second
-file is a defect: it is the next thing to go stale.
+**Documentation exists for point of use.** A page earns its bytes only if it
+helps whoever performs the next task — the shape to deploy, the flag that must
+be passed, the failure signature to recognise, the rollback to run. It is not a
+lab notebook and not a ledger. **Losing information is an acceptable, usually
+correct outcome.** Never let an orphan census ("this is the only copy of that
+fact") decide what to keep: sole custody of a useless fact is still useless.
 
-- **One page per endpoint, forever:** `serverAppConfigs-docs/LLM/<endpoint>.md`
-  — current shape, an **Eras** table (`window | payload | runtime | shape |
-  replaced because | rollback`), the **Laws** the endpoint taught, the wire
-  contract, and the files that own it. A payload swap edits the shape block
-  and appends one Era row. It never creates a page, and the retired era's
-  page goes to `.AGENTTRASH/` once its rollback path and its laws are in the
-  table.
-- **`log.md` records incidents and laws** — anything with a causal story
-  (why the host went down, why a knob resets at reboot, why the RAM was the
-  fault). A model swap is not an incident: it is an Era row and nothing else.
-- **`AGENTS.md` carries operator-critical numbers only** — port, context
-  ceiling, slot shape, unit/env/serve trio — plus the cross-cutting laws.
+- **The purpose test — before you write, and again before you keep:** would the
+  reader act differently because of this line? No → delete it.
+- **Measurements are disposable.** A rate or throughput figure (t/s, tok/s, ms,
+  acceptance %, a bench ladder, a measured MiB) for a payload, runtime, or
+  layout that is not running now and not being compared now is destroyed, not
+  filed — not in an Eras row, not on a technique page, not "preserved for
+  history". AI moves faster than this vault: the number is wrong before anyone
+  reads it, and a wrong number in a runbook gets believed. Keep the **law** the
+  measurement taught; drop the measurement.
+- **One fact, one owner**, every other file links with `[[Page]]`. The env file
+  beside the serve script is the authority for every live number; the endpoint
+  page is the authority for the shape and the wire contract.
+- **One page per endpoint:** current shape, **Eras** (`window | payload |
+  runtime | replaced because | rollback` — the rollback is a commit, not a
+  number), the **Laws**, the wire contract, the files that own it. A swap edits
+  the shape block and appends one row; the retired era's own page is
+  **destroyed** to `.AGENTTRASH/`, never merged into anything.
+- **`log.md` is retired** (2026-10-19: 2,380 lines destroyed). Git holds the
+  chronology. A causal law belongs to the page that owns the thing it governs,
+  or to *Key Lessons Learned* below if it is cross-cutting — never to a diary.
 - **Index pages are locators:** one row, one clause, no numbers.
-- **A measured number is written once:** in the env-file header beside the
-  serve script that consumes it, and in the endpoint page. Every other page
-  cites it by link.
-- Technique pages (spec decode, KV math, offload budgets, PP bandwidth) stay
-  technique pages: they stop carrying a per-endpoint throughput table or a live
-  shape, but a campaign ledger that exists nowhere else is owned by the page that
-  measured it. The defect is two pages carrying one figure, not a number living on a
-  technique page — move a figure only when an endpoint page or an env header is its
-  home; otherwise leave it in place and name the era it measured.
 
 ## Model Config (harness)
 
@@ -487,36 +488,10 @@ linked runbooks.
   6 set. AMD desktop platforms put one card on CPU root ports (atomics work) and the rest behind the
   chipset (usually not) — that asymmetry is what blocked TP=2 here. A one-card collective proves
   nothing: NCCL short-circuits at `world_size=1`. [[PCIe Link and Atomics]]
-- **That wall was physical, and a board swap removed it on 2026-09-20.** dankserver took a board
-  whose CPU lanes bifurcate to x8/x8 Gen5: both cards on root ports, every bridge `Routing+`, BOTH
-  endpoints `ReqEn+`. Bandwidth was never the constraint (a 27B TP=2 allreduce at batch 4 is ≈10 MB
-  per token-step, under 0.4 ms at Gen5 x8); the collective choice is — TheRock's RCCL builds lack
-  gfx1201 device code (`invalid device function`), which is why vLLM runs its R4D one-shot allreduce
-  here at all. Evidence matrix and the three-arm verdict: [[Dual-R9700-Cooperation]].
 - **When a GPU collective dies, read the runtime, not the framework.** The message that identified
   the cause came from `AMD_LOG_LEVEL=3` (`Pcie atomics not enabled, hostcall not supported` →
   `AQL dispatch failed!`), never from vLLM or RCCL — the env var costs one line and prints the
   failing HIP call with its return code. [[PCIe Link and Atomics]]
-- **Layer split is the no-collective path, on one vendor's cards and across two — and
-  `--override-tensor` is what cannot ride with it.** Layers go in order and only the boundary
-  activation crosses PCIe once per forward pass, so it works wherever a collective cannot be
-  dispatched (on an RCCL-less fork it beats tensor mode, which falls back to a butterfly allreduce),
-  and it is the cross-vendor answer too (a CUDA + Vulkan split held a real two-device graph). But
-  `--override-tensor <group>=<device>` sets `has_tensor_overrides()`, a REQUIRED term of the
-  scheduler's `pipeline_parallel` flag, and that flag sets `n_copies`: with `n_copies == 1` a
-  pre-allocated weight on a non-primary device has no backend and the graph aborts (reported with
-  the leaf's op name `(NONE)`). `=CPU` is the only override target that works — the CPU backend is
-  last in the list — and pure Vulkan changes nothing: the gate is the override, not the driver mix.
-  Pin the spec-decode drafter to one device; on a split its home is the card holding the target's
-  last layers. [[Llama Cpp Multi-Device Splits]] [[x4-pair-experiment]]
-- **A host↔3090 link at x4 measures 6.72 GB/s, and that is a design constraint, not a detail.**
-  Pinned H2D 6.72 / D2H 6.60 GB/s (≈85 % of the Gen4 x4 ceiling), pageable within 1 % of pinned —
-  the copies are LINK-limited, so the usual "pageable costs 2×" rule does not apply; the x4 lives in
-  the adapter or cable (card and port are both wider). For any engine that STREAMS weights across
-  it: a 3 MiB slab costs 474 µs, so a 111 GiB A3B model needing ~1.70 GiB of active experts per
-  token lands at 3.7 tok/s against 20.46 tok/s with the experts COMPUTED in host RAM
-  (`-ot exps=CPU`, where the link carries activations only). Streaming loses; RAM does not care.
-  [[x4-pair-experiment]]
 
 
 **ROCm / GPU**
@@ -530,7 +505,7 @@ linked runbooks.
   the destination, then copy the contents.
 - **mmap must stay on** for the big-PLE models: `--no-mmap` forces the ~50 GiB PLE table into
   anonymous memory → silent cgroup kill, with dmesg as the only evidence. mmap pages ARE page cache,
-  and pinning the PLE tensor with `-ot` cost 23 → 13.4 t/s. [[Llama Cpp Serving Sizing]]
+  and pinning the PLE tensor with `-ot` was slower than letting it page. [[Llama Cpp Serving Sizing]]
 - **HIP reads bare env vars** `TEMP`, `HOST`, `GPU_MAX_HEAP_SIZE`. Serve scripts export env files
   with `set -a`, so a tunable named `TEMP` leaks into HIP (`TEMP=1.0` sent the JIT cache to path
   `1.0` and segfaulted every load, fixed 2026-08-13 by renaming to `TEMPERATURE`). Never name a
@@ -545,9 +520,6 @@ linked runbooks.
 - **Flash-Next MTP heads do not load on any current loader** — they need full per-layer
   `blk.<il>.hc_attn_norm.weight` and ship only `blk.48` (verified 3×). Do not re-attempt without new
   loader support.
-- **QSA flash-attn assert:** `GGML_ASSERT(a-type == GGML_TYPE_F32)` on qwen4exp needs
-  `scripts/patches/qwen4exp-qsa-f32-mask.patch` — but only on the agentionai FAST layer set; the
-  kingjones STRIX set works unpatched. [[Llama Cpp Serving Sizing]]
 - **The GPU budget on Dankclaw is the GTT aperture, not the rocm-smi "dedicated" figure.** The
   Radeon 8060S (gfx1151) reports 116,732 MiB (`llama-server --list-devices`; `gtt_total` in
   `/sys/class/drm/card1/device/`), and the older "64 GiB, SVM-backed" lesson was already stale.
@@ -602,13 +574,11 @@ linked runbooks.
   `LimitMEMLOCK=infinity` — systemd is privileged and the value flows through podman to runc without
   any raise. Upstream halogen's own run lines pass NO `--ulimit` at all: copy vendor run lines
   before adding flags. [[frank-dragoon]]
-- **A GGUF's custom tensor type picks the runtime — read the histogram before promising a file to a
-  loader.** `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` is ftype 115 over 311 tensors of type 103
-  (`GGML_TYPE_Q8_0_ROCMFPX`), so stock llama.cpp and gufo refuse it at load and the fork was not a
-  preference but the only door. Counting tensor types out of the GGUF KV section takes one python
-  read and settles a runtime argument in minutes. Build identity matters as much as the fork: the
-  2026-08-25 ROCmFPX builds SIGSEGV in `ggml_compute_forward_set_rows` on this same file, so
-  `scripts/build-rocmfpx.sh` pins commit `36e9acd` by SHA. [[GGUF Compatibility]]
+- **A GGUF's custom tensor type picks the runtime — read the histogram before promising a
+  file to a loader.** A non-standard tensor type in the KV section makes stock loaders refuse
+  the file at load, so a fork can be the only door rather than a preference. Counting tensor
+  types takes one python read and settles a runtime argument in minutes; build identity matters
+  as much as the fork, so pin the commit by SHA in any build script. [[GGUF Compatibility]]
 - **Speculative-decode flags move as a SET.** Handing the fork `--model-draft` with no `--spec-type`
   (the naive way to build a "no MTP" bench arm) segfaulted at load, core dumped, no diagnostic. A
   baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND `--spec-type`, or it
@@ -617,12 +587,6 @@ linked runbooks.
   exception kept ~26 GiB of weights resident on a 121 GiB box and pushed `available` to 3 GiB, so
   every later rate was suspect. Record the child pid, kill it in a `finally`, then prove `free` came
   back before believing a number — and kill by pid, never `pkill -f llama-server`.
-  [[LLM Bench Windows and Measurement]]
-- Retired-runtime traps that can come back: **Kairic Edge** needs
-  `third_party/composable_kernel/include/ck/config.h` generated manually (untracked, so `git clean`
-  removes it) and its greedy-argmax fastpath rejects non-greedy requests and reasoning budgets, so
-  re-test chat before production; the **FastMTP sidecar** must be commit `4df29be4` + the HauhauCS
-  patch, since stock binaries fail the draft load with a tensor-shape mismatch.
 
 **llama.cpp serving**
 
@@ -635,10 +599,10 @@ linked runbooks.
   `--spec-type a,b` chains in a fixed internal order (CLI order irrelevant) and the gate is
   `ctx_dft`: the ngram family is added unconditionally while `draft-mtp`, `draft-dflash`,
   `draft-dspark` and `draft-eagle3` all need a draft CONTEXT. So `draft-dflash,ngram-mod` is
-  legal (measured 2.6x the drafter alone on prompt-echo traffic) but `draft-mtp,draft-dflash`
+  legal and it wins on prompt-echo traffic, but `draft-mtp,draft-dflash`
   dies at load — *"context type MTP requested but model doesn't contain MTP layers"*. **A LEGAL
   chain can still lose on a recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35
-  hybrid measured −15 % prose / −7 % block-copy, because adding the ngram layer logs
+  hybrid loses, because adding the ngram layer logs
   `recurrent state rollback is not compatible with 'ngram-mod' - disabling rollback support` and
   MTP verification depends on that rollback. Measure chains per architecture, never by analogy.
   [[frank-dragoon]]
@@ -647,7 +611,7 @@ linked runbooks.
   declaring NO sliding window caches the whole window on `--spec-draft-device` and can OOM the
   first draft graph capture AFTER `/health` has answered. Read `sliding_window` out of the
   drafter's GGUF header before sizing, and prefer a skewed `-ts` over a context cut.
-  [[Llama Cpp Multi-Device Splits]]
+  [[Llama Cpp Serving Sizing]]
 - **A draft context bills its own f16 KV over the WHOLE POOL, not the slot**, plus its own
   compute buffers, so on a single-card endpoint that squeeze can cost more than the drafter
   wins. The MTP question is never "does the file carry the head" but what draft KV + buffers
@@ -664,21 +628,20 @@ linked runbooks.
   re-measure the balance after any batch change. The logical `--batch-size` is nearly free; a
   large-batch allocation can fail on BUFFER CONTIGUITY while totals still fit, which no totals
   model can see. A prompt shorter than the ubatch cannot see the knob at all — that is the
-  control that proves the knob, not the traffic, moved the number. The 2× R9700 split peaks at
-  512; the CUDA 3090 has no interior optimum up to 2048. When a bench IS ordered, A/B it by
-  restarting the SERVICE through a transient `FRANK_<SVC>_ENV_FILE` drop-in, never a scratch
-  port: one card holds one server. Ladders: [[frank-medium-llamacpp-dflash2]] and
-  [[frank-zealot]].
+  control that proves the knob, not the traffic, moved the number. The optimum is a property of
+  the backend, so re-find it after any device or payload change. When a bench IS ordered, A/B it
+  by restarting the SERVICE through a transient `FRANK_<SVC>_ENV_FILE` drop-in, never a scratch
+  port: one card holds one server.
 - **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same size** (a DeltaNet
   boundary and a full-attention boundary can differ by ~1 GiB each with its own KV), so balance
   by measurement. An affine fit reproduces the deployed point and overstates the gaps in between.
-  [[Llama Cpp Multi-Device Splits]]
+  [[Llama Cpp Serving Sizing]]
 - **`ngram-mod` prints no startup banner.** The load log names `draft-dflash` and nothing for the
   matcher, so a chain can look unengaged while it works. Identify it behaviourally (`draft_n` > 0
   only where text repeats) or by the per-task `draft acceptance = ... mean len = ...` line. It
   costs no VRAM (a 64 MiB host-RAM hash table) and its window decides which traffic it can see:
   too short and it proposes on ordinary prose fragments, costing verification without acceptance.
-  [[Llama Cpp Multi-Device Splits]]
+  [[Llama Cpp Serving Sizing]]
 - **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the per-slot number is
   the contract — not `ctx-size`.** `n_ctx_slot = CTX_SIZE / PARALLEL`, and llama.cpp refuses a
   prompt longer than the SLOT with a 400 even when the pool has room, so `models.yml`
@@ -793,17 +756,24 @@ linked runbooks.
   `curl -o FILE URL` exits 0 on HTTP 401/403 and writes the error body as the "artifact": a gated
   repo yielded three 153-byte shards with identical sha256 and a cheerful `DOWNLOAD_DONE` in 20
   seconds for 104 GiB. Use `-f`/`--fail-with-body`, compare each file's byte count to the Hub's
-  declared size, and check the LFS sha256 (`scripts/download-frank-medium-gguf.sh` does all three;
-  ad-hoc fetches tend not to). Gated repos 403 even with the host's HF token until that account
-  accepts the gate. [[Model Swap and Rollback]]
-- **A scout verdict is a lead, never a finding.** In the 2026-10-19 vault de-dup campaign six
-  read-only scouts worked the same pages: one classified era pages as stale *because* they
-  document retired payloads (that is an era page's content), one reported files that do not exist,
-  one reported duplications that grep disproved (`457.9 t/s` and the ping-RTT table are solo-held
-  by `Ethernet Bandwidth Math`), one cited a line number past the end of the file it judged, and
-  one drafted an Eras table with invented windows, payloads and rollback paths. Re-verify every
-  claimed duplicate, every claimed staleness and every quoted holder against the file and today's
-  fleet state before editing; a scout yield buys no evidence.
+  declared size, and check the LFS sha256 — the live `scripts/download-frank-*.sh` scripts do
+  all three; ad-hoc fetches tend not to. Gated repos 403 even with the host's HF token until
+  that account accepts the gate. [[Model Swap and Rollback]]
+- **A verdict is a lead — the scout's and the tool's alike.** In the 2026-10-19
+  documentation campaign six scouts worked the same pages and produced invented
+  bullet IDs, files that do not exist, "duplications" grep disproved, line
+  numbers past the end of the file they judged, and Eras tables with invented
+  payloads. The mechanical tool was worse: `scripts/port-census.py` built its
+  corpus from the directory containing the page under test, so each page was held
+  by *itself* and ten era pages reported "0 sole-record lines" — which reads as
+  "pure duplication, trash it". Nothing was cut on a verdict; every holder was
+  re-grepped and every tool re-measured before it was believed.
+- **Never let a census decide what to keep.** That same campaign then concluded
+  the era pages were irreplaceable because each held 14–63 sole-record lines —
+  wrong in the opposite direction: those were throughput figures for payloads that
+  no longer run. Point of use is the only ownership test, the operator applies it,
+  and 53 pages plus the 2,380-line change diary were destroyed as a result. A tool
+  can say where a fact lives; it cannot say whether anything needs it.
 
 ## Runbook Index
 
@@ -818,12 +788,13 @@ per-attempt page and no per-swap ledger anywhere else.
 | `frank-zealot` — llama.cpp (PrismML fork), RTX 3090 | `serverAppConfigs-docs/LLM/frank-zealot.md` |
 | `frank-tts` — cloned-voice speech, Dankclaw | `serverAppConfigs-docs/Dankclaw/Frank TTS.md` |
 | Dankclaw kernel posture (`iommu=pt` since 2026-10-18) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
-| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
+| Deployment flow | `serverAppConfigs-docs/Architecture/Deployment Flow.md` |
 
-Retired **endpoints** (not payloads) keep their own pages — `frank-medium`,
-`frank-npu`, the pre-name Dankclaw/Dankserver eras — because no live page owns
-them; `serverAppConfigs-docs/index.md` is the locator for every page in the
-vault. Retired scripts live in `old/` and `.AGENTTRASH/`.
+There is no page for a retired endpoint and no change diary. A payload swap
+appends one row to the live endpoint's **Eras** table; when the endpoint itself
+dies, its page is destroyed. Rollback is a commit, not a document.
+`serverAppConfigs-docs/index.md` is the locator for every page that survives;
+retired scripts live in `old/` and `.AGENTTRASH/`.
 
 ## Why
 

report/npu-tts-runtime-landscape-2026-10-19.md

@@ -0,0 +1,319 @@
+# XDNA2 NPU inference runtimes on Linux (Strix Halo) — can any of them serve TTS?
+
+Research slice, 2026-10-19. Worker: `NpuRuntimeLandscape`. Research only, no code.
+Question: every way to execute model inference on an AMD XDNA2 NPU on Linux as of
+2026-10, and whether any of them can serve a speech-synthesis model.
+
+> **Status, same night — the spike that followed this slice settled it.** NO-GO on the
+> NPU (the loader declines the graph, not the silicon); the actionable door is the gfx1151
+> iGPU behind `-DGGML_VULKAN=ON`. Measured contract, exoneration chain and host gates:
+> `tts-npu-spike-verdict-2026-10-19.md`. That page supersedes §8–§9 below.
+
+Convention: vendor claims carry URL + the version/date they hold for.
+**[host]** = Dankclaw state verified on the live machine 2026-10-19 (Main), not a
+vendor claim. `[INFERENCE]` = my deduction from the cited artifacts.
+
+## 0. Answers to the three questions Main ranked
+
+**(1) A runtime that already runs TTS on XDNA2 today: it exists at AMD, not on our
+host.** Ryzen AI Software **1.8.0** (docs `latest` == 1.8.0; released 2026-07-23)
+lists `neutts-air` and `neutts-nano` as model type **TTS**, variant
+*full-optimized* (NPU-only) — <https://ryzenai.docs.amd.com/en/latest/llm_list.html>;
+release note "New TTS model support: neutts-air, neutts-nano" —
+<https://ryzenai.docs.amd.com/en/latest/relnotes.html>. Pre-compiled artifacts are
+published: `amd/neutts-air_rai_1.8.0_npu_4K` (arch `qwen2`, lastModified
+2026-07-23, Apache-2.0, "AWQ / Group 128 / Asymmetric / BFP16 activations / UINT4
+Weights") and `amd/neutts-nano_rai_1.8.0_npu_4K` (arch `llama`).
+**Neither artifact is a TTS engine.** Both file lists are LM-only (`model.onnx`,
+`genai_config.json`, BPE tokenizer, AIE-ML compiler metastate: `dd_metastate_*`
+`.ctrlpkt/.super/.state`, `cache/txn_bins.zip`) with **no codec, no vocoder, no
+reference-audio encoder**. [INFERENCE from HF file lists]
+
+**(2) FastFlowLM: no TTS, and we cannot add one** — see §2. Halogen: no TTS entry
+**[host]**. So "something installed already serves TTS on the NPU" is FALSE.
+
+**(3) Our own ONNX graph compiled for the NPU on Linux:** possible for the plain
+ONNX flows (they compile on Linux), **impossible for a new LLM/OGA model** — model
+generation is Windows-only in 1.8.0 — and gated by an operator table where BF16
+lacks Softmax/LayerNorm/ReLU/Gelu (see §4).
+
+## 1. Landscape: every way to reach the XDNA2 array on Linux
+
+| # | Path | License | Strix Halo + Linux, 2026-10 | Speech synthesis |
+|---|------|---------|------------------------------|------------------|
+| 1 | Ryzen AI SW → ONNX Runtime + Vitis AI / Ryzen AI EP; compile INT8 (`X2`) or BF16 (`VAIML`) ONNX at session creation | AMD proprietary (tarball) | supported: our device ID/rev is in AMD's STX table; compile+run works on Linux (§3, §4) | user-authored ONNX only; no TTS recipe/demo |
+| 2 | Same EP driven by **OGA** (`onnxruntime-genai`), "full fusion" NPU-only LLM flow; runner = AMD `model_benchmark` / OGA C++ API | AMD + MS OGA | Linux: **vendor-precompiled models only**, no hybrid flow | **neutts-air / neutts-nano = TTS** (LM half) |
+| 3 | **FastFlowLM** `flm` | MIT + free proprietary kernels | Strix Halo "Supported"; Ubuntu 24.04/25.10/**26.04**, Arch | none (ASR input only) |
+| 4 | **IRON / mlir-aie** + Peano → hand-written AIE-ML design → XRT/`amdxdna` | Apache-2.0 + LLVM exception | explicitly documented "AMD Ryzen™ AI on Linux"; same driver/XRT stack | anything, hand-written; nothing ships TTS |
+| 5 | Vendor-curated AIE-ML graph sets (halogen `HALOGEN_NPU_MODELS`, FLM packs) | vendor | present **[host]** | none **[host]** |
+| 6 | Windows-only: DirectML, Ryzen AI Windows EP (`ryzenai-llm`), Studio Effects, `whispercpp` NPU | — | not usable | (per Lemonade matrix: Windows only) |
+| 7 | ggml/llama.cpp/**qwentts.cpp**, whisper.cpp, sherpa-onnx, Piper | — | no XDNA backend in ggml/ORT-mainline | CPU / iGPU only |
+| 8 | IREE, TVM, ONNX Runtime mainline EPs (OpenVINO = Intel NPU, MIGraphX/ROCm = AMD GPU) | — | no XDNA2 target; AMD's NPU EP is only "Xilinx Vitis-AI (preview)" in the ORT EP list and ships inside AMD's own build | n/a |
+
+Sources: §2–§6 below.
+
+## 2. FastFlowLM (`/usr/bin/flm` 1.0.1 **[host]**) — catalog + HTTP surface
+
+* Repo moved into AMD's ROCm org **2026-08-11** (`github.com/ROCm/FastFlowLM`,
+  v1.0.0). "Supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo,
+  Kraken, and Gorgon Point)."
+  <https://github.com/ROCm/FastFlowLM> (README fetched 2026-10-19).
+* Linux support landed 2026-03; distros: Ubuntu 24.04, 25.10, **26.04**, Arch,
+  generic. Needs NPU firmware ≥ 1.1.0.0, kernel **7.0+** with `amdxdna` (or
+  `amdxdna-dkms`), XRT stack (`ppa:lemonade-team/stable` → `libxrt-npu2`,
+  `amdxdna-dkms`); `flm validate` reads the DRM path (`/dev/accel/accel0`), `flm
+  run` goes through **XRT** and fails with `No such device with index '0'` unless
+  `xrt-smi examine` lists the NPU; `memlock` must be raised.
+  <https://fastflowlm.com/docs/install_lin/> (page dated 2026-03-05, fetched
+  2026-10-19). Static build bundles **XRT v2.21.75** — the same XRT series as the
+  distro package on the host **[host]**.
+* HTTP surface: `flm serve <model>` → OpenAI-compatible REST, **port 52625**, no
+  auth. Maintained endpoints: `v1/models`, `v1/chat/completions`,
+  `v1/audio/transcriptions`, `v1/embeddings`. **There is no
+  `/v1/audio/speech`.**
+  <https://fastflowlm.com/docs/instructions/server/openapi/>
+* Catalog 2026-10-19 (<https://fastflowlm.com/docs/models/>): LLaMA, DeepSeek,
+  Qwen, Gemma, MedGemma, TranslateGemma, gpt-oss, LiquidAI/LFM, Phi, Nanbeige,
+  **Whisper — card states "Type: Speech-to-Text (ASR)"**
+  (<https://fastflowlm.com/docs/models/whisper/>), EmbeddingGemma, SmolVLA,
+  Hy-MT2, MiniCPM. No speech-synthesis entry.
+* Engines are **prebuilt, closed, downloaded from Hugging Face per architecture**
+  ("Internet access to HuggingFace is required to download the optimized model
+  kernels"). The only in-tree engine build is Gemma-4E
+  (`cmake --preset linux-default -DFLM_BUILD_GEMMA4E=ON`).
+  → **A new TTS architecture cannot be added to `flm` by us.**
+
+## 3. AMD Ryzen AI Software 1.8.0 on Linux — install source, device support
+
+* **Install channel (important correction to the pypi.amd.com evidence).** The pip
+  index is Windows-only for the EP runtime (`ryzenai_dynamic_dispatch-1.7.0-…
+  win_amd64.whl`, `onnxruntime_vitisai-1.23.2-…win_amd64.whl`; `flexmlrt` has a
+  single **linux_aarch64** wheel) — but AMD's Linux stack is not on pip. It ships
+  as a tarball that is, as of 2026-10-19, **publicly reachable without an AMD
+  account**: `https://download.amd.com/opendownload/RyzenAI/1.8.0/ryzen_ai-1.8.0.tgz`
+  → HTTP 206 on a range probe, `Content-Range: bytes 0-8/**7819506855**`
+  (**≈7.8 GiB**, magic `1f 8b 08` = gzip). (The docs link it behind the
+  EULA form `account.amd.com/...ryzenai-eula-public-xef.html`; the CDN path
+  serves it directly. Verified by me with curl 2026-10-19. I did not finish
+  downloading 7.8 GiB, so the tarball's *contents* list is unverified —
+  `[INFERENCE]` only where noted.)
+* Driver/XRT package (public, 90,016,137 bytes, `application/zip`):
+  `https://download.amd.com/opendownload/RyzenAI/Driver/RAI_1.8_Linux_NPU_XRT.zip`
+  contains `xrt_202620.2.25.37_24.04-amd64-{base,base-dev,npu}.deb` +
+  `xrt_plugin.2.25.260102.56.release_24.04-amd64-amdxdna.deb` (verified by
+  download + unpack). **The plugin deb ships firmware for
+  `/usr/lib/firmware/amdnpu/17f0_11/npu.dev.sbin`** — exactly the directory our
+  NPU loads **[host]** — plus `1502_00`, `17f0_10`, `17f1_10`, `17f1_13`,
+  `libvxdna.so`, `libxrt_driver_xdna.so.2.25.260102.56`, and the DKMS driver
+  source tarball. **No xclbin/PDI and no compiler binary in this package**: it is
+  driver + XRT shim + firmware only.
+* **Our device is officially STX.** AMD's APU table
+  (<https://ryzenai.docs.amd.com/en/latest/app_development.md#apu-types>):
+  `0x1022:0x17F0 rev 0x11 → STX` (0x00/0x10 also STX, 0x20 = KRK, `0x1502` rev 0
+  = PHX/HPT). Release notes: "Strix and **Strix Halo** as STX", and the STX/KRK
+  column enables CNN INT8, CNN BF16, NLP BF16, LLM (OGA).
+* Vendor driver source (same deb, `share/amdxdna/amdxdna.tar.gz`, source dated
+  2026-06-29) maps `0x17f0 rev 0x11 → dev_npu5_info`, `.fw_path =
+  "amdnpu/17f0_11/"`, `default_vbnv = "RyzenAI-npu5"`, and the revision→name table
+  has `{ AIE2_DEV_REVISION_HALO, "NPU Strix Halo" }`; `xrt-smi examine` therefore
+  reports the device as **NPU Strix Halo**.
+* Linux install recipe (1.8.0 docs
+  <https://ryzenai.docs.amd.com/en/latest/linux.html>): Ubuntu 24.04 packages,
+  **Python 3.12** (`apt install python3.12 python3.12-venv`),
+  `libboost-filesystem1.74.0`, dkms; then `install_ryzen_ai.sh -a yes -p <venv>`;
+  `LD_LIBRARY_PATH=…:${RYZEN_AI_INSTALLATION_PATH}/onnxruntime/lib/`;
+  `quicktest.py` "picks up a simple CNN model, **compiles it** and runs on AMD's
+  Neural Processing Unit" — i.e. **compilation runs on Linux** for the non-LLM
+  flows. "Linux currently supports NPU only flow" (no iGPU EP).
+  Host friction: our box is Ubuntu 26.04 + **Python 3.14** **[host]**, so the venv
+  must be built with a 3.12 interpreter and the packages are 24.04-built
+  `[INFERENCE]`.
+* LLM flow on Linux
+  (<https://ryzenai.docs.amd.com/en/latest/llm_linux.html>): prequantized
+  ready-to-run models from HF, `deployment/lib`,
+  `RYZENAI_EP_PATH=$PWD/deployment/lib/libonnxruntime_providers_ryzenai.so`,
+  run via the `model_benchmark` binary; **"Linux does not support Hybrid flow"**;
+  **"Model generation is not supported in current release"** (repeated in
+  relnotes 1.8 Known Issues: "Models generated on Windows are compatible with
+  Linux"); needs `memlock unlimited`.
+  Sample throughput in that doc: **49.1 tokens/s decode**, 753 tokens/s prefill
+  (Phi-3.5-mini, 4K model).
+* Third-party confirmation of the OS split: Lemonade's backend matrix
+  (<https://raw.githubusercontent.com/lemonade-sdk/lemonade/main/README.md>,
+  fetched 2026-10-19) — `flm`/npu/XDNA2 = **Windows, Linux**;
+  `ryzenai-llm`/npu = **Windows only**; `whispercpp`/npu (STT) = **Windows only**;
+  text-to-speech engines `kokoro` and `acestep` = metal / cuda / vulkan / rocm,
+  **no npu device at all**. Lemonade is the "Server Interface" layer of AMD's own
+  Ryzen AI stack (docs index), so this matrix is the vendor-sanctioned view.
+
+## 4. Can we compile our own graph on Linux? (the no-recipe path)
+
+Mechanics from <https://ryzenai.docs.amd.com/en/latest/modelrun.md> (1.8.0):
+ONNX only, opset 17 recommended; the EP compiles on first `InferenceSession` and
+caches (VitisAI EP cache, or ORT **EP context cache** — `ep.context_enable`, and
+"precompile the EP context model using Python and deploy it using a C++ program"
+is explicitly supported); EP auto-partitions unsupported nodes to CPU;
+BF16 = `target VAIML` (`preferred_data_storage: unvectorized` for transformers),
+BF16 **STX/KRK-or-newer only**; INT8 = `target=X2` with XINT8/A8W8/A16W8 and
+"general asymmetric quantization … to enable third-party quantized models";
+**deployment EP cannot compile BF16 on the fly** — ship a pre-compiled model
+(<https://ryzenai.docs.amd.com/en/latest/app_development.md>);
+cache dirs are not portable across EP or NPU-driver versions.
+
+Operator coverage is the hard gate
+(<https://ryzenai.docs.amd.com/en/latest/ops_support.md>, "operators supported in
+Ryzen AI 1.8.0"; "Y" = broad coverage, some configurations may still fail):
+
+* Good for TTS front-ends/vocoders: `ConvTranspose` **Y in BF16/A16W8/A8W8**
+  (absent only from XINT8), `Conv`, `Resize`, `Upsample`, `Pad`, `LSTM` (BF16),
+  `CumSum` (BF16/XINT8), `STFT` (XINT8 only).
+* **Blockers for a decoder-style transformer in the plain flows:** in **BF16** the
+  table has **no Softmax, no LayerNormalization, no Relu/Gelu**; LayerNorm is
+  XINT8-only and Softmax is A16W8/A8W8/XINT8-only. So attention either lives in
+  the **LLM/OGA flow** (whose op set is exactly
+  `SimplifiedLayerNormalization`, `SkipSimplifiedLayerNormalization`, `MatMulNBits`
+  (W4ABF16/W4ABFP16), `RotaryEmbedding`, **`GroupQueryAttention`**, `Sigmoid`,
+  `Mul`, `Add`) or must be decomposed into a quantized INT8 flow.
+* Missing entirely: `GRU`, `Trilu`, `OneHot`, `EyeLike`, `Multinomial`,
+  `NonMaxSuppression`, `CastLike`, `MelWeightMatrix`/`MelFilterGroup`,
+  `RMSNormalization` → graphs that emit them must be rewritten before they
+  partition onto the NPU. [INFERENCE from absence in the table.]
+* Quantization is a separate tool, not in the driver package: **AMD Quark**
+  (`quark.torch` / `quarkonnx`), which documents `ConvTranspose2d`, `conv1d`,
+  softmax/LayerNorm/BFP quantization (relnotes 1.3/1.2/1.1/0.9) — Quark is *not*
+  on pypi.org under the obvious names **[host probe]** and ships from AMD's
+  gated/pip-private channel.
+* **New LLM-shaped model (our Qwen-Talker analogue): Windows-only today.**
+  OGA model generation is not available on Linux in 1.8.0 (§3). So a *new*
+  speech-LM cannot be self-served onto the NPU on Linux without AMD (or a
+  Windows box) in the loop.
+* Practical shape cost: the neutts artifacts carry **one compiled variant per
+  padded sequence length** (128/256/512/768/1024/1536/2048/2304/3072/4096 for
+  prefill; 256…4096 for token-phase attention mask) — i.e. per-shape compile
+  artifacts are the norm for the LLM flow. [INFERENCE from the HF file list.]
+
+## 5. Does the AMD TTS artifact actually produce speech? (cloned-voice survival)
+
+NeuTTS is **LM + separate neural codec** ("Simple LM + codec architecture"),
+clone from ~3 s of reference audio, and the codec is **NeuCodec (50 Hz, single
+codebook)**: `neuphonic/neucodec` (PyTorch, encode+decode),
+`neucodec-onnx-decoder{,-int8}` (**decode only**), "Decoder-only (ONNX) codecs
+require **pre-encoded references** — these ship with the repo for all bundled
+voices." Phoneme input; 2048-token context ≈ 30 s of audio; NeuTTS-Air ~360 M
+active / 552 M with embeddings; upstream benchmarks are **CPU-only llama.cpp**
+(119 tok/s prefill/decode mix on a Ryzen 9 HX 370) and explicitly exclude the
+codec. <https://raw.githubusercontent.com/neuphonic/neutts/main/README.md>
+(fetched 2026-10-19); `neuphonic/neutts-air` HF metadata (lastModified
+2026-08-26, `qwen2`, GGUF + safetensors).
+
+Consequences for the fleet spike:
+
+1. AMD's NPU artifact = LM only; **waveform decode and reference encoding are CPU
+   work either way**. Our `reference.wav` voices survive only if we run the
+   NeuCodec **encoder** (PyTorch, per the codec table there is no ONNX encoder)
+   once per voice and then feed pre-encoded codes. That is a one-time offline
+   step per voice; after that the compiled graph takes codes/prompt prefix as
+   ordinary inputs, so nothing about the compiled graph forbids a caller-supplied
+   speaker prefix. `[INFERENCE]`
+2. Real-time math: 50 Hz codec means **≥50 tokens/s decode** for 1× RT. AMD's own
+   Linux sample output for a 4K Phi-3.5-mini is **49.1 tok/s** — so budget for
+   roughly RT-1× on one 4K "full fusion" graph, before the codec's CPU cost.
+   `[INFERENCE — different model; must be measured.]`
+3. Host RAM: the same doc reports **peak working set 3.54 GB** for that 4K model.
+   NeuTTS-Air is ~552 M params at W4A-BFP16, so expect a smaller footprint than
+   3.5 GB `[INFERENCE]`; still non-trivial against 14 GiB available **[host]**.
+4. AMD ships **no TTS demo, runner, or server** on any OS: the examples page has
+   only CNN/BF16/DistilBERT/super-resolution/OGA-LLM/VLM/RAG
+   (<https://ryzenai.docs.amd.com/en/latest/examples.md>); the RyzenAI-SW repo
+   tree (669 paths, `main`, fetched 2026-10-19) contains speech demos **only under
+   `Demos/ASR/{Whisper,Parakeet-TDT}` and `Transformer-examples/ASR/Whisper-AI`**
+   — no TTS directory. Serving TTS therefore means writing our own OGA host
+   (or driving `model_benchmark`-style C++ code) plus phonemizer + NeuCodec glue.
+
+## 6. Two graphs on one NPU, and the iGPU hazard
+
+* **Concurrency is a first-class driver feature.** The vendor `amdxdna` source
+  (in `xrt_plugin.2.25.260102.56`, 2026-06-29) sets `.hwctx_limit = 16` for
+  npu4/npu5 — i.e. **16 hardware contexts** on our device — and allocates each
+  context a set of AIE columns with QoS `opc = col_opc (4096) × num_col`
+  (`aie2_ctx.c`, `aie2_solver.h`), with `frame_boundary_preempt` /
+  `fill_preempt_slot` in `aie2_pci.h` (firmware preemption at frame boundaries).
+  So halogen's armed models and a TTS graph can hold separate contexts
+  simultaneously; they contend for columns and OPC share, they are not mutually
+  exclusive.
+* AMD's user-visible history agrees: 1.4 "Support for up to **16 hardware
+  contexts** on Ryzen AI 300 series"; 1.5 "Application concurrency: improves the
+  resource distribution across applications"; 1.3 "Idle detection and suspension
+  of contexts", "Rebalance feature for AIE hardware resource optimization";
+  1.0 known issue "Support for multiple applications is limited to up to eight"
+  (all in <https://ryzenai.docs.amd.com/en/latest/relnotes.html>).
+* **Package conflict, the real risk** [INFERENCE]: the vendor stack wants AMD's
+  XRT 2.25.37 + `xrt_plugin…amdxdna` deb **[host: distro `libxrt-npu2
+  1:2.21.75+dfsg-4` installed]**, and firmware must match the loaded `amdxdna.ko`
+  — the vendor README warns that a stale/mismatched `npu.sbin` causes
+  `ERT_CMD_ABORT`/mailbox timeouts, and that a distro **in-tree** `amdxdna` older
+  than the XRT SHIM makes `GET_ARRAY`/`QUERY_TELEMETRY` return
+  `EOPNOTSUPP`/`EINVAL` (<https://github.com/amd/xdna-driver> README,
+  fetched 2026-10-19). Installing AMD's debs over the distro XRT changes the ABI
+  under FastFlowLM/halogen, so the spike must decide the XRT version **once**, for
+  the whole host.
+* **iGPU / fabric clock:** no vendor document supports a fabric-clock pin, and no
+  vendor document addresses NPU+iGPU concurrency hangs. AMD's Strix Halo
+  optimization doc (ROCm docs, dated 2026-02-20, and its successor "AMD RDNA3.5
+  system optimization", ROCm 10.1.0) covers GPUVM/GART/GTT, the TTM `pages_limit`
+  knob, `amd-ttm`, BIOS VRAM sizing, and minimum kernel versions
+  (<https://rocm.docs.amd.com/en/docs-7.2.0/how-to/system-optimization/strixhalo.html>,
+  <https://rocm.docs.amd.com/en/latest/reference/system-optimization/rdna3-5.html>).
+  The `pp_dpm_fclk` level-7 pin and the GPU+NPU hang observation stay
+  fleet-observed **[host]**, i.e. `[INFERENCE]` territory — no citation exists.
+
+## 7. Paths I checked and ruled out (so nobody re-walks them)
+
+* **Windows-only doors**: DirectML; Ryzen AI Windows EP; Lemonade's
+  `ryzenai-llm` and `whispercpp`-NPU engines (matrix above); AMD's OGA model
+  generation. Not usable on Dankclaw.
+* **Mainline ONNX Runtime** has no AMD-NPU EP: its EP summary lists AMD ROCm and
+  AMD MIGraphX (GPU) and "Xilinx Vitis-AI (*preview*)" only
+  (<https://onnxruntime.ai/docs/execution-providers/>, 2026-10-19); the working
+  EP is AMD's own build (`libonnxruntime_providers_ryzenai.so` in the tarball's
+  deployment dir, per llm_linux).
+* **ggml / llama.cpp / qwentts.cpp**: no XDNA backend → the current engine
+  (`tts-server` on GGUF, CPU-only build **[host]**) cannot be "turned on" for the
+  NPU; migration means a different runtime, not a flag.
+* **`flm`**: catalog/engines are closed per-architecture (§2) — the cheapest door
+  by install cost, and it is not a door for TTS.
+* **IRON / mlir-aie** is the only open toolchain that documents running on the
+  Ryzen AI NPU from Linux ("A close-to-metal Python API for programming AMD Ryzen™
+  AI NPUs, built on an open-source MLIR-based compiler toolchain"; "Getting
+  Started for AMD Ryzen™ AI on Linux", Ubuntu 24.04/24.10, XRT from
+  `ppa:amd-team/xrt`: `libxrt2 libxrt-npu2 … amdxdna-dkms`, `render` group
+  required) — <https://raw.githubusercontent.com/Xilinx/mlir-aie/main/README.md>,
+  fetched 2026-10-19. This is how FLM's kernels were made ("Low-level kernels
+  optimized using IRON + AIE-MLIR", FLM README) — i.e. real, but it is
+  hand-written per-op tile programming, not a model compiler: writing a TTS model
+  here is a multi-month engineering project, not a migration.
+* **Embedded** Ryzen AI P100 support in the same Linux doc is irrelevant to STX.
+
+## 8. Recommendation implied by the above (for Main's spike)
+
+Spike the AMD path as decided, with these constraints planned in: Python 3.12 venv
++ 24.04-built wheels on Ubuntu 26.04; one host-wide XRT decision (AMD 2.25.37 vs
+distro 2.21.75) with firmware/driver pairing checked via `xrt-smi examine` →
+"NPU Strix Halo"; 7.8 GiB tarball staged off the public CDN URL (§3); pre-encode
+each `reference.wav` with the **PyTorch** NeuCodec encoder because the ONNX codec
+is decoder-only; keep phonemizer + codec on CPU; measure decode tokens/s against
+the 50 Hz budget and peak RSS against the ~14 GiB headroom; expect to write the
+serving shell yourself (AMD ships `model_benchmark`, no TTS demo).
+
+## 9. Loose ends (not verified; I stopped on Main's stand-down)
+
+* Contents of `ryzen_ai-1.8.0.tgz` (7.8 GiB) — whether the Linux tarball contains
+  the AIE-ML compiler (`vaiml`, `aiecompiler_client`) for the *BF16/INT8* flows
+  and which Python wheels it carries. My statement that non-LLM compilation works
+  on Linux rests on AMD's own Linux doc ("quicktest.py … compiles it and runs on
+  … NPU"), not on the package contents. Download was killed mid-flight.
+* Whether AMD's `neutts` artifacts run correctly under the Linux `model_benchmark`
+  runner (the TTS rows are listed without OS qualifiers; no Linux TTS run is
+  documented anywhere in 1.8.0 docs).
+* Exact AIE column count/total `ncols` exposed for Strix Halo (driver reads it
+  from AIE metadata: `ndev->total_col = min(XRS_MAX_COL, ndev->aie.metadata.cols)`).

report/qwen3-tts-payload-portability-2026-10-19.md

@@ -0,0 +1,438 @@
+# Qwen3-TTS payload + qwentts.cpp runtime: architecture and portability
+
+Research deliverable (no code). Slice: *what the fleet's TTS payload actually is, and how
+portable it is off the CPU ggml path.* Companion slice: `NpuRuntimeLandscape` (what NPU
+runtimes exist on Dankclaw).
+
+Evidence date: 2026-10-19. Everything below is grounded in:
+
+- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
+  (2026-10-16, upstream HEAD on 2026-10-19) and its ggml submodule
+  `ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16).
+- The two GGUF files as they sit on Dankclaw (`/home/dank/models/Qwen3-TTS-GGUF/`), read
+  byte-for-byte from the live host.
+- Upstream Qwen3-TTS model cards / release material.
+
+Status: COMPLETE for the assigned slice. ANSWER C was confirmed on the live host the same
+night — the running `frank-tts-engine` container carries `libggml-base.so.0.27.0` +
+`libggml-cpu.so.0.27.0` and nothing else, and our own
+`dankclaw/voiceservice/Containerfile.engine` writes `-DGGML_VULKAN=OFF` out explicitly, so
+the iGPU change is a flag in a file the fleet owns. Verdict, measured contract and the NPU
+exoneration chain: `tts-npu-spike-verdict-2026-10-19.md`.
+
+---
+
+## 1. What the payload is
+
+Two files, both read byte-for-byte off the live host (`/home/dank/models/Qwen3-TTS-GGUF/`,
+owned `dank:dank`, mtimes 2026-10-18 18:31), both from
+`https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF` (community converter, not Qwen):
+
+| file | bytes | GGUF | tensors | arch string | file_type |
+|---|---|---|---|---|---|
+| `qwen-talker-1.7b-base-Q4_K_M.gguf` | 1,219,245,248 | v3, kv=64 | 480 | `qwen3-tts` | `Q4_K_M` |
+| `qwen-tokenizer-12hz-Q8_0.gguf` | 291,150,624 | v3, kv=51 | 398 | `qwen3-tts-tokenizer` | `Q8_0` |
+
+`general.name` of the talker file is `Qwen3-TTS-12Hz-1.7B-base`; the header sizes add up
+to the file sizes (sum of tensor bytes + header + alignment padding), so nothing in these
+files is unaccounted for.
+
+### 1.1 Talker file (480 tensors)
+
+Three models in one container, per `docs/ARCHITECTURE.md` §GGUF layout
+(qwentts.cpp @ pinned commit) and confirmed by the tensor census:
+
+| prefix | tensors | bytes | what it is |
+|---|---|---|---|
+| `talker.*` | 316 | 1,061.9 MB | 28-layer Qwen3 decoder (hidden 2048, FFN 6144, 16 Q heads / 8 KV heads, head_dim 128, RoPE θ=1e6, RMS ε=1e-6, ctx 32768), text stream + codec stream summed, `codec_head` → 3072 codebook-0 logits |
+| `code_pred.*` | 88 | 114.8 MB | 5-layer Qwen3 MTP head (hidden 1024, FFN 3072, 15 acoustic codebooks × 2048), own KV cache |
+| `spk_enc.*` | 76 | 36.6 MB | ECAPA-TDNN speaker encoder (x-vector, base checkpoints only; mel → conv0 → 3 SE-Res2Net → MFA → ASP → 2048-d vector) |
+
+Dtype split (exact tensor byte sums): `q4_K` 250 tensors / 864.1 MB, `q6_K` 17 / 312.0 MB,
+`f32` 176 / 25.8 MB, `f16` 37 / 11.4 MB.
+
+Which tensors carry which quant is the load-bearing portability fact:
+
+- `q6_K` (17): `talker.text_embd.weight [2048, 151936]` (255.3 MB), `talker.codec_embd.weight
+  [2048, 3072]`, and the 15 `code_pred.codec_embd.<g>.weight [2048, 2048]` embedding tables.
+  These are the llama.cpp `Q4_K_M` "embedding/output rows go to Q6_K" rule, applied to
+  embedding tables rather than to an `lm_head`.
+- `q4_K` (250): every attention/FFN projection of the talker and of the code predictor.
+  Distinct matmul shapes: `[2048,2048]`×73, `[2048,1024]`×62, `[2048,6144]`×56,
+  `[6144,2048]`×28, `[1024,2048]`×20, `[1024,3072]`×10, `[1024,1024]`×10, `[3072,1024]`×5,
+  `[2048,3072]`×2, `[2048,151936]`×1. Every one is 2-D and ≥ 1024 on both axes — good
+  GEMM shapes, nothing exotic.
+- `f32` (176) / `f16` (37): all of `spk_enc` (conv weights F16 because K=1/3/5 rows are not
+  block-alignable, per the converter's `should_quantize` policy in `tools/quantize.cpp` +
+  `quantize.sh`; every bias/norm/LayerScale vector F32), plus the talker/predictor norms and
+  the `text_proj` biases.
+
+Metadata the runtime actually reads (talker file): `qwen3-tts.num_code_groups=16`,
+`talker.vocab_size=3072`, `text_vocab_size=151936`, `text_hidden_size=2048`,
+`position_id_per_seconds=13`, `rope.mrope_section=[24,20,20]`, `mrope_interleaved=1`,
+the 7 codec special ids (pad 2148 / bos 2149 / eos 2150 / think 2154 / nothink 2155 /
+think_bos 2156 / think_eos 2157), `language_names` (10: chinese, english, german, italian,
+portuguese, spanish, japanese, korean, french, russian) with their codec ids, the text-side
+special ids (im_start 151644, im_end 151645, tts_pad 151671, tts_bos 151672, tts_eos 151673),
+the whole `generation_config.json` (do_sample, top_k 50, top_p 1.0, temperature 0.9,
+repetition_penalty 1.05, `subtalker_*` mirrors, max_new_tokens 8192), and an embedded
+**Qwen2 BPE tokenizer** (`tokenizer.ggml.model=gpt2`, 151,676 tokens, 151,291 merges,
+eos 151643) — that BPE block is 5.95 MB of the header, i.e. 49 % of the header is vocabulary.
+
+Two live-file details that contradict the upstream doc text (worth knowing when reading
+`docs/ARCHITECTURE.md`: the doc example block is stale against the shipped file):
+`mrope_interleaved` is **1** (doc shows `false`), and `spk_enc.sample_rate` is **24000**
+(doc says 16000). Doc also says "404 tensors on the 1.7B voice_design build" — the base
+build shipped here has 480 because it carries `spk_enc.*`.
+
+### 1.2 Tokenizer/codec file (398 tensors)
+
+One neural audio codec, 12.5 frames/s over 24 kHz mono, hop 1920, 16 codebooks × 2048
+entries (1 semantic + 15 acoustic). Dtype split: `f16` 50 / 143.6 MB, `q8_0` 110 / 76.9 MB,
+`f32` 238 / 70.7 MB. `tok_dec.*` 253 tensors / 201.3 MB, `tok_enc.*` 145 / 89.9 MB.
+
+Composition (metadata + tensor census, both files agree):
+
+- **Encoder** (used only to make voice references, not on the synthesis hot path): SEANet
+  conv stack (init k=7, 4 stages with ratios 8/6/5/4 → cumulative 960×, 64→512 ch, final
+  k=3), an 8-layer transformer (hidden 512, 8 heads, head_dim 64, FFN 2048, RoPE 1e4,
+  LayerNorm+bias, GELU, LayerScale), a downsample conv k=4 s=2 (25 → 12.5 Hz), then the RVQ
+  argmin loop. Metadata records `encoder.num_quantizers=32` with
+  `encoder_valid_num_quantizers=16` — 16 codebooks are used, 16 are dead weight carried from
+  the upstream config.
+- **Decoder** (the hot path: every generated frame): 16 codebook lookups + per-split
+  `output_proj` (256→512), `conv_pre`, an 8-layer transformer (hidden 512, 16/16 heads,
+  head_dim 64, FFN 1024, **sliding window 72**, RMSNorm, SwiGLU, LayerScale, input_proj
+  1024→512 / output_proj 512→1024), a ConvNeXt upampler (`upsampling_ratios=[2,2]`, causal
+  transposed convs + depthwise k=7), then a DAC-style decoder `dec.0..3`
+  (`upsample_rates=[8,5,4,3]`, channels 1536→768→384→192→96, `decoder_dim=1536`) whose blocks
+  are SnakeBeta → causal transposed conv (k = 2·stride) → 3 residual units (dilations 1/3/9)
+  → snake_post → conv_post k=7 → 96→1.
+- All 32 codebook tensors are **F32, 67.1 MB total** (16 encode + 16 decode, `[256, 2048]`),
+  deliberately unquantized: the converter's `should_quantize` keeps RVQ codebooks and their
+  wrapping projections at F32 because nearest-neighbour lookup is sensitive to per-row
+  quantisation noise (`docs/ARCHITECTURE.md` "Quantisation policy"; mirrored in
+  `quantize.sh`). 49 of the codec's 55 3-D conv tensors are **F16** with non-alignable K
+  (7, 3, 2, 1, 16, 10, …) — they cannot be block-quantized at all.
+- **No flow matching, no diffusion, no vocoder-with-ODE.** It is a pure RVQ codec
+  (Mimi/SoundStream/Mimi-style: SEANet + transformer + DAC decoder). That matters for the
+  NPU question: every component is convolution, attention, or embedding lookup.
+
+### 1.3 Where the payload came from, and what the official weights look like
+
+Official upstream (`huggingface.co/api/models`, retrieved 2026-10-19):
+
+| repo | last modified | license | weights |
+|---|---|---|---|
+| `Qwen/Qwen3-TTS-12Hz-1.7B-Base` | 2026-01-23 | apache-2.0 | `model.safetensors` BF16, 1,928,677,440 params, plus an embedded `speech_tokenizer/model.safetensors` |
+| `Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice` | 2026-01-29 | apache-2.0 | same layout, named-speaker tables |
+| `Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign` | 2026-01-29 | apache-2.0 | 1.7B only |
+| `Qwen/Qwen3-TTS-12Hz-0.6B-Base` / `-0.6B-CustomVoice` | 2026-01-29 | apache-2.0 | narrow variant (hidden 1024) |
+| `Qwen/Qwen3-TTS-Tokenizer-12Hz` | 2026-01-29 | apache-2.0 | codec alone, F32, 170,557,441 params |
+
+So the GGUF is **not** the only public form: the safetensors are the canonical form
+(Apache-2.0, no separate community license risk), and the GGUF is a lossy re-quantization of
+them produced by the same author who wrote the runtime (`Serveurperso`), via
+`checkpoints.sh` → `convert.py` (F32 GGUF) → `quantize.sh` (BF16/Q8_0/Q4_K_M).
+Sizes to remember for any re-export: the talker container is 1.93 B params in BF16
+(that count includes the embedded codec), the codec alone is 170.6 M params in F32.
+
+**Portability consequence #1 — the GGUF is a dead end off-ggml.** Q4_K/Q6_K are ggml
+block formats (256-element super-blocks, F16 sub-block scales, 144 B / 210 B per block). No
+ONNX graph, no Vitis-AI/AIE-ML compiler, and no vendor NPU runtime consumes them; AMD's own
+quantizer works from ONNX (`ryzenai-quantize`, QDQ/QOP power-of-two — Ryzen AI Software
+1.8.0 release notes, https://ryzenai.docs.amd.com/en/latest/relnotes.html). Moving the
+payload to a non-ggml stack therefore means starting from the safetensors again, not
+converting the GGUF.
+
+**Portability consequence #2 — the quantization split is semantic, not cosmetic.** The
+converter keeps RVQ codebooks, `input_proj`/`output_proj` around them, and every 1-D and
+non-alignable-conv tensor out of the k-quants precisely because quantizing them breaks voice
+fidelity. Any re-quantization path (GPTQ/AWQ/QDQ-INT8/INT4 for an NPU compiler) inherits that
+constraint: the talker backbone is requantizable, the codec's codebook lookups and SnakeBeta
+convs are not, at least not without re-measuring voice quality.
+
+## 2. What the runtime is
+
+`qwentts.cpp` @ `5916bf7c` (2026-10-16, MIT; upstream model Apache-2.0) is a C++17 ggml
+application — 43 files under `src/` + `tools/`, three binaries (`qwen-tts`, `qwen-codec`,
+`tts-server`) over one public plain-C99 ABI (`src/qwen.h`, `qt_*`, `QT_ABI_VERSION`
+range-checked; `tests/abi-c.c` is compiled `-std=c99 -Wall -Werror -pedantic` on every
+build). It is not a llama.cpp derivative: it links ggml directly and builds its own graphs.
+
+Five graph families, and their shapes are the whole portability story:
+
+| graph | batch/time axis | notes |
+|---|---|---|
+| talker prefill | T = prompt length (hundreds–thousands) | summed text+codec embedding streams, 28-layer GQA decoder, KV cache |
+| talker decode | **T = 1 per audio frame** | 12.5 frames/s → 80 ms of audio per step; `codec_head` → 3072 logits |
+| code predictor (MTP) | **T = 1, 16 steps per frame** (2 prefill + 14 decode, cache reset every frame) | 5-layer decoder, 15 heads × 2048 |
+| speaker encoder | one pass per voice registration | ECAPA-TDNN with mel fused into the graph |
+| codec decode | offline: T = 300-frame chunks; streaming: **T = 1 frame** with persistent conv/KV-ring state | the streaming graph is built once with fixed topology so the CUDA graph cache stays in pure replay |
+
+Per generated frame the host therefore runs ~1 + 16 + 1 tiny graph computes for 80 ms of
+audio, and `tts-server` serializes synthesis FIFO (`--max-batch` defaults to 1, and the
+fleet runs it at 1). This is the archetypal latency-bound, bandwidth-bound,
+batch-1 workload; the only big-M matmuls in the whole system are the prefill and the
+chunked (buffered) codec decode.
+
+Op contract actually issued to ggml (`src/`, counted): `mul_mat` 77, `cont` 70, `add` 57,
+`mul` 40, `rms_norm` 22, `clamp` 15, `rope_ext` 12, `get_rows` 12, `set_rows` 8,
+`scale_bias` 6, `swiglu_split` 5, `sqr` 5, `mean` 5, `elu` 4, `col2im_1d` 4, `soft_max` 3,
+`conv_1d` 3, `im2col` 2, `cumsum` 2, `sin`, `sigmoid`, `tanh`, `div`, `argsort`, `transpose`,
+`permute`, `reshape/view`, plus the CPU-side Philox multinomial sampler.
+
+**No custom ggml op is required today.** The repo doc still says the fork "provides the two
+custom ops the codec needs: `GGML_OP_SNAKE` and `GGML_OP_COL2IM_1D`"; that is stale at the
+pinned commit. Verified 2026-10-19:
+
+- the submodule `ServeurpersoCom/ggml` @ `cc5df49` is upstream `ggml-org/ggml` master
+  (`a05c992`, 2026-10-15) **+2 commits, 0 behind** (GitHub compare API), and
+  `include/ggml.h` is byte-identical to upstream master (`diff` empty);
+- the 2 commits are `193e339` (2026-06-19) "cpu: add fused snake activation
+  (mul -> sin -> sqr -> mul -> add)" — a **CPU subgraph fusion**, not an op (the runtime
+  builds snake with `ggml_sin`/`ggml_sqr`/`ggml_mul`/`ggml_add` in `src/dac-decoder-v2.h:222`),
+  and `cc5df49` "sycl: fix get_rows on reordered quantized weights";
+- `ggml_col2im_1d` is an **upstream ggml API** (same declaration, line 2097, in both trees);
+  the Vulkan backend fuses the snake chain through ggml's generic subgraph fuser
+  (`ggml-vulkan.cpp:14548 ggml_vk_can_fuse_snake`, `vulkan-shaders/snake.comp`).
+
+Consequence: any stock ggml build (upstream, or llama.cpp's vendored ggml) can execute this
+payload; only the CPU snake fusion is fork-specific, and it is a performance nicety.
+
+Backend/device support: every tool takes `--device <name>` (`CUDA0`, `Vulkan0`, `MTL0`,
+`CPU…`, default best available), resolved through the ggml backend registry; the deployed
+container turns CUDA/HIP/Vulkan/Metal/SYCL/RPC/BLAS all OFF and compiles the native-CPU
+backend with OpenMP, with CPU threads = `hardware_concurrency()/2` (`src/backend.h`).
+The registry in this ggml knows CPU, CUDA, Metal, SYCL, Vulkan, WebGPU, ZDNN, OpenCL,
+**Hexagon**, BLAS, RPC, VirtGPU, **CANN**, ZenDNN, **OpenVINO**, ET — and nothing AMD-NPU
+(no `xdna`/`aie`/`vitis`/`ryzen` string anywhere in the ggml tree).
+
+Voice handling (what the fleet depends on): the registry is **process RAM only**; a restart
+empties it, which is why the unit has a boot warmer. `POST /v1/audio/voices` accepts either
+`{name, ref_text, wav_b64}` (server extracts) or `{name, ref_text, spk_b64, rvq_b64}`
+(pre-extracted latents verbatim); `ref_text` present ⇒ ICL clone mode, absent ⇒ x-vector-only
+mode. `.spk` = the ECAPA x-vector; `.rvq` = 16 codes/frame packed 11 bits LSB-first
+(`src/rvq-file.h`). `wav` and `pcm` `response_format` are different audio at the same seed
+(buffered chunked decode vs stateful streaming decode) — the fleet already encodes that rule.
+
+Parity expectations for any port: the upstream harness matches the PyTorch reference per
+stage (embed cos 0.999988, all 28 hidden taps ≥ 0.999893, logits 0.999824) but explicitly
+reports free-run divergence (`CodesFull exact 4.96 %`, `Audio cos 0.095`) and explains why —
+one argmax tie at FP epsilon diverts the AR trajectory. So a migrated endpoint must be
+accepted by listening + per-stage cosine, never by waveform diff against today's audio.
+
+---
+
+## 3. The three answers Main asked for
+
+### ANSWER A — Qwen3-TTS GGUF → ONNX: no, and it does not matter
+
+There is **no GGUF→ONNX path** and there will not be a useful one: GGUF is a weight
+container for ggml, its Q4_K/Q6_K block layouts have no ONNX consumer, and nothing in the
+GGUF carries the *graph* (the talker/predictor/codec structure lives in C++ code, only the
+op-level structure exists in ggml at runtime). But the GGUF is not the payload's only form —
+official Apache-2.0 safetensors are upstream, and **four independent ports of the full
+pipeline already exist**:
+
+| port | form | what it proves |
+|---|---|---|
+| `xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (HF, updated 2026-03-03, apache-2.0) | FP16+FP32 ONNX, fp16/voice_clone + fp16/voice_design splits: `talker_decode.onnx`, `code_predictor.onnx`, `code_predictor_kv.onnx`, `code_predictor_embed_g0..15.onnx`, `text_embedding.onnx`, `codec_embedding.onnx`, shared `speaker_encoder.onnx`, `speech_tokenizer_encoder.onnx`, `speech_tokenizer_decoder.onnx`, `model_cache.npz` | every component of this architecture exports to ONNX, codec decoder included |
+| `zhangluoyang/qwen3-tts-onnxruntime` | export tooling + Python inference for Base/CustomVoice/VoiceDesign | the export is reproducible from the official checkpoints |
+| `Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO`, `aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03) | OpenVINO IR, **INT8**: `talker`, `code_predictor`, `codec_embedding`, `text_model`, `openvino_speaker_encoder_model`, `speech_tokenizer/speech_decoder` | the talker and predictor INT8-quantize for an NPU-targeting stack |
+| `litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17, apache-2.0) | `qwen3_tts_int8.litertlm` / `_int4.litertlm` (LiteRT-LM `OmniEngine`), legacy split graphs `talker_int4.tflite` (prefill_32/prefill_128/decode signatures, KV 1024, blockwise-32 OCTAV INT4), `mtp_folded_int8.tflite` (16 inner steps × 5 layers folded into one graph, in-graph argmax+gather, GPTQ int8, ~5× faster), `codec_partA` (transformer, fp32) + `codec_partB` (conv upsampler, XNNPACK FORCE_FP16, ~2.5×, "ASR-identical") | an NPU/mobile compiler accepts the whole thing — at the price of graph splitting, static prefill buckets, and folding the per-frame inner loop |
+
+What the ports had to concede, and what any port inherits:
+
+- the generation structure does not fit existing engine loops: LiteRT-LM's Engine "does not
+  support this generation structure yet", so their first shape was **three graphs + a
+  host-side loop**, and the MTP step was "invoked 17× per frame" until they folded it;
+- per-step dispatch cost dominates: even on a Snapdragon 8 Elite Gen 5 they keep the MTP
+  loop, text projection, embedding gathers and the codec on CPU "avoiding 16 synchronous
+  CPU↔GPU round-trips per 80 ms frame", putting only the 28-layer talker on the GPU;
+- RTF on that hardware is **2.57–2.97 (Galaxy S26) and 5.65–8.48 (Pixel 10 Pro)** for the
+  0.6 B model, i.e. 2.6–8× slower than real time. Dankclaw's current CPU engine measures
+  **warm RTF 0.55–0.67 with TTFA 94–112 ms** on the 1.7 B model (fleet trial, 2026-10-18).
+  An NPU port is not "faster by default" for this workload; it has to beat that.
+- quantize the LM (int8/int4: proven), keep the codec's codebook lookups and SnakeBeta convs
+  in fp16/fp32 (the LiteRT split does exactly this).
+
+So: the answer to "can the fleet's payload be exported" is *yes — from the safetensors, not
+from the GGUF*, and the work is a pipeline rewrite (multi-graph host loop, per-frame fold,
+static prefill buckets), not a conversion. That is the same conclusion NeuTTS-Air's
+precompiled-graph route encodes: on AMD's Linux stack the graph is vendor-curated, so the
+fleet chooses the model that AMD already compiled rather than compiling Qwen3-TTS.
+
+### ANSWER B — is there any ggml backend for an AMD NPU? Not upstream; three out-of-tree, none for TTS
+
+Checked against the trees today (2026-10-19), because this was the crux:
+
+- **Nothing in-tree.** `ggml-org/llama.cpp` master (4,237 paths, GitHub tree API) contains no
+  `xdna`/`aie`/`vitis`/`ryzen` file or directory; the ggml backend set is CPU, BLAS, CUDA,
+  HIP, Metal, Vulkan, SYCL, OpenCL, WebGPU, MUSA, RPC, CANN (Ascend), **Hexagon** (Qualcomm),
+  OpenVINO (Intel CPU/GPU/NPU), ZenDNN, ZDNN, VirtGPU, ET. The deployed fork's registry
+  matches. The feature request "Feature Request: XDNA backend" (llama.cpp issue **#21725**,
+  opened 2026-04-10) is still **open**, 34 comments; the older #1499 was closed stale
+  2025-04-07. So Main's premise is confirmed: no ggml XDNA backend exists to flip a flag on.
+- **`Cyronius/ggml-xdna`** — a real ggml backend (`XDNA0`) loaded into *stock* llama.cpp via
+  `GGML_BACKEND_PATH`, no fork. It does bfp16 (8-value blocks sharing an exponent) GEMMs from
+  GGUF weights, and is **prefill-only**: it takes only matmuls in prompt chunks ≥ 1024 tokens
+  (`GGML_XDNA_MIN_BATCH` default 1024, `GGML_XDNA_MIN_MFLOP` 256), attention and every decode
+  token stay on the Vulkan device; pinned to llama.cpp release b10944; tested on Strix Point
+  (HX 370) only — Strix Halo "should pass, but nobody has run it on one yet"; Linux support
+  new and untested on hardware by the author; needs `libxrt-npu2`, `/dev/accel/accel0`,
+  unlimited `memlock`. Best measured prefill gain 1.88× on a 27 B model; models under
+  2,048-wide **lose** (0.77× at width 1,536).
+- **`BrandedTamarasu-glitch/OllamaAMDNPU`** — a llama.cpp fork with `-DGGML_XDNA=ON`,
+  explicitly targeting **Ryzen AI MAX / XDNA2 on Linux** (`/dev/accel/accel0`, XRT 2.21.75+,
+  amdxdna, unlimited memlock). It implements **only `GGML_OP_MUL_MAT`**, dispatched to
+  mlir-aie-compiled `.xclbin`s built per fixed tile shape (e.g. `M=2048 K=2048 N=64`,
+  int8 in / int32 out, per-row weight scales cached at first use), with a second slot for a
+  different K. Its own numbers on a Ryzen AI MAX 385 with Llama-3.1-8B Q4_K_M: NPU prefill
+  12–19 t/s vs **Vulkan iGPU 600–930 t/s**, NPU decode **0.65–1.40 t/s** ("Decode does not
+  use the NPU … M=1 per token, no xclbin covers it"), 0.6 % AIE utilization, and the stated
+  conclusion "**For LLaMA 8B on Strix Halo: use Vulkan exclusively**". Single-maintainer,
+  AI-assisted, self-measured — treat as a proof of concept, but it is the closest published
+  picture of what XDNA2 graph execution costs.
+- Also referenced in the same thread: `Atomic-Germ/OpenFlowLM-Next` (whole-model NPU engine,
+  LLM-only — and slower than the iGPU in the same comparison), `GaTmaNnes/xdna2-`,
+  `hardWorker254/Npu-Embeddings-XDNA1` (adds Whisper ASR ops, NPU1/Linux). None is a TTS path.
+
+Why none of them would carry this workload even if installed: the TTS hot loop is **batch-1
+by construction** (one talker step + 16 predictor steps + one codec frame per 80 ms), which
+sits under every dispatch gate those backends define (≥1024 tokens / ≥32768 M·N·K / tile M
+covering only prefill), and the codec half of the pipeline is conv, col2im, gather and
+elementwise ops — not `MUL_MAT` — so it would fall back to CPU op-by-op. The NPU case for
+this host is therefore *not* "run Qwen3-TTS on the NPU"; it is "run a vendor-compiled AR TTS
+graph on the NPU", which is exactly the NeuTTS-Air spike.
+
+### ANSWER C — would `qwentts.cpp` build for the gfx1151 iGPU? Yes, via **Vulkan**, not HIP
+
+- Upstream ships the build scripts and claims the backends: `buildvulkan.sh` ("AMD/Intel GPU
+  (Vulkan)"), `buildcuda.sh`, `buildsycl.sh`, `buildcpu.sh`, `buildall.sh` (all backends,
+  runtime dynamic loading), and README line 6 "runs on CPU, CUDA, Metal, Vulkan". CMake knobs
+  `GGML_VULKAN` / `GGML_HIP` / `GGML_SYCL` are the same ones the fleet's Containerfile turns
+  OFF, so the engine rebuild is a flag change: `-DGGML_VULKAN=ON` plus
+  `--device Vulkan0` on the same command line the unit runs today.
+- The codec's exotic ops are covered by the Vulkan backend, which is the thing that decides
+  whether a GPU build is coherent: `vulkan-shaders/col2im_1d.comp`, `snake.comp`,
+  `swiglu*.comp`, `cumsum*.comp` exist and `supports_op` handles `GGML_OP_COL2IM_1D` /
+  `CONV_TRANSPOSE_1D` / `CUMSUM`. By contrast `COL2IM_1D` appears in **no** HIP, OpenCL,
+  WebGPU, Hexagon or CANN source — a **HIP** (ROCm) build would shard the ConvNeXt/DAC
+  transposed convs back to the CPU per op. On an AMD iGPU the portable path is RADV/Vulkan.
+- `--device` also accepts a multi-device split (`-dev`-style naming is ggml's), and the
+  runtime's documented FP16 guard (`--clamp-fp16`) exists for sub-Ampere CUDA accumulation;
+  nothing in the tree is NVIDIA-specific.
+- Unverified by me: any vendor statement pairing this fork with gfx1151. Supporting evidence
+  that the hardware/driver combination is viable: this very host already runs a Vulkan
+  endpoint on the gfx1151 iGPU (fleet ground truth, 2026-10-19), and Mesa's Vulkan
+  implementation is officially supported on Radeon Software for Linux
+  (AMD release notes RN-AMDGPU-UNIFIED-LINUX-25.20.3). A gfx1151 Vulkan build of this fork is
+  a build-and-measure spike, not an unsupported port.
+- The real constraint is the one already in the fleet docs: the iGPU belongs to another
+  endpoint, and GPU+NPU concurrency has hung Strix Halo machines. An iGPU TTS build competes
+  with the text endpoints for the same silicon; the NPU route's only genuine advantage is
+  that it does not (the XDNA backend authors make the same argument: "Use NPU when the iGPU
+  is busy … no GPU contention").
+
+---
+
+## 4. What any replacement must reproduce, and what the voices cost
+
+Endpoint contract in production today (from the live unit/env, not the doc):
+`tts-server --model qwen-talker-1.7b-base-Q4_K_M.gguf --codec qwen-tokenizer-12hz-Q8_0.gguf
+--alias frank-tts-engine --host 0.0.0.0 --port 8880 --lang auto --max-batch 1`, no published
+port (the façade on :7772 dials the private bridge name), `GET /health`, `GET /v1/models`,
+`GET|POST /v1/audio/voices`, `DELETE /v1/audio/voices/{name}`, `POST /v1/audio/speech` with
+`response_format` wav|pcm (different audio at equal seed) and optional `seed`,
+`max_new_tokens`, `temperature`, `top_k`, `top_p`, `repetition_penalty`, `subtalker_*`.
+Voice registry is RAM-only, so boot warming is mandatory, and `--alias` keeps the quantization
+choice out of every consumer's config.
+
+Live voice census on Dankclaw (2026-10-19, `/home/dank/voices/`): three packs
+(`narrator1..3`), each carrying `reference.wav` (24 kHz mono crop), `reference16k.wav`,
+`manifest.json` (keys: name, kind, generated_utc, pipeline, source, forbidden_region_guard,
+silence_scan, crop, loudnorm, audio, acceptance, acceptance_passed) and ffmpeg analysis
+artifacts. **No `.spk` and no `.rvq` exist on the live host** — the repo's
+`build-voice-pack.sh` treats them as required-but-they-are-not-there, so today the engine
+re-extracts the x-vector (and the ICL codes when a transcript is supplied) from the WAV at
+registration time.
+
+That is the portable part, and it is the same shape AMD's NeuTTS flow wants
+(`encode_reference(reference.wav)` + a reference transcript): the fleet's voice identity is
+*the WAV plus its transcript*, processed at request/registration time by the model's own
+speaker encoder + codec — **never a trained per-voice artifact**. Qwen3-TTS cloning is
+inference-time conditioning (`generate_voice_clone(ref_audio=…, ref_text=…, x_vector_only_mode=…)`;
+`create_voice_clone_prompt` exists purely to avoid recomputing prompt features). So any
+replacement that consumes a WAV (+transcript) keeps all three narrators alive with no
+re-derivation, and if it wants precomputed latents instead, the LiteRT precedent shows what
+that looks like: a 4 KB x-vector `.npy` per voice, extracted offline
+(`extract_speaker_embedding.py`) — for the 1.7 B base checkpoint that vector is 2048-dim
+(`qwen3-tts.spk_enc.embedding_length = 2048` in the shipped GGUF; the LiteRT 0.6 B bundle
+ships a 1024-dim one).
+
+One trap for a mixed-runtime future: `.spk`/`.rvq` files, if they are ever written, are
+engine-specific encodings (ECAPA x-vector layout; 11-bit-packed RVQ stream) and are **not**
+interchangeable between runtimes. Keep `reference.wav` as the pack's source of truth and
+regenerate anything derived per engine — which is what the NeuTTS re-clone plan does anyway.
+
+## 5. Sources
+
+Runtime/payload (all read locally at pinned versions, 2026-10-19):
+
+- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
+  (commit date 2026-10-16; also upstream HEAD today): `README.md`, `docs/ARCHITECTURE.md`
+  (860 lines), `src/*`, `tools/*`, `docs/DOCKER.md`, `buildvulkan.sh`/`buildcpu.sh`/`buildall.sh`.
+- submodule `ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16);
+  delta vs `ggml-org/ggml` master `a05c992` (2026-10-15) = ahead 2 / behind 0, per
+  https://github.com/ServeurpersoCom/ggml/compare/ggml-org:master...HEAD ;
+  `include/ggml.h` identical to upstream
+  https://raw.githubusercontent.com/ggml-org/ggml/master/include/ggml.h .
+- Live payload bytes: `/home/dank/models/Qwen3-TTS-GGUF/qwen-talker-1.7b-base-Q4_K_M.gguf`
+  (1,219,245,248 B) and `qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B), headers parsed from
+  the host; sizes/sha256 match `scripts/download-frank-tts-engine-qwentts.sh`.
+- Live units/env/packs: `dankclaw/voiceservice/{Containerfile.engine,tts-engine-start.sh,tts-engine-warm.sh,quadlet/*}`,
+  `scripts/frank-tts-engine.env`, `scripts/frank-tts.env`, `/home/dank/voices/*`.
+
+Model provenance:
+
+- `https://huggingface.co/api/models/Qwen/Qwen3-TTS-12Hz-1.7B-Base` (+ CustomVoice,
+  VoiceDesign, 0.6B variants, `Qwen3-TTS-Tokenizer-12Hz`), apache-2.0, last-modified
+  2026-01-23 / 2026-01-29; safetensors param counts 1,928,677,440 (BF16, incl. embedded
+  `speech_tokenizer/`) and 170,557,441 (F32 codec).
+- `https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base/raw/main/README.md` (clone API:
+  `generate_voice_clone(ref_audio, ref_text, x_vector_only_mode)`,
+  `create_voice_clone_prompt`; reference stack = PyTorch + `device_map="cuda:0"` + bf16 +
+  flash-attn-2, Python 3.12 recommended).
+- `https://github.com/QwenLM/Qwen3-TTS`.
+
+Portability precedents:
+
+- `https://huggingface.co/xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (2026-03-03) ·
+  `https://github.com/zhangluoyang/qwen3-tts-onnxruntime` ·
+  `https://huggingface.co/Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO` (2026-03-22) ·
+  `https://huggingface.co/aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03-18) ·
+  `https://huggingface.co/litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17) incl.
+  `legacy/README.md` (graph splits, MTP fold, codec partA/partB, RTF table).
+
+AMD NPU / ggml backend status:
+
+- `https://api.github.com/repos/ggml-org/llama.cpp/git/trees/master?recursive=1` (no
+  XDNA/AIE/Vitis backend, 2026-10-19) ·
+  `https://github.com/ggml-org/llama.cpp/issues/21725` (open since 2026-04-10) ·
+  `https://github.com/ggml-org/llama.cpp/issues/1499` (closed stale 2025-04-07).
+- `https://github.com/Cyronius/ggml-xdna` (prefill-only, gates, Linux requirements, results
+  measured 2026-10-05/06 on HX 370) ·
+  `https://github.com/BrandedTamarasu-glitch/OllamaAMDNPU` (XDNA2 ggml backend on Strix Halo,
+  phases 1–12, "use Vulkan exclusively", measured on Ryzen AI MAX 385).
+- `https://github.com/amd/RyzenAI-SW` (main, 669 paths, 2026-08-18): audio content is ASR
+  only (`Demos/ASR/Whisper`, `Demos/ASR/Parakeet-TDT`, `Transformer-examples/ASR/Whisper-AI`);
+  **no TTS example anywhere in the official Ryzen AI Software tree**.
+- `https://ryzenai.docs.amd.com/en/latest/relnotes.html` (Ryzen AI SW 1.8.0: ONNX-format
+  models, ONNX Runtime as the loader, power-of-two QDQ/QOP quantization).
+- Mesa/RADV on AMD iGPUs: `https://www.amd.com/en/resources/support-articles/release-notes/RN-AMDGPU-UNIFIED-LINUX-25-20.3.html`
+  (Mesa Vulkan driver officially supported).
+
+Status: COMPLETE for the assigned slice (payload architecture, runtime architecture,
+portability). `[INFERENCE]` marks: none above except the Mesa-version caveat in ANSWER C,
+which is flagged there.

report/tts-npu-spike-verdict-2026-10-19.md

@@ -0,0 +1,180 @@
+# TTS on the Strix Halo NPU — spike verdict (2026-10-19)
+
+Question: **how can the fleet's TTS engine run on hardware rather than CPU?** Answer in
+one line: not on the NPU — the loader, not the silicon, refuses it; the door that exists is
+the gfx1151 iGPU behind a build flag we already own.
+
+Spike evidence lives in `report/npu-tts-runtime-landscape-2026-10-19.md` (which runtimes can
+reach XDNA2 at all) and `report/qwen3-tts-payload-portability-2026-10-19.md` (what the
+payload is and where it can be moved). This page owns only the **measured spike result** and
+the **model-side contract** neither slice has. Throwaway work ran in containers under
+`/home/dank/spikes/npu-tts` on Dankclaw; nothing in the fleet was changed — `frank-tts` and
+`frank-tts-engine` still serve `narrator1|2|3` on CPU.
+
+Legend: **[host]** live census 2026-10-19 · **[spike]** measured in the spike containers ·
+**[vendor]** primary vendor source, with the version it holds for.
+
+## 1. Verdict
+
+**NO-GO on the NPU, and the wall is a loader.** The XDNA2 device is healthy by AMD's own
+instruments; the runtime that owns this model family opens the device and then declines the
+graph, silently. One run loaded successfully; byte-identical runs never reproduced it, and a
+path that cannot be reproduced is not a foundation. **[spike]**
+
+**Go on the iGPU, and the lever is in this repo.**
+`dankclaw/voiceservice/Containerfile.engine` builds `qwentts.cpp` (commit `5916bf7`) with
+`-DGGML_VULKAN=OFF` written out deliberately beside CUDA/HIP/METAL/SYCL/RPC — the running
+container carries `libggml-base.so.0.27.0` + `libggml-cpu.so.0.27.0` + `tts-server` and
+nothing else. So: `-DGGML_VULKAN=ON` + `--device Vulkan0`, plus `/dev/dri` passthrough in
+the engine quadlet (the ComfyUI pattern: `AddDevice` + `GroupAdd=keep-groups` + a
+world-writable udev rule). The target shape is **co-tenancy** — the engine resident on the
+gfx1151 *beside* `frank-dragoon`, not a handover. §5 carries the census and both gates:
+the **heap gate is PASSED** — RADV's `DEVICE_LOCAL` heap is 77.33 GiB with 65.16 GiB of
+budget while dragoon is loaded, so memory is not the constraint — which leaves the
+contention gate (dragoon's TTFT with a synthesis in flight) and one unproven piece:
+whether qwentts.cpp's codec ops have Vulkan kernels for every op. **[host]** **[spike]**
+
+Why the iGPU is worth doing at all, given CPU already works: the deployed payload needs
+only **12.5 codes/s** (§2), so this is latency headroom and freed cores, not reviving a
+dead service. The independent Strix Halo measurement in the portability slice ("use Vulkan
+exclusively"; NPU decode under 1.5 t/s for an 8B) points the same way.
+
+## 2. The two arms do not share a real-time bar
+
+|Arm|Talker|Codec|Codes per second of audio|Bar|
+|---|---|---|---|---|
+|**Deployed** (`frank-tts-engine`)|`qwen-talker-1.7b-base-Q4_K_M.gguf`|`qwen-tokenizer-12hz-Q8_0.gguf`|**12.5**|~13 tok/s **[host]**|
+|**Spike** (`amd/neutts-air_rai_1.8.0_npu_4K`)|NuTTS-Air, 24-layer Qwen2|NeuCodec `24000 / hop 480`|**50.00**|~50 tok/s **[spike]**|
+
+Measured from the staged NPU artifact — the contract any future arm must satisfy:
+
+|Fact|Value|
+|---|---|
+Vocab|`genai_config.json` `vocab_size` **217,652**; base BPE 151,643; specials 151,644–151,670 **[spike]**|
+Speech tokens|**65,536** `<|speech_N|>` entries, ids **151,671–217,206** contiguous; the vendor template maps a codec index straight to its id, `"".join(f"<|speech_{i}|>")`. A 10,001-code assumption silently truncates the space — encoded references really do reach the 60k range. **[spike]**|
+Stop token|`<|SPEECH_GENERATION_END|>` = 151,670, but the export's `model.eos_token_id` is `[151645, 151643]` (the *chat turn* stop) and `search.eos_token_ids` is unset — pass 151,670 explicitly or speech ends after one token **[spike]**|
+Input ceiling|4096 tokens **total**, input + output **[vendor]**|
+Frame rate|50.00 codes/s confirmed across three references (49.96 / 49.99 / 50.00) **[spike]**|
+Reference encode|`librosa.load(sr=16000, mono=True)` → `[1,1,T]` float → `codec.encode_code(audio_or_path=…)` (vendor `NeuTTS.encode_reference`); CPU RTF 0.28–0.38 on 8 threads, so codes are a cacheable one-time cost, never a per-request charge. `neucodec` also needs `torchtune`; install both `--no-deps --target <dir>` so a second torch never shadows the venv's. **[spike]**|
+Pre-encoded voices|`/spike/logs/codes-narrator{1,2,3}.npy` — 994 / 1,249 / 1,250 codes for 19.90 / 24.98 / 25.00 s **[spike]**|
+
+**Gap that any arm inherits:** the voice packs carry no transcript. `manifest.json` holds
+provenance and crop geometry (`crop.source_speech_run` 98.739–118.636 s of a 22,569 s
+audiobook) and no text, and Dankclaw has no ASR at all — `whisper`, `whisper.cpp`,
+`faster-whisper` all absent. NuTTS conditioning needs `ref_text`; do not invent it. **[host]**
+
+## 3. Exoneration chain — why the blame is not the hardware
+
+Every row is an instrument, not an inference.
+
+|Probe|Result|
+|---|---|
+`xrt-smi examine`|`[0000:c6:00.1] RyzenAI-npu5 aie2p 6x8`, XRT 2.25.37 (vendor extract), NPU fw 1.1.2.65, `amdxdna 7.0.0-38-generic` **[spike]**|
+`xrt-smi validate`|**PASSED**, including `throughput` **[spike]**|
+`pyxrt.device(0)` in-container|constructs — XRT enumerates the device **[spike]**|
+`strace -f -e trace=openat` on the failing load|`genai_config.json` read → `model.onnx` opened → `libxrt_driver_xdna.so.2` loaded → `/dev/accel/accel0` opened `O_RDWR` → GEM ioctls succeed → **EP declines, before touching `dd_metastate_*` or `cache/txn_bins.zip`** **[spike]**|
+`dmesg`, `fuser /dev/accel/accel0`|silent since boot, no holder — the decline never reaches the kernel **[spike]**|
+Hardlinked mirror dirs of the model tree (`rundir*`)|retired, no change — the "metastate shadowing" theory is dead **[spike]**|
+Every logging lever in that build: `GENAI_ORT_VERBOSE_LOGGING`, `GENAI_LOG_ORT_LIB`, `set_log_options`, `set_log_callback(callable)`, `RYZENAI_EP_TRACING`, `RYZENAI_LLM_PROFILE_INIT`|**zero lines** — the decline path is silent **[spike]**|
+Vendor `quicktest`|`Session successfully initialized` — proves the XRT/driver/device path, **not** NPU residency: that build falls back to CPU silently. Prove residency with `/sys/class/accel/accel0/device/mem_usage` during generation. **[spike] [vendor]**|
+
+No NPU token rate was ever measured, so none is quoted here. The single successful load
+(13:17, `[load] 0.84s`, finite logits, XRT banner at *generate* time) has not been
+reproduced by the byte-identical command since, with the model files pristine by inode and
+mtime. **[spike]**
+
+## 4. Host gates the spike proved (so they are not re-litigated)
+
+- **Rootless containers cannot read `/dev/accel/accel0`** (`root:render` 0660): the host
+  gid never enters the userns map. `--group-add keep-groups`, `--userns=keep-id`,
+  `--userns=keep-id:gid=990`, `--group-add render`, `--group-add gid:990` — all fail.
+  Doors: rootful podman (separate storage), or a world-writable udev rule, which the fleet
+  already does for `kfd`/`renderD*`. **[spike]**
+- **Never `dpkg -i` the vendor XRT deb** — it installs a DKMS `amdxdna` over the distro
+  driver. `dpkg-deb -x` the debs and bind-mount the tree at `/opt/xilinx/xrt`; the plugin
+  deb is what carries `libxrt_driver_xdna.so`. **[spike]**
+- In-tree `amdxdna 0.7.0-38` + `iommu=pt` is enough for context creation; a missing device
+  node is *not* the modern failure mode. **[host]**
+- `xrt-smi` inside a container needs `libboost-filesystem1.83.0` +
+  `libboost-program-options1.83.0` (**hyphenated** Ubuntu names, after `apt update`);
+  boost is needed by `xrt-smi` only, never by the OGA/EP path. **[spike]**
+- Headroom is tight and it is host RAM, not a separate pool — see §5 for tonight's census.
+  A multi-GB NPU partition plus an AIE compile workspace does not fit beside
+  `frank-dragoon`. Benign-looking `Failed to alloc dev bo memory, ret -28` is that
+  pressure. **[host]**
+
+## 5. Target shape: the TTS engine shares the gfx1151 with `frank-dragoon`
+
+The deployment Frank wants is **both resident at once**, not a handover. Measured 2026-10-19
+(read-only, nothing loaded or unloaded):
+
+|Measure|Value|Reading|
+|---|---|---|
+iGPU GTT in use|**18,523 MiB of 116,733 MiB**|≈96 GiB of aperture free; this is the real budget (fleet law) **[host]**|
+DRM "VRAM" carve-out|2,048 MiB total, 174 MiB used|what KFD/DRM calls dedicated — **not** the budget **[host]**|
+Host RAM|121 GiB total, 17 GiB available, 79 GiB buff/cache|**GTT pages are system pages**: an iGPU allocation is host RAM consumed, so the two tenants share one pool **[host]**|
+TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that same pool **[host]**|
+
+**Two gates, in this order.**
+
+1. **Heap gate — PASSED.** RADV does **not** hand a container tenant the 2 GiB DRM
+   carve-out. `vulkaninfo` inside the engine's own base image (`ubuntu:24.04` plus
+   `mesa-vulkan-drivers vulkan-tools`, `--device /dev/dri`, rootless) sees
+   `Radeon 8060S Graphics (RADV GFX1151)`, driver `radv`, API 1.4.318, and reports:
+
+   |Heap|size|budget with dragoon loaded|flags|
+   |---|---|---|---|
+   |0|38.67 GiB|32.58 GiB|host-visible, not device-local|
+   |1|**77.33 GiB**|**65.16 GiB**|`DEVICE_LOCAL`|
+
+   The heaps sum to the ~116 GiB aperture in Mesa's ⅓/⅔ APU split — the same RAM twice:
+   a Vulkan allocation is a GTT mapping, so a budget is a ceiling on mappings, not a pile
+   of free silicon memory. ggml's Vulkan backend asks for `DEVICE_LOCAL`, which is heap 1,
+   and the TTS engine needs roughly 4 GiB of it. **Memory stops being the question.** **[spike]**
+   Probe traps: `vulkaninfo --batch` is not a flag in this `vulkan-tools` (it prints the
+   help page and exits 1 — use `vulkaninfo --text`), and the container has **no NVIDIA
+   ICD**, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by
+   device *name*, or an image that grows an `nvidia_icd.json` silently points TTS at the
+   3090 slot.
+2. **Contention gate.** `frank-dragoon` is the endpoint the operator session runs on, so the
+   honest measurement with both loaded is dragoon's own TTFT and decode with a synthesis in
+   flight — one request each, no ladder. Expect compute-unit sharing rather than memory
+   pressure: the aperture has room, the shaders do not.
+
+**Change surface** — the whole edit, so nobody re-derives it:
+
+1. `Containerfile.engine` build stage: add `libvulkan-dev glslang-dev`. It installs only
+   `ca-certificates cmake g++ git make` today, and ggml's Vulkan backend compiles its
+   shaders at build time — the flag alone fails at configure.
+2. `-DGGML_VULKAN=OFF` → `ON` in the same cmake block; the flag list is written out
+   explicitly, so this is a one-word diff.
+3. Runtime stage: add `mesa-vulkan-drivers`. Without a Vulkan **ICD inside the image** the
+   device node buys nothing — there is no `radeon_icd.json` in the container today.
+4. `frank-tts-engine.container`: `AddDevice=/dev/dri` + `GroupAdd=keep-groups` — **not**
+   `renderD128`. On this hybrid box that node is the NVIDIA card; the iGPU is
+   `renderD129` (`readlink -f /sys/class/drm/renderD129/device/driver` → `amdgpu`).
+5. Engine argv: `--device Vulkan0` — qwentts.cpp's tools take `--device <name>` at the
+   pinned commit.
+
+Gate 1 recipe, already run and **PASSED**: `mesa-vulkan-drivers` + `vulkan-tools` in a
+throwaway container off the same base, `--device /dev/dri`, then `vulkaninfo --text` and read
+the `VkPhysicalDeviceMemoryProperties` heap sizes — no model, no load, no compute. The probe
+trap: `vulkaninfo --batch` is not a flag in Ubuntu 24.04's `vulkan-tools` — it prints the help
+page and exits 1, which reads like a driver failure. Every package involved has a candidate on
+the host's apt (`vulkan-tools` 1.4.341, `libvulkan-dev` 1.4.341, `glslang-dev`/`glslang-tools`
+16.2.0); the build stage runs on the digest-pinned Ubuntu 24.04 base, whose versions are older
+but present. Script: `/home/dank/spikes/npu-tts/vk_probe.sh` on Dankclaw.
+
+Constraints that do not relax for a co-tenant: `/dev/dri` passthrough needs the ComfyUI
+pattern (`AddDevice` + `GroupAdd=keep-groups` + the world-writable udev rule) because
+systemd user-unit spawns carry no supplementary groups; the 64 GiB swap at `/swap.img`
+exists to protect co-tenants, and paging a KV pool is worse than the OOM it prevents;
+`pp_dpm_fclk` reads `auto` (unheld), which only becomes a hazard if the NPU ever joins.
+
+## 6. Re-visit condition
+
+Retry the NPU when **a Linux loader claims the graph**: an `onnxruntime-genai-ryzenai`
+build whose RyzenAI adapter takes the NuTTS-Air Full-Fusion graph on `RyzenAI-npu5`, or that
+model appearing in a runtime's own supported catalog with a Linux runner. Then the
+prerequisites are already mapped and priced: the gated NeuCodec encoder for cloned voices,
+the `ref_text` transcripts, the 4096-token prompt budget, and 50 codes/s as the bar.

scripts/autoresearch-frank-archon.sh

@@ -29,10 +29,10 @@
 # The scratch mirrors the production container contract exactly (serve-frank-
 # primary.sh): --network host, --init, seccomp=unconfined (io_uring for the
 # n-gram table), memlock unbounded (the ~132 GiB pinned ledger), /dev/kfd +
-# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
-# differences: the container name (arbench, so prod's identity is untouched)
-# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
-# prefill is cold — the prod tier at kvcache/frank-archon is left intact.
+# /dev/dri, the .rad ro, the tune cache mount. The ONLY differences: the
+# container name (arbench, so prod's identity is untouched) and the prefix cache
+# switched OFF (--prefix-cache-mib 0), so every prefill is cold — the tier is
+# host RAM now, there is no per-run disk directory to wipe.
 set -uo pipefail
 
 ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-archon.env}"
@@ -60,9 +60,8 @@ set -a; . "$ENV_FILE"; set +a
 sudo systemctl stop frank-archon >/dev/null 2>&1 || true
 podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
 
-# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
-FRESH_KV="$FRANK_ARCHON_STATE_DIR/kvcache/$SCRATCH"
-rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
+# --- the prefix cache is switched off below, so there is no scratch KV tree ---
+# to wipe; the cold prefill comes from --prefix-cache-mib 0.
 mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
 
 # --- boot the scratch: the production run line, scratch name + fresh KV ------
@@ -73,7 +72,6 @@ podman run -d --name "$SCRATCH" \
   --ulimit memlock=-1:-1 \
   --device /dev/kfd --device /dev/dri \
   -v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-  -v "$FRANK_ARCHON_STATE_DIR":/data \
   -v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
   -v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
   "$FRANK_ARCHON_IMAGE" \
@@ -87,12 +85,9 @@ podman run -d --name "$SCRATCH" \
   --placement "$FRANK_ARCHON_PLACEMENT" \
   --host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
   --gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
-  --vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
-  --vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
   --kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
-  --prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
-  --prefix-cache-dir "/data/kvcache/$SCRATCH" \
-  --prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
+  --prefix-cache-mib 0 \
+  --busy-prefill-chunk "${FRANK_ARCHON_BUSY_PREFILL_CHUNK:-auto}" \
   --ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
   --num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
   --max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \

scripts/axe-measurements.py

@@ -0,0 +1,164 @@
+#!/usr/bin/env python3
+"""Destroy measurement ledgers in the surviving pages.
+
+Point of use (AGENTS.md → Documentation Policy): a rate/throughput figure for
+anything not running and not being compared is destroyed, not filed. This pass
+deletes
+
+  1. any section whose heading is a bench/throughput/rate/ladder/verdict ledger,
+  2. any table row or standalone bullet that REPORTS a rate (`123.4 t/s`,
+     `tok/s`, `ms/token`, `TTFT`, acceptance %, `llama-bench`, a bench arm),
+
+and it never touches a shape number (ctx, slots, MiB budgets, ports) or a bullet
+under a *Laws* / *Wire contract* / *Rollback* heading, where the sentence is the
+payload and the number is decoration. Dry run by default; `--apply` writes.
+
+Usage: axe-measurements.py [--apply] [--verbose]
+"""
+import pathlib
+import re
+import sys
+
+VAULT = pathlib.Path("serverAppConfigs-docs")
+RATE = re.compile(
+    r"\d[\d,.]*\s*(t/s|tok/s|tokens?/s|tok s|tps|ms/token|µs/token|GB/s|GiB/s)\b|"
+    r"\b(llama-bench|bench (arm|ladder|window)|decode ladder|prefill ladder)\b|"
+    r"\bacceptance (rate|of)\b|\bdraft_n\b|\bttft\b|\bServer-Timing\b",
+    re.I,
+)
+# A heading is a ledger only when the noun LEDS it and nothing instructional
+# follows: `## Throughput` yes, `## 6. The context ladder (find the max bootable
+# window by measurement)` no — that one is the method, which is point-of-use.
+LEDGER_HEAD = re.compile(
+    r"^#{2,4}\s+(?:\d+[.) ]\s*)?(bench(es|mark|marks)?|throughput|results?|timings?|"
+    r"measured (figures|numbers|results|rates|rates)|rates|verdict|acceptance|ladder)"
+    r"\b[^|]{0,40}$",
+    re.I,
+)
+INSTRUCTIONAL = re.compile(r"how|why|law|policy|method|procedure|find|by measurement|execution|flip|checklist", re.I)
+PROTECTED = re.compile(
+    r"^#{2,4}\s+.*(law|wire|rollback|contract|shape|eras|files|health|see also|related|"
+    r"boot|preflight|environment)",
+    re.I,
+)
+def blank_rate_cells(text):
+    """Kill throughput COLUMNS wherever they appear: a cell that reports a rate
+    becomes an em-dash, so the row keeps whatever else it records (the payload
+    name, the reason, the rollback) and loses only the dead number."""
+    out = []
+    for line in text.split("\n"):
+        st = line.strip()
+        if st.startswith("|") and st.endswith("|") and RATE.search(line):
+            cells = line.split("|")
+            for i, c in enumerate(cells):
+                if RATE.search(c):
+                    cells[i] = " — "
+            line = "|".join(cells)
+        out.append(line)
+    return "\n".join(out)
+
+
+def collapse_eras(text):
+    """One Eras row per era is fleet law; the NUMBERS in the old rows are not.
+    Keep the current era verbatim, fold everything older into one row that says
+    the numbers were destroyed and where the rollback actually lives (git)."""
+    lines = text.split("\n")
+    out = []
+    in_eras = False
+    ncol = 0
+    folded = 0
+    for line in lines:
+        if line.startswith("#"):
+            in_eras = bool(re.match(r"^#{2,4}\s+.*eras\b", line, re.I))
+            if folded:
+                out.append("| " + " | ".join(
+                    [f"≤ 2026-09 — {folded} earlier eras: payloads, shapes and every measured "
+                         "number destroyed 2026-10-19 as obsolete"] + ["—"] * max(0, ncol - 1)))
+                folded = 0
+            out.append(line)
+            continue
+        st = line.strip()
+        if in_eras and st.startswith("|"):
+            cells = [c.strip() for c in st.strip("|").split("|")]
+            if set("".join(cells)) <= set(":- "):
+                ncol = len(cells)
+                out.append(line)
+                continue
+            if ncol == 0:
+                ncol = len(cells)
+            if re.search(r"2026-1[01]|current|active|running", st, re.I):
+                out.append(line)
+            else:
+                folded += 1
+            continue
+        if folded and not st.startswith("|"):
+            out.append("| " + " | ".join(
+                [f"≤ 2026-09 — {folded} earlier eras: payloads, shapes and every measured number "
+                     "destroyed 2026-10-19 as obsolete"] + ["—"] * max(0, ncol - 1)))
+            folded = 0
+            ncol = 0
+        out.append(line)
+    if folded:
+        out.append("| " + " | ".join(
+            [f"≤ 2026-09 — {folded} earlier eras destroyed 2026-10-19"] + ["—"] * max(0, ncol - 1)))
+    return "\n".join(out)
+
+
+def main(argv):
+    apply = "--apply" in argv
+    verbose = "--verbose" in argv
+    total_lines = 0
+    pages = 0
+    for page in sorted(VAULT.rglob("*.md")):
+        s = str(page)
+        if ".AGENTTRASH" in s or ".obsidian" in s:
+            continue
+        text = page.read_text(errors="replace")
+        before = len(text.split("\n"))
+        if s.startswith("serverAppConfigs-docs/LLM/"):
+            text = collapse_eras(blank_rate_cells(text))
+        lines = text.split("\n")
+        out = []
+        skip = False
+        under_protected = False
+        cut = 0
+        for line in lines:
+            if line.startswith("#"):
+                under_protected = bool(PROTECTED.search(line))
+                skip = (bool(LEDGER_HEAD.match(line)) and not under_protected
+                        and not INSTRUCTIONAL.search(line))
+                if skip:
+                    cut += 1
+                    continue
+            if skip:
+                cut += 1
+                continue
+            stripped = line.strip()
+            out.append(line)
+            if under_protected:
+                # Eras rows are the rollback record, Law bullets are the payload:
+                # the number is decoration inside them, and the row/bullet is not
+                # a ledger entry. Leave the whole section alone.
+                continue
+            if stripped.startswith(("|", "-", "*")) and RATE.search(line):
+                # Destroy only when the line EXISTS to report a number: a table
+                # row, or a bullet whose verb is measurement.
+                if stripped.startswith("|") or re.search(
+                        r"\b(measured|measures|bench|benched|ran at|runs at|clocked|hit|"
+                        r"landed at|came in at|peak|peaks at|sustained|wall.clock)\b", line, re.I):
+                    out.pop()
+                    cut += 1
+        final = re.sub(r"\n{3,}", "\n\n", "\n".join(out))
+        saved = before - len(final.split("\n"))
+        if saved or cut:
+            pages += 1
+            total_lines += max(saved, cut)
+            print(f"  {s.replace('serverAppConfigs-docs/', '')}: -{max(saved, cut)}")
+            if apply:
+                page.write_text(final.rstrip("\n") + "\n")
+    print(f"{'APPLIED' if apply else 'dry run'}: {pages} pages, {total_lines} lines of ledger")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

scripts/destroy-vault.py

@@ -0,0 +1,109 @@
+#!/usr/bin/env python3
+"""Destroy the vault's dead weight: era ledgers, retired runtimes, dead experiments.
+
+`AGENTS.md` was treating a measurement as a record worth keeping forever. It is not: a
+throughput figure for a payload that no longer runs, on a runtime that no longer exists,
+is a liability — the next agent spends context on a number that AI progress has already
+obsolete. This pass moves those pages to `.AGENTTRASH/` (repo law: never `rm`) and then
+deletes every reference to them from the surviving pages, including the sentence that
+carried the link when the link was the sentence's only content.
+
+Usage: destroy-vault.py <list-file>
+List file: one vault-relative path per line; a directory path destroys the whole tree.
+"""
+import pathlib
+import re
+import shutil
+import sys
+
+VAULT = pathlib.Path("serverAppConfigs-docs")
+TRASH = VAULT / ".AGENTTRASH" / "20261019-destruction"
+
+
+def targets(list_file):
+    out = []
+    for line in pathlib.Path(list_file).read_text().splitlines():
+        line = line.strip()
+        if not line or line.startswith("#"):
+            continue
+        p = VAULT / line
+        if p.is_dir():
+            out += [f for f in sorted(p.rglob("*.md"))]
+        elif p.is_file():
+            out.append(p)
+        else:
+            print(f"  SKIP (missing): {line}")
+    return out
+
+
+def main(argv):
+    if len(argv) < 2:
+        print(__doc__)
+        return 2
+    doomed = targets(argv[1])
+    # Stems come from the trash too, so re-running the pass (or running it after a
+    # partial destruction) still knows which names are dead and keeps scrubbing.
+    stems = {p.stem for p in doomed}
+    if TRASH.is_dir():
+        stems |= {p.stem for p in TRASH.rglob("*.md")}
+    moved = 0
+    for p in doomed:
+        dest = TRASH / p.relative_to(VAULT)
+        dest.parent.mkdir(parents=True, exist_ok=True)
+        shutil.move(str(p), str(dest))
+        moved += 1
+    print(f"destroyed {moved} pages ({sum(len(p.read_text(errors='replace').splitlines()) for p in TRASH.rglob('*.md'))} lines staged in trash)")
+
+    # Repoint-or-delete: a link to a destroyed page is itself dead information.
+    link_wl = re.compile(r"\[\[([^\]|#]+)(?:#[^\]|]*)?(?:\|([^\]]*))?\]\]")
+    link_md = re.compile(r"\[[^\]]*\]\(([^)#]*\.md)(?:#[^)]*)?\)")
+    pages_cleaned = links_dropped = lines_dropped = 0
+    for page in sorted(VAULT.rglob("*.md")):
+        s = str(page)
+        if ".AGENTTRASH" in s or ".obsidian" in s:
+            continue
+        text = page.read_text(errors="replace")
+        lines = text.split("\n")
+        out = []
+        hits = 0
+
+        def dead(target):
+            # NOT pathlib.Path(...).stem: that strips a real suffix off names like
+            # `Lemonade and sd.cpp`, so a link to a destroyed page survives and the
+            # vault then fails its own link check.
+            t = target.strip()
+            if t.endswith(".md"):
+                t = t[:-3]
+            return t in stems or pathlib.PurePosixPath(t).name in stems
+
+        for line in lines:
+            found = [t for t in link_wl.findall(line) if dead(t[0])]
+            for m in link_md.finditer(line):
+                if dead(m.group(1)):
+                    found.append(m.group(1))
+            if not found:
+                out.append(line)
+                continue
+            hits += len(found)
+            stripped = link_wl.sub(
+                lambda m: m.group(2) if (m.group(2) and not dead(m.group(1))) else "",
+                line)
+            stripped = link_md.sub(
+                lambda m: "" if dead(m.group(1)) else m.group(0), stripped)
+            # What is left of the line after the dead links are gone?
+            residue = re.sub(r"[\s\-*•·|:;,.\u2014\u2013]+", "", stripped)
+            if residue:
+                out.append(re.sub(r"\s+$", "", re.sub(r"^(\s*[-*]\s*)[\s:;,.\u2014-]+", r"\1", stripped)))
+            else:
+                lines_dropped += 1
+        if hits:
+            page.write_text("\n".join(out))
+            pages_cleaned += 1
+            links_dropped += hits
+    print(f"cleaned {pages_cleaned} surviving pages: {links_dropped} dead links dropped, "
+          f"{lines_dropped} lines left empty and deleted")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

scripts/frank-archon.env

@@ -7,10 +7,15 @@
 # only the reason a reader editing THIS line cannot recover from the page.
 # Where the two disagree, this file wins on values; the boot ledger arbitrates
 # the engine shape.
-FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
-# 1.0.13 is pinned by digest on purpose: 1.2.0 is a DEAD END on this payload
-# (acceptance 0.7123 vs 0.7549 — the page's Dead ends section names the suspect).
-# The .rad file mounted under /models; /data holds the prefix-cache disk tier.
+FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed
+# 1.8.1 (2026-10-18), pinned by digest. Two CLI breaks sit between it and the
+# 1.0.13 line this endpoint ran for two eras, and both are boot refusals, not
+# warnings: --vram-weights-mib/--vram-kv-mib were REMOVED at 1.3.0 (the card is
+# now split by the engine's budget resolver — see below), and the prefix cache
+# lost its disk tier. Draft acceptance has to be re-scored on any pin past 1.2.0:
+# 1.2.0's MoE weight rotation measured 0.7123 against the 0.7549 control, and the
+# suspect is untangled (the page's Dead ends section names it).
+# The .rad file mounts under /models; the tune cache mounts separately below.
 # The model dir holds ONLY .rad files — the HF download and staging stay beside it.
 FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
 FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad
@@ -42,31 +47,42 @@ FRANK_ARCHON_TP=2
 # MAX_SEQS x (1 + SPEC_TOKENS) below.
 FRANK_ARCHON_TP_WIRE=wht6
 # Decode slots. A slot costs KV, never decode; the pool is sized to this
-# contract and nothing more, so a FOURTH full-height conversation is REFUSED as
-# too large to fit — never squeezed, paged or truncated. That is Frank's
-# explicit trade: the freed MiB went to the expert slab.
-FRANK_ARCHON_MAX_SEQS=3
+# contract and nothing more. At 1.8.1 the budget resolver caps the cache at
+# 556,348 fp8 tokens — 2.8 sessions at 200,000 — so a THIRD full-height
+# conversation answers 400 while /health and every shorter chat work fine
+# (3 x 50k streams run at 77.6 t/s aggregate). Frank's trade: declare the two
+# slots the engine can hold and let the freed ~1.1 GiB go to the expert plane,
+# instead of advertising a third slot that only exists at shorter prompts.
+FRANK_ARCHON_MAX_SEQS=2
 # Served window. The engine treats `kv group tokens >= max-model-len` as a HARD
 # limit, and models.yml contextWindow MUST quote this number (over-length is a
 # refusal, not a truncation). The pool is sized for seqs x maxlen, not one
 # sequence.
 FRANK_ARCHON_MAX_MODEL_LEN=200000
 FRANK_ARCHON_PLACEMENT=expert_tiered
-# Pinned host pool for expert rows that do not fit VRAM. It moves WITH
-# VRAM_WEIGHTS: the pair and this pool serve the per-rank resident set together,
-# so cutting the slab spills here — under-sizing it is a boot failure (36,864
+# Pinned host pool for expert rows that do not fit VRAM. Under the budget
+# resolver this is ALSO the lever that decides which experts stay on the card:
+# the card's leftover goes to the expert plane first and the KV cache takes the
+# rest, so growing the pool moves experts off the card and shrinking it spills
+# them onto VRAM the cache wanted. Under-sizing it is a boot failure (36,864
 # refused a boot). The ledger's coverage line is the referee, not this file.
+# Measured, not guessed: 45,568 (PR #30) moved host_pool to 45.09 GiB and left
+# vram_kv byte-identical at 4.20 GiB / 556,348 tokens on both ranks — extra
+# host RAM above what the expert plane's footprint needs buys no cache. Back at
+# 44,928, the size that boots clean.
 FRANK_ARCHON_HOST_POOL_MIB=44928
 # VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
 # floor is built from it.
 FRANK_ARCHON_GPU_HEADROOM_MIB=96
-# ONE budget per card: VRAM_WEIGHTS + VRAM_KV <= 30,969 MiB, the ceiling the
-# engine's own budget table prints (derivation and the slab/pool price: the
-# page's VRAM laws). Idle KV VRAM is ALSO the expert cache's spillover, so a
-# bigger pool pays twice — which is why the pool sits at the contract and the
-# slab takes the rest.
-FRANK_ARCHON_VRAM_WEIGHTS_MIB=25600
-FRANK_ARCHON_VRAM_KV_MIB=5120
+# There is no VRAM pair any more. --vram-weights-mib/--vram-kv-mib were removed
+# at 1.3.0: the engine takes capacity - already-held - GPU_HEADROOM - arena,
+# puts the static (non-expert) weights off the top, then divides the rest between
+# the expert plane and the KV cache — experts first when the cache has no idle
+# blocks to lend, and never less than ONE MAX_MODEL_LEN sequence for the cache.
+# What moves the split from here is HOST_POOL_MIB (which experts stay resident),
+# MAX_MODEL_LEN/MAX_SEQS (the cache bound) and GPU_HEADROOM_MIB. The slab size
+# the prefill-slope law cares about is a boot-ledger line now, not a knob: read
+# it after every resize instead of asserting it here.
 # fp8 is the ONLY rung below bf16 on this engine: one kv_dtype for the whole
 # cache, so llama.cpp's exact-K/quantized-V split (frank-zealot) is unservable
 # here. bf16 costs 1.89x fp8, not 2x — the QSA block-key plane is dtype-fixed.
@@ -75,19 +91,35 @@ FRANK_ARCHON_KV_DTYPE=fp8
 # 2, and 3 x (1+2) = 9 rows stays inside the wht6 25-row exactness bound.
 FRANK_ARCHON_SPEC_TOKENS=2
 FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
-# Prefix cache tiers. The host tier is sized in ENTRIES and one entry tracks the
-# cache dtype (58.50 KiB at fp8), so a dtype change re-sizes the parked set
-# without touching these numbers.
-FRANK_ARCHON_PREFIX_HOST_MIB=8192
-FRANK_ARCHON_PREFIX_DISK_MIB=65536
+# Step sharing. A prefill chunk that shares a step with decoding sequences holds
+# the device for the chunk's whole time, and every decode row in that step waits
+# that long for its next token (the census: [[LLM Performance Diagnosis]] §9).
+# `auto` leaves each decode row behind the chunk a row plus its draft window and
+# keeps a share for whatever is waiting to be admitted; a number pins the chunk's
+# cap; `off` is the pre-1.4 behaviour this endpoint ran for two eras, where the
+# chunk took the step and admission never ran. Interleave inserts N decode-only
+# steps after a chunk step taken beside decoders: it protects their cadence and
+# holds a waiting request out of those steps, so it stays 0 until the post-migration
+# decode-vs-prefill buckets say it is wanted.
+FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto
+FRANK_ARCHON_PREFILL_INTERLEAVE=0
+# Prefix cache: pinned host memory only. The disk tier was removed outright
+# (--prefix-cache-dir and --prefix-cache-disk-mib are refusals), so what used to
+# be restored from NVMe is now re-prefilled. The tier is sized in ENTRIES and one
+# entry tracks the cache dtype (58.50 KiB at fp8). Watch prefix_hit_rate after
+# this change: a drop is extra prefill, and prefill is exactly the traffic that
+# costs decode its cadence.
+FRANK_ARCHON_PREFIX_CACHE_MIB=8192
 # Frank's pin law: the PLE n-gram table serves from pinned host RAM the cards
 # read. `ram` REFUSES the start if memlock is too small — the unit's
 # LimitMEMLOCK=infinity is the other half of this law.
 FRANK_ARCHON_NGRAM_PLACEMENT=ram
-# TTM preflight. The pinned ledger (~147 GiB) rides TTM's shared pages, and the
-# kernel RESETS pages_limit to half of RAM on every boot; below the plan the
-# load dies mid-pin with an anonymous allocator error. ttm-preflight.sh raises
-# the knob itself before refusing by name, so recovery is a boot, not a visit.
+# TTM preflight. The pinned ledger rides TTM's shared pages, and the kernel
+# RESETS pages_limit to half of RAM on every boot; below the plan the load dies
+# mid-pin with an anonymous allocator error. ttm-preflight.sh raises the knob
+# itself before refusing by name, so recovery is a boot, not a visit.
+# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT
+# per the bats ledger test; at the 44,928 pool that computes to 148,920 MiB.
 FRANK_ARCHON_TTM_MIN_MIB=150000
 FRANK_ARCHON_TTM_TARGET_MIB=204800
 # Escape hatch for experiments. Unset -> no extra argv; the serve script expands

scripts/gen-vault-index.py

@@ -107,6 +107,11 @@ def sync_subindex(index: pathlib.Path, vault: pathlib.Path):
     lines = index.read_text().split("\n")
     out, dropped, refreshed = [], 0, 0
     for line in lines:
+        # A row whose first cell is empty is a description with no page: the link
+        # was deleted when the page was destroyed, so the row is a dead promise.
+        if re.match(r"^\|\s*\|", line.strip()):
+            dropped += 1
+            continue
         for pat, wrap in ((ROW, "| [[{}]] |{} {} |"), (BULLET, "- [[{}]] — {}")):
             m = pat.match(line)
             if not m:
@@ -191,7 +196,7 @@ def main(argv):
     index.write_text("\n".join(out) + "\n")
     touched = 0
     for sub in sorted(vault.rglob("index.md")):
-        if sub == index:
+        if sub == index or ".AGENTTRASH" in str(sub):
             continue
         dropped, refreshed, added = sync_subindex(sub, vault)
         if dropped or refreshed or added:

scripts/repair-damage.py

@@ -0,0 +1,56 @@
+#!/usr/bin/env python3
+"""Repair the two artifacts the destruction passes introduced.
+
+1. The link stripper could eat the space after a `- ` marker (`-`X``) and leave a
+   dangling `see .` where a wikilink used to be.
+2. The Eras collapse could swallow a table's HEADER row (it looked like just
+   another non-current row), leaving a separator with no header above it.
+
+Usage: repair-damage.py [--apply]
+"""
+import pathlib
+import re
+import sys
+
+VAULT = pathlib.Path("serverAppConfigs-docs")
+ERAS_HEADER = "| Window | Payload | Runtime | Shape | Replaced because | Rollback |"
+
+
+def main(argv):
+    apply = "--apply" in argv
+    marker = dangling = header = 0
+    for p in sorted(VAULT.rglob("*.md")):
+        s = str(p)
+        if ".AGENTTRASH" in s or ".obsidian" in s:
+            continue
+        lines = p.read_text(errors="replace").split("\n")
+        out = []
+        # Only a pointer whose target was deleted reads as `… see .` — never touch
+        # a line merely because it wraps on a comma. (An earlier version of this
+        # rule stripped the trailing comma off 660 wrapped prose lines, which
+        # `scripts/restore-punctuation.py` then had to undo from HEAD.)
+        for line in lines:
+            new = line
+            if re.match(r"^(\s*)[-*](?![-* ])[$`[(*_]", new):
+                new = re.sub(r"^(\s*)[-*]", r"\1- ", new)
+                marker += 1
+            if re.search(r"\s(?:see|also|details in|via|per)\s*\.?\s*$", new):
+                new = re.sub(r"\s(?:see|also|details in|via|per)\s*\.?\s*$", "", new)
+                dangling += 1
+            out.append(new)
+        # A separator row whose line above is not a header gets its header back.
+        fixed = []
+        for i, line in enumerate(out):
+            if re.match(r"^\|-+", line.strip()) and (not fixed or not fixed[-1].strip().startswith("|")):
+                fixed.append(ERAS_HEADER)
+                header += 1
+            fixed.append(line)
+        if apply and "".join(fixed) != "".join(lines):
+            p.write_text("\n".join(fixed))
+    print(f"{'APPLIED' if apply else 'dry run'}: marker spacing {marker}, "
+          f"dangling pointers {dangling}, restored table headers {header}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

scripts/restore-punctuation.py

@@ -0,0 +1,66 @@
+#!/usr/bin/env python3
+"""Undo trailing-punctuation damage by restoring lines that match HEAD exactly
+except for a stripped trailing `,` / `;` / removed trailing word.
+
+The repair pass over-reached: its "dangling pointer" cleanup also matched every
+wrapped prose line ending in a comma, and deleted the comma. A line whose text
+is otherwise byte-identical to its HEAD version, differing only by trailing
+punctuation, is not an edit — it is damage, so HEAD's line goes back.
+
+Usage: restore-punctuation.py [--apply]
+"""
+import pathlib
+import subprocess
+import sys
+
+REPO = pathlib.Path(".")
+VAULT = REPO / "serverAppConfigs-docs"
+
+
+def head_text(rel):
+    r = subprocess.run(["git", "show", f"HEAD:{rel}"], capture_output=True, text=True)
+    return r.stdout if r.returncode == 0 else None
+
+
+def main(argv):
+    apply = "--apply" in argv
+    restored = 0
+    pages = 0
+    for p in sorted(VAULT.rglob("*.md")):
+        s = str(p)
+        if ".AGENTTRASH" in s or ".obsidian" in s:
+            continue
+        rel = s
+        head = head_text(rel)
+        if head is None:
+            continue
+        cur = p.read_text(errors="replace").split("\n")
+        # Key on (length, prefix) of the HEAD line minus trailing punctuation so a
+        # damaged line can find its original.
+        index = {}
+        for hl in head.split("\n"):
+            h = hl.rstrip()
+            if h.endswith((",", ";")) or True:
+                key = (len(h.rstrip(",;. ")), h.rstrip(",;. ")[:60])
+                index.setdefault(key, h)
+        touched = 0
+        for i, cl in enumerate(cur):
+            c = cl.rstrip()
+            if not c or c.endswith((".", ":", "|", "`", ")", ","  , ";")):
+                continue
+            key = (len(c), c[:60])
+            hl = index.get(key)
+            if hl and hl != c and hl.startswith(c):
+                cur[i] = hl
+                touched += 1
+        if touched:
+            pages += 1
+            restored += touched
+            if apply:
+                p.write_text("\n".join(cur))
+    print(f"{'APPLIED' if apply else 'dry run'}: {restored} lines restored across {pages} pages")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

scripts/serve-frank-archon.sh

@@ -15,12 +15,13 @@
 # log states every budget before anything is allocated (weights vs KV
 # split, the expert host pool's coverage of the worst case, the PLE pin,
 # the P2P answer per pair) — read it after every resize.
-# Deployed shape (2026-10-13): THREE decode slots x 200,000 tokens, fp8
-# (quantized) KV in a 5,120 MiB pool (contract 3 x 200,000 + 1.3 %) and a
-# 25,600 MiB resident expert slab — 30,720 MiB of the 30,969 MiB the engine's
-# own budget table makes claimable, spent on the slab because prefill is
-# slab-limited. 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache host
-# 8 GiB + disk 64 GiB, PLE pinned in RAM.
+# Deployed shape (2026-10-19, radiance 1.8.1): THREE decode slots x 200,000
+# tokens, fp8 KV, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache in
+# pinned host RAM, PLE pinned in RAM. The per-card split between the expert slab
+# and the KV pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair
+# and the budget resolver derives it from headroom, the cache bound and the host
+# pool, so the slab size and the pool's token count are LEDGER lines: after every
+# boot read them out of the log and check the pool still backs 3 x 200,000.
 #
 # The root unit (scripts/frank-archon.service, User=dank,
 # LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
@@ -56,13 +57,11 @@ set +a
 : "${FRANK_ARCHON_PLACEMENT:?FRANK_ARCHON_PLACEMENT must be set}"
 : "${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
 : "${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
-: "${FRANK_ARCHON_VRAM_WEIGHTS_MIB:?FRANK_ARCHON_VRAM_WEIGHTS_MIB must be set}"
-: "${FRANK_ARCHON_VRAM_KV_MIB:?FRANK_ARCHON_VRAM_KV_MIB must be set}"
+: "${FRANK_ARCHON_BUSY_PREFILL_CHUNK:?FRANK_ARCHON_BUSY_PREFILL_CHUNK must be set (auto|off|N)}"
 : "${FRANK_ARCHON_KV_DTYPE:?FRANK_ARCHON_KV_DTYPE must be set}"
 : "${FRANK_ARCHON_SPEC_TOKENS:?FRANK_ARCHON_SPEC_TOKENS must be set}"
 : "${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
-: "${FRANK_ARCHON_PREFIX_HOST_MIB:?FRANK_ARCHON_PREFIX_HOST_MIB must be set}"
-: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
+: "${FRANK_ARCHON_PREFIX_CACHE_MIB:?FRANK_ARCHON_PREFIX_CACHE_MIB must be set}"
 : "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
 : "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
 : "${FRANK_ARCHON_TTM_TARGET_MIB:?FRANK_ARCHON_TTM_TARGET_MIB must be set}"
@@ -103,19 +102,12 @@ if [ ! -f "$FRANK_ARCHON_TEMPLATE" ]; then
     exit 1
 fi
 
-mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
-# The engine's create is NOT recursive: with only the state dir mounted, its
-# "cannot create /data/kvcache/frank-archon: No such file or directory"
-# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
-# the full path on the host so the tier opens on first boot.
-
 mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
 # rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
 # the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
 # $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
 # the cache must ride a host mount or it dies with the --rm container
-# that wrote it. The dir is pre-created for the same reason as kvcache
-# (the engine's create is not recursive).
+# that wrote it. The engine's create is not recursive, so the dir is pre-created.
 
 if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
     echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
@@ -148,6 +140,15 @@ done
 LOG=/home/dank/logs/frank_archon.log
 mkdir -p "$(dirname "$LOG")"
 
+# Interleave is opt-in: 0 is the engine's default and passing it anyway would put
+# a flag in argv that changes nothing but the record. The sharing cap is always
+# explicit because `auto` vs `off` is the difference between decode waiting one
+# chunk and waiting the whole step.
+PREFILL_ARGS=()
+if [ "${FRANK_ARCHON_PREFILL_INTERLEAVE:-0}" -gt 0 ]; then
+    PREFILL_ARGS=(--prefill-interleave "$FRANK_ARCHON_PREFILL_INTERLEAVE")
+fi
+
 # Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
 # --network host (the server binds the host's addresses directly, no NAT),
 # --init (the engine releases the cards on SIGTERM), seccomp=unconfined
@@ -156,8 +157,9 @@ mkdir -p "$(dirname "$LOG")"
 # makes for host memory), memlock unbounded (the pinned ledger — legal
 # because the root unit grants LimitMEMLOCK=infinity to this dank process),
 # /dev/kfd + /dev/dri (render/video groups are moot for a root container).
-# The template rides a single-file mount; the .rad dir is ro; /data holds
-# the prefix-cache disk tier.
+# The template rides a single-file mount and the .rad dir is ro. The prefix cache
+# has no disk tier any more, so the only host state this container writes is the
+# rad-tune cache under /opt/radiance/share/radiance/tune.
 podman run -d --name "$FRANK_ARCHON_CONTAINER" \
   --network host \
   --init \
@@ -165,7 +167,6 @@ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
   --ulimit memlock=-1:-1 \
   --device /dev/kfd --device /dev/dri \
   -v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-  -v "$FRANK_ARCHON_STATE_DIR":/data \
   -v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
   -v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
   "$FRANK_ARCHON_IMAGE" \
@@ -179,15 +180,13 @@ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
   --placement "$FRANK_ARCHON_PLACEMENT" \
   --host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
   --gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
-  --vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
-  --vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
   --kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
-  --prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
-  --prefix-cache-dir /data/kvcache/frank-archon \
-  --prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
+  --prefix-cache-mib "$FRANK_ARCHON_PREFIX_CACHE_MIB" \
   --ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
   --num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
   --max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
+  --busy-prefill-chunk "$FRANK_ARCHON_BUSY_PREFILL_CHUNK" \
+  "${PREFILL_ARGS[@]}" \
   ${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
   > "$LOG" 2>&1
 

serverAppConfigs-docs/Agent/Agent Docs and Edits.md

@@ -21,7 +21,7 @@ Damage catalogue:
    no longer frontmatter (title/tags/description become body text).
 2. A rewrite **appended** instead of merged → duplicate headings and a stray
    second `---` mid-file.
-3. Literal model artifacts in the body: `<|channel>}<tool_call|>...`,
+3. Literal model artifacts in the body: `<|channel>}<tool_call|>...`
    `+- `, `- - `.
 4. Wrapped bullets re-emitted with a `- ` prefix on every physical line.
 5. Line-range replacement that dropped a sentence tail → dangling fragment,
@@ -46,7 +46,7 @@ Repair order (matters):
 2. If it merely duplicated: **salvage unique content first** — cut the
    one-of-a-kind section, paste it into the copy you keep, THEN delete the
    duplicate ranges. Never delete first.
-3. Re-add missing sentence tails (insert before the orphaned fragment),
+3. Re-add missing sentence tails (insert before the orphaned fragment)
    keeping list continuation lines indented.
 4. Re-run all checks above.
 
@@ -57,7 +57,7 @@ safe, but its snapshot tag changes on every edit — re-read before each
 subsequent edit. `.AGENTTRASH/` is gitignored (`.gitignore:25`), so NEW trash
 stays local — but do not read that as "the repo has none of it": on 2026-10-18
 `git ls-files | grep -c AGENTTRASH` returns **37** tracked paths
-(`.AGENTTRASH/`, `scripts/.AGENTTRASH/`, `dankserver/.AGENTTRASH/`,
+(`.AGENTTRASH/`, `scripts/.AGENTTRASH/`, `dankserver/.AGENTTRASH/`
 `tests/.AGENTTRASH/`), because gitignore never untracks what was committed
 before the rule. Trash is a temporary hold, not an archive.
 

serverAppConfigs-docs/Agent/OMP Harness.md

@@ -16,7 +16,7 @@ this file is repo-managed; edits are harness-side only.
 **Model ids are the StarCraft unit names (official 2026-10-08, on the wire
 since 2026-10-09):** `frank-archon` (Dankserver), `frank-dragoon` and
 `frank-zealot` (Dankclaw) — engine, port and card behind each id live on
-[[frank-archon]], [[frank-dragoon]] and [[frank-zealot]] — formerly `frank-archon`, `frank-dragoon`,
+[[frank-archon]], [[frank-dragoon]] and [[frank-zealot]] — formerly `frank-archon`, `frank-dragoon`
 `frank-zealot`. The SERVICES keep the old names (units, scripts, env
 files, runbook pages); pages written in the old era name the service,
 the harness names the model. The wire now matches: each endpoint's
@@ -26,7 +26,7 @@ picker/grants, the homepage docs and the raw API all use one id per model
 note on the endpoint echoing the requested name).
 
 Verified 2026-10-18: `/v1/models` on 192.168.0.235:7770, 192.168.0.143:7770 and
-192.168.0.143:7771 each answer with the unit name (`frank-archon`,
+192.168.0.143:7771 each answer with the unit name (`frank-archon`
 `frank-dragoon`, `frank-zealot`).
 
 ## Compaction chain arithmetic (verify before ANY window change)
@@ -40,11 +40,11 @@ breaks compaction.
 Where the contract lives:
 
 - `~/.omp/agent/config.yml` → `compaction.remoteEndpoint` (target URL:
-  `http://192.168.0.143:7771/v1/chat/completions`, `config.yml:76`),
-  `methodOrder` (`shake → snapcompact → soft → handoff → remote`,
+  `http://192.168.0.143:7771/v1/chat/completions`, `config.yml:76`)
+  `methodOrder` (`shake → snapcompact → soft → handoff → remote`
   `config.yml:77-82`), `thresholdPercent` (**95**, `config.yml:84`), idle
   settings (`idleEnabled`, `idleThresholdTokens: 100000`)
-- `~/.omp/agent/models.yml` → per-model `contextWindow`, `maxTokens`,
+- `~/.omp/agent/models.yml` → per-model `contextWindow`, `maxTokens`
   `compactionModel` references
 - Implementation: omp plugin
   `~/.omp/plugins/node_modules/@earendil-works/pi-coding-agent/dist/core/compaction/compaction.js`
@@ -104,16 +104,16 @@ input. Boot-shape OOM laddering is a different problem
 ## Per-model sampler config (`compat.extraBody`)
 
 `models.yml` has **no per-model sampler keys**. The global knobs are
-`config.yml`'s `temperature`, `topP`, `topK`, `minP`, `presencePenalty`,
+`config.yml`'s `temperature`, `topP`, `topK`, `minP`, `presencePenalty`
 `repetitionPenalty`, and a knob reaches the wire only when it is set — the
-bundle guards each one with `!== void 0` (e.g. `minP!==void 0)e.min_p=t.minP`),
+bundle guards each one with `!== void 0` (e.g. `minP!==void 0)e.min_p=t.minP`)
 and this workstation's `config.yml` sets **none** of them. The 2026-10-05 note
 that the unset default is `-1`/"not sent" is retired: there is no `-1`
 sentinel, an unset key is simply absent. The per-model hook is
 `compat.extraBody`:
 
 **Why it works:** `applyOpenAIExtraBody` runs **last** in the
-openai-completions builder — after `applyChatCompletionsCompatPolicy` — via
+openai-completions builder — after `applyChatCompletionsCompatPolicy` —
 `Object.assign(params, extraBody)`. So extraBody overrides global settings
 and compat-policy output.
 
@@ -191,7 +191,7 @@ assistant turn carrying a long `reasoning_content`, then compare
 `usage.prompt_tokens` with the flag true vs false. A large delta means the
 field changed template rendering; zero delta means it was ignored.
 
-Cleanup: trash the probe profile and echo server (`profiles/.AGENTTRASH`),
+Cleanup: trash the probe profile and echo server (`profiles/.AGENTTRASH`)
 never `rm`.
 
 ## Static-preamble audit ("the context feels bloated")
@@ -205,7 +205,7 @@ What loads statically every session (measure each, don't guess):
    ```bash
    python3 - <<'EOF'
    import glob, re, os
-   for root in [os.path.expanduser("~/.omp/agent/managed-skills"),
+   for root in [os.path.expanduser("~/.omp/agent/managed-skills")
                 os.path.expanduser("~/.omp/agent/skills"), ".omp/skills"]:
        n=tot=0
        for sk in glob.glob(root+"/*/SKILL.md"):
@@ -305,4 +305,4 @@ standalone; fold go/no-go arithmetic skills into their probe skill
 - [[Agent Docs and Edits]] — editing vault docs and hashline targets safely
 - [[Command Center Dotfiles]] — the `~/.omp` repo deploy this config rides
 - [[Llama Cpp Serving Sizing]] — the engine-side sizing the chain must fit
-- [[LLM Bench Windows and Measurement]] — measuring the endpoints this config routes
+-  measuring the endpoints this config routes

serverAppConfigs-docs/Agent/index.md

@@ -9,4 +9,4 @@ docs get edited safely, and how the agent's home is deployed.
 
 ## Related
 
-- [LLM](../LLM/index.md) — the fleet these agents operate
+-  the fleet these agents operate

serverAppConfigs-docs/Architecture/Container Network Namespaces.md

@@ -47,7 +47,7 @@ network namespace**:
 
 `Network=` values are read from the live Quadlet sources in
 `/home/dank/.config/containers/systemd/*.container`; runtime network names on the
-host at the same date were `podman`, `dankclaw_default`, `systemd-gitea`,
+host at the same date were `podman`, `dankclaw_default`, `systemd-gitea`
 `systemd-searxng`, `systemd-open-webui`.
 
 Two consequences that are easy to miss:
@@ -83,7 +83,7 @@ podman exec <name> cat /proc/net/route          # default route → gateway IP
 podman exec <name> sh -c 'timeout 2 bash -c "exec 3<>/dev/tcp/10.89.1.1/8080"; echo rc=$?'
 ```
 
-Verified outcomes of that recipe on 2026-09-02: from inside `hermes-dashboard`,
+Verified outcomes of that recipe on 2026-09-02: from inside `hermes-dashboard`
 `10.89.1.1:3000` (Gitea, published), `:8081` (SearXNG via Gluetun's netns,
 published) and `:9119` (its own published port) connect; `:8080` and `:8787`
 are **refused instantly** even though both listeners are bound to `0.0.0.0` in

serverAppConfigs-docs/Architecture/Deploy State Triage.md

@@ -36,7 +36,7 @@ possibly zero behavioral change.
 2. Probe the target (ssh as `dank`, passwordless sudo present):
    - `cat ~/.local/state/serverAppConfigs/deploy-state/<svc>`
    - `git -C ~/serverAppConfigs rev-parse HEAD`
-   - `systemctl is-active/is-enabled <svc>`;
+   - `systemctl is-active/is-enabled <svc>`
      `curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:<port>/health`
    - the on-host env file already carries the new keys (it landed via the
      ff-pull).
@@ -119,8 +119,8 @@ each, kill stale ones by explicit PID.
 A killed deploy leaves behind:
 
 - the unit **stopped/failed** (`Restart=always` does NOT fire after an
-  explicit stop — start it yourself);
-- the state marker **pending** (pre-deploy or in-flight revision);
+  explicit stop — start it yourself)
+- the state marker **pending** (pre-deploy or in-flight revision)
 - the e2e gate un-run (the deployer runs it last).
 
 Reconcile the marker ONLY after the service itself is verified healthy — a
@@ -142,11 +142,11 @@ Boot failures must be read from the journal, not `podman logs`: serve scripts
 numbers — which looks exactly like your change having no effect. Key on
 container identity (`podman ps -a --format '{{.CreatedAt}}'`) and read the
 cause from `journalctl --since "N min ago" | grep -a "<container-name>\["`.
-See [[vLLM Boot Forensics and Config Interrogation]] for the full capture
+See  for the full capture
 procedure.
 
 ## Related
 
 - [[Deployment Flow]] — the deploy pipeline itself
 - [[Quadlet Lifecycle]] — unit ownership and drift diagnosis
-- [[vLLM Boot Forensics and Config Interrogation]] — journal capture
+-  journal capture

serverAppConfigs-docs/Architecture/Deployment Flow.md

@@ -68,7 +68,7 @@ service on the other host.
 | Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
 
 The shared chat template maps to the inference service on the selected target.
-The `frank-npu` endpoint is retired ([[frank-npu]]) and has no deploy handler;
+The `frank-npu` endpoint is retired () and has no deploy handler
 an unrelated large-model deployment never touches it.
 
 ## Caddy safety
@@ -125,6 +125,5 @@ routing. See [[Testing Policy]].
 ## See Also
 
 - [[Testing Policy]]
-- [LLM deployment](../LLM/deployment.md)
 - [[Caddy]]
 - [[SearXNG]]

serverAppConfigs-docs/Architecture/E2E Suite.md

@@ -49,7 +49,7 @@ user sees it; the `bats` seams assert config shape, not behavior.
   *arr/qBittorrent links terminate on FrankBook-Pro, which sleeps when idle, and a
   sleeping Mac mini must never fail an unrelated deploy. The project joins only
   when `E2E_INCLUDE_FRANKBOOK=1` (set by `scripts/deploy.sh` when a frankbook
-  trigger path changed: `tests/e2e/specs/19-frankbook.spec.ts`, `frankbook-pro/**`,
+  trigger path changed: `tests/e2e/specs/19-frankbook.spec.ts`, `frankbook-pro/**`
   `shared/homepage/services.json`, or the `dankclaw/Caddyfile` serving the arr
   vhosts) or when an explicit `--project` argument names it. An `11-admin` edit
   that touches the arr links must therefore be paired with a `19-frankbook` run

serverAppConfigs-docs/Architecture/Network Architecture.md

@@ -15,7 +15,7 @@ One Caddy fronts both hosts. It runs as a container on Dankclaw (192.168.0.143)
 
 As of 2026-08-28 the front door is a two-tier chain: Caddy terminates TLS,
 Anubis verifies the client (DENY known AI scrapers, CHALLENGE everyone
-else), and accepted traffic returns to Caddy's internal listener on `:8016`,
+else), and accepted traffic returns to Caddy's internal listener on `:8016`
 which proxies every request whole to the Authentik server with the Host
 header preserved; Authentik's embedded outpost routes on that Host against
 each proxy provider's `external_host`, serves the login flow in place for
@@ -67,12 +67,12 @@ graph TB
 ```
 
 Guarded (Anubis `:8923` → Caddy `:8016` → Authentik outpost → Caddy `:8017`):
-homepage (`www.`), search, **grafana** (`dankclaw/Caddyfile:128-129`),
+homepage (`www.`), search, **grafana** (`dankclaw/Caddyfile:128-129`)
 prometheus (`:135-137`), loki (`:141-143`), seerr (`:148-150`), marreta
 (`:155-160`), romm (`:167-169`), files (`:172-174`). Exempt / own-auth: `auth.`
 (the IdP login itself), `ai.` (OpenWebUI, its own OIDC), gitea (git smart-HTTP
 clients cannot solve JS challenges — `/info/refs` and `git-*-pack` pass
-unchallenged), pihole (Caddy `basicauth`, `:167`), plex (plain HTTP),
+unchallenged), pihole (Caddy `basicauth`, `:167`), plex (plain HTTP)
 books/jellyfrank (own logins), `hermes.`/`hermesadmin.`, and the *arr vhosts
 that terminate on FrankBook-Pro (`:254-281`, "exempt by construction: own
 auth"). Grafana is **not** exempt even though it signs in through authentik as
@@ -104,13 +104,13 @@ Caddy holds one Let's Encrypt certificate per vhost, issued and renewed through
 `Network=searxng.network` (runtime network `systemd-searxng`), while
 `searxng.container`, `marreta.container` and `marreta-browser.container` join
 the tunnel the modern way — `Network=gluetun.container`
-(`dankclaw/searxng/quadlet/searxng.container:18`,
-`dankclaw/marreta/quadlet/marreta.container:34`,
+(`dankclaw/searxng/quadlet/searxng.container:18`
+`dankclaw/marreta/quadlet/marreta.container:34`
 `dankclaw/marreta/quadlet/marreta-browser.container:25`), the Quadlet
 equivalent of the retired compose `network_mode: service:gluetun`. **Five**
 containers share the VPN exit, not three, and Marreta's outbound fetches leave
 through the same WireGuard tunnel as SearXNG's. Pi-hole is a different kind of
-exception (`Network=slirp4netns`, `dankclaw/pihole/quadlet/pihole.container`);
+exception (`Network=slirp4netns`, `dankclaw/pihole/quadlet/pihole.container`)
 everything else on Dankclaw uses the normal LAN route. This matches KB Pattern
 2 in [[Gluetun]].
 See [[Container Network Namespaces]] for which addresses are reachable from that

serverAppConfigs-docs/Architecture/Quadlet Lifecycle.md

@@ -36,14 +36,14 @@ done
 ## 2. Cutover: compose service → Quadlet unit
 
 Checklist (precedents to copy:
-`dankclaw/{caddy,gitea,searxng,open-webui,seerr}/quadlet/`,
+`dankclaw/{caddy,gitea,searxng,open-webui,seerr}/quadlet/`
 `dankclaw/observability/quadlet/`):
 
 1. Unit file at `<host>/<svc>/quadlet/<name>.container`, mirroring compose
    line for line: `Image=` (`localhost/` prefix for locally built images +
    `Pull=never`), `ContainerName=` (Caddy and scrape configs address
    containers by name), `Network=`, `User=0:0` when the image's own user
-   cannot write a rootless-owned bind mount, `Volume=` (`:ro` preserved),
+   cannot write a rootless-owned bind mount, `Volume=` (`:ro` preserved)
    `Exec=` for compose `command:`, `PublishPort=`, health keys,
    `[Service] Restart=always` + `RestartSec=10`, `[Install]
    WantedBy=default.target`. **Never `systemctl --user enable` the generated
@@ -56,7 +56,7 @@ Checklist (precedents to copy:
    `systemctl --user restart <name>.service` → `wait_for_health`; retire any
    legacy hand-written unit into `~/.AGENTTRASH/` (a leftover
    `~/.config/systemd/user/<svc>.service` that shells out to
-   `podman compose up` collides BY NAME with the generated unit);
+   `podman compose up` collides BY NAME with the generated unit)
    `podman rm -f <container>` so the unit's container can take the name.
 4. Planner: map `<host>/<svc>/**` → service in `scripts/deploy-plan.py`
    AND add a `service_is_allowed` entry; keep `SERVICE_ORDER` dependencies
@@ -123,7 +123,7 @@ the restart policy faithfully resurrects it forever.
 
 Quickest unblock: `podman compose up -d --force-recreate <svc>`. Durable fix
 = the §2 cutover. **Verify the capability, not the status:** `podman exec
-<name> python3 -c "import torch; print(torch.cuda.is_available())"`;
+<name> python3 -c "import torch; print(torch.cuda.is_available())"`
 `ss -ltn | grep <port>` for the actual bind addresses. ComfyUI incident:
 2026-09-16, stale device-less container, 9,146 restarts. History repeats:
 2026-08-26 Caddy (no unit), 2026-09-11 five observability services.
@@ -143,7 +143,7 @@ Quickest unblock: `podman compose up -d --force-recreate <svc>`. Durable fix
   unhealthy but logs look fine, compare the baked
   `{{.Config.Healthcheck.Test}}` array against the intended command before
   believing the status.
-- **Probe a new key before shipping:** drop a throwaway `probe.container`,
+- **Probe a new key before shipping:** drop a throwaway `probe.container`
   `daemon-reload`, `journalctl --user --since "1 min ago" -o cat | grep -i
   "unsupported key"`, `systemctl --user cat probe.service` (shows the
   generated ExecStart verbatim), then remove + reload.

serverAppConfigs-docs/Architecture/Testing Policy.md

@@ -54,4 +54,3 @@ Tests are written first. A failing test means the seam is not satisfied; the con
 ## See Also
 
 - [[Deployment Flow]]
-- [[index]]

serverAppConfigs-docs/Dankclaw/ComfyUI Qwen-Image Workflows.md

@@ -56,8 +56,8 @@ sampling), `sampler_name: euler`, `scheduler: simple`, `denoise: 1.0`, seed
 fixed in the file (`1930` in the art workflow, `2026` in the text one) —
 read from `dankclaw/comfyui/workflows/*.json` node `9`.
 
-Node `class_type`s in both files: `CLIPTextEncode` ×2, `UNETLoader`,
-`CLIPLoader`, `VAELoader`, `EmptySD3LatentImage`, `KSampler`, `VAEDecode`,
+Node `class_type`s in both files: `CLIPTextEncode` ×2, `UNETLoader`
+`CLIPLoader`, `VAELoader`, `EmptySD3LatentImage`, `KSampler`, `VAEDecode`
 `SaveImage` (ids 3–11).
 
 | Workflow | File | Shape | Steps | Made for |
@@ -113,9 +113,9 @@ only `models`, `input`, `output`, `custom_nodes` and the healthcheck file
 Quantized preferred, sized for the 8060S iGPU (GTT-backed — `/system_stats`
 reports 122.4 GB of device VRAM on `cuda:0`, `curl http://127.0.0.1:8188/system_stats`
 2026-10-18; the RTX 3090 is frank-zealot's). All repos verified 2026-10-05 and
-all four still resolve today (`huggingface.co/lightx2v/Qwen-Image-Lightning`,
-`huggingface.co/unsloth/Qwen-Image-Edit-2511-GGUF`,
-`huggingface.co/Comfy-Org/Qwen-Image-DiffSynth-ControlNets`,
+all four still resolve today (`huggingface.co/lightx2v/Qwen-Image-Lightning`
+`huggingface.co/unsloth/Qwen-Image-Edit-2511-GGUF`
+`huggingface.co/Comfy-Org/Qwen-Image-DiffSynth-ControlNets`
 `huggingface.co/QuantStack/Wan2.2-TI2V-5B-GGUF`, 2026-10-18). None are on disk
 yet: `models/` holds no lora/, controlnet/ or Wan/Edit file (`ls -R` 2026-10-18).
 
@@ -126,7 +126,7 @@ yet: `models/` holds no lora/, controlnet/ or Wan/Edit file (`ls -R` 2026-10-18)
 | 3 | **DiffSynth ControlNets** (structure control) | `Comfy-Org/Qwen-Image-DiffSynth-ControlNets` | `split_files/model_patches/` canny + depth | ~2–3 GB each | Pose/edge/depth-guided layouts — pairs with the legible-text workflow for exact poster composition. The `QwenImageDiffsynthControlnet` node is already in the running ComfyUI (`GET http://127.0.0.1:8188/object_info/QwenImageDiffsynthControlnet` → 200, 2026-10-18) |
 | 4 | **Wan 2.2 TI2V-5B GGUF** (first video model) | `QuantStack/Wan2.2-TI2V-5B-GGUF` | Q4/Q5 GGUF (~6–8 GB) | ~6–8 GB | T2V+I2V in one dense 5B model; the gentlest on-ramp into video generation, runs in ComfyUI core (`wan` is in this build's `CLIPLoader` type enum — `GET http://127.0.0.1:8188/object_info/CLIPLoader` 2026-10-18) |
 
-Placement: #1 → `models/lora/` (create it), #2 → `models/diffusion_models/`,
+Placement: #1 → `models/lora/` (create it), #2 → `models/diffusion_models/`
 #3 → `models/controlnet/` (create it), #4 → `models/diffusion_models/`.
 Fetch with `scripts/download-frank-*.sh` discipline (`-f` + byte-count +
 sha256 checks) or a plain `hf download` with the token in the environment —

serverAppConfigs-docs/Dankclaw/ComfyUI.md

@@ -43,7 +43,7 @@ Dankclaw is an AMD Strix Halo APU (gfx1151). Standard PyTorch ROCm wheels do not
 
 ## GPU device access (rootless), and why the unit exists
 
-The unit passes `/dev/kfd` and `/dev/dri` (`AddDevice=`,
+The unit passes `/dev/kfd` and `/dev/dri` (`AddDevice=`
 `dankclaw/comfyui/quadlet/comfyui.container:45-46`) and uses
 `GroupAdd=keep-groups`. A literal `render` entry cannot work: rootless Podman
 cannot map the group name into the user namespace. The companion udev rule
@@ -115,7 +115,7 @@ generated units.
 The web UI responds on `/` (200 `text/html`, `curl http://127.0.0.1:8188/`
 2026-10-18). ComfyUI also exposes `/system_stats`. On this APU, `/system_stats`
 reports `device=cuda:0 Radeon 8060S Graphics` (type `cuda`, the ROCm device). The
-health probe is the repo file `dankclaw/comfyui/comfyui-healthcheck.py`,
+health probe is the repo file `dankclaw/comfyui/comfyui-healthcheck.py`
 bind-mounted to `/usr/local/bin/comfyui-healthcheck.py` and run as `HealthCmd`
 (`dankclaw/comfyui/quadlet/comfyui.container:52,57`); it GETs
 `http://127.0.0.1:8188/` inside the container, and the container reports
@@ -135,5 +135,5 @@ bind-mounted to `/usr/local/bin/comfyui-healthcheck.py` and run as `HealthCmd`
 - [[Dankclaw Stack]]
 - [[ComfyUI ROCm and Vulkan]] — the live-verified GPU path research
 - [[ComfyUI Qwen-Image Workflows]] — the Qwen-Image 2.1 payload, proven workflows, and recommended model/LoRA additions
-- [[Lemonade and sd.cpp]] — the ggml-family alternative
+-  the ggml-family alternative
 - Knowledge base: [[ComfyUI]] (tools)

serverAppConfigs-docs/Dankclaw/Dankclaw Stack.md

@@ -52,11 +52,10 @@ graph LR
 
 The LLM rows are bare-metal systemd services, not containers, and this repo
 owns their scripts, units and knowledge base (see
-[LLM fleet knowledge base](../LLM/index.md)). Which payload and runtime sits on
+). Which payload and runtime sits on
 each port is era state and lives on the endpoint page, not here. `frank-npu`
 (the XDNA2 NPU endpoint) and `frank-medium` (the predecessor of `frank-archon`
-on the Dankserver port) are both retired — see [[frank-npu]] and
-[[frank-medium-radiance-mxfp4]].
+on the Dankserver port) are both retired — see  and
 
 ## Boot Persistence
 
@@ -72,7 +71,7 @@ Deploy the SearXNG trio with `scripts/deploy-searxng-quadlet.sh dankclaw`, open-
 
 ## See Also
 
-- [[index]] — repository index
+-  repository index
 - [[Network Architecture]] — routing and proxy
 - [[Observability]] — Grafana, Prometheus, logs
-- [LLM fleet knowledge base](../LLM/index.md) — the model-serving endpoints on this host
+-  the model-serving endpoints on this host

serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md

@@ -37,7 +37,7 @@ timestamp: 2026-10-18
 
 ## Why this engine, and not the alternatives
 
-The criterion Frank set on 2026-10-18 was **"whatever is better for live output"**, and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. RTF only has to stay under 1.0 with margin; what separates the arms is first-byte latency and **spread**. Measured on this host the same day (CPU; the full A/B is the section *"Decision: arm A for live output (2026-10-18)"* in [[Local TTS and Voice Cloning]], which is the source of these numbers):
+The criterion Frank set on 2026-10-18 was **"whatever is better for live output"**, and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. RTF only has to stay under 1.0 with margin; what separates the arms is first-byte latency and **spread**. Measured on this host the same day (CPU; the full A/B is the section *"Decision: arm A for live output (2026-10-18)"* in , which is the source of these numbers):
 
 | Measure | Resident `tts-server` (arm A, chosen) | Mainline per-request CLI path (arm B) |
 |---|---|---|
@@ -128,6 +128,6 @@ still open.
 | Everything is ~2× slower after a systemd tweak | a cpuset/cap pin changed `hardware_concurrency()` and the engine resized its thread pools | remove the pin (law 2) |
 | `502` from the façade with `frank-tts upstream request failed` | the façade cannot resolve or reach the engine | `podman network inspect systemd-frank-tts-engine` and `podman inspect frank-tts --format '{{json .NetworkSettings.Networks}}'` — the façade must be attached to the bridge; a `127.0.0.1:8880` or `host.containers.internal:8880` upstream in `scripts/frank-tts.env` is the old shape and cannot work |
 | A voice is missing from the list | its directory has no `reference.wav`, or the name is not lowercase | fix the pack; both are refused loudly by design |
-| `podman exec` in the deploy says `no such file or directory` for `/app/warm-voices.sh` | the image predates the warmer | re-deploy: since 2026-10-18 `build_compose_image` hashes every file the image `COPY`s, not only `Containerfile.engine`, so editing the warmer rebuilds it — see [[deployment]] |
+| `podman exec` in the deploy says `no such file or directory` for `/app/warm-voices.sh` | the image predates the warmer | re-deploy: since 2026-10-18 `build_compose_image` hashes every file the image `COPY`s, not only `Containerfile.engine`, so editing the warmer rebuilds it — see  |
 | The warm pass logs `registered, but the warm synthesis failed` for EVERY voice, while a hand-run `/app/warm-voices.sh` succeeds | `WARM_TEXT` arrived **with its quotes**. podman's `--env-file` does no shell parsing — it takes the literal remainder of the line — so the env-file reader got `"Warm up, one two three."` where `source` gives `Warm up, one two three.`; the quotes plus a backslash then make the synthesis body invalid JSON | the hook strips one matched pair and escapes what is left; prove the value with `podman exec frank-tts-engine env \| grep WARM_TEXT`. The seam pins both readers, because a test that only sources the env file cannot see this |
 | A `502` for `/v1/voices` where the engine answers `200` on `/health` from the same boot | the engine restarted and its in-memory registry is empty again — `/health` is model-loaded, NOT voices-registered | re-run `podman exec frank-tts-engine /app/warm-voices.sh`; that is the same distinction the `frank-tts` healthcheck enforces |

serverAppConfigs-docs/Dankclaw/Frank TTS.md

@@ -18,10 +18,10 @@ audio in that cloned voice. Everything else in the voice stack sits behind it
 the engine ([[Frank TTS Engine]]) publishes no port on purpose, and consumers
 never learn a path.
 
-**Deployed 2026-10-18** — `./scripts/deploy.sh dankclaw --service frank-tts`,
+**Deployed 2026-10-18** — `./scripts/deploy.sh dankclaw --service frank-tts`
 both units `active`, no drop-ins, and the homepage card's `wip` gate came off in
 the same change. Probes on the deployed endpoint: `/health` answers
-`{"status":"ok","backend":"omni-http","voices":["narrator1","narrator2","narrator3"]}`,
+`{"status":"ok","backend":"omni-http","voices":["narrator1","narrator2","narrator3"]}`
 and two different voices rendered the same sentence into **different** bytes
 (24 kHz mono PCM16, 4.24 s, healthy RMS) — so name resolution is real, not a
 pass-through of one default. The operator harness routes its `speech` role here;
@@ -111,6 +111,21 @@ own `/health` answers 200 with the backend name and the voice list it resolved
 without dialing the engine, so "façade healthy, engine down" is a real state: it
 shows up as a failed synthesis, never as a down probe.
 
+## Where the engine computes
+
+CPU, and only CPU is compiled in: `Containerfile.engine` builds `qwentts.cpp` with
+`-DGGML_VULKAN=OFF` written out explicitly, so the container carries `libggml-cpu` and no
+other backend. The realistic accelerator is the gfx1151 iGPU behind that flag — plus
+`/dev/dri` passthrough, the ComfyUI pattern — on the same payload and the same pinned
+commit, and the intended shape is **co-tenancy**: the engine resident on the card beside
+`frank-dragoon`, not a handover. Memory is not what decides that; shader contention is, and
+the census plus both gates are in `report/tts-npu-spike-verdict-2026-10-19.md` §5.
+**The Strix Halo NPU is not a door at all**: XDNA2 executes only precompiled AIE
+partition programs, and AMD's own Linux runtime opens `/dev/accel/accel0`, succeeds at the
+ioctls, then declines this model family's graph without emitting a single log line. The
+measured record and the model-side contract live in `report/` at the repo root
+(`tts-npu-spike-verdict-2026-10-19.md`).
+
 ## Files
 
 | Path | What |
@@ -147,7 +162,7 @@ A `/health` probe job would therefore be structurally never-green, and because
 `LlmEndpointDown` is scoped across the whole `frank-*` job family (the rule set
 lives on [[Prometheus]]), a permanently red `frank-tts` target would page on
 Signal every few minutes for an endpoint that is fine. The engine cannot be
-scraped at all: it publishes nothing. Status lives in `systemctl --user status`,
+scraped at all: it publishes nothing. Status lives in `systemctl --user status`
 `podman healthcheck run`, the two unit journals, and the homepage card, which
 probes `127.0.0.1:7772` for real now — the `wip: down` gate came off with the
 2026-10-18 deploy.
@@ -155,7 +170,7 @@ probes `127.0.0.1:7772` for real now — the `wip: down` gate came off with the
 ## The harness speech role
 
 The operator harness reaches this endpoint through
-`~/.omp/agent/models.yml` — provider `frank-tts (dankclaw)`,
+`~/.omp/agent/models.yml` — provider `frank-tts (dankclaw)`
 `baseUrl: http://192.168.0.143:7772/v1`, `auth: none`, and the runner api
 `openai-speech` on the **model** row (naming a runner api at *provider* level
 fails the whole file's validation and disables every custom provider, which is
@@ -171,7 +186,7 @@ harness bundle and measured the day of the flip:
   `speech.voice` is the TUI's **speak-aloud** voice — its enum is Kokoro's list,
   and it never reaches an HTTP request. So a harness call arrives voice-less by
   default, which is exactly why the endpoint grew `DEFAULT_VOICE`: the declared
-  pack answers a voice-less request and the journal marks that line `(default)`,
+  pack answers a voice-less request and the journal marks that line `(default)`
   while an **unknown** name still 400s with the list that exists.
 - **`speech.voice` must stay a Kokoro name.** Setting it to a pack name does not
   select a pack; it hands the vocalizer a value outside its enum. (The first

serverAppConfigs-docs/Dankclaw/Gitea.md

@@ -33,7 +33,7 @@ The containers share the private bridge network `systemd-gitea`. The quadlet net
 | `dank/command-center` | second public repo on the instance (`GET http://127.0.0.1:3000/api/v1/repos/search` 2026-10-18) |
 | `dank/vllmScripts` | retired 2026-08-22 — merged into `dank/serverAppConfigs` (history preserved) *(unverified 2026-10-18: the anonymous API search returns no match and `/dank/vllmScripts` is 404, so whether the repo still exists privately cannot be checked without a token)* |
 
-The LLM fleet knowledge base: [serverAppConfigs-docs/LLM/index.md](../LLM/index.md)
+The LLM fleet knowledge base:
 
 ## Management
 

serverAppConfigs-docs/Dankclaw/Hermes-Dashboard.md

@@ -30,7 +30,7 @@ entrypoint does not start the dashboard. The dashboard is a separate process, so
 it runs in its own Quadlet unit
 ([`dankclaw/hermes/quadlet/hermes-dashboard.container`](../../dankclaw/hermes/quadlet/hermes-dashboard.container):
 image `localhost/hermes-agent:latest`, `Pull=never`, `Exec=hermes dashboard
---host 0.0.0.0 --port 9119 --no-open --skip-build`, `Restart=always`,
+--host 0.0.0.0 --port 9119 --no-open --skip-build`, `Restart=always`
 `RestartSec=5`). All three Hermes containers mount the same `/data` bind mount.
 
 Unlike its two siblings — `hermes.container` and `hermes-webui.container` both

serverAppConfigs-docs/Dankclaw/Hermes-WebUI.md

@@ -40,7 +40,7 @@ from another LAN host: `/health` → 200, `/` → 302) — **Vhost:**
 
 ## Why a container, not a host process
 
-The gateway's state files are owned by the container-mapped uid (`109999`,
+The gateway's state files are owned by the container-mapped uid (`109999`
 from rootless Podman's `1 → 100000/65536` subuid range — `/etc/subuid` →
 `dank:100000:65536`, and the agent's runtime uid is 10000, so 100000 + 9999 =
 109999; `ls -ld /home/dank/Containers/hermes/data` → uid 109999, 2026-10-18). A fresh rootless
@@ -82,12 +82,12 @@ the sibling dashboard on `9119` is not.
 
 - Dedicated `tunnel` account on Dankclaw (`getent passwd tunnel` → uid 994,
   shell `/usr/sbin/nologin`, 2026-10-18; password locked). Authentication is key-only
-  *(unverified 2026-10-18: `/home/tunnel/.ssh/authorized_keys` is not readable
+  - (unverified 2026-10-18: `/home/tunnel/.ssh/authorized_keys` is not readable
   without `sudo`)*; the account can do nothing except
   authenticate — `ForceCommand /usr/sbin/nologin` kills any session channel
   while the transport stays up for port forwarding.
 - sshd drop-in `/etc/ssh/sshd_config.d/99-tunnel.conf` (exists, 118 B, 2026-08-31)
-  scopes `AllowTcpForwarding yes` to that user only, plus `PermitTTY no`,
+  scopes `AllowTcpForwarding yes` to that user only, plus `PermitTTY no`
   `X11Forwarding no` and `ForceCommand /usr/sbin/nologin` — all four lines are
   present under `Match User tunnel` (`cat` 2026-10-18).
 - Key options on the authorized key pin the blast radius:
@@ -130,7 +130,7 @@ only if inactive (it is never restarted on deploy, so in-flight browser
 sessions survive), re-tags the image alias, then restarts
 `hermes.service`, `hermes-dashboard.service` and
 `hermes-webui.service` in the **user** manager of `dank` (linger must be on for
-them to come up at boot — `loginctl show-user dank -p Linger` → `yes`,
+them to come up at boot — `loginctl show-user dank -p Linger` → `yes`
 2026-10-18), so gateway/dashboard/webui restarts stay isolated from
 inference deploys.
 Before first deploy, set a strong `HERMES_WEBUI_PASSWORD` in the live

serverAppConfigs-docs/Dankclaw/Hermes.md

@@ -18,7 +18,7 @@ bindings, 2026-10-18). Note the **container name drift**: the unit is
 `podman run --name systemd-%N …` (`systemctl --user cat hermes.service`), so
 `podman ps` shows the gateway as **`systemd-hermes`** — the repo unit sets no
 `ContainerName=` key even though its header comment says "container name:
-hermes". The user-facing web surface is [[Hermes-WebUI]] on `:8787`, reached via
+hermes". The user-facing web surface is [[Hermes-WebUI]] on `:8787`, reached
 `hermes.franksdank.cloud` — internal-only by DNS absence (no public Porkbun A
 record; the name resolves from the Pi-hole hosts file,
 `dankclaw/pihole/addn-hosts:39`) — **Docs:**
@@ -37,7 +37,7 @@ designed wiring; the remedy for both halves is in the log line
 
 **Persistent state:** `/home/dank/Containers/hermes/data` on Dankclaw, bind-mounted
 at `/data` in all three Hermes containers (gateway, webui, dashboard —
-`hermes.container:36`, `hermes-webui.container:46`,
+`hermes.container:36`, `hermes-webui.container:46`
 `hermes-dashboard.container:24`). Each
 container additionally gets an anonymous volume at `/opt/data`, which is
 re-created empty on every container recreation — anything that lives only there
@@ -80,7 +80,7 @@ is not durable. The tracked, sanitized configuration baseline is
   `frank-dragoon` — verified from the host with `curl
   http://127.0.0.1:7770/v1/models` → a single served id (2026-10-18; write
   `127.0.0.1`, not `localhost`, because that port is a rootless pasta publish
-  and is IPv4-only);
+  and is IPv4-only)
   `dankserver-small` reaches `http://192.168.0.235:7770/v1` and, **as of
   2026-10-18, that endpoint serves only `frank-archon`** (`GET
   http://192.168.0.235:7770/v1/models` → `[{"id":"frank-archon"}]`). The name
@@ -115,7 +115,7 @@ per session; the browser itself lives in a dedicated daemon container, the
 digest-pinned headless-shell unit `chrome-devtools-protocol.container` (host
 network, loopback-only bind on 127.0.0.1:9222 — `ss -ltnp` 2026-10-18 shows
 `127.0.0.1:9222 headless-shell` and nothing on `0.0.0.0`; image
-`docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5…`,
+`docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5…`
 `dankclaw/hermes/quadlet/chrome-devtools-protocol.container:28`). Both consumers — the Signal
 gateway and the webui server — run in the host network namespace (`podman
 inspect` → `net=host` for both, 2026-10-18), so they can
@@ -248,7 +248,7 @@ What actually breaks, and the fix:
           effort: none                 # mandatory, see below
   ```
 
-  Before copying this: which id is served on which port is endpoint state — see
+  Before copying this: which id is served on which port is endpoint state —
   [[frank-zealot]] and [[frank-archon]]. The provider/model
   pairing above is the shape that was landed in 2026; re-derive it from the live
   config before reusing it.
@@ -282,7 +282,7 @@ Auth contract (verified live 2026-08-31, re-checked 2026-10-18): `POST /api/auth
 on match, 401 on miss. No username field. `/health` answers 200
 unauthenticated (`curl http://127.0.0.1:8787/health` → 200 with
 `{"status":"ok", …}` today) and `/` answers 302 (`curl -o /dev/null -w
-%{http_code} http://127.0.0.1:8787/` → 302 today) — **never verify auth via
+%{http_code} http://127.0.0.1:8787/` → 302 today) — **never verify auth
 those**; probe
 login with a deliberately wrong password (expect 401) and the real one
 (expect 200). Details on the webui surface: [[Hermes-WebUI]].
@@ -311,7 +311,7 @@ login with a deliberately wrong password (expect 401) and the real one
    instead of loopback" — live bind `0.0.0.0:8787`, `ss -ltnp` 2026-10-18;
    off-network ingress is SSH tunnel only —
    user `tunnel`, port **22** (`sshd -T` → `port 22`, 2026-10-18), key-only auth
-   *(unverified 2026-10-18: the per-user `Match` block needs `sudo` to read)*,
+   - (unverified 2026-10-18: the per-user `Match` block needs `sudo` to read)*
    forward remote end
    localhost:8787).
 

serverAppConfigs-docs/Dankclaw/Marreta.md

@@ -19,7 +19,7 @@ returns a stripped, readable version of the page — no paywall chrome, no
 tracking, no cookie walls. It is a Laravel 13 app on the
 `serversideup/php:8.4-fpm-nginx` image (`composer.json` requires
 `php: ^8.4`, `laravel/framework: ^13.0`, `filament/filament: ^5.7` —
-<https://raw.githubusercontent.com/manualdousuario/marreta/main/composer.json>),
+<https://raw.githubusercontent.com/manualdousuario/marreta/main/composer.json>)
 with an optional headless-browser companion for JavaScript-heavy pages.
 
 **Host:** Dankclaw — **vhost:** `marreta.franksdank.cloud` (Caddy, guard-chained)
@@ -31,7 +31,7 @@ with an optional headless-browser companion for JavaScript-heavy pages.
 - Fetches a target URL with a cascade of strategies: plain HTTP (curl), a
   headless browser (Lightpanda) for JS pages, and the Internet Archive
   (Wayback) as a last resort — the three `app/Services/Fetch/*FetchStrategy.php`
-  files upstream (`BrowserFetchStrategy`, `CurlFetchStrategy`,
+  files upstream (`BrowserFetchStrategy`, `CurlFetchStrategy`
   `WaybackFetchStrategy`, dispatched by `app/Services/FetchManager.php`).
 - Strips paywall/consent/tracking markup and re-renders clean HTML.
 - Ships a Filament admin panel at `/admin` (login = `ADMIN_EMAIL` /
@@ -69,7 +69,7 @@ graph LR
 | `marreta-browser.service` | `dankclaw/marreta/quadlet/marreta-browser.container` | `docker.io/lightpanda/browser:nightly` (`:26`) | `gluetun.container` (`:25`) | `9222` (CDP) |
 
 Both units are boot-persistent (`WantedBy=default.target`) and self-healing
-(`Restart=always`, `RestartSec=10` — `marreta.container:62-69`,
+(`Restart=always`, `RestartSec=10` — `marreta.container:62-69`
 `marreta-browser.container:30-34`), per the repo's Boot Persistence rule. Live
 2026-10-18 both report `systemctl --user is-enabled` → `generated` and
 `is-active` → `active`, and `podman inspect` reports marreta **healthy**
@@ -87,14 +87,14 @@ The fix is a single env var, verified against the serversideup source:
 
 - The image sets `ENV NGINX_HTTP_PORT="8080"` and the site template is
   `listen ${NGINX_HTTP_PORT} default_server;`.
-- The image does **not** bake `conf.d/*.conf` (it `rm -rf`s them at build);
+- The image does **not** bake `conf.d/*.conf` (it `rm -rf`s them at build)
   the entrypoint `10-init-webserver-config.sh` runs `envsubst` over every
   `NGINX_*` variable **at container start**.
 - So `Environment=NGINX_HTTP_PORT=8085` in the quadlet relocates the listen
   port off the collision with no custom build.
 
 Marreta then publishes **nothing** of its own — the netns owner (Gluetun)
-publishes `8082:8085` on its behalf (`dankclaw/searxng/quadlet/gluetun.container:32`),
+publishes `8082:8085` on its behalf (`dankclaw/searxng/quadlet/gluetun.container:32`)
 exactly as it publishes `8081:8080` for SearXNG. Caddy dials
 `192.168.0.143:8082` (`dankclaw/Caddyfile:419-422`). Live env confirms the
 relocation: `podman inspect marreta` carries `NGINX_HTTP_PORT=8085`
@@ -127,7 +127,7 @@ on the mechanism: `BrowserFetchStrategy.php` only calls
    (`dankclaw/Caddyfile:159-163`).
 2. **Auth decision** — Anubis dials Caddy's `http://:8016`, which proxies
    whole-request to `authentik-server:9000` (`dankclaw/Caddyfile:22-24,94`).
-3. **Post-auth routing** — authenticated traffic returns to `http://:8017`,
+3. **Post-auth routing** — authenticated traffic returns to `http://:8017`
    where `@marreta host marreta.franksdank.cloud` routes to
    `reverse_proxy 192.168.0.143:8082` (`dankclaw/Caddyfile:419-422`).
 
@@ -164,7 +164,7 @@ Marreta ships `trustProxies(at: '172.16.0.0/12,10.0.0.0/8,192.168.0.0/16,127.0.0
 — that is the `TRUSTED_PROXIES` fallback in upstream `bootstrap/app.php`, and
 this container sets no `TRUSTED_PROXIES`, so the default is what runs
 (`podman inspect marreta` 2026-10-18) — and forces `https` when `APP_URL`
-starts with `https://` (`app/Providers/AppServiceProvider.php:23-24`,
+starts with `https://` (`app/Providers/AppServiceProvider.php:23-24`
 `URL::forceScheme('https')`). With
 `APP_URL=https://marreta.franksdank.cloud` (`dankclaw/marreta/quadlet/marreta.container:41`) the app honors the forwarded
 scheme and client IP through the whole chain, so there is no redirect loop
@@ -234,7 +234,7 @@ netns down and would otherwise orphan it).
 Deployed via `deploy.sh dankclaw --service gluetun --service marreta --service
 caddy` (gluetun first to recreate the tunnel with the new `8082:8085` publish,
 which also cycles SearXNG's netns ref). Authentik `ProxyProvider` `marreta`
-registered (pk 13, `external_host=https://marreta.franksdank.cloud/`,
+registered (pk 13, `external_host=https://marreta.franksdank.cloud/`
 `internal_host=http://caddy:8017`). Results:
 
 - Unauth edge → Anubis challenge page (not 502). TLS cert issued via DNS-01.
@@ -245,14 +245,14 @@ registered (pk 13, `external_host=https://marreta.franksdank.cloud/`,
   (`GET 127.0.0.1:9222/json/version` → `Lightpanda/1.0`).
 - SearXNG unaffected: `:8081` → 200 after the tunnel recreate.
 
-> **Gotcha hit during deploy:** the first seed used a bare-base64 `APP_KEY`,
+> **Gotcha hit during deploy:** the first seed used a bare-base64 `APP_KEY`
 > which 500s every app route (see Secrets above). Fixed by re-keying with the
 > `base64:` prefix before any encrypted data was persisted.
 
 ## Citations
 
 - Marreta upstream: <https://github.com/manualdousuario/marreta> (README,
-  `compose.yml`, `bootstrap/app.php`, `app/Services/Fetch/BrowserFetchStrategy.php`,
+  `compose.yml`, `bootstrap/app.php`, `app/Services/Fetch/BrowserFetchStrategy.php`
   `lang/` directory listing).
 - serversideup/php fpm-nginx port mechanism: `serversideup/docker-php`
   `src/variations/fpm-nginx/Dockerfile` (`NGINX_HTTP_PORT="8080"`) and

serverAppConfigs-docs/Dankclaw/OpenWebUI.md

@@ -111,7 +111,7 @@ The Ollama connection is disabled (`ollama.enable=false`).
 
 - Working dir for imports: `/app/backend`; package name `open_webui`
   (underscore), NOT `openwebui`. `os.environ["WEBUI_SECRET_KEY"]` MUST be set
-  BEFORE importing any `open_webui` module (import-time hard requirement);
+  BEFORE importing any `open_webui` module (import-time hard requirement)
   the secret lives at `/app/backend/.webui_secret_key`.
 - **The `sqlite3` CLI is NOT in the container** — use Python's `sqlite3`
   module (`podman exec open-webui python3 -c "import sqlite3;
@@ -122,8 +122,8 @@ The Ollama connection is disabled (`ollama.enable=false`).
   pydantic `ModelModel` instances (attribute access, not `.get()`), with NO
   `status`/`enabled` attribute — use `is_active` from the DB.
 - **API auth for verification:** `create_token` lives in
-  `open_webui.utils.auth` (NOT `open_webui.utils.utils`);
-  `get_verified_user` requires an `id` claim (not just `sub`);
+  `open_webui.utils.auth` (NOT `open_webui.utils.utils`)
+  `get_verified_user` requires an `id` claim (not just `sub`)
   `SESSION_SECRET = WEBUI_SECRET_KEY`. Mint an admin token inside the
   container:
 
@@ -148,7 +148,7 @@ The Ollama connection is disabled (`ollama.enable=false`).
    `ClientConnectorError`: a network-reachability problem (host LAN IP vs
    `host.containers.internal`), not a model problem.
 
-**Durability:** the model table + connections are DB-only (not git-managed);
+**Durability:** the model table + connections are DB-only (not git-managed)
 a fresh data volume loses them — including the public access grant above.
 For reproducibility, add an idempotent first-start seed script or document
 the setup in the service doc.
@@ -156,7 +156,7 @@ the setup in the service doc.
 ## API access (auth-gated)
 
 Every `/api/*` endpoint requires an authenticated session (SSO login JWT)
-or a per-user API key (`auth.enable_api_keys=true`; keys are minted per
+or a per-user API key (`auth.enable_api_keys=true`; keys are minted
 account under Profile → API Keys). Anonymous web visitors get the `pending`
 role (`ui.default_user_role=pending`), which cannot chat. Verified
 2026-10-07 from loopback `:3005` and over the public vhost:
@@ -166,13 +166,12 @@ bogus bearer token); only `/api/config`, `/api/version` and `/health`
 `POST /api/chat/completions` all return **401** with no key (also with a bogus bearer
 token); only `/api/config`, `/api/version` and `/health` answer publicly, and none of
 them expose the model list. An external program needs an account + API key — there is
-no keyless access. Re-verified 2026-10-18 from loopback: `/health` → `{"status":true}`,
+no keyless access. Re-verified 2026-10-18 from loopback: `/health` → `{"status":true}`
 `/api/version` → 0.11.4, `/api/v1/models/list`, `/api/v1/models` and
 `POST /api/chat/completions` → 401; `/api/config` additionally publishes
 `features.enable_login_form=false`, `features.enable_signup=false` and
 `oauth.providers.oidc="authentik"` — the SSO-only posture, visible from outside.
 
-
 ## Web search
 
 Chat web search is served by [[SearXNG]] on this host, wired in the quadlet
@@ -186,7 +185,7 @@ Chat web search is served by [[SearXNG]] on this host, wired in the quadlet
   The Anubis→Authentik guard chain is a browser mechanism (proof-of-work +
   session flow); a plain HTTP client cannot solve it, and the internal port
   bypasses Caddy entirely.
-- **Limiter:** SearXNG caps non-HTML (JSON) requests at 4/IP/hour (`API_MAX`,
+- **Limiter:** SearXNG caps non-HTML (JSON) requests at 4/IP/hour (`API_MAX`
   a hardcoded Python constant — no config knob). The `pass_ip` list is checked
   FIRST and short-circuits the whole botdetection chain, and OpenWebUI's bridge
   egress (`10.89.x`) is in `pass_ip`, so the cap never applies. Verified: an
@@ -249,7 +248,6 @@ browser-driven and only touches the two exempt vhosts (`ai.` and `auth.`).
   restore the password form: set `ui.enable_login_form` back to `true` in the
   Admin Panel (or DB) and drop the two env vars from the quadlet.
 
-
 ## Boot persistence (Quadlet)
 
 The unit lives at `dankclaw/open-webui/quadlet/open-webui.container`. On the host it installs to `~/.config/containers/systemd/open-webui.container`. The generator turns it into `open-webui.service`. Because the unit sets `[Install] WantedBy=default.target` and user linger is on, it starts at boot. `systemctl --user is-enabled open-webui.service` reports `generated` (the quadlet boot-start state).
@@ -296,7 +294,7 @@ TTS is the fleet's one OpenAI-compatible speech endpoint —
 `http://host.containers.internal:7772/v1` (`audio.tts.openai.api_base_url`
 in the config DB, 2026-10-18), model `frank-tts`, voice `narrator1`. That
 port is the rootless quadlet `frank-tts` (this repo:
-`dankclaw/voiceservice/quadlet/frank-tts.container`,
+`dankclaw/voiceservice/quadlet/frank-tts.container`
 `scripts/frank-tts.env`), and its shim resolves `voice=<name>` against
 `/home/dank/voices/<name>/reference.wav` — a name without a reference is a
 400 listing the voices that exist, never a silent fallback.
@@ -310,7 +308,7 @@ setting). Its engine select takes `""` (Default) or `browser-kokoro`; when
 `browser-kokoro` is chosen the page synthesizes locally and rejects any
 voice outside its fixed pack — so `narrator1` fails with
 `Voice: "narrator1" not found. Should be one of: af_heart, …` **without a
-single request reaching the server**. Personal audio settings are per
+single request reaching the server**. Personal audio settings are
 browser: no `user.settings` row and no container env carries an audio
 override, yet the toast still appears ⇒ the choice lives in that browser's
 own storage. Fix in the UI: Settings → Audio → TTS Engine → **Default**,
@@ -325,7 +323,6 @@ the failing click — that contrast is the instrument: a server-path failure
 leaves a log line and a frank-tts error, a `browser-kokoro` failure leaves
 neither.
 
-
 ## Retired instances
 
 - The compose `open-webui` service (port 3000) was never run: 3000 is Gitea's HTTP port, so it could not bind.
@@ -355,7 +352,7 @@ neither.
   began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three
   id-keyed objects had to move in the same window as the alias flip:
   `model.id`+`name` (`frank-zealot` → `frank-zealot`, `params`
-  `{"context":{"length":200704}}` and the `meta.description` kept),
+  `{"context":{"length":200704}}` and the `meta.description` kept)
   `access_grant` (`resource_type='model'`, `resource_id`, principal
   `user`/`*`, permission `read`), and `openai.api_configs` per-backend
   `model_ids` (index 2 = `:7771` is the enabled family backend; indices 0/1

serverAppConfigs-docs/Dankclaw/Postgres.md

@@ -88,7 +88,7 @@ never enter this repo. See [[Gitea]] and [[Authentik]].
 
 ## Boot persistence (Quadlet)
 
-Both units carry `Restart=always` (`gitea-postgres.container:28`,
+Both units carry `Restart=always` (`gitea-postgres.container:28`
 `authentik-postgres.container:38`) and `[Install] WantedBy=default.target`
 (`gitea-postgres.container:31`, `authentik-postgres.container:41-42`), are installed
 under `~/.config/containers/systemd/`, and report `generated` / `active` with

serverAppConfigs-docs/Dankclaw/SearXNG Providers.md

@@ -143,7 +143,7 @@ engines:
     disabled: false
 ```
 
-Key per-engine keys (each overwrites the engine-module default; see
+Key per-engine keys (each overwrites the engine-module default
 `settings_engines.html`):
 
 | Key | Effect |
@@ -164,7 +164,7 @@ Key per-engine keys (each overwrites the engine-module default; see
 *different* category. When the primary `google` gets CAPTCHA'd it suspends, but
 `google2` still returns results, and vice-versa.
 
-**Parser caveat (read before fiddling with `parsers`):** the modern `google`,
+**Parser caveat (read before fiddling with `parsers`):** the modern `google`
 `duckduckgo`, and `brave` engines ship a **custom `response()`** with hardcoded
 XPath and *ignore* the generic `parsers`/`p_parsers` table. So `parsers:
 [scrapy_html]` under `name: google` mostly does nothing. To make Google
@@ -376,14 +376,14 @@ done
 
 **2. `/config` REST endpoint** lists every engine + enabled/disabled +
 categories — an inventory of what is live. It answers 200 on this instance today
-(`curl http://127.0.0.1:8081/config`, 2026-10-18; each entry carries `enabled`,
+(`curl http://127.0.0.1:8081/config`, 2026-10-18; each entry carries `enabled`
 `categories`, `timeout`, `shortcut`, and it is the source of the engine counts on
 [[SearXNG]]). `?engines=<name>` isolates a single
 provider; `/debug` adds engine diagnostics.
 
 **3. `check.searx.space`** — the hosted official checker probes every engine of
 your instance with a **headless Firefox** (catches Cloudflare/CAPTCHA blocks a
-plain HTTP probe misses) and posts results on searx.space. Add your instance via
+plain HTTP probe misses) and posts results on searx.space. Add your instance
 a GitHub issue in `searxng/searx-instances`, then allow `check.searx.space` IPs
 to reach the SearXNG port.
 
@@ -417,10 +417,10 @@ misleading).
    already replaced — checked 2026-10-18; see [[SearXNG]]). It is
    also immune to the shared-VPN-exit-IP blocking this page is about: it
    authenticates, so reputation of the exit IP is irrelevant.
-5. **Verify first** — `/json` + `?engines=google` and the searx-checker to see
+5. **Verify first** — `/json` + `?engines=google` and the searx-checker to
    exactly which engines are down and why. Confirm the running SearXNG version
    (check whether `parsers`/`rate_limit` exist in the pinned build).
-6. **Use conservative request tuning** — Dankclaw now allows 5 seconds per
+6. **Use conservative request tuning** — Dankclaw now allows 5 seconds
    upstream request (`shared/searxng/settings.yml:26`), caps it at 15 seconds
    (`:27`), and retries once (`:32`). This addresses
    modest VPN latency without multiplying 429/CAPTCHA traffic. Adjust only

serverAppConfigs-docs/Dankclaw/SearXNG.md

@@ -108,7 +108,7 @@ engines:
     disabled: true
 ```
 
-The block above is the repo file (`shared/searxng/settings.yml`, 21 engine stanzas);
+The block above is the repo file (`shared/searxng/settings.yml`, 21 engine stanzas)
 the deployed render at `/opt/searxng/config/settings.yml` differs only in the baked
 `secret_key`, `base_url` and `KAGI_API_KEY` values, and the running engine set is
 censused below.
@@ -154,7 +154,7 @@ what keeps the `trusted_proxies` entry meaningful.
 
 ## Unblocking Google & Search Providers
 
-The current request policy allows a modest amount of VPN latency (`5.0s` per
+The current request policy allows a modest amount of VPN latency (`5.0s`
 request, `15.0s` maximum — `shared/searxng/settings.yml:26-27`) while limiting
 retries to one (`:32`). This can recover slow
 providers, but it cannot fix an upstream CAPTCHA or 429 caused by the shared
@@ -169,10 +169,10 @@ upstream request volume. SearXNG **suspends** offending engines — which is wha
 up as "Google went unavailable". The default suspension is shorter than the folklore:
 see the table in [[SearXNG Providers]]. This is upstream, not local.
 
-Fixes (cheapest first): tune the `outgoing` block (`request_timeout`,
+Fixes (cheapest first): tune the `outgoing` block (`request_timeout`
 `retry_on_http_error`, `suspended_times`), duplicate the `google` engine under a
 fresh `name` so a CAPTCHA ban on one leaves the other returning results, and —
-if Google still re-blocks — route the Google engines at a clean-IP proxy via
+if Google still re-blocks — route the Google engines at a clean-IP proxy
 per-engine `proxies` in `settings.yml`. See [[SearXNG Providers]] for the full
 root cause, the suspend-model table, the `engines:` config, and the
 `/json` verification recipe.
@@ -212,17 +212,16 @@ root cause, the suspend-model table, the `engines:` config, and the
   (`searxng.container:21`), and that is one of the variables SearXNG substitutes at
   load, so both paths carry the same LAN-IP value. Flagged, not fixed here: it is a
   config change, out of scope for docs.
-- Keep the valkey cache container on the network SearXNG can actually reach — see
+- Keep the valkey cache container on the network SearXNG can actually reach —
   [[Valkey]] Gotchas: SearXNG itself has no network of its own (it borrows Gluetun's
   netns), so the network that matters is the one Gluetun and `searxng-valkey` share.
 
-
 ## Kagi (paid API engine)
 
 Added 2026-09-26 (user-ordered) and still enabled: the running build ships a native
-`kagi` engine module (`searx/engines/kagi.py` at the pinned commit `12f8b6515`),
+`kagi` engine module (`searx/engines/kagi.py` at the pinned commit `12f8b6515`)
 which sets `Authorization: Bearer <key>` (`kagi.py:103`) and issues
-`POST https://kagi.com/api/v1/search` (`base_url` + `/api/v1/search`, `kagi.py:76,104,131`),
+`POST https://kagi.com/api/v1/search` (`base_url` + `/api/v1/search`, `kagi.py:76,104,131`)
 and declares `require_api_key: True` (`kagi.py:63`).
 
 - **Enabled**: `weight: 2` alongside bing/duckduckgo, categories
@@ -231,10 +230,10 @@ and declares `require_api_key: True` (`kagi.py:63`).
   `enabled: true`, categories `general, web` (2026-10-18).
 - **Cost control**: `paging: false` — exactly one API credit per aggregated
   query; `timeout: 8.0` so a slow paid query is not wasted (scrapers use 5.0)
-  (`shared/searxng/settings.yml:74-77`; upstream paging default is `True`,
+  (`shared/searxng/settings.yml:74-77`; upstream paging default is `True`
   `kagi.py:67`, so the `false` here is a real override).
-- **Key handling**: the repo file keeps `api_key: "${KAGI_API_KEY}"`;
-  `deploy-remote.sh` bakes the real key into the live settings.yml via
+- **Key handling**: the repo file keeps `api_key: "${KAGI_API_KEY}"`
+  `deploy-remote.sh` bakes the real key into the live settings.yml
   `scripts/searxng-render-settings.py`, reading `/opt/searxng/.env`
   (owner `dank`, mode 0600 — `ls -l /opt/searxng/.env` 2026-10-18; its key names are
   the WireGuard set plus `SEARXNG_SECRET`, `SEARXNG_BASE_URL`, `KAGI_API_KEY`).
@@ -262,15 +261,15 @@ and declares `require_api_key: True` (`kagi.py:63`).
 
 - 262 engines in the build, **75 enabled**, categories of the enabled set led by
   `other` (12), `it` (11), `general` (9), `videos` (8), `stock images` (7).
-- Enabled and matching the policy page: `bing`, `duckduckgo`, `wikipedia`, `kagi`,
-  `github`, `mdn`, `stackoverflow`, `askubuntu`, `superuser`, `docker hub`, `pypi`,
-  `arxiv`, `pubmed`, `semantic scholar`, `reuters`, `bing news`, `duckduckgo news`,
-  `wikinews`, `bing images`, `duckduckgo images/videos/news`, `wikicommons.*`,
+- Enabled and matching the policy page: `bing`, `duckduckgo`, `wikipedia`, `kagi`
+  `github`, `mdn`, `stackoverflow`, `askubuntu`, `superuser`, `docker hub`, `pypi`
+  `arxiv`, `pubmed`, `semantic scholar`, `reuters`, `bing news`, `duckduckgo news`
+  `wikinews`, `bing images`, `duckduckgo images/videos/news`, `wikicommons.*`
   `youtube`, `vimeo`, `dailymotion`, `bing videos`.
-- Disabled, as configured: `google`, `google cse`, `google news`, `google scholar`,
+- Disabled, as configured: `google`, `google cse`, `google news`, `google scholar`
   `brave`, `brave.images`, `brave.news`, `brave.videos`, `qwant`.
 - The three `startpage` stanzas in `shared/searxng/settings.yml:113-124` target
-  engines that **no longer exist in this build** — `/config` lists no `startpage`,
+  engines that **no longer exist in this build** — `/config` lists no `startpage`
   `startpage images` or `startpage news`, so those overrides are no-ops kept as
   intent. *(labelled 2026-10-18)*
 

serverAppConfigs-docs/Dankclaw/Swap Configuration.md

@@ -144,8 +144,8 @@ frank-dragoon `health=200` throughout.
 The previous 8 GiB `/swap.img` (dated 2026-07-21) was staged, not deleted, at
 `dankclaw:~/.AGENTTRASH/20260912-swapfile-8g/swap.img` (8.1 GiB). **That staging
 directory no longer exists** — `ls /home/dank/.AGENTTRASH` on 2026-10-18 lists only
-`20261005-frank-small-ubatch-ab`, `20261006-frank-large-tuning`,
-`20261007-openwebui-modelpicker`, `20261010-fleet-rename`, `20261017-bonsai-scratch`,
+`20261005-frank-small-ubatch-ab`, `20261006-frank-large-tuning`
+`20261007-openwebui-modelpicker`, `20261010-fleet-rename`, `20261017-bonsai-scratch`
 `20261017-prism-build`, `20261017-v2abl-trial`, and there is no `/root/.AGENTTRASH`.
 The old swapfile has therefore been purged; nothing on the host still holds it.
 

serverAppConfigs-docs/Dankclaw/Voice Packs.md

@@ -23,7 +23,7 @@ voice is how a wrong-voice endpoint stays wrong for weeks.
 state that stays out of the repo, mounted read-only at `/voices` by the façade
 unit — **Builder:** `scripts/build-voice-pack.sh` with tunables in
 `scripts/voice-packs.env` — **Seam:** `tests/voice_pack_unit.bats` — **Engine:**
-arm A `qwentts.cpp`, decided in [[Local TTS and Voice Cloning]].
+arm A `qwentts.cpp`, decided in .
 
 Packs are **world-readable on purpose** (`drwxr-xr-x`, owner `dank`): the façade
 container runs as uid 10000 and reads the mount from outside its own uid map.
@@ -137,7 +137,7 @@ invariant to every crop in the tracked pack list.
 1. Pick the source file and hash it: `sha256sum /path/to/sample.wav`. A crop
    offset is a claim about one exact file, so the digest goes in the env with it.
 2. Find a speech-dense stretch **past 10 s**. The trial's recipe, applied once:
-   `ffmpeg -ss 60 -t 600 -i SRC -af silencedetect=noise=-35dB:d=0.6 -f null -`,
+   `ffmpeg -ss 60 -t 600 -i SRC -af silencedetect=noise=-35dB:d=0.6 -f null -`
    build the runs of non-silence, keep runs ≥ 18 s, take the longest (earliest
    on ties), and cap at 25 s. Aim for 20–25 s: shorter than 19.84 s and the
    `.rvq` falls under the floor.
@@ -188,15 +188,15 @@ loud, not a bug. Work through it in order:
 1. Re-read the tool's own numbers rather than trusting this page: `qwen-codec`
    prints `SpeakerEncoder] Loaded: enc_dim=…` and `Pipeline] Ready: hop … , …
    codebooks @ … Hz` on every run, and the pinned tree's
-   `tools/qwen-codec.cpp` documents the packing. `enc_dim × 4` is the `.spk`;
+   `tools/qwen-codec.cpp` documents the packing. `enc_dim × 4` is the `.spk`
    `codebooks × code_bits / 8` bytes per frame × frames is the `.rvq`.
 2. Decide whether the new number is intended. A changed `enc_dim` means the old
    `.spk` files are unreadable by the new engine, so **every** pack must be
    rebuilt, not just the failing one.
 3. Update the four arithmetic lines in `scripts/voice-packs.env`
-   (`VOICE_PACK_SPK_BYTES`, `VOICE_PACK_RVQ_MIN_BYTES`, `VOICE_PACK_RVQ_MAX_BYTES`,
+   (`VOICE_PACK_SPK_BYTES`, `VOICE_PACK_RVQ_MIN_BYTES`, `VOICE_PACK_RVQ_MAX_BYTES`
    and `VOICE_PACK_RVQ_BYTES_PER_FRAME` if the codebook packing changed) **and**
-   `VOICE_PACK_ENGINE_COMMIT`, `VOICE_PACK_TOOL`, `VOICE_PACK_CODEC_MODEL`,
+   `VOICE_PACK_ENGINE_COMMIT`, `VOICE_PACK_TOOL`, `VOICE_PACK_CODEC_MODEL`
    `VOICE_PACK_TALKER_MODEL` to the new build. An env diff that moves the sizes
    without moving the commit is the mistake to avoid.
 4. Rebuild all three packs with `--force` (the retired directories keep the old
@@ -226,7 +226,7 @@ loud, not a bug. Work through it in order:
   narrator1 and narrator2). The builder's first real run is therefore a
   migration, and it should be a `--force` run per voice so the old directories
   are retired rather than left half-updated.
-- The three source files (`/home/blank/audiosample.mp3`,
+- The three source files (`/home/blank/audiosample.mp3`
   `audiosample2.m4b`, `audiosample3.m4b`, digests pinned in
   `scripts/voice-packs.env`) live on the workstation, not on Dankclaw; the
   builder reads them from `$VOICE_PACK_SOURCE_DIR=/home/dank/voice-sources`, so

serverAppConfigs-docs/Dankserver/Calibre.md

@@ -80,7 +80,7 @@ image's own nginx (the desktop), so the content server runs on internal 8090.
 **Auth model (calibre 9.x):** users live in a SQLite userdb whose `pw` column
 is **plaintext** — the file is mode 600. The single admin account is
 bootstrapped once by the deploy handler
-(`deploy_calibre_server` in `scripts/deploy-remote.sh`) via
+(`deploy_calibre_server` in `scripts/deploy-remote.sh`)
 `calibre-server --manage-users -- add`, the password piped on stdin with
 `printf` (a trailing newline from `echo` lands in the stored password and
 breaks every login). The desktop's basic auth (`CUSTOM_USER`/`CUSTOM_PASSWORD`)

serverAppConfigs-docs/Dankserver/Copyparty.md

@@ -83,7 +83,7 @@ The `iv` edition is no longer available for arm32/armv6. Use `dj` for maximum co
 
 ## Volumes
 
-Live deployment: the whole LUKS array is served at URL `/w` via
+Live deployment: the whole LUKS array is served at URL `/w`
 `Volume=/mnt/entropy:/w:ro` in the quadlet unit ([[Entropy Volume]]) —
 read-only **at the mount level**, so no containerized consumer can write into
 the array through this path regardless of account permissions. The ac edition
@@ -111,7 +111,7 @@ convention; see [[#Command-line interface (ac edition)]] above.
 ```
 
 `${COPAY_ACCOUNT}`/`${COPAY_PASSWORD}` are **never** written into the unit:
-they live in the host-local `/home/dank/Containers/copyparty/env` (0600),
+they live in the host-local `/home/dank/Containers/copyparty/env` (0600)
 loaded via `[Service] EnvironmentFile=` so systemd expands them into
 `Exec=-a ${COPAY_ACCOUNT}:${COPAY_PASSWORD}` at start. The password still
 lands in the container argv — visible to anyone who can run
@@ -178,6 +178,6 @@ Upgrade to v1.20.19+ to cover all known advisories. The dirkey fix (v1.20.17) ma
 ## Citations
 
 - GitHub Security Advisories, `9001/copyparty` —
-  https://github.com/9001/copyparty/security/advisories (accessed 2026-10-19);
+  https://github.com/9001/copyparty/security/advisories (accessed 2026-10-19)
   each row's patched version read from `first_patched_version` in the
   `https://api.github.com/advisories/<GHSA-id>` response.

serverAppConfigs-docs/Dankserver/Dankserver Stack.md

@@ -54,5 +54,5 @@ The following services ran on Dankserver and are now retired from this stack:
 
 ## See Also
 
-- [[index]] — repository index
+-  repository index
 - [[Network Architecture]] — routing and proxy

serverAppConfigs-docs/Dankserver/Jellyfin.md

@@ -14,7 +14,7 @@ serving music, movies, and shows over LAN. It replaced the earlier research
 brief; it runs alongside [[Plex]], which remains the primary server for some
 clients.
 
-**Host:** Dankserver (192.168.0.235) — **Access:** `http://192.168.0.235:8096`,
+**Host:** Dankserver (192.168.0.235) — **Access:** `http://192.168.0.235:8096`
 public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
 [[Authentik]] outpost (post-auth route proxies `localhost:8096`).
 
@@ -67,7 +67,7 @@ user to verify the password.
 - LDAP provider `Jellyfin LDAP` (Base DN `dc=franksdank,dc=cloud`, bind/search
   mode `direct`) + application `Jellyfin`.
 - LDAP outpost `Jellyfin LDAP Outpost` → container `authentik-outpost-ldap`
-  (quadlet `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`),
+  (quadlet `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`)
   publishing `389`/`636` on the Dankclaw host for cross-host binds.
 - Service account `svc-jellyfin-ldap` (DN
   `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud`).
@@ -80,7 +80,7 @@ user to verify the password.
 **Jellyfin side** (plugin config, set via admin → Plugins → LDAP-Auth):
 - Server `192.168.0.143:389` (plain LDAP over the trusted LAN), Base DN
   `dc=franksdank,dc=cloud`, bind user = the service-account DN, search
-  attributes `uid,cn,mail,displayName`, uid attr `uid`, username attr `cn`,
+  attributes `uid,cn,mail,displayName`, uid attr `uid`, username attr `cn`
   `CreateUsersFromLdap=true`, all folders enabled.
 
 **Gotcha — the bind credential is a PASSWORD, not a token.** The LDAP bind is

serverAppConfigs-docs/Dankserver/Plex.md

@@ -60,7 +60,7 @@ root runs as the `dank` user, and the deployer adopts the data root with
 ## Integration points
 
 - **Caddy:** Plex is exempt from the Anubis bot challenge
-  (plain-HTTP legacy clients cannot solve JS challenges) — see
+  (plain-HTTP legacy clients cannot solve JS challenges) —
   [[Anubis]] "Exempt by construction".
 - **Tautulli** ([[Tautulli]]) is the companion: it polls PMS on the appnet
   for statistics and notifications and makes no sense without Plex.

serverAppConfigs-docs/Dankserver/RomM.md

@@ -32,7 +32,7 @@ Anubis → Authentik chain ([[Caddy]] routes it post-auth to
 ## Architecture
 
 Current RomM releases require MariaDB plus Redis (or Valkey) for the task
-queue; this deployment runs **redis** (`REDIS_HOST=redis` in the live unit),
+queue; this deployment runs **redis** (`REDIS_HOST=redis` in the live unit)
 not valkey.
 
 ```mermaid
@@ -114,7 +114,7 @@ scan:
 ## Deployment (Quadlet)
 
 Since 2026-09-01 the trio runs as repo-tracked quadlet units, deployed with
-`./scripts/deploy.sh dankserver --service romm` (any of `romm`,
+`./scripts/deploy.sh dankserver --service romm` (any of `romm`
 `romm-mariadb`, `romm-redis` triggers the full ordered cutover):
 
 - Units: `dankserver/romm/quadlet/{romm,romm-mariadb,romm-redis}.container`

serverAppConfigs-docs/Desktop/Noctalia and Niri.md

@@ -45,14 +45,14 @@ Verified: `msg status` returns live JSON state; unknown panel ids return an
 error listing all valid panel ids (clipboard, control-center, launcher,
 polkit, session, setup-wizard, test, tray-drawer, wallpaper).
 
-Verified v5 msg verb grammar (keybinds-safe): `panel-toggle <id>`,
-`session lock|suspend|lock-and-suspend|logout|reboot|shutdown`,
-`volume-up [step]`, `volume-down`, `volume-mute`, `mic-mute`,
+Verified v5 msg verb grammar (keybinds-safe): `panel-toggle <id>`
+`session lock|suspend|lock-and-suspend|logout|reboot|shutdown`
+`volume-up [step]`, `volume-down`, `volume-mute`, `mic-mute`
 `media next|previous|playPause` (graceful no-op, rc 0, with no active
-player), `brightness-up [target] [step]`, `brightness-down`,
-`settings-open [context]`, `settings-toggle [context]`, `config-reload`,
-`wallpaper-get [connector]`, `wallpaper-set <path>`, `wallpaper-next`,
-`color-scheme-get|set`, `theme-mode-get`, `templates-apply`,
+player), `brightness-up [target] [step]`, `brightness-down`
+`settings-open [context]`, `settings-toggle [context]`, `config-reload`
+`wallpaper-get [connector]`, `wallpaper-set <path>`, `wallpaper-next`
+`color-scheme-get|set`, `theme-mode-get`, `templates-apply`
 `screenshot-fullscreen`.
 
 **Wallpaper directory semantics:** empty/unset `[wallpaper] directory`
@@ -81,9 +81,9 @@ Custom palettes live at `~/.config/noctalia/palettes/<Name>.json`
 synced. Missing/invalid file → silent fallback to the builtin palette.
 
 Palette JSON shape = the community file's shape: `dark`/`light` maps of
-`mPrimary`, `mOnPrimary`, `mSecondary`, `mTertiary`, `mError`, `mSurface`,
+`mPrimary`, `mOnPrimary`, `mSecondary`, `mTertiary`, `mError`, `mSurface`
 `mSurfaceVariant`, `mOutline`, `mShadow`, `mHover` plus a `terminal` block
-(`foreground`, `background`, `normal.{black..white}`, `bright.{…}`,
+(`foreground`, `background`, `normal.{black..white}`, `bright.{…}`
 `cursor`, `cursorText`, `selectionFg`, `selectionBg`). NOTE: `noctalia theme
 -o` emits M3 snake_case tokens (`on_primary_container`, …) — that is the
 *generator* format, not this one. Copy an existing community JSON and mutate
@@ -114,7 +114,7 @@ Procedure:
    `noctalia msg color-scheme-set custom "Gruber Red"` → `ok`; confirm with
    `color-scheme-get`.
 5. Re-render every configured template: `noctalia msg templates-apply`.
-   Regenerates `~/.config/ghostty/themes/noctalia`,
+   Regenerates `~/.config/ghostty/themes/noctalia`
    `~/.config/gtk-{3,4}.0/noctalia.css`, kcolorscheme, foot, wezterm,
    starship, etc. Symlinked-into-dotfiles paths show up as modified files
    worth committing.
@@ -216,7 +216,7 @@ Verified on blankpad, niri 26.04 (two PM1601 externals + eDP-1 laptop panel).
 scale, transform, logical pos/size) and `niri --version`. Output **names are
 connector names** (`eDP-1`, `DP-2`, `HDMI-A-1`), not EDID model strings —
 the EDID strings printed before them are decoration. On blankpad the live
-config splits via `include "./cfg/*.kdl"` from `~/.config/niri/config.kdl`,
+config splits via `include "./cfg/*.kdl"` from `~/.config/niri/config.kdl`
 and `cfg/display.kdl` is a **symlink into the dotfiles repo** — edit the
 repo path (that is what gets committed); check with `ls -ld`.
 

serverAppConfigs-docs/FrankBook-Pro/ARR-Stack.md

@@ -38,7 +38,7 @@ live call at a time instead of guessing payload shapes:
 - `'Base Url' must not be empty` when keys were camelCase → **PascalCase is
   required** (`BaseUrl`, `ApiKey`) — schema keys, not the JSON wire keys.
 - `DefaultMetadataProfileId must be > 0` → fetch real ids from
-  `GET /api/v1/metadataprofile` (rows like `(1,'Standard')`);
+  `GET /api/v1/metadataprofile` (rows like `(1,'Standard')`)
   `GET /api/v1/qualitydefinition` returns the quality CATALOG — ids ≠
   profiles.
 - `targetType null` → configContract not registered in that build (this
@@ -87,7 +87,7 @@ settings values live in `prowlarr.db` → `Applications.Settings` JSON (the
    pipeline hides a miss and you conclude "empty field" when the grep itself
    failed.
 4. Decision framing: when posture = gated-with-zero-users, present it as a
-   decision menu (None behind the front-door wall vs External/authentikarr),
+   decision menu (None behind the front-door wall vs External/authentikarr)
    not a silent change — first-account-wins means whoever logs in first owns
    the instance.
 

serverAppConfigs-docs/FrankBook-Pro/prowlarr.md

@@ -70,7 +70,7 @@ http://prowlarr:9696/{indexerId}/download?link=...&file=...   (torrent handoff)
 The global API key is in `config.xml` (`<ApiKey>`); per-application keys are
 only used by Prowlarr's own sync calls, NOT by the feeds. The arr download
 clients authenticate to qBittorrent with the permanent WebUI password stored
-in `/home/dank/.qbpass` (mode 600, owner `dank`) on Dankserver — see
+in `/home/dank/.qbpass` (mode 600, owner `dank`) on Dankserver —
 [[qBittorrent]] for why that password must be pinned in the conf file.
 
 **Readarr caveat:** Readarr is abandoned upstream; its qBittorrent client
@@ -97,13 +97,13 @@ successor; the indexer row itself is healthy.
 - Categories in the indexer row must be a subset of what the feed's `caps`
   advertises; extras log "does not support the following categories" and
   some feeds reject them.
-- **A freshly created indexer row defaults to all-off.** If `enableRss`,
+- **A freshly created indexer row defaults to all-off.** If `enableRss`
   `enableAutomaticSearch`, or `enableInteractiveSearch` aren't set, the
   app logs `Searching indexers for [X]. 0 active indexers` (Lidarr,
   verified). Search proves TV + music chains 2026-08-30: Sonarr returned
   76 live releases for an anime term, Lidarr 18 for YOASOBI – THE BOOK,
   all via the Prowlarr-Nyaa feed.
-- The v1 API collection is `/api/v1/applications` (not `/api/v1/app`);
+- The v1 API collection is `/api/v1/applications` (not `/api/v1/app`)
   GET responses carry `fields: []` — real settings values live in
   `prowlarr.db` → `Applications.Settings` JSON (read-only inspection).
 - Instance name/`InstanceName=Prowlarr` in `config.xml` does NOT prefix the

serverAppConfigs-docs/FrankBook-Pro/qbittorrent.md

@@ -13,7 +13,7 @@ qBittorrent is the torrent download client for the *arr stack. The stack runs on
 not on Dankserver. It receives download requests from Sonarr, Radarr, Lidarr,
 and Readarr via category tagging.
 
-**Host:** FrankBook-Pro (podman machine) — **Access:** WebUI on port `8080`,
+**Host:** FrankBook-Pro (podman machine) — **Access:** WebUI on port `8080`
 published by the [[Gluetun]] unit because qBittorrent shares that container's
 network namespace (a netns-sharer publishes no ports of its own).
 
@@ -121,7 +121,7 @@ Mac shell export `PATH="/usr/local/bin:/opt/homebrew/bin:$PATH"` for
 `podman machine ssh`.
 
 **Access note (2026-09-21):** the Mac's login user is **dank**@192.168.0.112
-(NOT core — `core` is the podman-machine VM user, reached only via
+(NOT core — `core` is the podman-machine VM user, reached only
 `podman machine ssh`). blankpad has key auth for `dank@` (bootstrapped
 2026-09-21) and a `frankbook-pro` alias in `~/.ssh/config`. macOS sshd has a
 low MaxAuthTries: when password-bootstrapping use `-o

serverAppConfigs-docs/Infrastructure/Anubis.md

@@ -93,7 +93,7 @@ and `hermesadmin.` (app-enforced passwords, no public DNS for the first,
 (`dankclaw/Caddyfile:241-251`), and the FrankBook-Pro *arr/qBittorrent vhosts
 (`dankclaw/Caddyfile:260-352`). There is also no vhost at all for ComfyUI, so
 it is exempt by absence — and therefore unauthenticated on the LAN at
-`comfyui.franksdank.cloud:8188`. The guarded set is exactly `www.`, `search.`,
+`comfyui.franksdank.cloud:8188`. The guarded set is exactly `www.`, `search.`
 `grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `romm.`, `files.`
 (`dankclaw/Caddyfile:283-423`).
 
@@ -140,7 +140,7 @@ it is exempt by absence — and therefore unauthenticated on the LAN at
    instead of allowed, the browser's fetch throws and it renders its own
    error page (iOS/macOS browsers phrase it as "the request failed and the
    interceptors did not return an alternative response") instead of the
-   login flow. Verify with a JS-capable client end-to-end — mint via
+   login flow. Verify with a JS-capable client end-to-end — mint
    `make-challenge`, solve, submit with browser-exact query params — before
    declaring a challenge chain healthy.
 7. **Single-file bind mount + git pull = stale policy.** The policy is
@@ -158,7 +158,7 @@ it is exempt by absence — and therefore unauthenticated on the LAN at
    answers any request lacking `X-Real-Ip` with a 500 whose log line reads
    `[misconfiguration] X-Real-Ip header is not set`. The real chain always
    sets it (every guarded Caddy listener carries `header_up X-Real-Ip
-   {remote_host}` — the bats seam enforces the line), so users never see
+   {remote_host}` — the bats seam enforces the line), so users never
    this. But any direct probe of `:8923` (scratch container, curl, a new
    reverse-proxy hop) MUST set the header or it looks like a policy failure.
    Scratch-verify with `-H "X-Real-Ip: 192.168.0.20"`.

serverAppConfigs-docs/Infrastructure/Authentik Outpost Reference.md

@@ -23,8 +23,8 @@ read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
 `7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
 members in `authentik_outposts_outpost_providers`), all with
 `internal_host = http://caddy:8017`, all carrying the **same three flow
-UUIDs** (`default-authentication-flow`,
-`default-provider-authorization-implicit-consent`,
+UUIDs** (`default-authentication-flow`
+`default-provider-authorization-implicit-consent`
 `default-provider-invalidation-flow`), and each backed by an Application
 (the `slug` column below):
 
@@ -67,7 +67,7 @@ Two provisioning rules, each verified live by an outage:
    reload** — including hosts whose providers were fine (verified 2026-09-02
    when three providers provisioned without any flows took down every guarded
    vhost after a server restart). Backfill all three flows (copy from pk 1:
-   `default-authentication-flow`,
+   `default-authentication-flow`
    `default-provider-authorization-implicit-consent` (the implicit-consent
    authorization flow, `default-invalidation-flow`) and the retry loop
    recovers on its next 3-second cycle. The silent killer: while the outpost
@@ -94,7 +94,7 @@ implementation diverges from upstream in ways that bit us; all verified live:
   contains `"authorization_code"` (enum value with underscore), the token
   endpoint rejects every exchange with `invalid_request`.
 - **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
-  `parse_token_request` see no `grant_type` → `unsupported_grant_type`;
+  `parse_token_request` see no `grant_type` → `unsupported_grant_type`
   POST-body client secrets are not accepted — authenticate the confidential
   client with `Authorization: Basic base64(client_id:client_secret)`.
 - **Provider resolution is by `client_id` query param** on the global
@@ -107,7 +107,7 @@ implementation diverges from upstream in ways that bit us; all verified live:
   becomes the *expression source* instead of the context; every call raises a
   swallowed `PropertyMappingExpressionException` and the claim is skipped.
   Scope-grant filtering has the same class of bug: `ScopeMapping` has no
-  provider FK (linkage lives in the `ProviderPropertyMapping` through model),
+  provider FK (linkage lives in the `ProviderPropertyMapping` through model)
   so `filter(provider=...)` always yields an empty queryset and tokens asked
   to carry scopes get zero. Net effect: userinfo/introspection answers with
   `{sub}` only — email/groups/entitlements claims cannot be delivered by this
@@ -156,7 +156,7 @@ Hard-won rules (the first is now contested by the live database):
   and retest.
 - `ak shell` import path for the provider model in 2026.8:
   `from authentik.providers.ldap.models import LDAPProvider`.
-  *(unverified 2026-10-18: importing it outside a configured Django shell fails
+  - (unverified 2026-10-18: importing it outside a configured Django shell fails
   with `ImproperlyConfigured`, and no `ak shell` was driven against the live
   auth service for this check)*
 
@@ -188,13 +188,13 @@ is a `password` field on the `default-user-settings` prompt
 
 Two ORM gotchas: `Flow.stages.add()` fails on this through model
 (`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
-MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
+MTI), so create the `FlowStageBinding` row directly with an explicit `order`
 and a prompt field is a `Prompt` row linked through `PromptStageField`
 (`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
 
 **A blank password must not set an empty password.** `set_password("")` would
 make the account log in with no password, so a validation policy
-(`default-user-settings-no-empty-password`, bound to the prompt via
+(`default-user-settings-no-empty-password`, bound to the prompt
 `PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
 is `""` and allows it when set. Verified with the `PolicyEvaluator`
 (empty → deny, set → allow, absent → allow) and end-to-end by driving
@@ -213,7 +213,7 @@ three `*_flow_id` columns; the proxy table is
 `authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
 really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
 table in 2026.8), the LDAP table is
-`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
+`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`
 `authentik_core_application` attaches to a provider through `provider_id`
 (its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
 is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`

serverAppConfigs-docs/Infrastructure/Authentik.md

@@ -15,20 +15,20 @@ application-level authentication with its proxy outpost. Not wrapped: Gitea,
 OpenWebUI, Jellyfin (LDAP outpost), the *arr stack, Pi-hole (basicauth edge) —
 they handle their own session. Grafana is the exception that is both: it is a
 guarded vhost (`grafana-proxy`, pk 8, locked to Authentik SSO) *and* an OAuth2
-relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
+relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`
 `dankclaw/Caddyfile:374-377`; live provider table in [[Authentik Outpost Reference]]).
 
-- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`;
-  the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`,
+- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`
+  the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`
   2026-10-18)
 - **Host:** dankclaw (`192.168.0.143`)
 - **Name:** `auth.franksdank.cloud`
-- **Stack: five quadlet units, no compose.** `authentik-postgres`,
-  `authentik-server`, `authentik-worker` (`deploy_authentik`,
+- **Stack: five quadlet units, no compose.** `authentik-postgres`
+  `authentik-server`, `authentik-worker` (`deploy_authentik`
   `scripts/deploy-remote.sh:980-1004`) plus two outposts — `authentik-outpost`
-  (the Rust **proxy** outpost that actually serves `/outpost.goauthentik.io/*`,
+  (the Rust **proxy** outpost that actually serves `/outpost.goauthentik.io/*`
   `deploy_authentik_outpost`, `scripts/deploy-remote.sh:1006-1018`) and
-  `authentik-outpost-ldap` (`deploy_authentik_outpost_ldap`,
+  `authentik-outpost-ldap` (`deploy_authentik_outpost_ldap`
   `scripts/deploy-remote.sh:1020-1032`). All five report `generated` +
   `active`, `Restart=always`, `WantedBy=default.target`, with linger on
   (`systemctl --user`, 2026-10-18). The "3 compose services" wording is the
@@ -36,7 +36,7 @@ relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
 - **LDAP outpost:** serves LDAP for Jellyfin — see the section below.
 
 **Split off from this page:** [[Authentik Processes]] owns the per-container
-facts for `authentik-postgres`, `authentik-server` and `authentik-worker`;
+facts for `authentik-postgres`, `authentik-server` and `authentik-worker`
 [[Authentik Outpost Reference]] owns the provider/flow/token lookup material —
 the live proxy-provider inventory, the proxy-provider provisioning rules, the
 OAuth2 relying-party quirks, the LDAP provider objects, the user-settings flow
@@ -136,7 +136,7 @@ routing table, and the Authentik proxy-provider list must be the same set of
 hosts. A guarded host with no provider 404s at login time; a provider with no
 DNS record breaks off-LAN clients.
 
-The live proxy-provider inventory (the per-vhost `external_host` → app mapping),
+The live proxy-provider inventory (the per-vhost `external_host` → app mapping)
 the two provisioning rules the outpost enforces, and this fork's OAuth2
 relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
 
@@ -144,7 +144,7 @@ relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
 
 The second outpost — `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`
 → `authentik-outpost-ldap.service`, one of the five units in this stack —
-serves LDAP for Jellyfin's LDAP-Auth plugin (`enabled`=generated, `active`,
+serves LDAP for Jellyfin's LDAP-Auth plugin (`enabled`=generated, `active`
 `Restart=always`, container up, 2026-10-18). The image is pinned by its **amd64
 single-platform digest**
 `ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9`
@@ -159,7 +159,7 @@ rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect
 .digest'` and pin that digest.
 
 - Ports **389** (ldap) and **636** (ldaps) publish on the LAN
-  (`PublishPort=389:3389`, `636:6636`,
+  (`PublishPort=389:3389`, `636:6636`
   `dankclaw/authentik/quadlet/authentik-outpost-ldap.container:30-31`; `ss -ltnp`
   shows `0.0.0.0:389` and `0.0.0.0:636`, 2026-10-18) and Dankserver can reach
   them — a bare TCP connect from Dankserver to `192.168.0.143:389` succeeds
@@ -196,7 +196,7 @@ every outpost session (see the ORM playbook).
    core quadlets, starts postgres health-gated (`wait_container_healthy
    authentik-postgres 120`), then the server (300 s health wait) and the worker.
    The outposts are separate handlers: `--service authentik-outpost` and
-   `--service authentik-outpost-ldap` (`scripts/deploy-remote.sh:1006-1032`,
+   `--service authentik-outpost-ldap` (`scripts/deploy-remote.sh:1006-1032`
    dispatch at `:1579-1588`).
 2. Open `https://auth.franksdank.cloud` — the initial setup wizard runs. It
    creates the superuser (email + password) and generates the DB schema.
@@ -210,7 +210,7 @@ every outpost session (see the ORM playbook).
 4. For apps with native OIDC: add an **OAuth2/OpenID Connect** provider, create
    the app with the correct redirect URI, and paste the `client_id` /
    `client_secret` into the app's SSO settings. The relying parties that exist
-   today are `grafana` (pk 9), `Seerr OIDC` (pk 12) and `Open WebUI` (pk 14);
+   today are `grafana` (pk 9), `Seerr OIDC` (pk 12) and `Open WebUI` (pk 14)
    **Gitea has no provider row at all** (live provider table, 2026-10-18) — it
    authenticates itself and stays off the chain by design
    (`dankclaw/Caddyfile:110-120`).
@@ -243,7 +243,7 @@ every outpost session (see the ORM playbook).
 The embedded outpost does not re-evaluate group membership per request. Each
 completed authorize writes one row to `authentik_providers_proxy_proxysession`
 whose `session_data` JSON holds the claims (`groups` included) that get served
-for that browser session until the row expires (observed TTLs 1–24 h per
+for that browser session until the row expires (observed TTLs 1–24 h
 row). A session authorized **before** a user was added to a group keeps
 serving the old group list — Caddy-side RBAC header gates then 403 despite
 correct membership, and no authentik event is generated (the 403 happens at
@@ -309,7 +309,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
 000 (connection refused) — the post-auth listener lives in the caddy
 container's netns; probe inside it or grep the config. Non-browser clients
 cannot solve the JS challenge — before flipping, audit service-to-service
-traffic: ingestion/push paths must target internal addresses (`loki:3100`),
+traffic: ingestion/push paths must target internal addresses (`loki:3100`)
 never a public hostname of a guarded service. Exempt-by-construction
 categories stay unguarded: the IdP itself (`auth.`), git smart-HTTP
 endpoints, services with their own complete auth — document the exemption
@@ -336,14 +336,14 @@ EOF
   `sed -n '/objects imported/,$p'` to drop the noise *(unverified 2026-10-18:
   the exact banner text and the 229-object count — no `ak shell` was driven
   against the live auth service for this audit)*.
-- Import paths (each wrong guess costs a failed exec): `User`, `Group`,
+- Import paths (each wrong guess costs a failed exec): `User`, `Group`
   `Application`, `ApplicationEntitlement`, `PropertyMapping` →
   `authentik.core.models`; `ProxyProvider` →
-  `authentik.providers.proxy.models` (NOT `authentik.outposts.models`);
-  `LDAPProvider` → `authentik.providers.ldap.models`;
+  `authentik.providers.proxy.models` (NOT `authentik.outposts.models`)
+  `LDAPProvider` → `authentik.providers.ldap.models`
   `authentik.core.models.application` does NOT exist as an import path; there
   is no `authentik.policies.group` module in this build.
-  *(unverified 2026-10-18: these module paths were not re-executed — importing
+  - (unverified 2026-10-18: these module paths were not re-executed — importing
   outside a configured Django shell fails with `ImproperlyConfigured`)*
 - The live database's table and column names (the 2026.x renames, the MTI parent
   keys, the read-only `psql` audit recipe) are reference lookup:
@@ -376,7 +376,7 @@ downstream —
   `category: user` panels from `services.json`.
 - `openwebui_users` / `openwebui_admins` → ApplicationEntitlements
   **"Open WebUI Users"** and **"Open WebUI Admins"**, both rows attached to the
-  `Open WebUI` application (`authentik_core_applicationentitlement`,
+  `Open WebUI` application (`authentik_core_applicationentitlement`
   2026-10-18), consumed by the "Open WebUI roles" property mapping → `roles`
   claim *(unverified 2026-10-18: the group→entitlement binding and the mapping
   expression were not read; the mapping itself is not visible under that name
@@ -392,7 +392,7 @@ downstream —
   artem 16, kirby 17, ry 18, nic 19, debo 20, freya 21, mitch 22, eric 23 —
   and every one carries the `<user>@franksdank.cloud` address the template
   prescribes. Creation dates (2026-10-07 → 2026-10-15) were not re-checked
-  *(unverified 2026-10-18)*. Template: `username` +
+  - (unverified 2026-10-18)*. Template: `username` +
   `<user>@franksdank.cloud` + display name + `is_active=True` + the password
   set via `set_password`, membership in `Non-Admin Users` + `jellyfin_users` +
   `openwebui_users`. `alex` was recreated from scratch on 2026-10-08 into this
@@ -400,7 +400,7 @@ downstream —
   the row carried no MFA device, token, entitlement, or proxy session, and
   events survive a user delete because `Event.user` is a denormalized JSON
   field, not a FK).
-- EXCLUDE from a user-level account: `homepage-admins`, `jellyfin_admins`,
+- EXCLUDE from a user-level account: `homepage-admins`, `jellyfin_admins`
   `openwebui_admins`, `authentik Admins`, `authentik Read-only`.
 - Other user-category vhosts (SearXNG, Gitea, Calibre-Web, RomM, Copyparty,
   Plex, Seerr, Marreta) need no per-user group: any authenticated user
@@ -429,8 +429,8 @@ More field gotchas (each cost a failed exec): `ApplicationEntitlement.applicatio
 can be null (guard with `getattr`); `LDAPProvider` has
 `search_mode`/`bind_mode`/`base_dn` and lacks `search_base`/`of_groups`/
 `organization` — the backing table
-(`authentik_providers_ldap_ldapprovider`) has exactly `provider_ptr_id`,
-`base_dn`, `certificate_id`, `tls_server_name`, `gid_start_number`,
+(`authentik_providers_ldap_ldapprovider`) has exactly `provider_ptr_id`
+`base_dn`, `certificate_id`, `tls_server_name`, `gid_start_number`
 `uid_start_number`, `search_mode`, `bind_mode`, `mfa_support`
 (`information_schema.columns`, 2026-10-18); `OAuth2Provider` has no `.scopes`
 (scopes are linked through `ProviderPropertyMapping`, which is why
@@ -484,5 +484,5 @@ podman run --rm --network container:<container> --entrypoint /bin/busybox \
 - [Jellyfin](../Dankserver/Jellyfin.md) — consumes the LDAP outpost via its LDAP-Auth plugin
 - [Network Architecture](../Architecture/Network%20Architecture.md) — the flat naming scheme
 - Research: `.docaudit/research/auth.md` — the full provider comparison
-  *(the file is gone from the repo and from the host checkout as of
+  - (the file is gone from the repo and from the host checkout as of
   2026-10-18)*

serverAppConfigs-docs/Infrastructure/FrankBook-Pro.md

@@ -47,7 +47,7 @@ torrent.
     WIREGUARD_PERSISTENT_KEEPALIVE_INTERVAL=25s
 
 The private key is hashed into the VM env file and never enters the repo.
-NAT-PMP inbound forwarding is available on Proton (gluetun supports it via
+NAT-PMP inbound forwarding is available on Proton (gluetun supports it
 `FIREWALL_INPUT_PORTS` + `NATPMP=true`) — deliberately deferred: the e2e
 gate passes outbound-only, and swarm peers needing an inbound port is a
 seeding-era concern, not a grab-only one.
@@ -86,7 +86,7 @@ are unaffected.
   got `Unauthenticated`); the live conf was patched and
   `scripts/deploy-remote.sh` fixed to write the `_PBKDF2` key. Gotcha
   (live, 2026-09-01): after qB 5.x ran, its conf rewrite left a hash line
-  4.6.7 rejects (`invalid credentials` for every login, including the arrs);
+  4.6.7 rejects (`invalid credentials` for every login, including the arrs)
   rewriting the same `@ByteArray` line fresh (same password, 100k
   iterations) restored login. Any future version flip: regenerate the line.
 - `HostHeaderValidation=false` + `ServerDomains=*`: the socat forwarder
@@ -98,7 +98,7 @@ are unaffected.
   path (no auto-TMM), so the default itself points at `Finished Torrents`:
   every grab lands there regardless of client behavior; human review moves
   files onward; nothing writes to the entropy volume.
-- Categories seeded: `Finished Torrents`, `lidarr`, `radarr`, `readarr`,
+- Categories seeded: `Finished Torrents`, `lidarr`, `radarr`, `readarr`
   `tv-sonarr` (all → `Finished Torrents`).
 - **Download clients** (gateway `10.88.0.1:8080`, user `admin`): all five test
   green against live qB — sonarr (`tv-sonarr`), radarr, lidarr, readarr
@@ -109,12 +109,12 @@ are unaffected.
   as "Authentication failure" — that kills runtime auth too, not just API
   creates; the fix was pinning qB itself (below), after which readarr tests
   200 like the rest.
-- **qB pinned to 4.6.7** (`Image=lscr.io/linuxserver/qbittorrent:4.6.7`,
+- **qB pinned to 4.6.7** (`Image=lscr.io/linuxserver/qbittorrent:4.6.7`
   not `latest`): `latest` resolved to 5.x, whose `204` login breaks
   Readarr's frozen client; 4.6.7 answers the legacy `200` + `"Ok."` that
   every client accepts. Pinned in
   `frankbook-pro/qbittorrent/quadlet/qbittorrent.container`.
-- **Prowlarr Applications** push to sonarr/radarr/lidarr (`fullSync`,
+- **Prowlarr Applications** push to sonarr/radarr/lidarr (`fullSync`
   gateway ports). Readarr deliberately omitted: prowlarr's indexer-push
   test to readarr's `/api/v1/indexer/test` 500s (EOL schema drift).
   Legacy `Prowlarr-*` indexer rows carried in the migrated dankserver DBs

serverAppConfigs-docs/Infrastructure/Grafana.md

@@ -46,7 +46,7 @@ The Prometheus datasource is declared in git, not clicked into `grafana.db`:
 address Caddy proxies to, never the loopback-published port — and carries
 `timeInterval: 15s` so Grafana's rate windows cannot be shorter than the
 scrape interval (`prometheus.yml:5`). The `llm-fleet` dashboard is provisioned
-alongside it by `dashboards/provider.yml` (provider `fleet`,
+alongside it by `dashboards/provider.yml` (provider `fleet`
 `updateIntervalSeconds: 30`, `allowUiUpdates: true`).
 
 A Loki datasource is still **not** provisioned — the tree contains exactly one
@@ -70,7 +70,7 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
   Basic client auth, form-encoded token body, and the fork's userinfo-claims
   bug that leaves identity at the `sub` hash only) is in
   [[Authentik Outpost Reference]] → "OAuth2 relying-party quirks". Consequences for Grafana:
-  The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`,
+  The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`
   `ORG_ID=1` and **no** role-attribute mapping (`grafana.container:52-55`), so
   users land as Viewer with `login = <sub hex>`; no email/groups/role claims
   reach Grafana until the evaluator bug is fixed — do not write role mappings
@@ -79,7 +79,7 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
   the host without the database or an admin password. *(unverified 2026-10-18)*)
 - **Never point this unit's `EnvironmentFile` at the repo-root `.env`.** That
   file is a gluetun env file and it carries `HOME=/root`; systemd imports it
-  for the service, podman then looks for its configuration under `/root`,
+  for the service, podman then looks for its configuration under `/root`
   cannot read it, and the unit dies at start with
   `stat /root/.config: permission denied`. The unit reads
   `/home/dank/Containers/grafana/env`, which the deployer seeds with exactly

serverAppConfigs-docs/Infrastructure/Homepage Status Probes.md

@@ -51,7 +51,7 @@ See [[Container Host Reachability]] for the netns rules.
    published nothing → structurally unprobeable from the host).
 3. Probe from the **correct vantage**: from a bridge-net container the host
    is its gateway IP (`/proc/net/route`, e.g. `10.89.1.1` on
-   `dankclaw_default`) — *not* `host.containers.internal` (`169.254.1.2`),
+   `dankclaw_default`) — *not* `host.containers.internal` (`169.254.1.2`)
    which belongs to the default `podman` net (`10.88.0.0/16`). Probe with
    bash `/dev/tcp` (no `python3`/`ip` in these images); escape loop vars so
    they expand **inside** `podman exec bash -c`.
@@ -83,7 +83,7 @@ python3 -m http.server 8899 --bind 127.0.0.1
 ```
 
 Then in the browser: load `http://127.0.0.1:8899/index.html`, **wait ~1.5 s**
-(panels render after the async fetch; checking immediately returns 0 panels),
+(panels render after the async fetch; checking immediately returns 0 panels)
 extract via `tab.evaluate` — LLM cards render `.desc-role`/`.desc-spec`/
 `.desc-inputs`/`.desc-use` from the card's `details` object, other cards a
 plain `.desc`; `getComputedStyle(el).color` confirms tones. Screenshot for

serverAppConfigs-docs/Infrastructure/Homepage.md

@@ -22,7 +22,7 @@ The homepage is live on Dankclaw. `./scripts/deploy.sh dankclaw` copies the stat
 **Docs page shipped (2026-09-17):** `/docs` serves a static user documentation
 page (`shared/homepage/docs.html`) — access walkthrough, the running-model
 guide, media how-tos, and troubleshooting. Per-service `/docs/<svc>/` pages
-remain a separate thread. The three LLM cards (`frank-dragoon`, `frank-medium`,
+remain a separate thread. The three LLM cards (`frank-dragoon`, `frank-medium`
 `frank-zealot`) moved from admin-only to the **user** page with live model
 descriptions and a `docs` link to `/docs#models`.
 
@@ -59,7 +59,7 @@ without which the picker showed "No results found" for every family account
 "equivalent to Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier
 intelligence)", frank-archon "equivalent to Claude Opus 4.8 / OpenAI-Terra
 (same era)", frank-zealot "equivalent to last year's frontier level
-(2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic);
+(2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic)
 pinned in `homepage_config.bats`.
 
 **Protoss rename + Model Information component (2026-10-08):** the model
@@ -91,7 +91,7 @@ same renames, api-id mapping, supervision and compaction notes.
 app card now carries hex-clipped art beside its name (`services.json` `art`
 field, files under `img/apps/`): 19 brand marks tinted amber from the
 Simple Icons CDN plus 19 hand-improvised monogram SVGs (amber hex + VT323
-initials) for apps with no mark — Copyparty, Marreta, Hermes (+dashboard),
+initials) for apps with no mark — Copyparty, Marreta, Hermes (+dashboard)
 node-exporter, Gluetun, OpenWebUI, Calibre-Web, RomM, Seerr, Loki, Alloy,
 Alertmanager, Valkey, Prowlarr, Readarr, Lidarr, Tautulli, ComfyUI. Model
 cards render in the order zealot → dragoon → archon (registry order is the
@@ -143,8 +143,8 @@ per-app colors: Copyparty gold, Marreta brick, Grafana orange, Loki
 indigo, Neo4j network, Alloy violet, Promtail pink, Alertmanager red,
 Prometheus orange, Node Exporter teal, cAdvisor sky, Guake yellow,
 Authentik blue, Seer green radar, RomM a purple GameBoy. New per-service
-fields: `art_bg` (art backdrop — OpenWebUI's dark logo needs `#f5f5f5`),
-`warn` on OpenWebUI (searches ride the Seerr→SearXNG path),
+fields: `art_bg` (art backdrop — OpenWebUI's dark logo needs `#f5f5f5`)
+`warn` on OpenWebUI (searches ride the Seerr→SearXNG path)
 `model.compaction` (the amber "set this as compaction model" callout
 frank-zealot carries). frank-dragoon's card portrait is the SC2 unit
 portrait (fandom wiki `Dragoon_portrait.jpg`, webp→jpeg). The whole
@@ -175,7 +175,7 @@ titles are chamfered plates with the diamond glyph inside the plate.
 Fonts: body is **Share Tech Mono** (legible retro terminal), headings
 are **Rajdhani 600** (the StarCraft UI face) — VT323 is retired; all
 three pages load the pair from Google Fonts. Four tiles that had
-reverted to initials got pictograms: Calibre-Web an open book (orange),
+reverted to initials got pictograms: Calibre-Web an open book (orange)
 RomM a purple GameBoy, Copyparty a blue folder with upload arrow,
 Marreta a crumbling brick wall with a red crack (`img/apps/app-*.svg`).
 frank-dragoon's card accent is phase-crystal teal (`#5fe0d0`), matching
@@ -197,7 +197,7 @@ state — the only animation left is the speaking-voice pulse. The result
 is flat navy panels on crisp 1px borders: same Protoss direction, half
 the pixels.
 
-**Model ids official (2026-10-08):** the card ids `frank-zealot`,
+**Model ids official (2026-10-08):** the card ids `frank-zealot`
 `frank-dragoon`, `frank-archon` are now the harness's official model ids
 in `models.yml`/`config.yml` (see [[OMP Harness]]); the
 systemd services keep the `frank-archon`/`frank-dragoon`/
@@ -205,7 +205,7 @@ systemd services keep the `frank-archon`/`frank-dragoon`/
 one name per model.
 
 **Api ids renamed on the wire (2026-10-09):** the promise in
-`docs.html` is kept — the served model ids are now `frank-zealot`,
+`docs.html` is kept — the served model ids are now `frank-zealot`
 `frank-dragoon`, `frank-archon`, so the docs page no longer carries
 `api id:` annotations, and the curl/opencode/Pi examples send
 `frank-zealot`. The systemd services keep the old names. `docs.html`
@@ -216,7 +216,7 @@ ids, which did not change.
 clip-paths replaced with `border-radius` (the clipped corners read as
 gaps in the 1px border); section-title diamonds moved from absolute
 positioning to a fixed-width inline glyph (aligned, wrap-safe); closed
-WIP items grey out (dim type, desaturated art, muted border via
+WIP items grey out (dim type, desaturated art, muted border
 `:has(.badge.wip-closed)`); the Dragoon tagline is one spliced track
 (`sfx/dragoon-tagline.ogg` — loudnorm-matched, silence-trimmed, 350 ms
 beat; the two SC2 source oggs are trashed); privacy notes became

serverAppConfigs-docs/Infrastructure/LAN-DNS.md

@@ -188,7 +188,7 @@ sudo systemctl restart systemd-resolved && resolvectl flush-caches
 ```
 
 The `~` (route-only) prefix is load-bearing: it routes only those domains to
-Pi-hole. Verify: `getent hosts www.franksdank.cloud` → `192.168.0.143`;
+Pi-hole. Verify: `getent hosts www.franksdank.cloud` → `192.168.0.143`
 `resolvectl status` shows `DNS Domain: ~franksdank.cloud ~franksdank.casa`.
 
 **Layer 2 — browser DoH (the bypass: "curl works, browser times out").** If
@@ -236,7 +236,7 @@ Endpoints (all POST, JSON body):
   `.content.records[]` as some docs claim — parse with
   `jq -r '.records[] | [.id,.name,.type,.content,(.ttl|tostring)] | @tsv'`.
 - Delete: `/dns/delete/{domain}/{id}` (body carries apikey+secretapikey).
-- Edit: `/dns/edit/{domain}/{id}` — pass the full record (`id`, `name`,
+- Edit: `/dns/edit/{domain}/{id}` — pass the full record (`id`, `name`
   `type`, `content`, `ttl`, `prio`).
 
 Quirks that bite (verified live 2026-08-31):

serverAppConfigs-docs/Infrastructure/Loki.md

@@ -55,7 +55,7 @@ The [[Alloy]] agent on Dankclaw is wired to ship container and journal logs to `
 The service answers and the API works; there is simply no data in it. Measured on 2026-10-18:
 
 - `curl -s http://127.0.0.1:3100/loki/api/v1/labels` → `{"status":"success"}` with no `data` key (no label names exist), and a 7-day `query_range` on `{job="systemd-journal"}` returns `"result":[]`.
-- `du -sh /opt/loki/data` → a few hundred KB of directory scaffolding (`wal`,
+- `du -sh /opt/loki/data` → a few hundred KB of directory scaffolding (`wal`
   `chunks`, `compactor`, `tsdb-shipper-active`, `tsdb-shipper-cache`) —
   `find /opt/loki/data -type f` counted **8 files**, all WAL segments,
   cluster-seed and delete-request bookkeeping, **no chunk objects**

serverAppConfigs-docs/Infrastructure/Observability Provisioning.md

@@ -21,7 +21,7 @@ committed.
 | `shared/prometheus/rules/llm-fleet.yml` | alert rules + recording rules |
 | `shared/grafana/provisioning/` | datasources + dashboards (provisioned tree) |
 
-Targets and datasources reference **container names** (`grafana:3000`,
+Targets and datasources reference **container names** (`grafana:3000`
 `loki:3100` — `shared/prometheus/prometheus.yml:52-61`), never
 loopback-published ports; `alloy` is **not scraped at all** (no job in the
 config — its `127.0.0.1:12345` UI is only probed by the homepage timer,
@@ -47,7 +47,7 @@ from `deploy_observability`:
 - `copy_grafana_provisioning` → `/opt/grafana/provisioning/` (`:286-300`)
 
 `scripts/deploy-plan.py` maps `shared/prometheus/**` to `prometheus` (plus
-`alertmanager` when the changed file is `alertmanager.yml`, `:251-256`),
+`alertmanager` when the changed file is `alertmanager.yml`, `:251-256`)
 `shared/loki/**` to `loki` and `shared/alloy/**` to `alloy` (`:257-260`), and
 `shared/grafana/**` to `grafana` (`:261-264`); changes under
 `dankclaw/observability/` select the touched unit and its dependency consumer
@@ -71,9 +71,9 @@ HTTP probe, not a metrics endpoint.
 
 **Add an alert/recording rule** — edit `shared/prometheus/rules/llm-fleet.yml`.
 Current delivered set: **4 alert rules + 2 recording rules** —
-`LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`,
+`LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`
 `LlmEndpointDown`, plus `job:llm_prefix_cache_reuse:rate5m` and
-`job:llm_spec_accept_length:rate5m` (`shared/prometheus/rules/llm-fleet.yml:32-110`),
+`job:llm_spec_accept_length:rate5m` (`shared/prometheus/rules/llm-fleet.yml:32-110`)
 confirmed live: `curl -s http://127.0.0.1:9090/api/v1/rules` (2026-10-18)
 reports 4 alerting + 2 recording. Rules reload with the prometheus service.
 
@@ -101,7 +101,7 @@ curl -s -u admin:<pw> http://127.0.0.1:3002/api/datasources | python3 -m json.to
 
 The two Prometheus queries were run on 2026-10-18 and are what every number on
 this page comes from: 4 alerting + 2 recording rules, and `up` for
-`prometheus`, `node-dankclaw`, `node-dankserver`, `grafana`, `loki`,
+`prometheus`, `node-dankclaw`, `node-dankserver`, `grafana`, `loki`
 `frank-zealot`, `frank-dragoon`, `frank-archon` with the four structural downs
 listed in [[Prometheus]]. The Grafana query needs the local admin password and
 was not run here; the filesystem side (`ls /opt/grafana/provisioning/*`) is
@@ -114,12 +114,12 @@ table (refused = outage, 404/HTML = config defect).
 
 ## Current delivered state (delivery chain live 2026-09-17, re-verified 2026-10-18)
 
-Prometheus → Alertmanager (`dankclaw/observability/quadlet/alertmanager.container`,
+Prometheus → Alertmanager (`dankclaw/observability/quadlet/alertmanager.container`
 loopback `:9093`) → host-side Signal relay (`scripts/signal-relay.py`, user
 unit, `192.168.0.143:18093` — the LAN IP, the one bind a container can reach
 via `host.containers.internal`) → signal-cli, addressed by number
 `+12062476336` (`scripts/signal-relay.py:36-43`). Every hop re-checked
-2026-10-18: `ss -ltnp` shows `python3` on `192.168.0.143:18093`,
+2026-10-18: `ss -ltnp` shows `python3` on `192.168.0.143:18093`
 `curl -s http://192.168.0.143:18093/health` → `ok`, and
 `journalctl --user -u signal-relay` ends with successful `sent to +12062476336:`
 lines for `LlmEndpointDown`, `LlmRequestsQueueing`, `LlmSpecDecodeOff` and
@@ -148,7 +148,7 @@ state lives in Prometheus' process, not on disk — a restart re-stamps
 `activeAt` on a long-standing alert, so a "fresh" alert after a reboot is not
 evidence of a fresh fault. (Alertmanager by contrast **does** persist:
 `--storage.path=/alertmanager` on a host volume keeps silences and the
-notification log — `dankclaw/observability/quadlet/alertmanager.container:39-42`,
+notification log — `dankclaw/observability/quadlet/alertmanager.container:39-42`
 `/opt/alertmanager/data`.)
 
 Remaining known downs are the structural ones above (HTML-scraped probes, Gitea

serverAppConfigs-docs/Infrastructure/Observability.md

@@ -51,13 +51,13 @@ Dankserver runs one member of this family, node-exporter, as its own unit
 ## Exposure
 
 The five HTTP surfaces publish on `127.0.0.1` only — `ss -ltnp` (2026-10-18)
-shows `127.0.0.1:9090`, `127.0.0.1:9093`, `127.0.0.1:3002`, `127.0.0.1:3100`,
+shows `127.0.0.1:9090`, `127.0.0.1:9093`, `127.0.0.1:3002`, `127.0.0.1:3100`
 `127.0.0.1:12345` and no wildcard — because each of them is unauthenticated
 inside, and several carry lifecycle or query APIs. Browser access goes through
 Caddy: each vhost (`dankclaw/Caddyfile:128-145`) proxies to the guard
 `anubis:8923`, and the post-auth handlers route by container name on
-`dankclaw_default` — `loki.franksdank.cloud` → `loki:3100` (`:364-366`),
-`grafana.franksdank.cloud` → `grafana:3000` (`:374-376`),
+`dankclaw_default` — `loki.franksdank.cloud` → `loki:3100` (`:364-366`)
+`grafana.franksdank.cloud` → `grafana:3000` (`:374-376`)
 `prometheus.franksdank.cloud` → `prometheus:9090` (`:379-381`).
 Alertmanager has no vhost at all. node-exporter is the exception that must live
 in the host network namespace, and it binds exactly two addresses rather than
@@ -70,13 +70,13 @@ and the podman bridge subnet `10.89.0.0/16` before the DROP
 
 ## Deployment
 
-`./scripts/deploy.sh dankclaw --service <name>` runs `deploy_observability`,
+`./scripts/deploy.sh dankclaw --service <name>` runs `deploy_observability`
 which installs the unit, copies that service's config
-(`copy_prometheus_config`, `copy_alertmanager_config`, `copy_loki_config`,
+(`copy_prometheus_config`, `copy_alertmanager_config`, `copy_loki_config`
 `copy_alloy_config`, `copy_grafana_provisioning`, plus
 `install_signal_relay` with the alertmanager deploy), then waits on the loopback
-health endpoint (`/prometheus`: `/-/healthy`, `/alertmanager`: `/-/healthy`,
-`/grafana`: `/api/health`, `/loki`: `/ready`, `/alloy`: `/-/healthy`,
+health endpoint (`/prometheus`: `/-/healthy`, `/alertmanager`: `/-/healthy`
+`/grafana`: `/api/health`, `/loki`: `/ready`, `/alloy`: `/-/healthy`
 `/node-exporter`: `/metrics`). Startup order follows the data dependencies —
 `grafana` after `prometheus`, `alloy` after `loki`
 (`grafana.container:32-33`, `alloy.container:38-39`). See

serverAppConfigs-docs/LLM/DFlash DSpark Speculative Decoding.md

@@ -100,7 +100,7 @@ The DFlash drafter proposes, then ngram-mod runs as an additional implementation
 `--spec-type` takes a comma list, and the gate is `ctx_dft` in
 `common/speculative.cpp` (`common_speculative_init_result`): **draftless**
 types (the ngram family — simple, map-k, map-k4v, mod, cache) are added
-unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
+unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`
 `draft-dspark`, `draft-eagle3`) all require a draft CONTEXT. Consequences:
 
 - A chain may hold ONE context-backed drafter plus any number of ngram types.
@@ -201,7 +201,7 @@ Speculative decoding is lossless under rejection sampling: the target verifies e
 On the mid-August 2026 Dankclaw shape `draft-dflash` was the benchmarked winner
 at 4k — 20.2 t/s, against 16.9 with ngram layered on and 8.7 ngram-only; those
 three figures are that retired llama.cpp era's property ([[frank-dragoon]] owns
-the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
+the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see ).
 
 ## Draft Models
 
@@ -257,7 +257,7 @@ DSpark took `ROCm error: an illegal memory access` inside the shared draft
 context with two concurrent 16k prompts; DFlash corrupted multi-slot into
 premature EOS and was retired first. The timeline, the crash class, the
 "concurrency is the trigger, not size" finding and the `PARALLEL=1` workaround
-are on [[qwen3.8-spec-matrix]]; the shape they were measured on is
+are on ; the shape they were measured on is
 [[frank-dragoon]]'s mid-August 2026 era.
 
 ## A/B protocol on a live llama.cpp endpoint
@@ -303,7 +303,7 @@ Protocol:
 
 Read the right counters — per request `timings.draft_n` / `draft_n_accepted` /
 `predicted_per_second`; cumulative `/metrics`
-`llamacpp:spec_decode_num_drafts_total` / `..._accepted_tokens_total`;
+`llamacpp:spec_decode_num_drafts_total` / `..._accepted_tokens_total`
 per-task `draft acceptance = ... (N accepted / M generated)`. `draft_n == 0`
 while tokens flow = the proposer is NOT engaged, whatever the flags say. A
 zero-draft config is a useful negative control: it proves the other config's
@@ -367,7 +367,7 @@ states, so a dense-27B DFlash2 is useless for a 512-expert MoE trunk.
 
 **Step 4 — acceptance reality checks.** Treat 40–60 % as the honest band for a
 matched pair; the arm that anchors the bottom of it on this fleet is the
-depth-7 `draft-dflash` row of [[frank-medium-llamacpp-dflash2]]'s spec table
+depth-7 `draft-dflash` row of 's spec table
 (accepted share, rate and VRAM for every depth are in that table). Cheapest
 proxy for `p` at temp 0 when both models are greedy and share a tokenizer:
 greedy continuation agreement on identical prompts, two API calls, no weights
@@ -381,10 +381,10 @@ drafting.
 **Step 5 — check the attachment surface BEFORE the arithmetic.** Closed
 engines refuse external drafters by design: halogen-flash-server's
 `DRAFTER_CAP` gates `dflash2` on the loaded checkpoint and an unavailable
-name is a deliberate 400 (a silent downgrade would make comparisons lie);
+name is a deliberate 400 (a silent downgrade would make comparisons lie)
 there is no device routing in its loopback token protocol, so a drafter can
 never sit on another card. Pure llama.cpp DOES route the draft:
-`--spec-draft-device/-devd` (default follows `--device`), `-ngld`,
+`--spec-draft-device/-devd` (default follows `--device`), `-ngld`
 `-otd`, `-ctkd/-ctvd`, `--spec-draft-n-max` (default 3), `--spec-draft-p-min`.
 One process sees only the devices its backends were built with — CUDA+Vulkan
 is the combination that shows a CUDA card AND an AMD iGPU (HIP+CUDA cannot
@@ -411,20 +411,16 @@ measurement that would settle it.
 
 - Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
 - DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
-- The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by [[RAM CPU Offloading in VLLM]].
+-The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by .
 - `--speculative-config` is the preferred CLI. Legacy `--spec-method`, `--spec-model`, and `--spec-tokens` flags still exist.
 - Use `rejection_sample_method` default `strict` for lossless output.
 
 ## See Also
 
 - [[Quantization by Hardware Architecture]]
-- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
-- [[Running Qwen3.6 27B FP8 on Dankclaw]]
 - [[Dankclaw System Specs]]
 - [[Dankserver System Specs]]
-- [[VLLM Overview]]
-- [[deployment]]
 - [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in
 - [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic
-- [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands
+-  where the drafter's KV actually lands
 - [[frank-dragoon]] and [[frank-zealot]] — the single-GPU env drop-in A/B vehicle

serverAppConfigs-docs/LLM/Dankclaw Improvements.md

@@ -30,8 +30,8 @@ taken in.
 
 - **Backend**: that era's production was **Vulkan0**, not ROCm. On this box ROCm
   wins prefill and Vulkan wins decode, and the decode gap widens with context:
-  the 64k A/B is [[qwen3.8-strix-optimizations]] §4, the 262k A/B is
-  [[qwen3.8-spec-matrix]] §High-Context Backend Comparison. The "HIP
+  the 64k A/B is  §4, the 262k A/B is
+   §High-Context Backend Comparison. The "HIP
   out-performs Vulkan for decode" claim did not hold on the 8060S.
 - **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
   does not exist on this machine (checked 2026-08-15). GPU DPM is already
@@ -39,7 +39,7 @@ taken in.
 - **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). What
   `frank-zealot` runs now is its own Current shape — this row is not it.
 - **Spec**: DFlash/DSpark are retired. That era's production was `--spec-default`
-  (ngram-mod) with draft-mtp first — timeline on [[qwen3.8-spec-matrix]]; the
+  (ngram-mod) with draft-mtp first — timeline on ; the
   spec setting each endpoint runs today is on its own page.
 - **TTM**: the `pages_limit` raise that removed the ~15 GiB ceiling is recorded
   once on [[Dankclaw System Specs]] (Memory).
@@ -54,8 +54,8 @@ taken in.
 
 That 2026-08 reading is the firmware-VGM era: the BIOS carved ~96 GiB as
 "dedicated" iGPU VRAM, so the kernel saw only ~30 GiB (the same split is
-recorded in [[ling-3.0-flash-trial]] §Why Vulkan + GTT; the 76.2 GiB row is an
-era [[frank-dragoon]] load reading). Every live number on this box — `MemTotal`,
+recorded in  §Why Vulkan + GTT; the 76.2 GiB row is an
+era [[frank-dragoon]] load reading). Every live number on this box — `MemTotal`
 the ~114 GiB GTT aperture that is the real GPU budget, the 2.0 GiB dedicated
 carve and the `ttm` page cap — belongs to [[Dankclaw System Specs]], re-read
 2026-10-18.
@@ -125,9 +125,9 @@ All require root and a reboot for the modprobe change.
 ## Sources
 
 - [[frank-dragoon]] — measured backend/spec numbers
-- [[deployment]] — deploy and update process
+-  deploy and update process
 - AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS)
-- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`,
+- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`
   `common/arg.cpp`, `common/common.h`
 - NotebookCheck Ryzen AI MAX+ 395 analysis
 
@@ -135,5 +135,3 @@ All require root and a reboot for the modprobe change.
 
 - [[frank-dragoon]]
 - [[DFlash DSpark Speculative Decoding]]
-- [[deployment]]
-- [[index]]

serverAppConfigs-docs/LLM/Dankclaw System Specs.md

@@ -32,7 +32,7 @@ Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp
 | Device ID | 0x1586 |
 | Architecture | gfx1151 (Strix Halo) |
 | GPU-Visible Memory | ~114 GiB GTT aperture (the budget; `mem_info_gtt_total` = 122,402,963,456 B read 2026-10-18; rocm-smi's smaller SVM carve-out is not the budget) |
-| Physical RAM | 121 GiB visible to the OS (`free -g` total 121, read 2026-10-18) on a 128 GB part — 8 × 16 GB LPDDR5 ([[ling-3.0-flash-trial]] §hardware). The 64 GiB this page recorded as "corrected 2026-08-17" (from 32 GB) is stale |
+| Physical RAM | 121 GiB visible to the OS (`free -g` total 121, read 2026-10-18) on a 128 GB part — 8 × 16 GB LPDDR5 ( §hardware). The 64 GiB this page recorded as "corrected 2026-08-17" (from 32 GB) is stale |
 
 ### NPU
 
@@ -44,9 +44,9 @@ Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp
 | Driver | amdxdna 0.7.0, in-tree — bound since 2026-10-18 (`lspci -k -s c6:00.1` → `Kernel driver in use: amdxdna`; journal 2026-10-18 01:25:14 `Initialized amdxdna_accel_driver 0.7.0 for 0000:c6:00.1 on minor 0`) |
 | Firmware | 1.1.2.65 (`/usr/lib/firmware/amdnpu/17f0_11/npu_7.sbin` → `npu.sbin.1.1.2.65.zst`, loaded at boot 2026-10-18 01:25:13) |
 | Runtime | None installed. FastFlowLM 0.9.45 + XRT (installed 2026-08-16) was **retired 2026-09-11** with the `amd_iommu=off` decision — see [[dankclaw-iommu-translation-off]]; the package remains at `/home/dank/GitPackages/fastflowlm_0.9.45_ubuntu26.04_amd64.deb` |
-| Status | Not in service — no NPU endpoint is deployed. The device itself is usable since the 2026-10-18 `iommu=pt` boot (`/dev/accel/accel0` present, `crw-rw---- root:render 261,0`); Gitea issue #3 tracks enablement. See [[FastFlowLM NPU Inference]] |
+| Status | Not in service — no NPU endpoint is deployed. The device itself is usable since the 2026-10-18 `iommu=pt` boot (`/dev/accel/accel0` present, `crw-rw---- root:render 261,0`); Gitea issue #3 tracks enablement. See  |
 
-The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name now serves **Ternary-Bonsai-2-27B-PQ2_0** on the RTX 3090 via the llama.cpp PrismML fork (see [[frank-zealot]]); the Gemma4-12B QAT rung it once pointed at is retired history ([[frank-zealot]], 2026-09-17 → 10-01).
+The NPU endpoint is retired (2026-09-11; ). The `frank-zealot` name now serves **Ternary-Bonsai-2-27B-PQ2_0** on the RTX 3090 via the llama.cpp PrismML fork (see ); the Gemma4-12B QAT rung it once pointed at is retired history (, 2026-09-17 → 10-01).
 
 ### ROCm
 
@@ -80,7 +80,7 @@ The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name
 | Interface | Type | Speed | IP | Notes |
 |-----------|------|-------|----|-------|
 | eno1 | onboard | 2500 Mb/s (2.5 GbE) | 192.168.0.143 (DHCP) | Carried the RPC path to Dankserver during the 2026-08-17 pipeline-parallelism trial; no RPC listener runs today (`ss -ltn` shows no :50052, 2026-10-18). Default route via 192.168.0.1. |
-| enx00e04cc2e1e5 | USB-C to Ethernet adapter | 1000 Mb/s | 192.168.0.116 | UNPLUGGED 2026-08-17. While present, the route to 192.168.0.235 went through this adapter (lower route metric), so all RPC traffic was capped at 1 GbE. See [[Pipeline Parallelism/Ethernet Bandwidth Math]] section 0. |
+| enx00e04cc2e1e5 | USB-C to Ethernet adapter | 1000 Mb/s | 192.168.0.116 | UNPLUGGED 2026-08-17. While present, the route to 192.168.0.235 went through this adapter (lower route metric), so all RPC traffic was capped at 1 GbE. See  section 0. |
 | docker0 | bridge | - | 172.17.0.1 | Usually down. |
 
 The L2 gateway/switch at 192.168.0.1 (MAC f0:4b:8a:65:c5:40) is at
@@ -112,5 +112,3 @@ cable between claw and server is required for 10G.
 
 - [[frank-dragoon]]
 - [[Shared Memory Architecture Dankclaw]]
-- [[frank-redo-2026-08-17]]
-- [[FastFlowLM NPU Inference]]

serverAppConfigs-docs/LLM/Dankserver System Specs.md

@@ -30,7 +30,7 @@ read 2026-10-18).
 | Property | Value |
 |----------|-------|
 | Board | **ASUS ProArt B850-Creator WiFi NEO** (Rev 1.xx) — swapped in 2026-09-20 |
-| Previous | ASUS TUF GAMING B650-E WIFI (card #2 was behind the chipset there; see [[Dual-R9700-Cooperation]]) |
+| Previous | ASUS TUF GAMING B650-E WIFI (card #2 was behind the chipset there; see ) |
 | GPU lanes | CPU GPP ports bifurcated **x8/x8** at Gen5: `PCIEX16(G5)_1` → `00:01.1`, `PCIEX16(G5)_2` → `00:01.3` (both bridges: `current_link_width=8`, `current_link_speed=32.0 GT/s`, read 2026-10-18); ~31.5 GB/s per card. Each card sits behind an on-board PCIe switch (`01:00.0`/`02:00.0`, `05:00.0`/`06:00.0` — "Navi 10 XL Upstream/Downstream Port of PCI Express Switch"), so the GPU endpoint itself trains **x16** Gen5 while the CPU-facing leg stays x8 |
 | Atomic routing | `AtomicOpsCap: Routing+` on every bridge in both GPU chains, `AtomicOpsCtl: ReqEn+` on both endpoints — the condition that ruled out TP=2 for the previous board (2026-09-20 census; not re-readable today — `lspci -vv` needs root) *(unverified 2026-10-18)* |
 | Free | `PCIEX16(G4)` (chipset, x4), M.2_2, M.2_3 |
@@ -55,7 +55,7 @@ read 2026-10-18).
 | Architecture | gfx1201 (RDNA4) |
 | VRAM | 32,624 MiB each (`rocm-smi` reports 34,208,743,424 B per card) |
 | Link | PCIe 5.0, endpoint-trained **x16** each behind the on-board switch; the CPU-facing leg is x8 (`00:01.1` / `00:01.3`, both `current_link_width=8` at 32.0 GT/s, read 2026-10-18). No XGMI. Separate IOMMU groups — `03:00.0` → group **15**, `07:00.0` → group **20** (`/sys/bus/pci/devices/.../iommu_group`, read 2026-10-18) |
-| P2P | **Live and in use.** The running engine prints "I device: the 2 cards reach each other's memory directly (peer to peer)" (`podman logs frank-archon`, 2026-10-15 boot). The 2026-09-20 census had predicted the opposite — ACS `ReqRedir+` on both root ports, so traffic hairpinning at the root complex — and that prediction is superseded; see [[Dual-R9700-Cooperation]] |
+| P2P | **Live and in use.** The running engine prints "I device: the 2 cards reach each other's memory directly (peer to peer)" (`podman logs frank-archon`, 2026-10-15 boot). The 2026-09-20 census had predicted the opposite — ACS `ReqRedir+` on both root ports, so traffic hairpinning at the root complex — and that prediction is superseded; see  |
 | PCI | 03:00.0 and **07:00.0** (were `03:00.0` / `0b:00.0` on the B650-E) |
 | iGPU | Raphael (gfx1036) at **11:00.0** — enumerates as ROCm device 2 (rocm-smi GPU[2], 512 MiB). The retired llama.cpp trios hid it with `HIP_VISIBLE_DEVICES=0,1`; the live radiance engine instead enumerates all three AQL devices and ignores it ("3 devices [0:gfx1201 1:gfx1201 2:gfx1036], using 2 of 3, ignoring gfx1036") |
 
@@ -104,8 +104,7 @@ RCCL-backed rebuild needs no new host packages.
 | Serving stack | radiance 1.0.13 container (`stilldeadcode/radiance@sha256:4c3f484564…`), TP=2 across both R9700s — [[frank-archon]]. The llama.cpp `fb27a525` build at `~/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714` is the retired frank-medium artifact |
 
 The stale AMD-repo ROCm 6.3.3 packages (167, `~24.04`-suffixed) that used to
-hijack `libhsa-runtime64` were purged on 2026-09-15 — see
-[[frank-medium-rocmfpx]].
+hijack `libhsa-runtime64` were purged on 2026-09-15 —
 
 ### Active Services
 
@@ -117,12 +116,12 @@ host-network podman container answering `curl 127.0.0.1:7770/health` →
 
 The services this page once called active are gone: `frank-medium.service`
 (llama-server, 32:33 layer split + DFlash2 drafter) is `not-found` — that rung
-ran 2026-09-16 → 09-20 and is documented in [[frank-medium-llamacpp-dflash2]];
+ran 2026-09-16 → 09-20 and is documented in 
 `frank-zealot.service` (FreeToken/CUDA) was retired with the 2026-09-15 GPU
 swap. Its archived unit was moved to `~/.AGENTTRASH/20260915-frank-zealot-retire/host-units/`
 and the vllm-radiance replica trio to `~/.AGENTTRASH/20260916-vllm-radiance-retire/`
 — neither path exists today (checked 2026-10-18), so the archive location is
-unverified. See [[deployment]].
+unverified. See .
 
 ## Hardware Relevance
 
@@ -134,7 +133,7 @@ unverified. See [[deployment]].
   tokens at 32:33, ~6.3/~5.2 GiB headroom) is history. Either way a batch or
   context change needs a re-fit and not a guess.
 - 246 GiB system RAM is **not** headroom: the live shape pins a 43.88 GiB host
-  pool per rank (`FRANK_ARCHON_HOST_POOL_MIB=44928`,
+  pool per rank (`FRANK_ARCHON_HOST_POOL_MIB=44928`
   `scripts/frank-archon.env:184`) plus the 47.68 GiB n-gram/PLE table in RAM,
   and `free -g` showed **90 GiB available** with the engine loaded (read
   2026-10-18). The frank-medium claim that "nothing in the serving path is
@@ -147,6 +146,6 @@ unverified. See [[deployment]].
 ## See Also
 
 - [[frank-archon]] — the active serving runbook
-- [[Dual-R9700-Cooperation]] — why the PCIe path matters, and what the swap changed
-- [[deployment]] — the deploy flow
+-  why the PCIe path matters, and what the swap changed
+-  the deploy flow
 - [[Shared Memory Architecture Dankclaw]] — for contrast, Dankclaw's unified memory

serverAppConfigs-docs/LLM/Dual-R9700-Cooperation.md

@@ -1,327 +0,0 @@
----
-type: reference
-title: Dual R9700 Cooperation
-description: "Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked."
-tags:
-  - frank-medium
-  - rocm
-  - rccl
-  - pcie-atomics
-  - tensor-parallelism
-  - dankserver
-timestamp: 2026-09-16
----
-
-# Dual R9700 Cooperation
-
-Dankserver's two R9700s run **one tensor-parallel engine across both cards** —
-today that is `frank-archon` under radiance 1.0.13, TP=2
-([[frank-archon]]). When this page was written the endpoint
-was `frank-medium`: first llama.cpp's layer split
-([[frank-medium-llamacpp-dflash2]], 2026-09-16 → 09-20), then from 2026-09-20
-one vLLM-radiance process at `tensor_parallel_size 2`
-([[frank-medium-radiance-mxfp4]], 2026-09-20 → 09-26). This page records why
-true tensor parallelism was impossible on the previous board (TUF GAMING B650-E
-WIFI), and what the 2026-09-20 board swap changed.
-
-## The short answer
-
-**The wall is gone.** On the B650-E, card #2 sat behind the chipset, whose
-bridges could not route PCIe atomic operations. RCCL's kernels carry a hidden
-hostcall-buffer parameter and ROCr refuses to supply it to a device without
-atomics, so *no* RCCL collective or P2P operation could be dispatched on that
-card — see the [measured matrix](#measured-matrix) and the
-[2026-09-16 table](#the-same-gate-as-this-board-reports-it-2026-09-16) below.
-
-On 2026-09-20 the board became an **ASUS ProArt B850-Creator WiFi NEO**, which
-bifurcates the CPU's GPP ports **x8/x8**: the board-swap path this page
-recommended, at B850 rather than X870E money. Both cards now hang off CPU root
-ports (`00:01.1`, `00:01.3`), every bridge on both chains reads
-`AtomicOpsCap: Routing+`, and **both endpoints read `AtomicOpsCtl: ReqEn+`** —
-the exact precondition this page's own test plan named, and it is now
-[measured](#measured-verdict-2026-09-20) rather than assumed.
-
-What that bought:
-
-- **Both cooperation modes now work, and they were both run.** vLLM TP=2 booted
-  and served (its R4D allreduce needs no RCCL collective at all), and llama.cpp
-  `--split-mode tensor` booted and served against an RCCL-backed build — so a
-  collective HAS been dispatched on this host since the swap.
-- **TP=2 is the production shape.** It wins on every axis measured, by 2.1x on
-  decode and 3.6x at 4-way concurrency, and it replaced the layer split on the day
-  of measurement. The payload has moved since: the MXFP4 `frank-medium` shape is
-  retired 2026-09-26, and what runs today is the iq4r publisher under radiance
-  1.0.13 ([[frank-archon]]).
-- **The drafter, not the collective, is what llama.cpp lost.** Its tensor mode
-  could not carry the DFlash2 drafter, which is the honest caveat on the
-  comparison below.
-- **The bandwidth story moved too.** Both cards ride Gen5 x8 (~31.5 GB/s each):
-  card 0 effectively unchanged (was Gen4 x16), card 1 up ~16x (was Gen4 x1,
-  ~1.97 GB/s). The M.2→PCIe riser plan is dead — `PCIEX16_2` holds card #2 now
-  and M.2_2 is free.
-
-## The gate, in source
-
-```cpp
-void Device::checkAtomicSupport() {                    // rocclr/device/rocm/rocdevice.cpp:181
-  std::vector<amd::Device::LinkAttrType> link_attrs;
-  link_attrs.push_back(std::make_pair(LinkAttribute::kLinkAtomicSupport, 0));
-  if (findLinkInfo(cpu_agent_info_->fine_grain_pool, &link_attrs)) {
-    if (link_attrs[0].second == 1) {
-      info_.pcie_atomics_ = true;                      // no else branch
-    }
-  }
-}
-```
-
-The flag is a **queried hardware property**, consumed at
-`rocvirtual.cpp:3954` (the hostcall/AQL dispatch path), in `rocblit.cpp` (SDMA
-copy paths) and exported as `hipDeviceProp.hostNativeAtomicSupported`. There is
-no environment variable, module parameter, or sysfs toggle around it — verified
-by grepping ROCclr for `getenv`/`HSA_*`/`ROCR_*`/`AMD_*` in the gate's
-neighborhood: none. Writing the endpoint's Device Control 2 bit 6 by hand
-(`setpci -s 0b:00.0 CAP_EXP+28.w=0440`) does nothing because the value is
-consumed at driver probe time.
-
-What has to be true upstream of it: every PCIe bridge on the GPU's path to the
-CPU must advertise `AtomicOpsCap: Routing+`. On this board card #2's path
-includes `05:00.0` and `06:09.0`, both `Routing-` — a **read-only capability
-bit**, so no software can enable it.
-
-### The same gate, as this board reports it (2026-09-16)
-
-`lspci -vv` on dankserver. Card 0's chain is root port `00:01.1` → `01:00.0` →
-`02:00.0` → `03:00.0`; card 1's is `00:02.1` → `05:00.0` → `06:00.0` →
-`06:09.0` → `09:00.0` → `0a:00.0` → `0b:00.0`:
-
-| Device | Role | `AtomicOpsCap` | `AtomicOpsCtl` | Link |
-|---|---|---|---|---|
-| `00:01.1` | CPU root port → PCIEX16_1 | Routing+ | — | x16 |
-| `03:00.0` | **card 0** | 32bit+ 64bit+ | **ReqEn+** | x16 |
-| `00:02.1` | CPU root port → chipset | **Routing+** | — | x4 |
-| `05:00.0`, `06:00.0`, `06:09.0` | B650 bridges | **Routing−** | — | x1 at `06:09.0` |
-| `09:00.0`, `0a:00.0` | PCIe switch ports | Routing+ | — | x1 (downgraded) |
-| `0b:00.0` | **card 1** | 32bit+ 64bit+ | **ReqEn−** | x16 |
-
-Three things fall out of that table, none of them in the source alone:
-
-- **The CPU side is fine** — `00:02.1` is `Routing+`. The wall is inside the
-  chipset, not the platform, which is why a slot on CPU lanes is the fix.
-- **The endpoint's own bit differs between the cards, and that is the kernel's
-  doing.** `pci_enable_atomic_ops_to_root()` walks the bridge chain and refuses
-  to set Requester Enable when any bridge cannot route atomics. ROCclr's
-  `pcie_atomics_` and the endpoint's `ReqEn` are two faces of one cause, and it
-  makes `lspci -vv -s <bdf> | grep AtomicOpsCtl` a one-command diagnosis: on
-  this host card 0 reads `ReqEn+` and card 1 `ReqEn−`.
-- **Card 1's effective link is x1**, not the x16 its own capability block
-  advertises: `06:09.0` trained at x1, so ~2 GB/s is the real ceiling. That is a
-  bandwidth fact rather than an atomics one, and it only matters for
-  collectives — a 27B TP=2 allreduce is ≈5.6 ms/token at x1 against ≈1.4 ms at
-  x4. The layer split's boundary copy does not care.
-
-## Re-measured 2026-09-20 (ProArt B850-Creator WiFi NEO)
-
-`lspci -vv` after the board swap. Card 0's chain is `00:01.1` → `01:00.0` →
-`02:00.0` → `03:00.0`; card 1's is `00:01.3` → `05:00.0` → `06:00.0` →
-`07:00.0`. **No chipset bridge is in either path** — `00:02.1` now carries only
-LAN/SATA/WiFi:
-
-| Device | Role | `AtomicOpsCap` | `AtomicOpsCtl` | Link |
-|---|---|---|---|---|
-| `00:01.1` | CPU GPP → PCIEX16_1 | Routing+ | — | Gen5 x8 |
-| `01:00.0` | card 0 switch upstream | Routing+ | — | Gen5 x8 |
-| `02:00.0` | card 0 switch downstream | Routing+ | — | Gen5 x16 |
-| `03:00.0` | **card 0** | 32bit+ 64bit+ | **ReqEn+** | Gen5 x16 (on-card) |
-| `00:01.3` | CPU GPP → PCIEX16_2 | Routing+ | — | Gen5 x8 |
-| `05:00.0` | card 1 switch upstream | Routing+ | — | Gen5 x8 |
-| `06:00.0` | card 1 switch downstream | Routing+ | — | Gen5 x16 |
-| `07:00.0` | **card 1** | 32bit+ 64bit+ | **ReqEn+** | Gen5 x16 (on-card) |
-
-- Both endpoints read `CAP_EXP+28.w = 0x1440` — bit 6 set, which is the
-  kernel's `pci_enable_atomic_ops_to_root()` having walked a chain with no
-  `Routing-` bridge. Same one-command diagnosis, opposite answer.
-- `dmesg` agrees on bandwidth: `252.056 Gb/s available PCIe bandwidth, limited
-  by 32.0 GT/s PCIe x8 link at 0000:00:01.1` (and `.13`) — ~31.5 GB/s per
-  card. DMI confirms the bifurcation: `PCIEX16(G5)_2`, "8x or x8".
-- `rocm-smi --showtopo` reported `PCIE` between the cards (Navi 48 has no XGMI),
-  they were in separate IOMMU groups (15 and 20 — re-read 2026-10-18: still 15
-  and 20), and both root ports had ACS `ReqRedir+`, so this page inferred P2P
-  would hairpin at the root complex. **That inference is superseded:** the live
-  engine reports the two cards reaching each other's memory directly, which
-  [[frank-archon]] and [[Dankserver System Specs]] record. Plain P2P traffic
-  measured 45.27 GB/s bidirectional on the *old* board — DMA was never the gate,
-  atomics were.
-- Kernel 7.0.0-31-generic with `iommu=pt` then; this host runs 7.0.0-38-generic
-  with `iommu=pt` today (`/proc/cmdline`, read 2026-10-18). Both were ruled out
-  as causes ([above](#ruled-out-empirically-do-not-re-derive)) and this host's
-  IOMMU posture is unchanged.
-
-**Still unproven (as of 2026-09-20).** The gate that matters is ROCclr's
-`checkAtomicSupport()` reading the `kLinkAtomicSupport` link attribute; the ReqEn
-bit is the same cause seen from the kernel side, but no RCCL kernel had been
-launched since the swap. Until the 2-rank probe runs — or a `--split-mode row`
-boot that does *not* log `internal AllReduce init failed ... falling back to
-meta-backend butterfly` — "collectives work here" was a strong expectation, not a
-measurement. **RCCL never got that measurement**; what settled the operational
-question instead is production traffic — radiance's TP=2 wire carries cross-rank
-all-reduces over the direct P2P path and prints its own precision, and the
-measured `wht6` banner with its exact-step bound is on [[frank-archon]].
-
-## Measured matrix
-
-Bare 2-rank `torch.distributed` (`backend=nccl`) on the two R9700s, no vLLM, no
-model. Rank 0 = card #1 (CPU lanes), rank 1 = card #2 (chipset):
-
-| Operation | rank 0 (card #1) | rank 1 (card #2) |
-|---|---|---|
-| `broadcast` | OK | **FAIL** `DistBackendError` |
-| `all_reduce` | OK | **FAIL** |
-| `barrier` | OK | **FAIL** |
-| `send` / `recv` (P2P) | blocked, then timeout | **FAIL** |
-| `all_gather` (after failure) | group unusable | — |
-
-The `AMD_LOG_LEVEL=3` trace names the cause: `Pcie atomics not enabled, hostcall
-not supported` → `AQL dispatch failed!` → `hipLaunchKernel` returns
-`hipErrorIllegalState` for `ncclDevKernel_Generic_4(...)`. Nothing in vLLM or
-RCCL mentions atomics; the runtime log is the only place it appears.
-
-**`send`/`recv` failing is the decisive result**: it kills pipeline parallelism
-as an option, because vLLM's PP stage handoff is exactly that primitive.
-
-## Software escapes: closed, and the last one is now moot
-
-Every row below was written under the old board. The last row — the only door
-that was ever open — stopped mattering on 2026-09-20, because the hardware gate
-it existed to route around no longer applies to this host.
-
-| Idea | Finding |
-|---|---|
-| vLLM's IPC custom allreduce | **Gated off by a table keyed to CUDA sm versions** — `{9.0, 10.0, 10.3, 10.7}`; ROCm reports capability `12.0` for gfx1201, so the gate is `False`. `custom_ar` itself is compiled into the image. |
-| ...and if that table were patched | Still insufficient: `broadcast`, `barrier`, and the TP logits `all_gather` have no non-RCCL path. |
-| The radiance fork's own allreduce | Already present and *on by default* (`RADIANCE_USE_R4D_AR`, size gate `RADIANCE_AR_MAX_KB`): a one-shot P2P push allreduce wrapping `CudaCommunicator.all_reduce`, exchanging IPC handles over the **gloo** group. With the gate raised it covers prefill and decode, and `ncclDevKernel` disappears from the profile. But it does not intercept `PyNcclCommunicator.__init__`'s **warmup allreduce**, nor `pynccl_comm.all_gather` — both direct RCCL calls. |
-| NCCL symmetric memory | Unavailable: the shipped RCCL advertises NCCL 2.30.4 but exposes no `ncclDevComm*` API, so torch's symmetric-memory path cannot run. |
-| Prefill/decode disaggregation | Needs a KV-transfer connector plus coordination collectives; both out. |
-| gloo for GPU work | CPU/TCP collectives — orders of magnitude too slow for per-layer TP. |
-| **Rebuild RCCL so its kernels do not declare a hostcall buffer** | **OPEN — unverified.** The claim is that the hostcall requirement is triggered by device-side `printf`/`assert`, and that `-mprintf-kind=buffered` avoids it. Counter-evidence: RCCL's device source at `src/device/` contains **no live** `printf`, `assert`, or `__assertfail` (checked), so what makes its kernels declare `HiddenHostcallBuffer` is unexplained. The claim traces to search-result summaries, not a primary source. Cheap to settle once a host is available: recompile librccl with `-mprintf-kind=buffered`, dump a kernel's declared arguments (`roc-obj-extract` + `llvm-readelf --notes`) and look for `hidden_hostcall_buffer`; if it is gone, a 2-rank probe should dispatch. **Stop and mask the service first** — see the note below. |
-
-## Bandwidth: why x4 is enough
-
-A 27B TP=2 allreduce at batch 4 moves roughly
-$2 \times \text{batch} \times \text{hidden} \times 2\ \text{bytes} \approx 160\ \text{KB}$
-per layer per forward pass; across ~62 layers that is **≈10 MB per token-step**,
-or **~1.4 ms at PCIe 4.0 x4** (~7.9 GB/s), and under 0.4 ms at the Gen5 x8
-~31.5 GB/s each card now has. TP is compute-bound here, not interconnect-bound.
-That is why the cheap option (the M.2 riser) and the expensive one (the board
-swap) were both considered adequate on bandwidth alone — and why the board that
-landed, at x8/x8, clears the arithmetic with room to spare.
-
-## Ruled out empirically (do not re-derive)
-
-- Kernel versions 7.0.0-28 / -30 / -31 — byte-identical failure.
-- `iommu=pt`, and `amd_iommu=off` is unrelated (that was a Dankclaw prefill knob).
-- P2P hardware: 45.274 GB/s bidirectional between the cards works fine — plain
-  memory traffic is not the problem; *atomic* routing is.
-- A from-source librccl rebuild: identical failure at the identical call path.
-- Every NCCL transport/algorithm knob.
-- A one-card test proves nothing: NCCL short-circuits at `world_size=1`, which
-  is exactly why the two replicas work at all.
-- Independent corroboration: StillDeadcode/vllm-radiance Codeberg issue #22 —
-  different host, same image, kernel 7.0.0-30, same `enqueue.cc` failure.
-
-## Measured verdict (2026-09-20)
-
-Both cooperation modes were run on this host on 2026-09-20 — the same salted
-10-prompt battery (`scripts/bench-conversation.py`), the service stopped and
-masked between arms. The arm table is [[frank-medium-radiance-mxfp4]]'s bench of
-record (arms R = radiance TP=2 + DFlash2 as deployed, I = the llama.cpp layer
-split that preceded it, L = llama.cpp `--split-mode tensor` against RCCL); this
-page keeps only the verdict. **TP=2 won on every axis — roughly 2.1× decode and
-3.6× at 4-way concurrency over the layer split — and replaced the layer split the
-day of measurement.**
-
-Arm L carries a caveat no flag can remove: llama.cpp's tensor mode could not load
-the DFlash2 drafter (it aborts at `ggml-backend.cpp:941` — a draft weight
-pre-allocated on the non-primary device has no backend once `n_copies == 1`), so
-it decoded **without** speculation while the other arms decoded with it; and part
-of R's margin over I is the 4-bit MXFP4 checkpoint, not the parallelism.
-
-Two numbers elsewhere on this page are therefore not interchangeable:
-
-- The 272 t/s recorded 2026-09-16 is **prefill**, on a **butterfly** allreduce,
-  with RCCL off, on the old board. It was never a TP decode number, and its
-  paired decode figure is in "The third path" below.
-- The butterfly tensor arm and the RCCL tensor arm a day later are the same shape
-  family on different collective paths; the measured pair lives with
-  [[frank-medium-radiance-mxfp4]]'s arms.
-
-RCCL needs PCIe atomics on *every* participating GPU — the gate the old board
-failed. vLLM's R4D one-shot allreduce dispatches no collective at all, which is
-why TP=2 runs today rather than merely "being unblocked": the image ships TheRock
-RCCL, and TheRock carries no gfx1201 device code (`invalid device function`), so
-radiance could not have used RCCL on this host even with atomics routed. Ubuntu's
-RCCL 2.27 does carry gfx1201 kernels, and that is what arm L linked against.
-
-**Rule for any future multi-GPU collective experiment: stop and mask the
-production service first**, per the benchmarking policy, or run it on a host that
-is not serving users. The production service is `frank-archon`, a **root** unit,
-so on Dankserver that is `sudo systemctl stop frank-archon && sudo systemctl
-mask frank-archon` (`scripts/frank-archon.service`; never `disable` — that
-deletes the unit file, and `mask` is what the bench policy prescribes). The
-2026-09-20 wording below — `systemctl --user stop frank-medium && systemctl
---user mask frank-medium`, frank-medium being a rootless quadlet — is retired
-with that service, which today reports `not-found` (read 2026-10-18).
-
-This is not theoretical: on 2026-09-16 a bare RCCL P2P probe was run against the live
-service; the host went hard down ~20 minutes later (no ARP, no ICMP, required a
-power cycle). The mechanism is not proven, but a hung AQL dispatch wedging the
-GPU driver is a plausible cause and this host has precedent for going
-unreachable until a power cycle.
-
-
-## The third path (production in the layer-split era): llama.cpp's layer split
-
-Everything above is about *collectives*, and vLLM cannot proceed without them
-in any of its parallelisms — TP needs a per-layer allreduce, and PP's stage
-handoff is a p2p `send`/`recv` of the hidden state, the primitive this host
-also fails. **llama.cpp's `--split-mode layer` needs no collective at all**:
-layers are distributed across the cards and the boundary activation is a
-plain per-backend copy, once per forward pass. No RCCL kernel is ever launched
-on the second card, so the atomics gate never comes up.
-
-That is what `frank-medium` ran from 2026-09-16 to 09-20
-([[frank-medium-llamacpp-dflash2]]): one process, one model, both cards,
-262144 context. Its measured layer-split rates are that page's; this host's
-tensor arm under the butterfly fallback measured 272 t/s prefill / 26.3 tok/s
-decode. **That tensor number is not a prediction for an RCCL-backed build** — it
-is what tensor split costs when the allreduce degrades to a meta-backend
-butterfly, and it is the number the board-swap path exists to retire.
-
-The ranking therefore changed twice. On 2026-09-16: software already gets both
-cards onto one model, so the hardware fix stopped being a capability question
-and became a performance one — *tensor* parallelism, ~40 % decode, plus one
-pooled KV instead of two independent 122k-token pools. On 2026-09-20 the
-hardware half of that question was answered by the board swap; the llama.cpp
-measurement half never ran, and `GGML_HIP_RCCL` still defaults `OFF` in
-`scripts/build-llama-rdna4.sh:41` (re-read 2026-10-18) for exactly the reason
-this page documents — moot while no llama.cpp shape is deployed on this host.
-
-## What would change the answer
-
-- ~~An RCCL build whose kernels stop declaring a hostcall buffer~~ — **moot
-  since 2026-09-20.** That door only mattered while the hardware gate could not
-  be passed; the board swap passes it. The `-mprintf-kind=buffered` hypothesis
-  and its counter-evidence stay in the escape table for the record.
-- The symmetric-kernel path inside RCCL, if a release ever ships the device API
-  (`ncclDevComm*`) that torch's symmetric-memory path needs. The shipped TheRock
-  nightly (`~/.local/share/rocm-7.14.0a20260624/lib/librccl.so.1`) still does
-  not.
-- ~~Firmware or an AGESA update that makes the chipset bridges advertise atomic
-  routing~~ — arrived as new silicon instead.
-- A GPU-side workaround: none found in ROCclr, and the gate is per-device,
-  queried once at probe.
-
-## See Also
-
-- [[Pipeline Parallelism/index|Pipeline Parallelism]] — two-machine research (the PP variant of this problem)
-- [[Dankserver System Specs]] — hardware inventory
-- [[Observability]] — the scrape/dashboard/alert stack around these endpoints

serverAppConfigs-docs/LLM/FastFlowLM NPU Inference.md

@@ -1,448 +0,0 @@
----
-type: reference
-title: FastFlowLM NPU Inference
-description: "The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook."
-tags:
-  - fastflowlm
-  - npu
-  - xdna2
-  - dankclaw
-  - research
-timestamp: 2026-09-03
----
-
-> **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
-> See [[frank-redo-2026-08-17]] and [[deployment]] for the active configuration.
-> The `frank-zealot` name and port 52625 then belonged to the vLLM
-> service on Dankclaw (nameistoken Quark W8A8); this page is archive.
->
-> **The NPU endpoint is retired — 2026-09-11.** On Dankclaw today
-> `frank-npu.service` is `masked`/`inactive` and nothing listens on 52625
-> (`ss -ltn`, read 2026-10-18). The runtime is still installed
-> (`fastflowlm 1.0.1`, `libxrt2`/`libxrt-npu2`, `/opt/fastflowlm`,
-> `~/.config/flm`, `/usr/bin/flm` — all read 2026-10-18) and the device is
-> usable again since the 2026-10-18 `iommu=pt` boot (`amdxdna` bound,
-> `/dev/accel/accel0` present — [[dankclaw-iommu-translation-off]]), but no
-> service runs on it. `frank-zealot` today is the llama.cpp PrismML endpoint on
-> the RTX 3090 ([[frank-zealot]]). See [[frank-npu]]
-> for the retirement record. Everything below is the 2026-08 research record.
-
-# FastFlowLM NPU Inference
-
-FastFlowLM (FLM) runs LLMs entirely on the AMD Ryzen AI NPU. It targets XDNA2 chips: Strix, Strix Halo, Kraken, and Gorgon Point. The project describes itself as "Ollama, but deeply optimized for NPUs." As of 2026-08-16, FLM was installed on Dankclaw, the NPU validated, and `frank-zealot` (port 52625) served in production — that arrangement ended with the 2026-09-11 retirement (see the banner above). See Promotion.
-
-## Verdict (2026-08-16)
-
-- The NPU is a complement, not a replacement. It fits dense models up to about 9B.
-- Qwen3.8-27B (dense) cannot run on the NPU. No FLM build exists and the memory budget is too small.
-- Qwen3.6-35B-A3B has an official FLM build (Q4_KS). AMD benchmarks it on a 96 GB machine. It probably does not fit the 30 GiB budget of Dankclaw.
-- Realistic first candidates: Qwen3.5-9B, Qwen3-8B, gpt-oss-20b (at the budget edge).
-- **Landed 2026-08-16:** gpt-oss-20b serves on the NPU at 19.3 t/s decode, concurrently with the PP head. See Measured Results.
-- The memlock limit was the setup blocker. Fixed 2026-08-16 via `/etc/security/limits.d/99-flm-memlock.conf`.
-
-## What FastFlowLM Is
-
-FLM is an NPU-first inference runtime. It started as a 2025 university project (University of Rhode Island, Ken Qing Yang). The team joined AMD on 2026-07-17. The repository moved to the ROCm organization on 2026-08-11 as v1.0.0.
-
-Key properties, per the project README and docs:
-
-- Runs fully on the NPU. No GPU or CPU load during inference.
-- CLI mode (`flm run <tag>`) and server mode (`flm serve <tag>`).
-- Server mode exposes a REST API and an OpenAI-compatible API. Default port is 52625.
-- Models are not GGUF. FLM downloads pre-compiled kernel packs (built with AMD IRON + AIE-MLIR) from Hugging Face.
-- Context lengths up to 256k tokens on listed models.
-- Vision, audio, embedding, and MoE model support.
-- Runtime is about 17 MB. Orchestration code is MIT-licensed. Kernels are free for commercial use.
-- Models store in `~/.config/flm/` (Linux). Override with `FLM_MODEL_PATH`.
-
-## NPU Hardware on Dankclaw
-
-Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU
-with 50 TOPS of compute and, crucially, **no dedicated VRAM** - it shares system memory
-via the IOMMU SVA mapping. The budget this page reasons about - 29 GiB usable at
-4800 MT/s LPDDR5X - is the 2026-08 host state; the live figure is [[Dankclaw System Specs]].
-
-Live verification on Dankclaw, 2026-08-16:
-
-```
-$ flm validate
-[FLM]  Fetching models from: /opt/fastflowlm/share/flm/model_list.json
-[Linux]  Kernel: 7.0.0-29-generic
-[Linux]  NPU: /dev/accel/accel0 with 8 columns
-[Linux]  NPU FW Version: 1.1.2.65
-[Linux]  amdxdna version: 0.7
-[ERROR]  Memlock limit is too low (8MB). Please raise the limit or set to infinity.
-```
-
-- Kernel 7.0.0-29-generic shipped the `amdxdna` driver in-tree, loaded. The host now runs 7.0.0-38-generic and the driver is still bound (`lspci -k -s c6:00.1` → `Kernel driver in use: amdxdna`, read 2026-10-18).
-- Firmware 1.1.2.65 meets the 1.1.0.0 minimum.
-- The XRT runtime packages (`libxrt2`, `libxrt-npu2`) are installed — still installed on 2026-10-18 (`dpkg -l` → `libxrt2 1:2.21.75+dfsg-4`, `libxrt-npu2 1:2.21.75+dfsg-4`).
-- The only failure is the memlock ulimit (8 MB). FLM needs `unlimited`.
-- Ubuntu 26.04 is not on the FLM supported-distro list (24.04, 25.10, Arch, generic). Validation passes anyway. Treat 26.04 as "generic Linux, working".
-
-## Memory Model
-
-The NPU allocates from system RAM. AMD/Microsoft enforce an internal cap of about half of total system RAM for NPU use. The gpt-oss model card states a cap of about 15.6 GB on a 32 GB machine, which is half.
-
-Budget math for this fleet:
-
-| Machine | RAM | NPU budget (approx. half) |
-|---------|-----|---------------------------|
-| Dankclaw (Strix Halo) | 30 GiB | ~15 GiB |
-| Dankserver (Ryzen 7 7700X) | 62 GB | no NPU |
-| FLM bench box (Kraken Point) | 32 or 96 GB | ~16 or ~48 GB |
-
-The half-of-RAM gate scales with host RAM - live figures in [[Dankclaw System Specs]]
-and [[Dankserver System Specs]]. The live gate on Dankclaw is still `flm validate`
-(it owns the current number).
-
-Weight sizes (approximate, Q4-class):
-
-| Model | Quant | Weights | Fits ~15 GiB budget? |
-|-------|-------|---------|----------------------|
-| Qwen3.5-9B | Q4_1 | ~5.5 GB | yes |
-| Qwen3-8B | Q4_1 | ~5 GB | yes |
-| gpt-oss-20b (MoE, 3.6B active) | Q4_1 | ~15.1 GB | at the edge |
-| Qwen3.6-35B-A3B (MoE, 3B active) | Q4_KS | ~20 GB | no, unproven at 30 GiB |
-| Qwen3.8-27B (dense) | any Q4 | ~16 GB+ | no build exists |
-
-Do not load a model larger than the budget first. An oversized load can starve the unified pool and crash the box while production services still run.
-
-## Supported Models (Qwen family, plus gpt-oss)
-
-All Qwen cards use Q4_1 unless noted. Context lengths are the FLM-published maxima.
-
-| Tag | Base model | Context max | Notes |
-|-----|-----------|-------------|-------|
-| `qwen3:0.6b` | Qwen3-0.6B | 32k | thinking toggle |
-| `qwen3:1.7b` | Qwen3-1.7B | 32k | thinking toggle |
-| `qwen3:4b` | Qwen3-4B | 32k | tool calling |
-| `qwen3:8b` | Qwen3-8B | 32k | tool calling |
-| `qwen3-tk:4b` | Qwen3-4B-Thinking-2507 | 256k | thinking |
-| `qwen3-it:4b` | Qwen3-4B-Instruct-2507 | 256k | tool calling |
-| `qwen3vl-it:4b` | Qwen3-VL-4B-Instruct | 256k | vision |
-| `qwen2.5-it:3b` | Qwen2.5-3B-Instruct | 32k | |
-| `qwen2.5vl-it:3b` | Qwen2.5-VL-3B-Instruct | 256k | vision |
-| `qwen3.5:0.8b` | Qwen3.5-0.8B | 256k | vision, thinking toggle |
-| `qwen3.5:2b` | Qwen3.5-2B | 256k | vision, tool calling |
-| `qwen3.5:4b` | Qwen3.5-4B | 256k | vision, tool calling |
-| `qwen3.5:9b` | Qwen3.5-9B | 256k | vision, tool calling |
-| `qwen3.6-moe:35b-a3b` | Qwen3.6-35B-A3B | 256k | **Q4_KS**, MoE, vision, thinking, tool calling |
-| `gpt-oss:20b` | gpt-oss-20b | 128k | Q4_1, ~15.1 GB in FLM |
-
-Other families: LLaMA, DeepSeek, Gemma, MedGemma, TranslateGemma, LiquidAI/LFM, Phi, Nanbeige, Whisper, EmbeddingGemma, SmolVLA.
-
-## Published Benchmarks (FLM, vendor numbers)
-
-All vendor results come from AMD Ryzen AI 7 350 (Kraken Point, 40 TOPS NPU) boxes under the default NPU power mode (Performance). Our NPU is a 50 TOPS Strix Halo XDNA2. Treat these numbers as directional, not as our expected speed.
-
-Qwen3.5, FLM v0.9.38, Kraken Point 32 GB DRAM. Decode t/s by starting context length:
-
-| Model | 1k | 2k | 4k | 8k | 16k | 32k |
-|-------|-----:|-----:|-----:|-----:|-----:|-----:|
-| Qwen3.5-0.8B | 39.2 | 38.0 | 36.3 | 33.1 | 28.1 | 21.6 |
-| Qwen3.5-2B | 26.8 | 26.2 | 25.4 | 23.7 | 21.3 | 17.0 |
-| Qwen3.5-4B | 15.0 | 14.6 | 14.2 | 13.3 | 11.8 | 9.6 |
-| Qwen3.5-9B | 9.3 | 9.2 | 9.0 | 8.5 | 7.8 | 6.9 |
-
-Prefill t/s for the same run:
-
-| Model | 1k | 2k | 4k | 8k | 16k | 32k |
-|-------|-----:|-----:|-----:|-----:|-----:|-----:|
-| Qwen3.5-0.8B | 983 | 1257 | 1471 | 1584 | 1579 | 1447 |
-| Qwen3.5-2B | 803 | 1004 | 1142 | 1223 | 1225 | 1151 |
-| Qwen3.5-4B | 378 | 440 | 479 | 493 | 487 | 450 |
-| Qwen3.5-9B | 284 | 333 | 362 | 379 | 378 | 357 |
-
-Qwen3.6-35B-A3B (Q4_KS), FLM v0.9.45, Kraken Point **96 GB DRAM**:
-
-| Metric | 1k | 2k | 4k | 8k | 16k | 32k |
-|--------|-----:|-----:|-----:|-----:|-----:|-----:|
-| Decode t/s | 13.65 | 13.41 | 13.09 | 12.51 | 11.24 | 9.51 |
-| Prefill t/s | 78.98 | 118.04 | 156.43 | 197.93 | 218.84 | 221.96 |
-
-- PP and NPU together cost 2.2 GiB + NPU model + ~2 GiB of headroom. The then-current
-  PP split (head + 3090 worker, 2026-08 era - see [[frank-redo-2026-08-17]]) decoded at
-  14-18 t/s for the 27B dense model, so an NPU offload would free the head or upgrade
-  the worker's model.
-
-The vendor also claims "over 10x more power-efficient" versus GPU. That is a marketing claim. We have not measured it.
-
-## Fit to the Frank Program
-
-| Our model | NPU verdict |
-|-----------|-------------|
-| Qwen3.8-27B UD-Q8_K_XL (Dankclaw, dense) | No. No FLM build. ~16 GB+ even at Q4. Exceeds the ~15 GiB budget. |
-| Qwen3.6-35B-A3B (Dankserver, MoE) | Unproven. Official build exists. Vendor bench ran on 96 GB RAM. A 30 GiB box has ~15 GiB for the NPU. Test last, with the service stopped. |
-| Qwen3.5-9B / Qwen3-8B | Yes, the realistic tier. ~5-6 GB. Leaves room for KV and the OS. |
-| gpt-oss-20b (MoE) | **Landed.** Weights are 13.8 GB on disk (q4nx). The model holds ~15 GiB resident on the NPU. Decode is 19.3 t/s. See Measured Results. |
-
-Proposed role: a small, always-on, low-power endpoint for triage, summarization, embeddings, or agent side-tasks on the Dankclaw NPU. It frees the iGPU and the 3090 for the 27B/35B workloads. It does not replace either.
-
-## Install State on Dankclaw (verified 2026-08-16)
-
-- `fastflowlm 1.0.1` installed (deb, upgraded from 0.9.45-12-ga6bd056 on 2026-08-17). `flm` at `/usr/bin/flm`. The 0.9.45 runtime refused the qwythos pack, which declares `flm_version` 1.0.0.
-- `libxrt2` / `libxrt-npu2` (Xilinx Runtime) installed.
-- `/dev/accel/accel0` present. `flm validate` passed fully. (Re-read 2026-10-18: `/dev/accel/accel0` is still present and `fastflowlm 1.0.1` is still installed, but the endpoint is retired — `frank-npu.service` masked, no listener on 52625.)
-- Memlock fixed: `/etc/security/limits.d/99-flm-memlock.conf` sets soft and hard memlock to unlimited. A fresh sshd session picks it up via pam_limits. No reboot needed.
-- `gpt-oss:20b` pulled (4/4 files verified, 13.8 GB `model.q4nx` + tokenizer) into `~/.config/flm/models/GPT-OSS-20B-NPU2/`.
-- `qwythos:9b` (Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-NPU2) installed 2026-08-17 via `flm-add.py` (downloaded from the HF repo; stdlib-only, no pip needed). Tag registered in the user registry `~/.config/flm/model_list.json` (a copy of the v1.0.1 system registry + the custom entry); kernels symlinked `~/.config/flm/xclbins/Huihui-...-NPU2 -> /opt/fastflowlm/share/flm/xclbins/Qwen3.5-9B-NPU2`. The service env exports `FLM_CONFIG_PATH` and `FLM_XCLBIN_PATH` so the tag resolves (without them flm hits the unknown-tag fallback, see the incident below).
-- `flm serve gpt-oss:20b --host 0.0.0.0 -p 52625 --pmode performance` runs as a nohup process. It is not a systemd unit. It does not survive a reboot.
-- The `llama-server` (frank-dragoon, the PP head) holds an fd on `/dev/accel/accel0` from its own startup (device enumeration, not compute). FLM opened the same device concurrently without conflict. Verified live 2026-08-16.
-
-## Measured Results (2026-08-16, gpt-oss-20b on the Dankclaw NPU)
-
-Conditions: `--pmode performance`, PP head (27B, port 7770) serving concurrently, 5.5 GiB system RAM headroom steady-state, NPU resident footprint ~15 GiB.
-
-| Run | Prompt tok | Compl tok | Wall s | Prefill t/s | Decode t/s | KV occ % |
-|-----|-----------:|----------:|-------:|------------:|-----------:|---------:|
-| 1 (smoke) | 74 | 63 | 7.2 | 19.5 | 19.3 | 1.7 |
-| 2 (160-tok prompt) | 161 | 256 | 17.4 | 40.8 | 19.3 | 5.1 |
-| 3 (95-tok prompt, 512 out) | 95 | 512 | 30.8 | 24.3 | 19.3 | n/a |
-
-Decode is flat at 19.3 t/s across all three runs. Output includes `reasoning_content` (gpt-oss is a reasoning model). At `max_tokens 256` one run spent its whole budget thinking and returned empty content. Give it 500+ tokens or strip reasoning in the client.
-
-Long-context prefill is not yet measured (runs 1-3 used under 200 prompt tokens). The 35B-A3B step remains untested.
-
-## Model Bakeoff (measured 2026-08-16, 4 models)
-
-All runs on the Dankclaw NPU (50 TOPS Strix Halo), single stream, `--pmode performance`, PP head (frank-dragoon) serving concurrently on 7770. Decode t/s is the FLM-reported rate. Vision accuracy = ground-truth elements out of 4 (text SPECTRUM, red circle left, blue square right, three green triangles bottom).
-
-| Model | Tag | Weights | Decode t/s (short) | 8k prefill t/s | 8k decode t/s | Vision decode t/s | Vision accuracy | Max ctx (registry) |
-|-------|-----|---------|-------------------:|---------------:|--------------:|------------------:|----------------|--------------------|
-| Gemma4-E4B | `gemma4-it:e4b` | 6.8 GB + 912 MB audio | 12.2 | 582 | 9.9 | 11.7 | 4/4 | 32768 |
-| Qwen3.5-9B | `qwen3.5:9b` | 7.3 GB | 7.2 | 343 | 6.6 | 7.3 | 4/4 | 32768 |
-| GPT-OSS-20B | `gpt-oss:20b` | 13.8 GB | 19.3 | 378 | 16.9 | n/a (no vision) | 0/4 | **8192** |
-| Qwen3.6-35B-A3B | `qwen3.6-moe:35b-a3b` | 22.2 GB | 12.7 | 207 | 11.2 | 12.7 | 4/4 | 32768 |
-
-Notes:
-
-- gpt-oss:20b is the fastest decoder (MoE, 3.6B active) but its KV cache fills at ~8k tokens (98.85% occupancy at 7,926; requests past ~8k return 400). It has no vision encoder.
-- 35B-A3B loads 28-29 GiB resident. It works but leaves ~1.5 GiB headroom with the PP head running. Do not run it alongside the head at full load.
-- Vision prefill for the two vision models with ~720 token image input: gemma 141 t/s, qwen3.5 176 t/s, 35B-A3B 68 t/s.
-- All four comply with the same image at 4/4 elements except gpt-oss, which cannot see it.
-
-## Promotion (2026-08-16)
-
-`frank-zealot` then ran the NPU endpoint on Dankclaw (2026-08; the name was reused
-twice after this window - see the banner - and the NPU unit today is `frank-npu`,
-masked). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`),
-wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`.
-`LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap
-`FLM_MODEL` in the env to switch.
-
-### Boot policy (2026-08-17)
-
-The `frank-zealot` unit is disabled for boot. Only the PP pair (`frank-dragoon` + `frank-pp-worker`) starts at boot. The unit file was removed from `/etc/systemd/system`; the running FLM service keeps serving until stopped or rebooted. To restore the endpoint, run these commands on Dankclaw:
-
-```
-sudo ln -sf /home/dank/vllmScripts/scripts/frank-zealot.service /etc/systemd/system/frank-zealot.service
-sudo systemctl daemon-reload
-sudo systemctl enable frank-zealot
-```
-
-Live config (2026-08-16): `FLM_PMODE=turbo`, `FLM_ASR=1` (loads Whisper-V3-Turbo-NPU2; exposes `/v1/audio/transcriptions`), `FLM_CTX_LEN=131072` (verified max — Gemma4 natively supports 128K and the NPU budget fits it; KV occupancy measured at 0.11% for a 123-token exchange, decode 12.0 t/s, 13 GiB headroom).
-
-The PP split was renamed `frank-pp-head` → `frank-dragoon` (2026-08-16, commit `48142f5`).
-
-Model swap (2026-08-16, commit `50c55ee`): `FLM_MODEL=qwen3.6-moe:35b-a3b`, `FLM_CTX_LEN=262144` (native max), `FLM_PMODE=turbo`, `FLM_ASR=1` kept. Decode measured 12.5-13.1 t/s. The Gemma4 live config above (131072 ctx) applied to `gemma4-it:e4b`. Swap `FLM_MODEL` and `FLM_CTX_LEN` back to restore it.
-
-Model swap (2026-08-17, commit `c85ff6c`): `FLM_MODEL=qwythos:9b` (Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-NPU2, Q4NX 7.76 GB). The pack installs through `flm-add` (user registry `~/.config/flm/model_list.json` + user xclbins `~/.config/flm/xclbins`, kernels symlinked from official `Qwen3.5-9B-NPU2`). Runtime upgraded to FLM 1.0.1 first (see Install State). `FLM_CTX_LEN=262144` native max (Qwen3.5-9B hybrid: 8 of 32 layers full attention, so KV stays small; occupancy 0.008% at 21 tokens). `FLM_PMODE=turbo`, `FLM_ASR=1` kept. The env file now exports `FLM_CONFIG_PATH` / `FLM_XCLBIN_PATH` — required, or flm treats the tag as unknown and silently serves llama3.2:1b. `qwen3.6-moe:35b-a3b` files stay installed for rollback. pi client entry (`~/.pi/agent/models.json` on dankware): id `qwythos:9b`, ctx 262144, input `[text]` (pack has no vision weights), compat `supportsDeveloperRole: false` kept.
-
-## Parallel Streams (measured 2026-08-16)
-
-The NPU serializes generations. It runs exactly one stream at a time. Extra streams queue in FIFO order. FLM server flags `--socket` (default 10) and `--q-len` (default 10) cap queue depth only. They do not add concurrency.
-
-Concurrent-request test, gpt-oss-20b, 128-token cap per stream:
-
-| Streams | Wall s | Aggregate t/s | Per-stream wall s |
-|--------:|-------:|--------------:|-------------------|
-| 1 | 10.6 | 12.1 | 10.6 |
-| 2 | 21.1 | 12.0 | 10.8 / 21.1 |
-| 4 | 41.1 | 11.6 | 10.9 / 21.6 / 32.4 / 41.1 |
-
-The 4-stream wall times form a near-perfect 10.7 s ladder. Each stream self-reports 19.3 t/s decode, counting only its own active slices. Trust per-stream wall time, not the self-reported speed, under concurrency.
-
-Consequence: the NPU endpoint is a single lane. It suits low-frequency background work (triage, summarization, agent side-tasks). It does not suit interactive chat for several simultaneous users. The 27B llama.cpp service, by contrast, runs PARALLEL=2 with true concurrent slots.
-
-## Test Runbook
-This runbook targets a retired endpoint; keep it as the procedure if the NPU is
-ever revived.
-
-**Host RAM availability.** The gate is half of system RAM by default - a relic of the
-install: the endpoint ran on a machine that presented 30 GiB of system RAM; on today's
-box that ceiling is far larger (live figure: [[Dankclaw System Specs]]). Check free RAM
-first, or stop
-the service the way the bench policy requires (`sudo systemctl stop
-frank-dragoon && sudo systemctl mask frank-dragoon` — self-healing would
-otherwise restart it). The NPU shares the unified pool with everything else on
-the box.
-
-```mermaid
-graph TD
-    A[1. Raise memlock to unlimited] --> B[2. flm validate + xrt-smi examine]
-    B --> C{NPU visible to XRT?}
-    C -- no --> D[Fix driver or firmware path]
-    C -- yes --> E[3. flm run qwen3.5:4b smoke test, /verbose]
-    E --> F{Generates tokens?}
-    F -- no --> D
-    F -- yes --> G[4. flm serve on a scratch port, never 7770]
-    G --> H[5. Bench decode + prefill t/s]
-    H --> I{Speed acceptable?}
-    I -- no --> J[Record results, stop]
-    I -- yes --> K[6. Attempt gpt-oss:20b]
-    K --> L{Loads in ~15 GiB?}
-    L -- no --> J
-    L -- yes --> M[Bench, record]
-    M --> N[7. Last: qwen3.6-moe:35b-a3b, service stopped]
-    N --> O[Record, decision]
-```
-
-1. Raise memlock. Add `* soft memlock unlimited` and `* hard memlock unlimited` to `/etc/security/limits.conf`. Then reboot. For a systemd unit, set `LimitMEMLOCK=infinity` instead.
-2. Verify the NPU. Run `flm validate`. Run `xrt-smi examine`. Both must show the NPU. `flm validate` uses the kernel device. `flm run` uses XRT. A passing validate does not guarantee XRT sees the device.
-3. Smoke test. Run `flm run qwen3.5:4b`. Toggle `/verbose` for performance reporting. Type `/bye` to exit.
-4. Server mode. Run `flm serve qwen3.5:4b`. The default port is 52625. Use a scratch port if 52625 is taken. Never 7770.
-5. Bench. Measure decode and prefill t/s at 1k/4k/16k starting context. Record to this page.
-6. Step up in size: 9B, then gpt-oss-20b.
-7. Attempt `qwen3.6-moe:35b-a3b` only after the smaller runs succeed, and only with the production service stopped. Watch `free -h` during the model download and load.
-
-## Registering packs absent from the catalog (NPU2)
-
-Use when adding a non-catalog NPU2 pack (an extraction or fine-tuned model)
-to the FLM registry. Registry facts: user registry
-`/home/dank/.config/flm/model_list.json`, shape
-`{"model_path": "models", "models": {"<tag>": {"<size>": {name, url,
-file_url, size, flm_min_version, files[], vlm, default_context_length,
-max_prefill_len, details{format:"NPU2", family, …}, label[]}}}}`; serve
-resolves tags through `FLM_CONFIG_PATH` at startup, and `flm list` with the
-env var set lists custom tags (`⏬` = not downloaded).
-
-1. Back up the registry JSON first (`cp -a`).
-2. Insert the entry via a python json edit + atomic `os.replace`. Set
-   `files` to the FULL HF file list — **including `vision_weight.q4nx` even
-   for text-only packs**: the qwen3.5-family loader opens it unconditionally
-   and a missing file makes the first request HANG silently (no error log).
-3. **Do not use `flm pull <tag>` for custom entries** — it throws
-   `json.exception out_of_range key '<tag>:<size>' not found` while building
-   the download list. Place files manually into
-   `/home/dank/.config/flm/models/<PackName>/` with
-   `curl -sfL "https://huggingface.co/<repo>/resolve/main/<file>?download=true"`
-   — plain `raw/main/<file>` returns 135-byte LFS pointer stubs, not
-   weights; verify byte counts against the HF tree API before trusting a
-   download.
-4. Kernel packs: copy from the same-shape family pack (shape-match
-   `config.json` on hidden_size/num_hidden_layers/head_dim first) into
-   `/opt/fastflowlm/share/flm/xclbins/<PackName>/` via `sudo cp -a`
-   (root-owned dir).
-5. Deploy/restart the serve unit (the deploy path, not ad-hoc restarts) so
-   the new process loads with all files present.
-6. Verify with a REAL completion request (`model: "<tag>:<size>"`) —
-   `/v1/models` alone proves registration, not loadability.
-
-Hard constraints: only ONE `flm serve` process can hold the NPU hardware
-context — a second concurrent serve process fails every load with
-`DRM_IOCTL_AMDXDNA_CREATE_HWCTX err=-22` (aie2 mgmt msg opcode 0x2 status
-0x2000003); never run scratch/verify instances while production holds the
-NPU. Tag switching happens INSIDE one serve process; the first request
-after a model switch pays ~6–11 s cold-load cost (visible in
-`usage.load_duration`). Over ssh, never `pkill -f "<pattern>"` with a
-pattern present in your own command line — use bracket-guarded patterns
-(`[q]wen3.5:4b`) or exact PIDs.
-
-## Tag load-failure diagnosis (200 with an error body)
-
-`POST /v1/chat/completions` answers **HTTP 200** with a flat
-`{"error":"Failed to load <tag> model!"}` and no `choices`; `/v1/completions`
-returns a nested OpenAI-shaped `server_error` (code 500) also under HTTP
-200. A status-code-only client reads this as a successful empty completion —
-**always check for the `error` key**. Two distinct causes, distinguished via
-`journalctl -u <flm-service> --full`:
-
-1. **Unknown tag** (not in the registry): immediate error, nothing loaded.
-   Correct behavior, not a bug.
-2. **Registered tag whose kernel pack is missing**: FLM resolves the tag,
-   downloads missing weight files on the fly (first request hangs ~90–110 s),
-   then fails `Failed to load model: No such file
-   '/home/dank/.config/flm/xclbins/<PackName>/<kernel>.xclbin'`; subsequent
-   attempts fail fast (~2 s). Root cause: the serve script exports
-   `FLM_XCLBIN_PATH=/home/dank/.config/flm/xclbins` (user dir), so every
-   kernel lookup resolves under the user dir, which only contains symlinks
-   someone linked. **A tag listed in `/v1/models` is NOT proof it is
-   loadable on this host.** Fix: one symlink per missing pack
-   (`ln -s /opt/fastflowlm/share/flm/xclbins/<PackName>
-   /home/dank/.config/flm/xclbins/<PackName>`) — verified live: after the
-   symlink the first request loaded and answered in 7.4 s wall.
-
-Switch-cost behavior: the `model` field routes per-request (no restart
-needed); a request whose model differs from the resident one triggers a full
-reload (7–11 s wall for an 8-token completion on ~4B-class packs);
-alternating tags pays the swap each time. Single-lane NPU: concurrent
-requests queue FIFO behind the current lock holder (`NPU Locked!` →
-`NPU Lock Released!`). Probe recipe: `/v1/models` for registered tags; grep
-the journal for `Failed to load model` (the quoted xclbin path names the
-missing pack dir exactly); diff `ls ~/.config/flm/xclbins/` (symlinks =
-installed) against `/opt/fastflowlm/share/flm/xclbins/` (available).
-
-## Incident: qwen3.6-moe unreachable (2026-08-16)
-
-Symptom: a client on dankware could not reach `qwen3.6-moe:35b-a3b` on port 52625. The journal shows three separate causes in the 14:19-15:03 window.
-
-1. **EXEC 203 crash loop (14:19-14:20).** The exec bit was lost on `serve-flm-small.sh` on the target. The unit failed `status=203/EXEC: Permission denied` four times, then started after the bit was restored. The git mode is `100755` (commit `3f42210`), and a `git pull` restores the bit. Verify with `git ls-files -s scripts/serve-flm-small.sh`.
-2. **Wrong model ID (14:29-14:52).** The client config still used the model ID `frank-zealot` (a service alias). FLM rejects that tag with `[ERROR] Model tag 'frank-zealot' is not supported.` Each such request also triggered the unknown-tag fallback below.
-3. **Swap churn (14:48-15:03).** The swap `gemma4-it:e4b` → `qwen3.6-moe:35b-a3b` restarted the service several times. Each load of the 23 GB model takes about 1.5-2.5 minutes at 262144 ctx. The endpoint does not serve during a load.
-
-Fix: the client model ID must be the FLM registry tag `qwen3.6-moe:35b-a3b`. The working pi agent entry in `~/.pi/agent/models.json` is `baseUrl http://192.168.0.143:52625/v1`, `id qwen3.6-moe:35b-a3b`, `contextWindow 262144`, `reasoning true`, `input [text, image]`. Verified end-to-end from dankware (192.168.0.226) at 15:32 on 2026-08-16.
-
-### Unknown-tag fallback (FLM behavior)
-
-When a request names a tag that is not in the FLM registry, FLM does not return a clean 404. It logs the ERROR, then falls back to the default model `llama3.2:1b`:
-
-- If the `llama3.2:1b` files are missing, FLM downloads them (1.2 GB on 2026-08-16) while the request hangs.
-- FLM then loads the 1B model in place of the requested model.
-- The files now exist in `~/.config/flm/models/Llama-3.2-1B-NPU2` (1.3 GB) on dankclaw. A future unknown-tag request loads the 1B model silently, with no download and no error.
-
-**Correction (live-probed 2026-08-30):** the fallback no longer fires on the
-current serve path — an unknown tag now answers HTTP 200 with a flat
-`{"error":"Failed to load <tag> model!"}` body and nothing is loaded or served;
-`/v1/completions` answers a nested `server_error` (code 500) under HTTP 200.
-
-Keep client model IDs as FLM registry tags. Never send a service alias. Optional cleanup: move `~/.config/flm/models/Llama-3.2-1B-NPU2` (an incident artifact) to `.AGENTTRASH` so a stray tag re-downloads loudly instead of serving the wrong model.
-
-### Request compatibility: the `developer` role (found 2026-08-16)
-
-pi sessions failed against this endpoint with `Stream ended without finish_reason` while curl worked. Root cause, verified by field bisection:
-
-- The FLM Qwen chat template accepts only the roles `system`, `user`, and `assistant`. Any other role raises `Unexpected message role.` in the template.
-- FLM reports the error differently per mode. Non-streaming: HTTP 500 with the template traceback. Streaming: HTTP 200, and the error JSON is the first body. A streaming client sees a stream with no chunks and no `finish_reason`.
-- pi sends `role: "developer"` when `model.reasoning && compat.supportsDeveloperRole` (pi source, `openai-completions.js`). The default `supportsDeveloperRole` is true for standard OpenAI-compatible providers, so a reasoning model without a `compat` block always triggers the bug.
-- Fix: `"compat": { "supportsDeveloperRole": false }` on the model entry in `~/.pi/agent/models.json`. pi then sends `role: "system"`. Verified with a headless pi session: full round trip, `finish_reason: stop`.
-- FLM ignores unknown body fields. `reasoning_effort`, `store`, `max_completion_tokens`, and `tools: []` all arrive in pi requests and pass through without error.
-
-## Risks and Open Questions
-
-- **Ubuntu 26.04 is unofficial.** The distro list stops at 25.10. Validation passes, but breakage would fall on the "generic Linux" path.
-- **The 30 GiB budget is untested at the very top.** gpt-oss-20b (15 GiB resident) works. 35B-A3B (~20 GB weights) is still an unknown. Vendor data covers 32 GB and 96 GB boxes only.
-- **The FLM serve process was not under systemd** (2026-08-era risk). A reboot
-  of Dankclaw killed it. The fleet law today is the opposite: every deployed
-  container/service ships a boot-persistent, self-healing unit, and the retired
-  NPU unit is deliberately `masked` (`frank-npu.service`, read 2026-10-18) so it
-  cannot come back at boot.
-- **gpt-oss reasoning budget.** The model can spend its full token budget on `reasoning_content` and return empty content. Clients must handle or cap this.
-- **No llama.cpp GGUF path.** FLM uses its own pre-compiled kernels. Our UD-Q8_K_XL and HauhauCS builds are not importable. NPU work means FLM-specific model tags.
-- **NPU production role (2026-08-16, retired 2026-09-11).** `frank-zealot` then
-  served `qwen3.6-moe:35b-a3b` on port 52625; the endpoint was retired with the
-  `amd_iommu=off` decision that day and the unit is masked today. The remaining
-  NPU content (benchmarks, runbook steps) is research on a device that is again
-  usable since 2026-10-18 but serves nothing.
-
-## Citations
-
-1. ROCm/FastFlowLM README (chip support, Linux support date, ROCm move, quick start, model storage paths): https://github.com/ROCm/FastFlowLM , accessed 2026-08-16.
-2. FastFlowLM docs overview: https://fastflowlm.com/docs/ , accessed 2026-08-16.
-3. FastFlowLM Qwen model cards (tags, quantization, context lengths): https://fastflowlm.com/docs/models/qwen/ , accessed 2026-08-16.
-4. FastFlowLM gpt-oss model card (15.1 GB weight size, ~15.6 GB NPU cap, 32 GB inconsistency note): https://fastflowlm.com/docs/models/gpt-oss/ , accessed 2026-08-16.
-5. FastFlowLM Qwen3.5 benchmarks (v0.9.38, Kraken Point 32 GB): https://fastflowlm.com/docs/benchmarks/qwen3.5_results/ , accessed 2026-08-16.
-6. FastFlowLM Qwen3.6 benchmarks (v0.9.45, Kraken Point 96 GB): https://fastflowlm.com/docs/benchmarks/qwen3.6_results/ , accessed 2026-08-16.
-7. AMD blog, "FastFlowLM Joins AMD to Advance AI Inference" (2026-07-17): https://www.amd.com/en/blogs/2026/fastflowlm-joins-amd-to-advance-ai-inference.html
-8. Wikipedia, "Zen 5" article, Strix Halo section (Ryzen AI MAX+ 395: XDNA2 NPU, 50 TOPS, 256-bit LPDDR5X) via local ZIM `wikipedia_en_all_maxi_2026-02`.
-9. Live system checks on Dankclaw, 2026-08-16: `flm validate` output, `dpkg -l` package list, kernel 7.0.0-29-generic, `/proc/3052096/fd` device fd inspection.

serverAppConfigs-docs/LLM/FreeToken Builds.md

@@ -1,97 +0,0 @@
----
-type: reference
-title: FreeToken Builds
-description: "Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit."
-tags: [freetoken, build, dankclaw, vllm, mtp]
-timestamp: 2026-10-05T00:00:00-07:00
----
-
-# FreeToken Builds
-
-FreeToken = the official franksdank fork of FreeTok Tokenworks' freetoken
-vLLM-MTP runtime. Two build paths exist, both on **Dankclaw**, with
-different dependency traps. Both were verified end-to-end 2026-09-11/12.
-
-## Common verification (do not skip)
-
-**Verify the build by real completion, not by version string.** A version
-bump alone does not move data through the model — a build can print a new
-version and still not have completed a real token. Before declaring a
-FreeToken build deployable, run an actual completion (any tiny prompt) and
-read the returned content:
-
-```python
-import json, urllib.request
-req = urllib.request.Request("http://<host>:<port>/v1/chat/completions",
-    json.dumps({"model": "frank-zealot", "messages": [{"role": "user",
-    "content": "Hi"}]}).encode(), {"Content-Type": "application/json"})
-print(json.load(urllib.request.urlopen(req, timeout=600)))
-```
-
-A 200 with a real `choices[0].message.content` is the only build green.
-
-## CUDA build (the torch/vLLM dependency trap)
-
-Both paths build on **Dankclaw** with the same preinstall state: Python
-3.12.10 in the system venv and **Torch 2.10.0.dev** (nightly, repo
-download.pytorch.org/whl/nightly/cu130, installed with
-`uv pip install --index-strategy unsafe-best-match "torch==2.10.0.dev<date>+cu130"
---index-url https://download.pytorch.org/whl/nightly/cu130` — use
-`--index-strategy unsafe-best-match`, `uv` otherwise refuses the dev
-version). Verify the exact pair first; mismatched preinstalled deps are the
-#1 build-failure cause.
-
-Then install the **matching vllm dev wheel**:
-
-```bash
-uv pip install --index-strategy unsafe-best-match \
-  --index-url https://wheels.vllm.ai/<commit-sha>/vllm-0.6.6.dev0+g<shortsha>.cu130-cp312-cp312-linux_x86_64.whl \
-  --extra-index-url https://wheels.vllm.ai/nightly/cu130
-```
-
-The wheel URL pattern is fixed:
-`https://wheels.vllm.ai/<full-commit>/vllm-0.6.6.dev0+g<8-char>.cu130-cp312-cp312-linux_x86_64.whl`
-— read it back from vLLM's CI-run page (`run details →
-build_and_install_package` job, "install the vllm wheel" step). **The trap:
-vLLM refuses versions < its install_requires floor at runtime, so the wheel
-version must be ≥ the runtime's floor.** A wheel built at a lower base
-(0.5.x) downloads but `import vllm` fails with a version constraint error —
-that failure looks like a build failure and is not one; upgrade the wheel,
-not the code.
-
-If `uv pip install -e . --no-deps` refuses (some repos pin the
-`--no-deps` branch off), fall back to `python3 setup.py` inside the venv,
-after `python -m venv --copies .venv && source .venv/bin/activate` and
-`uv pip install --no-build-isolation "torch==…"` — the venv must NOT
-re-resolve torch (a second torch from PyPI d1 breaks the CUDA extension ABI).
-
-## ROCm build (gfx1151 via TheRock nightly)
-
-Same venv discipline, ROCm wheels from TheRock's nightly index:
-
-```bash
-uv pip install --index-strategy unsafe-best-match \
-  --index-url https://rocm.nightlies.amd.com/v2/gfx1151-dcgpu-supports_fbsgens_split_0717-rc1/ \
-  "torch==2.10.0.dev<date>+rocm" "triton==3.6.0" "pytorch-triton==3.6.0"
-```
-
-Then `vllm` itself (`uv pip install -e . --no-build-isolation --no-deps`,
-repo branch `specs-mtp-dev`, commit `2ee440d`), after copying the two
-prebuilt extension wheels into the tree (the repo's `setup.py` skips
-compiling them when found). The ROCm build also needs
-`install-fastrak-gfx1151.sh` from the repo's scripts and
-`FASTRACK_HOME=/home/dank/FastTrack-runtime` in the unit env. FreeToken's
-x86_64-gfx1151 prebuilt extension (`freetoken-ops`) ships on HF hub
-`franksdank/freetoken-ops-gfx1151` — fetch it once and point
-`FREETOKEN_OPS_HOME` at it; never rebuild the extension from source unless
-the extension API changed.
-
-`--index-strategy unsafe-best-match` is load-bearing on both paths: the
-nightly indexes carry version pairs that `uv`'s strict resolver otherwise
-rejects, and the error (`No candidate found for pytorch-triton`) reads like
-a missing package.
-
-## Related
-
-- [[Model Swap and Rollback]] — the swap/deploy story these builds serve
-- [[Llama Cpp Serving Sizing]] — the harness measurement discipline

serverAppConfigs-docs/LLM/GGUF Compatibility.md

@@ -14,7 +14,7 @@ Studio refuse custom types at load (`unknown tensor type`); a fork-private type
 means exactly one engine can serve the file. Measured on this fleet repeatedly:
 `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` is ftype 115 with 311 tensors of type 103
 (`GGML_TYPE_Q8_0_ROCMFPX`) — only charlie12345/ROCmFPX reads it. The full type
-tables live in [[Llama Cpp Runtimes]] and [[Llama Cpp Code Diff]]
+tables live in  and
 (ggml_type 100–108, ftype families 100–106 / 110–113 / 118).
 
 ## 1. Read the header, do not guess
@@ -69,14 +69,14 @@ filenames lie about quantization and about MTP/vision heads.
    ```
 
    Validate before trusting: the parse is right iff the tensor-info block ends
-   somewhere sane and names look real (`blk.0.attn_q.weight`,
+   somewhere sane and names look real (`blk.0.attn_q.weight`
    `token_embd.weight`, `output_norm.weight`). EOF throw = wrong enum or
    offset — do not work around it.
 
 3. **Diff what engines actually bind on**: `general.architecture`; tensor
    count; tensor name list in order; per-tensor `ggml_type` and dims; geometry
-   KVs (`<arch>.block_count`, `.embedding_length`, `.feed_forward_length`,
-   `.attention.head_count[_kv]`, `.rope.*`, `.ssm.*`, `.context_length`,
+   KVs (`<arch>.block_count`, `.embedding_length`, `.feed_forward_length`
+   `.attention.head_count[_kv]`, `.rope.*`, `.ssm.*`, `.context_length`
    `.nextn_predict_layers`); MTP tensors (`nextn` in the name); vision tensors
    (`visual`, `v_` prefixes). A same-family fine-tune quantized by the same
    tool usually comes out byte-identical in every one of these — that is the
@@ -117,8 +117,8 @@ re-download.
 
 ## Related
 
-- [[Llama Cpp Runtimes]] — which fork ships which custom types
-- [[Llama Cpp Code Diff]] — the enum tables and code-level differences
-* [[kingjones-q8agent-rocmfpx]] — the era that served the ftype-115 file, and
+-  which fork ships which custom types
+-  the enum tables and code-level differences
+*  the era that served the ftype-115 file, and
   [[frank-dragoon]] for what runs there now
 - [[Llama Cpp Serving Sizing]] — pricing what a payload costs once it loads

serverAppConfigs-docs/LLM/Heterogeneous Split Triage.md

@@ -93,7 +93,7 @@ of bytes), so it is tempting. Before proposing it:
   (compare GGUF `tokenizer.ggml.tokens`, never trust the family name), or it
   must be a head **trained for that target** (EAGLE/DFlash/MTP are
   conditioned on the target's hidden states and are not portable between
-  models);
+  models)
 - a plain drafter costs **K sequential forwards per round**, so it must be
   10–50× faster than the target. Break-even with `r = t_draft/t_target`:
   `rate_ratio = A/(K·r + 1)`, `A = (1−p^(K+1))/(1−p)`; at `r ≈ 0.4–0.5`
@@ -107,10 +107,10 @@ of bytes), so it is tempting. Before proposing it:
   (draft gets its own device list) plus `-ngld`/`-otd`/`-ctkd -ctvd`.
 
 The llama.cpp half of that bullet is checkable upstream: `--spec-draft-device`
-(`-devd` / `--device-draft`, line 4259), `--spec-draft-ngl` (`-ngld`, 4268),
+(`-devd` / `--device-draft`, line 4259), `--spec-draft-ngl` (`-ngld`, 4268)
 `--spec-draft-override-tensor` (`-otd`, 4137) and `--spec-draft-type-k/v`
 (`-ctkd`/`-ctvd`, 4111/4124) all exist in
-`common/arg.cpp` ([source](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/common/arg.cpp),
+`common/arg.cpp` ([source](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/common/arg.cpp)
 read 2026-10-18).
 
 ## 6. Expressing a split (llama.cpp, the usual vehicle)
@@ -122,8 +122,8 @@ read 2026-10-18).
 
 - buffer types match by name at load (`ggml_backend_buft_name`), so device
   names work as `-ot` values;
-- `-cmoe`/`-ncmoe` are **CPU-only** (`--cpu-moe` / `--n-cpu-moe`,
-  `common/arg.cpp:2764,2771`); device-side MoE placement is `-ot`;
+- `-cmoe`/`-ncmoe` are **CPU-only** (`--cpu-moe` / `--n-cpu-moe`
+  `common/arg.cpp:2764,2771`); device-side MoE placement is `-ot`
 - several backends in one binary is fine (CUDA + Vulkan is the practical
   pair; HIP + CUDA do not coexist *(unverified 2026-10-18 — upstream build docs
   do not settle the combination either way)*) — `GPU_BACKEND_A=OFF` in the
@@ -155,7 +155,7 @@ Rank and report; prefer the boring answer when it wins:
 1. **Two independent endpoints** (zero shared traffic) — often the real
    answer to "what's a useful combination", especially over a narrow link.
 2. **One process, weights-resident, KV co-located with attention on the
-   fast pool** — promising and quantifiable (+% decode at long context);
+   fast pool** — promising and quantifiable (+% decode at long context)
    needs a dual-backend build and a baseline measurement.
 3. **Hot-expert cache / per-token weight streaming** — affordable only if
    the miss set per token is small (a few experts; check
@@ -169,5 +169,5 @@ Rank and report; prefer the boring answer when it wins:
 ## Related
 
 - [[PCIe Link and Atomics]] — the link and collective walls underneath
-- [[Llama Cpp Multi-Device Splits]] — the deployed split shapes
+-  the deployed split shapes
 - [[Llama Cpp Serving Sizing]] — KV and pool arithmetic

serverAppConfigs-docs/LLM/Hybrid CPU GPU Processing Dankserver.md

@@ -1,225 +0,0 @@
----
-type: reference
-description: "Dankserver uses NVIDIA GPU with system RAM for hybrid processing."
-tags:
-  - llm
-timestamp: 2026-08-28
----
-
-# Hybrid CPU GPU Processing Dankserver
-
-## Overview
-
-Dankserver has **no NVIDIA GPU**: the RTX 3090 left the box on 2026-09-15 and the host now runs two AMD Radeon AI PRO R9700s (`lspci | grep -ci nvidia` → 0, read 2026-10-18 — [[Dankserver System Specs]]). This page's vLLM-era hybrid guidance is reference material, not the serving path: the live service is radiance 1.0.13 as `frank-archon`, and its hybrid mechanism is a pinned host pool (43.88 GiB per rank, `scripts/frank-archon.env:184`) plus the CPU-resident 47.68 GiB n-gram/PLE table, not vLLM KV swapping ([[frank-archon]]).
-
-## Architecture
-
-### Model Weights
-
-In the setup this page describes, model weights stay on GPU and the GPU must hold all parameters. As a statement about vLLM it is no longer true: current vLLM ships CPU weight offload as `cpu_offload_gb`, plus the grouped prefetch path (`offload_group_size`) — `vllm/config/offload.py:23,54` ([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/config/offload.py), read 2026-10-18). What remains true is the capacity arithmetic: offloaded weights cost PCIe traffic per token (see [[PCIe Link and Atomics]] §2).
-
-### KV Cache
-
-KV cache moves between GPU and CPU:
-
-1. Active KV cache pages stay in GPU VRAM
-2. Inactive pages swap to CPU RAM
-3. Swapped pages write to disk swap space if RAM is full
-4. Pages reload to GPU when needed
-
-### Compute Distribution
-
-- GPU handles all matrix multiplication and attention computation
-- CPU handles tokenization, detokenization, and input processing
-- CPU handles KV cache swap operations
-- GPU worker processes run on CPU cores via multiprocessing
-
-## Optimal Settings for Dankserver
-
-### GPU Memory Utilization
-
-Set `--gpu-memory-utilization` based on your GPU:
-
-| GPU | Recommended Value |
-|-----|-------------------|
-| 24 GB (RTX 3090/4090) | 0.7-0.8 |
-| 40 GB (A100) | 0.8-0.9 |
-| 80 GB (A100 80GB) | 0.85-0.95 |
-| 80 GB (H100) | 0.9-0.95 |
-
-Dankserver has neither NVIDIA card today: it carries 2× 32 GB (32,624 MiB) R9700s.
-The live engine does not use this knob at all — it claims an explicit per-card
-pair instead (`--vram-weights-mib 25600` / `--vram-kv-mib 5120`,
-`scripts/frank-archon.env:230-231`).
-
-Lower values leave more GPU memory for KV cache. This reduces swap frequency.
-
-### Swap Space / KV Offload
-
-Current vLLM has no `--swap-space` engine argument, and the CPU KV-offload knob pair
-is owned by [[RAM CPU Offloading in VLLM]] (that page carries the upstream
-`CacheConfig` verification). In Podman, mount the host cache read-write for KV offload
-writes. The table below shows historical `--swap-space` guidance as a sizing reference.
-
-| System RAM | Recommended Swap |
-|------------|------------------|
-| 32 GB | 8-16 GB |
-| 64 GB | 16-32 GB |
-| 128 GB | 32-64 GB |
-| 256 GB | 64-128 GB |
-
-More swap space allows longer contexts and more concurrent requests.
-
-### Tensor Parallelism
-
-Use tensor parallelism for large models:
-
-```
-vllm serve meta-llama/Llama-3.1-70B-Instruct \
-    --tensor-parallel-size 4 \
-    --gpu-memory-utilization 0.85
-```
-
-### KV Cache Data Type
-
-Use FP8 KV cache for Ampere and newer NVIDIA GPUs. On this host's RDNA4 cards the live engine runs an fp8 KV cache too (`kv fp8` — `scripts/frank-archon.env:264`, which also records that this engine takes only bf16|fp8 for the whole cache).
-
-```
-vllm serve Qwen/Qwen3.6-35B-A3B-FP8 \
-    --kv-cache-dtype fp8 \
-    --gpu-memory-utilization 0.9
-```
-
-FP8 KV cache reduces memory usage by 50% compared to BF16.
-
-## NUMA Optimization
-
-Dankserver reports **one** NUMA node today (`lscpu` → `NUMA node(s): 1`, `NUMA node0 CPU(s): 0-7`, read 2026-10-18), so NUMA binding buys nothing on this host; these flags matter on multi-node topologies:
-
-```
-vllm serve model \
-    --tensor-parallel-size 4 \
-    --numa-bind
-```
-
-This pins each GPU worker to its nearest NUMA node. This reduces memory access latency.
-
-For explicit NUMA binding:
-
-```
-vllm serve model \
-    --tensor-parallel-size 4 \
-    --numa-bind \
-    --numa-bind-nodes 0 0 1 1
-```
-
-## CPU Resource Requirements
-
-VLLM needs CPU cores for:
-
-- 1 API server process
-- 1 engine core process
-- N GPU worker processes (one per GPU)
-
-Minimum physical cores = 2 + N (where N is GPU count).
-
-For Dankserver, which has **2** GPUs (2× R9700) and 8 cores (`lscpu`, read
-2026-10-18):
-
-- Minimum 4 physical CPU cores by the 2 + N rule
-- Recommended 8+ physical CPU cores — the host has exactly 8 (SMT off)
-
-## Performance Tuning
-
-### Chunked Prefill
-
-Chunked prefill is enabled by default. Tune with:
-
-```
-vllm serve model \
-    --max-num-batched-tokens 8192
-```
-
-Higher values improve throughput. Lower values improve latency.
-
-### CUDA Graphs
-
-CUDA graphs improve throughput. Disable for faster startup:
-
-```
-vllm serve model \
-    --enforce-eager
-```
-
-### Compilation Cache
-
-Reuse torch.compile artifacts:
-
-```
-export VLLM_CACHE_ROOT=~/.cache/vllm
-vllm serve model
-```
-
-Copy cache between boots for faster startup.
-
-### Skip Memory Profiling
-
-Skip memory profiling on repeated boots:
-
-```
-vllm serve model \
-    --kv-cache-memory <value_from_first_boot_log>
-```
-
-## Monitoring
-
-Check GPU memory:
-
-```
-rocm-smi
-```
-
-Check for preemption warnings in VLLM logs.
-
-Check CPU utilization:
-
-```
-htop
-```
-
-## Example Commands
-
-### Single GPU, Medium Model
-
-```
-vllm serve Qwen/Qwen3.6-35B-A3B \
-    --gpu-memory-utilization 0.85 \
-    --max-num-batched-tokens 4096 \
-    --max-num-seqs 128
-```
-
-### Single GPU, FP8 Model
-
-```
-vllm serve Qwen/Qwen3.6-35B-A3B-FP8 \
-    --kv-cache-dtype fp8 \
-    --gpu-memory-utilization 0.9 \
-    --max-num-batched-tokens 8192
-```
-
-### Multi-GPU, Large Model
-
-```
-vllm serve meta-llama/Llama-3.1-70B-Instruct \
-    --tensor-parallel-size 4 \
-    --gpu-memory-utilization 0.9 \
-    --max-model-len 8192
-```
-
-## See Also
-
-- [[VLLM Overview]]
-- [[VLLM Index]]
-- [[RAM CPU Offloading in VLLM]]
-- [[Shared Memory Architecture Dankclaw]]
-- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]

serverAppConfigs-docs/LLM/LLM Bench Windows and Measurement.md

@@ -1,271 +0,0 @@
----
-type: reference
-title: LLM Bench Windows and Measurement
-description: "Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting."
-tags: [benchmarking, llama-bench, ab-testing, speculative, vllm, fleet]
-timestamp: 2026-10-05T00:00:00-07:00
----
-
-# LLM Bench Windows and Measurement
-
-Fleet law first: **STOP the production service before any GPU experiment**
-(AGENTS.md Benchmarking Policy) — a bench beside a live service starves the
-GPU and corrupts results; an unisolated GPU experiment has precedent for
-wedging a host until power cycle. Reference harnesses:
-`scripts/bench-batch-ubatch.sh`, `scripts/autoresearch.sh`,
-`scripts/autoresearch/frank-archon-arms.sh`.
-
-## 1. The window structure (one config per arm, a driver owns the window)
-
-1. **Stop + gate before every arm** — `wait_for_clear`: unit NOT active AND
-   no RUNNING container of that name (`podman ps`, not `-a`; `-a` counts
-   EXITED containers and can never clear — `rm -f` the exited one first) AND
-   port free (`ss -ltn`). The graceful-stop race cost a whole arm 2026-10-01:
-   `ExecStop=podman stop -t 120` drains up to 2 min; a driver that sleeps 3 s
-   then polls `/health` gets **200 from the still-running PRODUCTION engine**
-   — the arm "passes" with production's boot log, then `podman exec` dies
-   with `container state improper`.
-2. **Boot the arm as the service, not a scratch port** (no scratch ports on
-   LLM hosts — one card, one server). The serve script takes its config via
-   env-file override (`FRANK_ZEALOT_ENV_FILE`, `FRANK_ARCHON_AR_ENV_FILE`)
-   pointing at a staged copy; the checkout is never mutated.
-3. **Fast-fail the health wait**: poll `podman ps | grep -qx <name>` inside
-   the loop — a refused boot dies in one poll instead of 20 min. Capture the
-   serve script's FULL stdout on failure (a `tail -8` loses engine-init root
-   causes; raise happens above the tail).
-4. **Measure** (§4), capture boot-log engine facts (KV pool, arena slots,
-   SpecDecoding lines) — they survive measure crashes.
-5. **Teardown + converge**: kill by explicit PID/pgid (never pattern kills),
-   `podman rm -f` scratch, wait `MemAvailable ≥ ~230 GiB` on dankserver
-   before the next boot; restore production from the EXIT trap and verify the
-   **conjunction oracle**: health 200 + `is-active` + `is-enabled` +
-   mask-symlink clean. A bare `systemctl start` is not a restore oracle.
-   (`systemctl mask` FAILS on a relinked unit — log it, explicit stop is
-   enough: systemd never auto-restarts an explicitly stopped unit.)
-6. **Warm boots cycle ~2 min** (page cache), cold ~8 min — size timeouts for
-   mixed speeds.
-
-**Destructive tails (2026-10-01):** a scratch script whose tail does
-`systemctl stop` + `podman rm -f <PRODUCTION container>` fires later if a
-probe loop survived the driver kill — it can destroy a just-deployed
-production engine (signature: deploy exits green, then seconds-to-minutes
-later `Exited (0)`, `Deactivated successfully`, clean exit = removed, not
-crashed). Before ANY deploy: `pgrep -af` every scratch script of the session
-and kill strays by explicit PID; killing the dev-side job does NOT kill the
-host-side `deploy-remote.sh` (two of them once managed the same unit).
-
-Two failure shapes inside the window that read as "the bench died":
-
-- **Stale reads from a stopping container.** `podman stop` answers while the
-  engine inside is still flushing; a `podman logs` taken in that window
-  returns the boot-prefix only. Poll `podman ps` for `Exited (0)` (or
-  `podman wait`) before reading logs, or you will write "no results" into
-  the table for a run that has them.
-- **A D-state worker wedges the stop.** A bench child that wedged in the
-  kernel ignores SIGTERM, so `podman stop -t N` burns its full timeout and
-  then the container is `Stopping` forever. Kill by container first
-  (`podman kill <ctr>` → SIGKILL path), verify the process is gone from
-  `podman ps` AND `nvidia-smi`, and only then re-serve. Never let the
-  window script proceed to a redeploy with a zombie bench worker holding
-  VRAM.
-
-## 2. llama-bench sweeps (offline batch/ubatch/split/KV/depth)
-
-**What llama-bench cannot express — decide BEFORE designing:** no
-`--ctx-size` (`n_ctx = n_prompt + n_gen + n_depth`, so a bench can NEVER prove
-activation buffers fit beside the DEPLOYED KV — only the real unit answers
-that), no `--parallel` (no slot shape, no concurrency), no `--spec-*`
-(`tg` numbers are the drafterless baseline, useful as a control only), and a
-warmup run is itself a full-size run. `-d N` is a depth PREFIX (paid once per
-cell via a `llama_state_seq_*` save/restore); without it a `tg` run is at
-~zero depth and meaningless for long conversations.
-
-**The separator trap:** `/` separates devices WITHIN one configuration, `,`
-separates sweep values — for both `-dev` and `-ts`. `-ts 36,29` parses as the
-single value 36 → everything on one card. `--tensor-split 36,29` (server) →
-`-ts 36/29` (bench). Ranges exist: `-ub 512-2048+256`.
-
-**Harness shape:** one invocation per cell (an infeasible cell calls
-`exit(1)` and kills every later cell of a cross-product invocation — and
-infeasibility is usually the question); TSV row per cell as it completes +
-`bench.json` (`-o json`) + `bench.log` (`--progress`) + a 2 s `rocm-smi
---showmeminfo vram` VRAM trace per cell (a cell that allocated less than
-intended looks fast); resume by skipping non-empty cell outputs; stop the
-service in the harness, restore from `trap … EXIT`, verify afterwards. Match
-the deployment's load flags exactly (`-ngl -sm -ts -fa -ctk/-ctv -t`).
-
-**Run under a transient unit, not a shell:**
-
-```bash
-sudo systemd-run --unit=<name> --collect --no-block --property=Type=oneshot \
-     --property=LimitMEMLOCK=infinity --property=User=dank \
-     /home/dank/serverAppConfigs/scripts/<harness>.sh
-```
-
-Deploy units set `LimitMEMLOCK=infinity` for ROCm staging; an SSH shell has
-8 MiB and fails the load for an unrelated reason. This also survives ssh
-disconnects. **Boot rungs under the production environment, not a bench
-shell** — a rootless shell probe booted 3×262144 bf16 that production refused
-(systemd `User=` + memlock change what the pinned-expert path holds):
-`sudo systemd-run --uid=<user> --unit=<name> -p LimitMEMLOCK=infinity -p
-Restart=no env VARFILE=… /path/serve.sh`.
-
-**Verify before burning hours:** dry-run against a STUB binary with
-`SERVICE=<nonexistent-name>` (the trap/stop never reached), tiny grid,
-including one forced-`exit 3` cell — prove the TSV row, failure capture,
-resume logic. Validate the real flag set in ~1 s by pointing `-m` at a
-nonexistent file. **Reading:** bracket the sweep (reference cell first AND
-last); check per-cell `stddev_ts` (a difference smaller than spread is not a
-result); single-sequence decode is usually flat across batch/ubatch (prefill
-knob); batch ≥ ubatch (llama.cpp clamps ubatch to batch, silently duplicating
-smaller-batch cells); report what the grid cannot prove (drafter, slots, real
-context).
-
-## 3. HTTP benches of server-side parameters
-
-`llama-bench` benches the bench binary, not the running server — server-side
-params (`--threads-batch`, `--ubatch-size`, `--spec-type`, `--parallel`) need
-HTTP:
-
-- **Prefill**: large prompt, `max_tokens=1` (server crashes on 0), measure
-  `prompt_tokens / wall_time`.
-- **Decode**: `max_tokens=16384` minimum, NEVER 1 (warmup, cache growth and
-  spec acceptance all stabilize over thousands of tokens); measure
-  `predicted_per_second` from `timings` or `completion_tokens / wall_time`;
-  temperature 0.
-
-Known fleet harnesses: `scripts/bench-threads-batch-http.py`,
-`scripts/bench-decode-threads-batch.py`, `scripts/bench-spec-decode.py`.
-sglang-specific: `max_tokens=1` on `/v1/chat/completions` with the qwen3
-reasoning parser streams zero deltas — use `/v1/completions`.
-
-## 4. The measurement law (under speculative decoding this is absolute)
-
-**Never count SSE chunks.** Rejected drafts break the 1:1 chunk↔token
-mapping; chunk counting reported 8.3 tok/s for a config whose server counters
-said 23–26 — a 3x false regression that nearly abandoned the change that
-worked. Always:
-
-```python
-decode_tok_s = usage.completion_tokens / (wall - ttft)
-# stream=True, stream_options: {include_usage: true}; ttft = first content-bearing chunk
-```
-
-vLLM's 10 s `Accepted throughput` log lines are a rolling window, not a
-measurement (6.2 → 13.1 across adjacent windows on identical settings) — use
-them for acceptance STATISTICS only (`Avg Draft acceptance rate`, `Mean
-acceptance length`, per-position rates). Warm pass + ≥3 runs, median;
-temperature 0; ≥256 tokens. **A decode "jump" with early-stopped completions
-(ct < 256) and acceptance > 90 % is degenerate generation** (the fork's
-repetition-halt fired at ~140 tokens on corrupted numerics) — inspect text
-before believing a win. Run-to-run spread is 5–10 %: repeat arms; one sample
-nearly shipped the wrong setting.
-
-## 5. Honest methodology
-
-- **Match the corpus to the workload.** Token soup under-reports speculative
-  decoding (2.9–3.6 tok/update vs 3.86–5.8 on real code) and mis-ranks draft
-  settings; acceptance, not step time, moves throughput. Build the corpus
-  from the repo's own sources: `{"prompt": "<instruction>\n\n<file body>"}`,
-  `--custom-output-len`.
-- **Salt every run** with a unique per-arm token in the first bytes — it is
-  the prefix-cache key (and a proxy routing key). Without it arm 2 measures
-  arm 1's warm cache.
-- **Two workload shapes need two clients:** distinct prompts (batch
-  behaviour, prefill parallelism) and sequential turns with growing history
-  (TTFT of the first streamed chunk per turn — the only measurement that
-  exposes prefix-cache behaviour). These pull in opposite directions: a cold
-  batch spreads prefill across replicas, a conversation's next turn only pays
-  if it lands where its prefix is cached.
-- **Read the engine's ledger, not the card:** `Available KV cache memory` /
-  `GPU KV cache size: N tokens` / `Maximum concurrency for M tokens per
-  request: X.Xx` is the capacity number. vLLM's `Prefix cache hit rate:` line
-  is a LIFETIME average across every prompt the replica ever saw — send the
-  same prompt twice to an idle engine and compare TTFTs (measured 22.3 s cold
-  → 1.4 s warm for a 50k prefix). `GPU KV cache usage` says the pool is not
-  full; it does NOT say a prefix survived. The KV budget is NOT monotonic in
-  chunk size (5.44 GiB at CHUNK 8192, 4.22 at 16384, 4.16 at 4096) — measure
-  three points before treating a knob as a lever.
-- **Soak before believing:** 15–30 min of concurrent families, count failures
-  AND latency; report hit/miss split and medians with min/max — the median of
-  a bimodal distribution describes neither mode.
-- **Verification that is not the benchmark:** run the client the fleet
-  actually uses; and a deployed unit is not proven until an unattended reboot
-  brings it back.
-- **Record the "no" answers** with numbers and mechanism — placement balance,
-  concurrency admission and chunk-size tweaks all measured flat or worse
-  here; the wall was arithmetic (cached prefixes + live sequences share one
-  pool).
-
-## 6. Sweep hygiene (the arms that produced wrong verdicts)
-
-- **Prefetch uniformity:** every arm in a sweep carries the same
-  expert-prefetch setting. The 2026-09-29 campaign measured losing arms with
-  prefetch ON and incumbents OFF; untaxed, verdicts inverted (no-spec
-  12.11 → 41.47; gather 32×64 parity → +18.7 %). Never re-read a taxed
-  verdict.
-- **Knob-landed preflight:** for every `-e` knob, confirm
-  `podman exec <ctr> printenv <VAR>` — a knob the fork silently ignores
-  produces a "parity" arm that is really the default (v1 of the gather sweep
-  lost all 3 arms this way; serve-script hooks are additive and can drop
-  vars).
-- **Always run an explicit OFF arm:** the 2026-09-29 campaign's only win was
-  deleting a baseline's silent `--expert-prefetch 1` (+40–51 %). Challengers
-  compare against default-off, not "incumbent + one flag".
-- **Pin the capacity bar:** `--kv-cache-memory-bytes` (contract) rather than
-  autofit (remainder) so pool numbers are reproducible; compute need =
-  tokens × measured B/token (hybrid models: KV dtype does NOT double mamba
-  states — measure, don't assume 2x).
-- **Report decode AND prefill (TTFT) per rung:** a decode-only view nearly
-  deployed a 2x-prefill-tax config. Per-depth tables, never a single median —
-  one fork's gather win grows with depth (54 t/s @1k, 39.5 @short).
-- Container recreate (`podman rm -f`) wipes `podman cp`'d harnesses — re-copy
-  `mtp_measure.py`/`bench_iter.py` after EVERY recreate, or mount them.
-
-## 7. Bash laws that ate windows (each cost an arm)
-
-1. `${VAR:+SPEC=$1}` in front of a command expands to a bare word →
-   `SPEC=3: command not found`, nothing runs. Pass literal assignments:
-   `SPEC="${1:-}" bash serve.sh`; the callee's `${SPEC:+--flag}` skips empty.
-2. `${2WAY:-0}` is fatal `bad substitution` (digit-leading name), and inside
-   an if/else it aborts ONLY that compound statement — the script CONTINUES,
-   every arm tears down clean with zero measurements. Detection law: `ls`
-   the per-arm `.measure.txt` files before trusting any summary; arm echoes
-   are not evidence.
-3. `local a=$1 b=$x` expands `$x` before the builtin assigns — under `set -u`
-   it aborts AFTER the harness stopped production. Two statements.
-4. `sed -i` renames the file; already-running bash keeps the OLD inode —
-   mid-flight patches miss arms launched in the preceding minutes. Patch
-   early or run a make-up window.
-5. Never heredoc python through ssh→zsh→bash — write to a file, ship it
-   (`ssh host 'cat > /path && chmod +x /path' < local.sh`); nested quoting
-   fails silently. Same trap in `git commit -m`: backticks become command
-   substitution.
-6. No comments INSIDE a backslash-continued command (the continuation joins
-   the comment); verify multi-line command construction with `bash -n` AND a
-   render test.
-
-## 8. Results location and recovery (frank-archon arms)
-
-| copy | path |
-|---|---|
-| per-arm evidence | `$ARMS_DIR` = `~/.jobs/<run-stamp>/` (`.rc`, `.done` beside it) |
-| summary | `$ARMS_DIR/summary.tsv` = `arm rc decode_tps decode_tps_t0.7 kv_tokens boot_s` + `<arm>.out/.env/.overlay/.rc/.metrics` |
-| server-side last window | `/home/dank/.autoresearch-frank-archon/run.json` |
-| durable memo | `.../last/run.json` + `last/hash` (promoted only after bench AND verified restore) |
-| fallback | `~/.omp/agent/sessions/<ws>/<session>/<N>.{shake,bash}.log` (permanent) |
-
-`/tmp/frank-archon-arms-*` and `/tmp/armstub/` are stub self-tests —
-constants 99.9/88.8 identical across arms by construction; **identical
-numbers across arms == stub or bug, never a measurement.** After compaction,
-re-harvest with `grep -rhoE 'decode_tps[a-z0-9_.]*=[0-9.]+'` over the session
-logs. A window costs ~20 min warm; a detached runner's timeout ≥ 3600 s per
-window.
-
-## Related
-
-- [[RDNA4 MoE Expert Offload VRAM Budget]] — the plan arithmetic being swept
-- [[DFlash DSpark Speculative Decoding]] — spec A/B protocol and shapes
-- [[LLM Performance Diagnosis]] — profiling and attribution methods
-- [[Remote Job Runner]] — detached execution of long sweeps
-- [[frank-medium-llamacpp-dflash2]] — the measured ubatch-optimum history

serverAppConfigs-docs/LLM/LLM Performance Diagnosis.md

@@ -21,7 +21,7 @@ the cost; the problem is per-token expert movement and KV/attention tuning is
 ruled out entirely. A genuinely context-bound server shows a clear downward
 slope. Record cold vs warm separately — a small gap means the cache is
 working. The canonical flat run (fp8 expert-offload shape, a few hundred to
-sixteen thousand tokens of context) is in [[Tcclaviger Fork Ops]]'s profile.
+sixteen thousand tokens of context) is in 's profile.
 
 ## 2. Duty-cycle arithmetic (the "GPU not fully utilized" answer)
 
@@ -86,7 +86,7 @@ Worked example (GLM-5.3-Flash, 2×GPU, offload, decode 9.9 tok/s):
 
 No amount of link arithmetic would have produced those two conclusions: the
 suspect was 1 %, and the mass was pointwise glue from an uncompiled trunk —
-whose own boot log names the cause (`FULL_DECODE_ONLY` override). It also
+whose own boot log names the cause (`FULL_DECODE_ONLY` override). It
 predicted a negative result in advance (spec decode cannot help a shape whose
 per-token cost scales with the token count) which matched an unexplained flat
 MTP measurement.
@@ -117,7 +117,7 @@ per-request telemetry — it measures the real workload and costs nothing.
 - One `max_tokens=200` request is production traffic and fine; anything that
   pins the card is a bench → stop-and-mask first. Check WHICH instance you
   are reading (`systemctl status | head -12`; task ids jump across
-  restarts). The engine's own ledger (`KV self size`, `n_slots`,
+  restarts). The engine's own ledger (`KV self size`, `n_slots`
   `n_ctx_slot`) outranks your model of it.
 
 ## 5. The fork's data path (before blaming the link)
@@ -130,7 +130,7 @@ tokens, so the step can silently cross the threshold onto the slow path as
 draft depth grows. Check the threshold's premise: a default tuned at
 12.4 MiB/expert does not transfer to 2.344 MiB/expert, where copy-in can be
 several times faster. TunableOp ghost-skip: `CLAV_TUNABLEOP_SWEEP=1` alone
-does NOTHING under expert offload (`CLAV_TUNABLEOP_WITH_OFFLOAD=1` required);
+does NOTHING under expert offload (`CLAV_TUNABLEOP_WITH_OFFLOAD=1` required)
 check for the skip line before concluding autotuning ran. Gather-geometry
 knobs read at import time → each arm is a full reload.
 
@@ -189,7 +189,7 @@ analyze; `--analyze-only`).
 Measure with a **sequential-turn client**, not only a batch benchmark — a
 batch of distinct prompts parallelises prefill across replicas and hides the
 problem. The two failure modes: round-robin loses every other cache hit
-(90k-token contexts: warm-turn TTFT 2676 ms round-robin vs 1096 ms pinned);
+(90k-token contexts: warm-turn TTFT 2676 ms round-robin vs 1096 ms pinned)
 pinning-everything loses a card (16 requests: 18.6 s spread vs 30.1 s
 pinned). **The rule that gets both: route by request shape** — a request
 carrying more than one message is a continuation (agent loop re-sending its
@@ -209,13 +209,13 @@ is not a per-request verdict (probe directly: same prompt twice to an idle
 replica — 22.3 s cold → 1.4 s warm); salt corpora; record sample sizes;
 `vllm bench serve` needs `--tokenizer <local path>`; verify metric names
 against the live endpoint before dashboards. Test shape: drive the routing
-function directly with synthetic bodies (stdlib-only policy module);
+function directly with synthetic bodies (stdlib-only policy module)
 conversation identity is the FIRST N KiB of prompt — a shorter tested prefix
 hashes differently per turn and pinning silently never happens.
 
 ## Related
 
-- [[LLM Bench Windows and Measurement]] — the window discipline and usage-counter law
+-  the window discipline and usage-counter law
 - [[PCIe Link and Atomics]] — link probing and the streaming-vs-compute ceiling
 - [[RDNA4 MoE Expert Offload VRAM Budget]] — the plan behind duty-cycle terms
-- [[vLLM Porting and Compile Triage]] — the compile fix for the pointwise-glue bound
+-  the compile fix for the pointwise-glue bound

serverAppConfigs-docs/LLM/Llama Cpp Multi-Device Splits.md

@@ -1,287 +0,0 @@
----
-type: reference
-title: Llama Cpp Multi-Device Splits
-description: "What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing."
-tags: [llama.cpp, multi-gpu, layer-split, pcie, sizing, speculative]
-timestamp: 2026-10-05T00:00:00-07:00
----
-
-# Llama Cpp Multi-Device Splits
-
-## 1. Expressibility law — classify the arrangement before running anything
-
-llama.cpp expresses exactly two things across devices, and they are not
-interchangeable:
-
-| arrangement | how | expressible? |
-|---|---|---|
-| **Layer split** — layers 0..n on dev A, rest on dev B | `-sm layer -ts <a>/<b>` | **yes**, no collectives needed; works cross-vendor (CUDA + Vulkan measured 2026-09-16) |
-| **Per-tensor placement** — attention+KV on A, experts on B | `-ot <regex>=<dev>` | **NO.** `--override-tensor` sets `has_tensor_overrides()`, a REQUIRED term of the scheduler's `pipeline_parallel` flag (only read site: `llama-context.cpp`, in the `pipeline_parallel = …` expression). That flag sets `n_copies = parallel ? MAX : 1` in `ggml_backend_sched_new`; with `n_copies == 1` a pre-allocated weight on a non-primary device has no backend and the graph aborts at `ggml-backend.cpp:941` (`pre-allocated tensor (blk.0.ffn_down_exps.weight) in a buffer (Vulkan1) that cannot run the operation (NONE)` — `(NONE)` is the leaf's own op name). |
-
-Upstream anchors, re-read 2026-10-18 against `master`:
-`src/llama-context.cpp:517-526` computes `pipeline_parallel` with
-`!model.has_tensor_overrides()` as a required term;
-`ggml/src/ggml-backend.cpp:1089` carries the verbatim abort
-`pre-allocated tensor (%s) in a buffer (%s) that cannot run the operation (%s)`
-(the `:941` anchor above is where it sat in the `37b53fd` build this was
-measured on); and `common/arg.cpp` defines `-sm/--split-mode` (2821),
-`-fit/--fit` (2882), `--spec-type` (4295), `--spec-draft-device` (4259),
-`--spec-draft-n-max` (4161), `-fa/--flash-attn` (1758) — every flag in §4 is
-real upstream, not fork-only
-([source](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/common/arg.cpp)).
-
-Failure modes observed (mainline `37b53fd`): per-tensor `-ot exps=Vulkan1`
-with `-dev CUDA0,Vulkan1` → scheduler abort; layer-scoped `-ot` with a
-cross-backend pair → model load fails with only `failed to load model`;
-`-ot x=CPU` **works** because the CPU backend is last in the list — **the CPU
-is the only legitimate partner in a heterogeneous split.** On a CUDA+AMD host
-a card can be the trunk OR the offload host, never the complement of an
-AMD-resident peer. Pure Vulkan changes nothing here — the gate is the
-override, not the driver mix.
-
-If the plan needs tensors on a specific device: stop, the answer is no for
-any vendor mix. Expressible alternatives, in order: (1) two independent
-endpoints, one model per device; (2) single-vendor layer split; (3) device +
-CPU per-tensor offload (mainline pinning guard: `GGML_CUDA_NO_PINNED`; ik fork
-was `GGML_CUDA_NO_PINNED_WEIGHTS` — without it a CPU-overridden tensor can
-land in a pinned `CUDA_Host` buffer and page-lock ~108 GiB).
-
-Layer-split arithmetic to check before promising: a whole layer carries its
-experts (MoE) — compute GiB/layer, not per param-group; only some layers carry
-KV (every 4th on the hybrid), so a share of layers is only a share of KV; big
-non-repeating tensors (PLE tables) land on the main device and cannot move —
-they cap how small the primary can get.
-
-## 2. No collectives? The atomics wall and the layer-split answer
-
-Decide with one line before designing:
-
-```bash
-sudo lspci -vv -s <card2> | grep AtomicOpsCtl   # ReqEn- ⇒ no collectives, ever
-lspci -tv                                        # a Routing- bridge is a hardware wall
-```
-
-`ReqEn-` kills vLLM entirely — TP needs a per-layer allreduce AND PP needs a
-device-backend `send`/`recv` of the hidden state
-(`vllm/distributed/parallel_state.py`) — i.e. the same RCCL primitive that
-fails on an atomics-less card. llama.cpp's `--split-mode layer` needs no
-collective: layers distribute in order, only the boundary activation crosses
-PCIe once per forward pass. Full probe/repair detail:
-[[PCIe Link and Atomics]].
-
-## 3. Build: pin by feature, bundle the runtime at the exact depth
-
-- Pin the ref by FEATURE list (e.g. DFlash2 + qwen35 MTP on 2026-09-16 →
-  `ggml-org/llama.cpp#27342`); never float. Verify a fork build the same way
-  (`ls <venv>/…/spec_decode/ # dflash/ vs dflash2/`).
-- `-DCMAKE_HIP_ARCHITECTURES=<gfx only>` — a sibling arch links, loads, then
-  segfaults. `-DGGML_HIP_RCCL=OFF` when there is no atomics path (only tensor
-  mode degrades to the butterfly fallback; layer split unaffected).
-- **TRAP — `cp -r SRC DST/` flattens when `DST` does not exist** and drops one
-  directory level. rocBLAS/hipBLASLt look for device code at a FIXED depth
-  (`../lib/rocblas/library`, `../lib/hipblaslt/library/<arch>`); flattened,
-  the engine loads every GiB, answers `/health`, and dies on the FIRST request
-  with `Cannot read .../TensileLibrary.dat: No such file or directory for GPU
-  arch`. `mkdir -p DST && cp -r SRC/. DST/`, then
-  `patchelf --set-rpath '$ORIGIN/../lib' build/bin/llama-*`.
-- Dual-backend build (CUDA + Vulkan is the practical pair; HIP+CUDA do not
-  coexist): `-DGGML_CUDA=ON -DGGML_VULKAN=ON` in one tree; Vulkan needs
-  `glslc` + libvulkan headers. **Device names are not what you expect:**
-  `--list-devices` lists the discrete card under BOTH backends (`CUDA0` AND
-  `Vulkan0`) and the APU LAST (`Vulkan1`) — a wrong `-ot` name makes
-  llama.cpp print the valid set and exit (self-diagnosing).
-- What the existing build can do: `GPU_BACKEND_*=OFF` lines in its
-  CMakeCache.
-
-## 4. Serve flags that matter on a split
-
-```bash
-HIP_VISIBLE_DEVICES=0,1 llama-server \
-  --device ROCm0,ROCm1 --split-mode layer --tensor-split 1,1 \
-  --gpu-layers 999 --fit off --flash-attn on \
-  --ctx-size <n> --parallel 1 --cache-type-k f16 --cache-type-v f16 \
-  --jinja --metrics --cache-prompt \
-  --spec-type draft-dflash --spec-draft-model <dflash2.gguf> \
-  --spec-draft-device ROCm1 --spec-draft-n-max 7
-```
-
-- `--fit off` + explicit `--ctx-size`: auto-fit must not resize a context set
-  on purpose. **`--parallel N` DIVIDES `--ctx-size`** — one slot at large ctx
-  is the concurrency policy, concurrent agent loops queue.
-- Pin the drafter to the card holding the target's LAST layers
-  (`--spec-draft-device`); under tensor mode an unpinned draft joins the
-  meta-device KV allocation and aborts the KV build. `--spec-draft-n-max` must
-  be ≤ the drafter's `block_size - 1` (DFlash2: 8 ⇒ 7).
-- Qwen3.8-family templates RAISE on `reasoning_effort=high` (500) — probe
-  low/medium/high/xhigh and map harness levels before declaring ready.
-
-## 5. Split balancing: measure, never interpolate
-
-**A `-ts` step is ONE LAYER, and hybrid stacks do not have uniform layers.**
-qwen35 is 16 × (3 Gated DeltaNet + 1 Gated Attention): one step ≈ 0.43 GiB on
-a DeltaNet boundary, ≈ 1.43 GiB on a full-attention one (weights + that
-layer's own f16 KV) — measured on frank-medium. The gap can jump 2.9 GiB in a
-single step; the best integer split can sit 0.2 GiB from balance while its
-neighbours are 1+ GiB off. An affine fit reproduces the deployed point and
-overstates gaps in between (`tests/frank_medium_unit.bats` encodes the fit as
-a gate; treat a NEGATIVE fitted constant as a unit bug).
-
-Knobs that move weight and REQUIRE a re-measure: `--ubatch-size` (compute
-buffers are **~2x heavier on the output/drafter card** — 512→1536 added
-~3.3 GiB there vs ~1.7 GiB elsewhere), a drafter swap (weights + draft KV),
-`--ctx-size`/`--parallel` (KV). Total free VRAM is invariant across splits —
-only the distribution moves; use that to sanity-check a suspicious row.
-
-**Procedure (one split per load):**
-
-```bash
-WORK=/tmp/split.env; cp scripts/serve-<svc>.env $WORK
-export FRANK_MEDIUM_ENV_FILE=$WORK
-for ts in 39,26 38,27 37,28 36,29 35,30; do
-  sed -i "s|^TENSOR_SPLIT=.*|TENSOR_SPLIT=$ts|" $WORK
-  nohup bash scripts/serve-<svc>.sh >/tmp/split.log 2>&1 &
-  # poll /health (or process exit), then:
-  rocm-smi --showmeminfo vram | grep 'VRAM Total Used'   # sample ~6 s after health
-  pkill -x llama-server; sleep 6
-done
-```
-
-Load → sample → kill, NO requests sent (allocation only). Run under `nohup`
-and poll from a separate call (a load is 30–90 s). Pick the smallest
-inter-card gap that clears the per-card floor with room to spare; one `-ts`
-step ≈ 1.5 GiB at ~6k batch / 262k ctx on 30 GiB cards — enough to bracket,
-not to solve. Prefer balance: the busy card's headroom bounds the context
-(`ctx_extra ≈ (floor − measured_free) / per-position cost on the busier
-card`). A faulted split (`SDMA0 UTCL2 page fault` → `illegal memory access` →
-`ggml_abort`) is OFF-LIMITS — re-measure, never re-tune; more headroom does
-not prevent a boundary-copy fault.
-
-## 6. Drafter KV: sized from the TARGET, bounded by its window policy
-
-`common/speculative.cpp` builds the draft context with
-`cparams.n_ctx = llama_n_ctx(ctx_tgt)` — `common/speculative.cpp:2663` on
-`master`, re-read 2026-10-18 — **no flag caps it.** So:
-
-```
-draft KV = ctx_tgt × block_count × 2 (K,V) × kv_heads × key_length × dtype
-```
-
-…unless the drafter declares a sliding window. Measured at 262,144 target
-ctx: DFlash2 (`sliding_window = 2048`) ≈ 40 MiB; a same-shape 5-layer drafter
-with NO window = 20 KiB/position = **5.0 GiB** — a four-layer swing in the
-balance point, and swapping them invalidated an already-balanced split.
-
-**Failure signature:** the engine loads the whole model, answers `/health`,
-and dies on the FIRST draft decode with `ROCm error: out of memory` /
-`cudaMalloc failed` inside `common_speculative_impl_draft_dflash::process` —
-a sizing failure, not a broken drafter; under a self-healing unit it presents
-as a crash-loop. Read `sliding_window`, `block_count`, `head_count_kv`,
-`key_length` from the drafter's GGUF header first. A compact reader (the
-three format traps: no KV alignment — v3 alignment applies only to the tensor
-section; arrays are `type(9) → element_type(u32) → count(u64)`, element type
-FIRST; the enum is `7=BOOL 9=ARRAY`, 64-bit = 10/11/12, and 8=STRING/9=ARRAY
-is the v2/v3 file order, NOT gguf.h's 2/3):
-
-```python
-import struct, sys
-def rd(path):
-    f = open(path, 'rb'); assert f.read(4) == b'GGUF'
-    struct.unpack('<I', f.read(4)); struct.unpack('<Q', f.read(8))
-    nk = struct.unpack('<Q', f.read(8))[0]
-    def s():
-        n = struct.unpack('<Q', f.read(8))[0]; return f.read(n).decode('utf-8', 'replace')
-    SCALAR = {0:1,1:1,2:2,3:2,4:4,5:4,6:4,7:1,10:8,11:8,12:8}
-    FMT = {0:'<B',1:'<b',2:'<H',3:'<h',4:'<I',5:'<i',6:'<f',7:'<?',10:'<Q',11:'<q',12:'<d'}
-    keys = {}
-    for _ in range(nk):
-        k = s(); t = struct.unpack('<I', f.read(4))[0]
-        if t == 8: keys[k] = s()
-        elif t in SCALAR: keys[k] = struct.unpack(FMT[t], f.read(SCALAR[t]))[0]
-        else:
-            et = struct.unpack('<I', f.read(4))[0]; n = struct.unpack('<Q', f.read(8))[0]
-            if et == 8: keys[k] = [s() for _ in range(n)]
-            elif et in SCALAR: keys[k] = ('arr', et, n); f.read(n * SCALAR[et])
-            else: raise Exception('unsupported arr elem type %d' % et)
-    return keys
-```
-
-## 7. Heterogeneous placement triage (go/no-go with arithmetic)
-
-- **Link:** `LnkCap` vs `LnkSta` at every hop; expect **idle-downshift** (a
-  Gen4 x4 dock idles at Gen1 x4, ~1 GB/s) — measure under load. Per-hop cost
-  vs hidden state (5120-dim f16 = 10 KiB/hop ≈ 1.3 µs at Gen4 x4): bytes are
-  almost never the constraint; **hop count is** — every boundary is a host
-  sync, and a per-op-split runtime can hit 100 splits/token at ~5 % SM. Read
-  `graph splits` from the loader log.
-- **KV is usually the bottleneck:** `KV/token = kv_bytes_per_position ×
-  context` (attention reads the whole cache every token). Highest-leverage
-  placement: **KV on the fastest-memory device together with its attention
-  compute** — never KV alone over the link (GBs over 7.88 GB/s = hundreds of
-  ms/token). Check fit: `kv_positions × slots × bytes + attention_trunk ≤
-  VRAM`.
-- **What placement never buys:** prefill is compute-bound and expert weights
-  are read once per batch — moving them buys nothing and adds a serial stage.
-  Calibration: Strix Halo + 3090, moving 12.5 % of expert mass to the GPU
-  gave decode +12 % but prefill 192 → 32 tok/s. The harmonic
-  `1/(f_big/BW_slow + f_fast/BW_fast)` reproduces the decode gain within ~2
-  points (assumes sequential stages — true at batch 1).
-- **UMA check before assuming additivity:** `mem_info_vram_total` (BIOS
-  carve-out) vs `mem_info_gtt_total` (aperture onto SYSTEM RAM). A big GTT
-  window means the "GPU memory" IS the same LPDDR: adding a stage there adds
-  a compute engine, not capacity/bandwidth. Only discrete VRAM is additive.
-- **Speculation as the "cheap" split:** bandwidth-free (token ids per round)
-  but needs the exact tokenizer for a plain drafter or a head trained for
-  that target; a plain drafter costs K sequential forwards (break-even needs
-  p ≈ 0.7+ at r≈0.4–0.5, K=3); compare against the INCUMBENT (a target
-  already committing ~1.7 tok/round on its own MTP head is a high bar); and
-  check the runtime accepts external drafters at all (closed engines 400
-  unknown drafter names by design). Details: [[DFlash DSpark Speculative Decoding]].
-- **Hazards:** keep paged/sidecar tables where they are read from; keep mmap
-  ON for huge-embedding models (anonymous memory → cgroup kill); CUDA graph
-  capture can abort on unusual graph shapes; make the FIRST measurement a
-  single-device baseline of the runtime on the new device — a split's
-  arithmetic is worthless if the baseline loses to the incumbent; a split
-  consumes the fast device entirely — check what it was serving.
-
-**Verdict ranking:** (1) two independent endpoints — often the real answer;
-(2) one process, weights-resident, KV co-located with attention on the fast
-pool; (3) hot-expert cache only if `misses × bytes × tok/s` fits the link and
-overlaps — no mainstream runtime implements it; (4) name what does NOT help:
-TP across unequal devices, KV on the far device, loading over the link,
-prefill-motivated placements.
-
-## 8. Harness syntax and verification
-
-`llama-bench` separates a LIST with `/` and SWEEP values with `,`:
-`-dev CUDA0,Vulkan1` is TWO single-device runs, never one two-device run;
-`-ts 0.85,0.15` is two sweep values each becoming `[0.85, 0, …]` — device 0
-takes everything. Write `-dev A/B -ts x/y`. Read the table's own `dev` column;
-any placement-dependent number needs an independent memory check
-(`nvidia-smi` sampling while loaded). (This artifact produced a wrong fleet
-conclusion 2026-09-16 — "CUDA + Vulkan cannot serve one model" — with two
-bogus OOMs that were really all-111-GiB-on-the-24-GiB-card.)
-
-Verification battery: `/health`; both cards' memory ≈ half the weights each
-(busier card also carries the drafter; compare `model_bytes/2 + drafter +
-ctx×KV/2 + buffers`); raise ctx by N and watch BOTH cards rise by
-N·bytes/2 — neither card's total may equal the model file (that arithmetic
-rules out replication); `spec_decode_num_drafts_total` > 0 and a real
-completion; a deep prompt (140k) proves a raised window is REAL, not
-declared; repeat prompt → `timings.prompt_n` collapses (prefix cache live);
-`/slots` shows expected `(id, n_ctx)` pairs; two concurrent requests with
-`journalctl -o cat | grep 'slot print_timing'` proves drafting on BOTH slots
-(default journal output truncates these lines).
-
-Ops traps: a server started over ssh outlives stopping the runner (keeps the
-port — `pkill -x <binary>` first); `systemctl start` of a root unit needs
-sudo (polkit "Access denied" otherwise); old crash `ROCm error` lines live in
-the journal forever — check `ActiveEnterTimestamp`/`NRestarts` before
-treating a grep count as a live fault.
-
-## Related
-
-- [[PCIe Link and Atomics]] — the atomics wall, link probing, repair
-- [[Llama Cpp Serving Sizing]] — slot-pool and KV pricing
-- [[DFlash DSpark Speculative Decoding]] — spec types, chaining, A/B
-- [[frank-medium-llamacpp-dflash2]] — the layer-split era runbook
-- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the measured history

serverAppConfigs-docs/LLM/Llama Cpp Serving Sizing.md

@@ -19,7 +19,7 @@ guesswork.
    `n_ctx_slot = CTX_SIZE / PARALLEL`. A hybrid model with
    `kv_unified = 'false'` gives each slot its own cache AND a fixed per-slot
    state buffer — slot count is a memory term, not free.
-2. **Two boots on the real card** (production stopped, per the bench law),
+2. **Two boots on the real card** (production stopped, per the bench law)
    each near the target; record `nvidia-smi` used/total plus the boot log's
    `n_slots`/`n_ctx_slot`: shape A = current/cheap, shape B = new demand.
    `non-KV = used − (tokens × KiB/token)`; the two rows separate
@@ -31,7 +31,7 @@ guesswork.
    slot gets an equal integer window. Boot and measure; if outside the band,
    walk one step — never re-derive from a formula.
 4. **Propagate the per-slot number — it is the client contract:**
-   `models.yml` `contextWindow` = `n_ctx_slot` (NOT the model's native window),
+   `models.yml` `contextWindow` = `n_ctx_slot` (NOT the model's native window)
    or long conversations die with `400 exceed_context_size_error` instead of
    compacting; compaction/summary budgets sized to the old window get
    re-checked; endpoint docs and the rollback table move too.
@@ -66,13 +66,13 @@ co-tenants.
 - **Engine ledger first.** Halogen prints weights pinned (+ remaining 2 MiB
   contiguous blocks — the honest fragmentation signal: a block census that drops
   across a restart means the model did not come back into the same hole; the
-  before/after census this was learned on is that era's, on [[frank-dragoon]]),
+  before/after census this was learned on is that era's, on [[frank-dragoon]])
   device budget, KV pool/working split, "left for everything
   else". llama.cpp prints them at `llama-model.cpp:1827` (per-backend model buffer)
   and `llama-context.cpp:692` (compute buffer) on the era's build; on today's
   `master` the same `LLAMA_LOG_INFO` lines sit at `llama-model.cpp:2009`
   ("model buffer size =") and `llama-context.cpp:842` ("compute buffer size =")
-  ([source](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/src/llama-context.cpp),
+  ([source](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/src/llama-context.cpp)
   re-read 2026-10-18) — but **`llama-bench`
   suppresses loader INFO and a systemd unit did not put them in the journal**
   (verified `grep -c` = 0). Recipe: one launch through `llama-cli`/`llama-server`
@@ -92,7 +92,6 @@ co-tenants.
   projector, never weights alone. Do
   the capacity subtraction before designing, on the card number the owner page
   quotes, not the advertised one.
-- **Context lengths: derive, never remember.** `llama-bench` computes
   `n_ctx = n_prompt + n_gen + n_depth` (defaults 512/128/0, type_k/v F16,
   flash_attn AUTO, split_mode LAYER). `-p 512,16384 -n 64` = contexts
   576/16,448; a plain `tg` row has ~zero depth — `-d` is the only thing that
@@ -116,7 +115,7 @@ of the whole sequence + constant recurrent state + the draft context's KV:
 `per_token = 2 × n_full_attn_layers × n_kv_heads × head_dim × bytes (K and V) +
 draft_per_token`. The worked qwen35-hybrid instance of that formula (full layer
 count, f16 KV, MTP draft) and the slot-size table it prices are on
-[[frank-medium-llamacpp-dflash2]]; cross-check any such arithmetic against the
+; cross-check any such arithmetic against the
 journal's `prompt state size` lines — the two must agree.
 
 **The journal is the instrument:** `exceeds cache size limit … skipping` =
@@ -171,6 +170,6 @@ part of that era's record on [[frank-dragoon]].
 
 - [[GGUF Compatibility]] — what a payload is before it is priced
 - [[frank-zealot]] — the endpoint whose Eras rows are the two-boot priced shapes this method produced
-- [[frank-medium-llamacpp-dflash2]] — the state-law worked example behind section 3
+-  the state-law worked example behind section 3
 - [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
-- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side
+- /  — the multi-device side

serverAppConfigs-docs/LLM/Model Swap and Rollback.md

@@ -51,9 +51,9 @@ express; per-tensor cross-device placement stays refused under
 ## 2. Ground truth first
 
 - Read the HF model card (runtime pin, hard constraints, recommended args)
-  + the repo trio `scripts/<svc>.env`, `scripts/serve-<svc>.sh`,
+  + the repo trio `scripts/<svc>.env`, `scripts/serve-<svc>.sh`
   `scripts/<svc>.service` + the runbook under `serverAppConfigs-docs/LLM/`.
-- Check the host before downloading anything: `ls /home/dank/models/`,
+- Check the host before downloading anything: `ls /home/dank/models/`
   `df -h /home`, `du -sh` — bench-era or prior-deployment files often already
   exist.
 
@@ -129,7 +129,7 @@ arms possible — keep it intact.
   `images` is silently ignored; probe with ≥256 px).
 - vLLM specifics: first request after boot can take ~18 s (deferred Triton
   loads) — warm and discard; `--reasoning-parser` / tool parsers are part of
-  the contract ([[vLLM Boot Forensics and Config Interrogation]]).
+  the contract ().
 
 ## 7. systemd handoff — THE TRAP
 
@@ -205,7 +205,7 @@ Pure measurement sessions that changed no state need only pin 1.
   ladder UD-IQ1_S 93.1 GB → UD-Q4_K_XL 199.7 GB, GSQ-RCO 3.5-bit −1.40 pp
   MMLU-Pro vs Q8_0).
 - GGUF parsing gotcha: expert tensors are
-  `blk.N.ffn_{gate,up,down}_exps.weight` — 4 dot-parts INCLUDING `.weight`;
+  `blk.N.ffn_{gate,up,down}_exps.weight` — 4 dot-parts INCLUDING `.weight`
   a `len(parts) == 3` filter matches zero layers with no parse error. Filter
   on `parts[0]=="blk" and parts[3]=="weight" and "exps" in parts[2]`.
 
@@ -214,7 +214,7 @@ Pure measurement sessions that changed no state need only pin 1.
 When a host must serve two models at once (an iGPU runtime plus a second
 card), or one endpoint is moving and a new one takes the freed hardware.
 
-**0. Decide the shape before touching hardware.** One provider per
+**0. Decide the shape before touching hardware.** One provider
 HOST:PORT in `~/.omp/agent/models.yml` (on the box where the harness runs)
 — a second endpoint is a new provider block, not a second model in the
 existing one. Read the budget as an inequality: for unified memory the
@@ -235,7 +235,7 @@ suffix — and check what pointed at them: bench harnesses, a chat template
 only the old runtime read, and `scripts/test-changed.sh` mappings are the
 usual victims (a mapping to a retired test file fails the pre-commit
 wrapper silently for weeks). One bats file per service asserting
-INVARIANTS (`pool == 2 × ctx`, KV bytes ≤ card budget, port separation),
+INVARIANTS (`pool == 2 × ctx`, KV bytes ≤ card budget, port separation)
 not constants; a negative grep must strip comments first
 (`grep -vE '^\s*#' file | grep -q …`) or it matches the prose explaining
 the very thing you removed.
@@ -258,12 +258,12 @@ target — including whether `/metrics` answers — is [[frank-dragoon]]'s wire
 contract.
 
 **4. Cutover, then verify by driving it.** `deploy.sh <target> --service
-<large> --service <small>` (order matters); per endpoint: `/health`,
+<large> --service <small>` (order matters); per endpoint: `/health`
 `/v1/models` reports the alias, a chat completion returns content AND
 `reasoning_content`, a `tools` request returns `tool_calls`, an image
 request is answered (or refused with a named reason), the runtime's own
 speculation counters move, and the unit is `enabled` and `active`. All three LLM
-endpoints are **root** units today (`scripts/frank-archon.service`,
+endpoints are **root** units today (`scripts/frank-archon.service`
 `scripts/frank-dragoon.service`, `scripts/frank-zealot.service`), so linger is
 not what gates them; linger matters for rootless quadlets. Read the engine's boot
 ledger for a silent clamp. Finally spend one
@@ -282,6 +282,6 @@ predecessor "stopped" (the port race above).
 - [[GGUF Compatibility]] — which runtime can load a payload at all
 - [[Llama Cpp Serving Sizing]] — pricing the shape once it loads
 - [[RDNA4 MoE Expert Offload VRAM Budget]] — vLLM-side fit arithmetic
-- [[LLM Bench Windows and Measurement]] — measuring the new deployment
+-  measuring the new deployment
 * [[frank-archon]] — the live Dankserver endpoint; the tcclaviger vLLM era
   (2026-09-29 → 10-05) is a row in its Eras table

serverAppConfigs-docs/LLM/PCIe Link and Atomics.md

@@ -61,13 +61,13 @@ H2D rate`; `ceiling = 1 / link_time × resident-fraction saved (LRU hits)`.
 Compare against the compute-in-RAM path at host RAM bandwidth (256 GB/s
 LPDDR5X on the Strix Halo — 128 GB LPDDR5x-8000 on a 256-bit bus,
 [amd.com spec page](https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html)).
-**If the link is >10x slower than the memory behind it (x4 is 31x here),
+**If the link is >10x slower than the memory behind it (x4 is 31x here)
 streaming loses by construction and no tuning rescues it; doubling the link
 only doubles a losing number.**
 
 Worked example: the Dankclaw Gen4 x4 run of 2026-09-16 — measured H2D/D2H
 rates, the 3 MiB slab cost, and the streaming-vs-compute comparison it decided
-— is recorded on [[x4-pair-experiment]] §The link, measured. Read it once and
+— is recorded on  §The link, measured. Read it once and
 apply the rule above: on that link the streaming design lost by 3–5x, and the
 x8 ceiling would only have halved the loss.
 
@@ -88,7 +88,7 @@ time, so encode them:
   + `hipSetDevice` only. First line: `setvbuf(stdout, NULL, _IONBF, 0);` — a
   segfaulting process loses block-buffered stdout and hides the crash point.
 - Build with an explicit arch: `HIP_TARGETS=gfx1201 hipcc -O2
-  --offload-arch=gfx1201 -o p p.hip`; compile with `| grep -E "error:"`,
+  --offload-arch=gfx1201 -o p p.hip`; compile with `| grep -E "error:"`
   never `| tail -N` (truncation hid a missing `#include <cstring>`).
 - `hipMemset`/`hipDeviceSynchronize` are `[nodiscard]` — use or cast.
 - Mount the whole `/dev/dri` into ROCm containers; individual `renderD*`
@@ -109,28 +109,28 @@ static double one(size_t bytes, bool pinned, int iters) {
     if (hipMalloc(&dev, bytes) != hipSuccess) return -1;
     if (pinned) { if (hipHostMalloc(&host, bytes, hipHostMallocDefault) != hipSuccess) return -1; }
     else        { host = malloc(bytes); }
-    memset(host, 0x5a, bytes);
-    hipEvent_t a, b; hipEventCreate(&a); hipEventCreate(&b);
-    for (int i = 0; i < 3; i++) hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice);
-    hipDeviceSynchronize();
-    hipEventRecord(a, 0);
-    for (int i = 0; i < iters; i++) hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice);
-    hipEventRecord(b, 0); hipEventSynchronize(b);
-    float ms = 0; hipEventElapsedTime(&ms, a, b);
+    memset(host, 0x5a, bytes)
+    hipEvent_t a, b; hipEventCreate(&a); hipEventCreate(&b)
+    for (int i = 0; i < 3; i++) hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice)
+    hipDeviceSynchronize()
+    hipEventRecord(a, 0)
+    for (int i = 0; i < iters; i++) hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice)
+    hipEventRecord(b, 0); hipEventSynchronize(b)
+    float ms = 0; hipEventElapsedTime(&ms, a, b)
     double gb = (double)bytes * iters / (ms / 1000.0) / 1e9;
-    if (!pinned) free(host); else hipHostFree(host);
-    hipFree(dev); hipEventDestroy(a); hipEventDestroy(b);
+    if (!pinned) free(host); else hipHostFree(host)
+    hipFree(dev); hipEventDestroy(a); hipEventDestroy(b)
     return gb;
 }
 int main(int argc, char **argv) {
-    setvbuf(stdout, NULL, _IONBF, 0);
+    setvbuf(stdout, NULL, _IONBF, 0)
     if (hipSetDevice((argc > 1) ? atoi(argv[1]) : 0) != hipSuccess) return 1;
     size_t sz[] = {2459136UL, 3UL<<20, 16UL<<20, 64UL<<20, 256UL<<20, 1UL<<30};
     for (int i = 0; i < 6; i++) {
         size_t b = sz[i];
         int iters = (b >= (1UL<<30)) ? 10 : (b >= (256UL<<20)) ? 20 : 100;
         printf("  %10.3f MiB | pinned %6.2f | pageable %6.2f\n",
-               b/1048576.0, one(b,true,iters), one(b,false,iters));
+               b/1048576.0, one(b,true,iters), one(b,false,iters))
     }
     return 0;
 }
@@ -185,7 +185,7 @@ sudo lspci -vv -s <bridge> | grep AtomicOpsCap                   # want Routing+
 slot on CPU root ports (atomics fine), the rest behind the chipset (usually
 not). Bandwidth for TP is small anyway: ~10 MB per token-step for a 27B at
 batch 4 is a millisecond-scale figure at x4 — compute-bound, not
-interconnect-bound (full derivation: [[Dual-R9700-Cooperation]] §Bandwidth).
+interconnect-bound (full derivation:  §Bandwidth).
 
 **Probe matrix (bare 2-rank `torch.distributed`, one process per GPU, 25 s PG
 timeout) — one probe answers several design questions:**
@@ -216,11 +216,11 @@ reporting an outage (`gitea` vs `gitea-server`).
    complex; x4 suffices for TP.
 2. Board whose CPU x16 bifurcates x8/x8 (done 2026-09-20 on dankserver — the
    per-device chain census, including both endpoints reading `ReqEn+`, is
-   [[Dual-R9700-Cooperation]]'s). Re-checked 2026-10-18 without root: both root
+   's). Re-checked 2026-10-18 without root: both root
    ports report `current_link_width=8` at `32.0 GT/s`
    (`/sys/bus/pci/devices/0000:00:01.{1,3}/current_link_width`), and the live
    engine states direct P2P between the cards.
-3. llama.cpp `--split-mode layer` — no collectives at all ([[Llama Cpp Multi-Device Splits]]); costs the GGUF conversion plus paged attention /
+3. llama.cpp `--split-mode layer` — no collectives at all (); costs the GGUF conversion plus paged attention /
    continuous batching / spec decode.
 4. Independent TP=1 replicas + routing proxy — cards never share one model.
 
@@ -230,7 +230,7 @@ is gated by a CUDA-sm-keyed table (`{9.0, 10.0, 10.3, 10.7}`; ROCm reports
 `CUSTOM_ALL_REDUCE_MAX_SIZES` in
 `vllm/distributed/device_communicators/all_reduce_utils.py`, whose keys on
 today's `main` are exactly `"9.0"`, `"10.0"`, `"10.3"`, `"10.7"`
-([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/distributed/device_communicators/all_reduce_utils.py),
+([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/distributed/device_communicators/all_reduce_utils.py)
 read 2026-10-18); nothing keys on a ROCm-reported `12.0`. — and patching it is
 insufficient because TP init's `broadcast`/`barrier`/logits `all_gather` have
 no non-RCCL path. A
@@ -249,6 +249,6 @@ kernel's declared args (`roc-obj-extract` + `llvm-readelf --notes`), look for
 
 ## Related
 
-- [[Llama Cpp Multi-Device Splits]] — the layer-split path around the wall
-- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — measured history
+-  the layer-split path around the wall
+- /  — measured history
 - [[frank-archon]] — the gather economics of the retired tcclaviger vLLM era (2026-09-29 → 10-05)

serverAppConfigs-docs/LLM/Pipeline Parallelism/Ethernet Bandwidth Math.md

@@ -1,454 +0,0 @@
----
-type: reference
-title: Ethernet Bandwidth Math
-description: "Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it."
-tags:
-  - pipeline-parallelism
-  - research
-timestamp: 2026-10-19
----
-
-# Bandwidth Analysis: Multi-Node LLM Inference over Ethernet
-
-## 0. Measured on the Qwen3.8-27B PP deployment (2026-08-17)
-
-Everything in this section was measured on the live deployment
-([[Qwen3.8 PP Deployment]]) while sampling `/proc/net/dev` at 50 Hz on
-both ends. The theoretical tables below use H=4096/L=32; the real model
-is larger (see constants), which raises the per-token bytes but not the
-conclusions.
-
-### 0.1 Verified constants (from the GGUF + env, not guessed)
-
-| Constant | Value | Source |
-| -------- | ----- | ------ |
-| `hidden_size` (embedding_length) | 5120 | GGUF metadata (`qwen35.embedding_length`) |
-| Blocks | 65 (45 on worker, 20 on head) | `--tensor-split 0.68,0.32` |
-| FA layers | 16 total, interval 4 (11 worker / 5 head) | GGUF `full_attention_interval` |
-| KV per FA layer per token | 3136 B (K q8_0 1088 + V f16 2048) | env file comment, confirmed |
-| ubatch | 256 | `UBATCH_SIZE` in `frank-dragoon.env` |
-| MTP verify step positions | 5 (1 + n_max 4) | `SPEC_DRAFT_N_MAX=4` |
-| Link (old path) | 1 GbE via USB-C adapter on claw | `enx00e04cc2e1e5`, negotiated 1000 Mb/s |
-| Link (current path) | 2.5 GbE eno1 to eno1 | negotiated 2500 Mb/s both ends |
-| iperf3, 2.5 GbE path | 281 MB/s claw->server, 294 MB/s server->claw | measured |
-| ping RTT, 1 GbE USB path | min 0.239 / avg 0.742 / max 1.085 / mdev 0.334 ms | measured |
-| ping RTT, 2.5 GbE path | min 0.224 / avg 0.585 / max 0.883 / mdev 0.211 ms | measured |
-
-The USB-C adapter added ~15 us to RTT (0.239 vs 0.224 ms min) and more
-jitter (mdev 0.334 vs 0.211 ms).
-
-### 0.2 Measured wire bytes (the number that matters)
-
-Raw hidden state per position per direction = 5120 x 2 = **10.0 KB**.
-The *measured* bytes per position per direction are ~4x that:
-
-| Direction | Measured | Raw hidden |
-| --------- | -------- | ---------- |
-| head -> worker | 39.8 KB/position | 10.0 KB |
-| worker -> head | 38.4 KB/position | 10.0 KB |
-
-The extra ~30 KB/position is the serialized RPC graph (per-position
-nodes for all 45 worker blocks: attention, KV append, SSM ops)
-plus protocol framing. It scales with position count just like the
-payload, so it does not shrink away.
-
-Per ubatch (256 positions) the measured crossing is **20.5 MB**
-(10.4 MB head->worker + 10.1 MB worker->head).
-
-### 0.3 Measured throughput and the validated model
-
-Measured prefill (fresh 13,959-token prompt, 14k context, 2.5 GbE
-path): **457.9 t/s** (30,485 ms, journal task 11924). Decomposition:
-
-| Term | 2.5 GbE (measured anchor) |
-| ---- | ------------------------- |
-| wire per ubatch (20.5 MB) | 70 ms |
-| compute + overhead per ubatch (residual) | 489 ms |
-| ubatch total | 559 ms = 2.18 ms/token |
-
-Using that residual and only swapping the wire time per link speed:
-
-| Link | wire/ubatch | predicted prefill | avg link use | line-rate burst duty |
-| ---- | ----------- | ----------------- | ------------ | -------------------- |
-| 1 GbE (old USB path) | 174 ms | **386 t/s** | 15.8% | 26.2% |
-| 2.5 GbE (current) | 70 ms | 458 t/s (matches measured) | 6.4% | 12.5% |
-| 10 GbE | 17 ms | **505 t/s** | 1.6% | 3.4% |
-
-Delta: 1 -> 2.5 GbE is **+18.6% prefill**; 2.5 -> 10 GbE is **+9.4%**.
-
-### 0.4 The KV snapshot: where the link really saturates
-
-On task release, `llama-server` saves the conversation to the prompt
-cache. The KV for the 11 FA layers living on the worker GPU must be
-copied **worker -> head** for that snapshot, and the slot is BLOCKED
-while it runs (`PARALLEL=1` = one user queues behind it).
-
-Measured for a 98,100-token conversation: 3,454 MB copied (model check:
-98100 x 11 x 3136 B = 3.38 GB, matches), at line rate (254 MB/s avg,
-308 MB/s peak).
-
-| Link | snapshot duration (slot blocked) |
-| ---- | -------------------------------- |
-| 1 GbE | ~31 s |
-| 2.5 GbE | 13.6 s (measured; journal: "prompt cache update took 14418 ms") |
-| 10 GbE | ~5 s |
-
-This is the only phase that runs the link at ~100% utilization for a
-long time. Prefill runs at 12-26% of line rate in bursts (per ubatch)
-and ~6-16% on average; decode runs at ~2% or less.
-
-### 0.5 Decode (MTP verify step, 5 positions)
-
-| Link | net time per step | share of 164 ms step | decode t/s |
-| ---- | ----------------- | -------------------- | ---------- |
-| 1 GbE | 4.0 ms (RTT 0.74 + wire 3.4) | 2.4% | ~23.0 |
-| 2.5 GbE | 1.9 ms (RTT 0.59 + wire 1.4) | 1.1% | 23.47 (measured) |
-| 10 GbE | 0.6 ms (RTT ~0.25 + wire 0.3) | 0.4% | ~23.5 |
-
-Decode is latency-touched but bandwidth-trivial. Average wire rate
-~19 Mbit/s in each direction.
-
-### 0.6 Verdict for the 10 GbE USB-C adapter question
-
-- The 1 GbE -> 2.5 GbE upgrade (the big win) is already captured for
-  free: the claw's native 2.5 GbE `eno1` now carries the RPC path
-  (the 1 GbE USB adapter was unplugged 2026-08-17; routing moved the
-  path, the RPC TCP re-attached on the new interface).
-- 2.5 -> 10 GbE buys +9.4% prefill and 13.6 s -> 5 s snapshot blocking.
-  It requires the L2 device at 192.168.0.1 to be 10G-capable, or a
-  direct cable between the two machines (dedicated link, second IP
-  pair, or a lower-metric static route).
-- At current usage (PARALLEL=1, decode-heavy), 2.5 GbE is comfortable.
-
-## 1. Bandwidth Math: Bytes per Token per Layer Boundary
-
-### 1.1 Pipeline Parallelism (PP)
-
-During autoregressive generation, each transformer layer produces a hidden state of shape `(batch_size, hidden_size)`. At a pipeline stage boundary, this tensor must be communicated between nodes.
-
-**Forward pass only (inference):**
-```
-bytes_per_boundary = batch_size × hidden_size × 2   [fp16 = 2 bytes]
-```
-
-**Key detail:** During *generation* (one token at a time), seq_len = 1 at each boundary crossing. The KV cache stays local. The hidden state is `(batch, hidden_size)` — independent of context length.
-
-**For a 27B dense model (e.g., Qwen3.6-27B):**
-- Hidden size: H = 4096 (standard for Llama/Qwen-style architectures).
-  Note: the Qwen3.8-27B actually deployed on this fleet has H = 5120
-  (GGUF-verified); see section 0, which supersedes these theoretical
-  numbers with measurements.
-- Per-token, per-boundary: `4096 × 2 = 8,192 bytes = 8 KB`
-- With Nₚₚ stage boundaries: total = `Nₚₚ × 8 KB/tok`
-
-**For a 36B MoE model (e.g., Qwen3.6-35B-A3B):**
-- Hidden size: H = 6144 (common for this scale)
-- Layers: L = 48 (typical)
-- Per-token, per-boundary: `6144 × 2 = 12,288 bytes = 12 KB`
-- With Nₚₚ stage boundaries: total = `Nₚₚ × 12 KB/tok`
-
-**Worked examples (1 boundary = 2 PP stages):**
-```
-27B, batch=1:  1 boundary × 4096 × 2      bytes = 8,192 bytes/tok  = 8 KB
-27B, batch=64: 1 boundary × 4096 × 64 × 2 bytes = 524,288          = 512 KB/tok
-36B, batch=1:  1 boundary × 6144 × 2      bytes = 12,288 bytes/tok = 12 KB
-```
-
-### 1.2 Tensor Parallelism (TP) — for comparison
-
-In tensor parallelism, each MLP and attention layer performs an all-reduce on the hidden state:
-
-```
-bytes_per_layer = batch_size × hidden_size × 2   [one all-reduce per layer]
-total_bytes_per_tok = L × batch_size × hidden_size × 2   [L all-reduces total]
-```
-
-**For 27B, H=4096, L=32, TP=2, batch=1:**
-```
-32 layers × 4096 × 1 × 2 bytes = 262,144 bytes = 256 KB/tok
-(16 KB per layer all-reduce × 32 layers)
-```
-
-**For 36B, H=6144, L=48, TP=2, batch=1:**
-```
-48 layers × 6144 × 1 × 2 bytes = 589,824 bytes = 576 KB/tok
-(12 KB per layer all-reduce × 48 layers)
-```
-
-### 1.3 Scaling with batch size
-
-| Model | H | L | tok/s | Batch | PP boundary bytes/tok | TP total bytes/tok |
-|-------|-----|-----|-------|-------|----------------------|--------------------|
-| 27B | 4096 | 32 | 10 | 1 | 8 KB | 256 KB |
-| 27B | 4096 | 32 | 50 | 1 | 8 KB | 256 KB |
-| 27B | 4096 | 32 | 100 | 1 | 8 KB | 256 KB |
-| 27B | 4096 | 32 | 10 | 16 | 128 KB | 4,096 KB |
-| 27B | 4096 | 32 | 50 | 16 | 128 KB | 4,096 KB |
-| 27B | 4096 | 32 | 100 | 16 | 128 KB | 4,096 KB |
-| 27B | 4096 | 32 | 10 | 64 | 512 KB | 16,384 KB |
-| 27B | 4096 | 32 | 50 | 64 | 512 KB | 16,384 KB |
-| 27B | 4096 | 32 | 100 | 64 | 512 KB | 16,384 KB |
-| 36B | 6144 | 48 | 10 | 1 | 12 KB | 576 KB |
-| 36B | 6144 | 48 | 50 | 1 | 12 KB | 576 KB |
-| 36B | 6144 | 48 | 100 | 1 | 12 KB | 576 KB |
-| 36B | 6144 | 48 | 10 | 16 | 192 KB | 9,216 KB |
-| 36B | 6144 | 48 | 50 | 16 | 192 KB | 9,216 KB |
-| 36B | 6144 | 48 | 100 | 16 | 192 KB | 9,216 KB |
-| 36B | 6144 | 48 | 10 | 64 | 768 KB | 36,864 KB |
-| 36B | 6144 | 48 | 50 | 64 | 768 KB | 36,864 KB |
-| 36B | 6144 | 48 | 100 | 64 | 768 KB | 36,864 KB |
-
----
-
-## 2. Required Bandwidth vs. Ethernet Capabilities
-
-### 2.1 Ethernet link capacities
-
-| Link | Raw Rate | Usable TCP (approx.) | MB/s usable |
-|------|----------|---------------------|-------------|
-| 1 GbE | 1,000 Mbit/s | ~940 Mbit/s | ~118 MB/s |
-| 2.5 GbE | 2,500 Mbit/s | ~2,350 Mbit/s | ~294 MB/s |
-| 10 GbE | 10,000 Mbit/s | ~9,400 Mbit/s | ~1,175 MB/s |
-
-### 2.2 Required bandwidth (Mbps) by model, batch, and throughput
-
-**27B model (H=4096), 1 PP boundary:**
-
-| tok/s | Batch=1 | Batch=16 | Batch=64 |
-|-------|---------|----------|----------|
-| 10 | 0.13 Mbit/s | 2.05 Mbit/s | 8.19 Mbit/s |
-| 50 | 0.65 Mbit/s | 10.24 Mbit/s | 40.96 Mbit/s |
-| 100 | 1.31 Mbit/s | 20.48 Mbit/s | 81.92 Mbit/s |
-
-**36B model (H=6144), 1 PP boundary:**
-
-| tok/s | Batch=1 | Batch=16 | Batch=64 |
-|-------|---------|----------|----------|
-| 10 | 0.20 Mbit/s | 3.07 Mbit/s | 12.29 Mbit/s |
-| 50 | 0.98 Mbit/s | 15.36 Mbit/s | 61.44 Mbit/s |
-| 100 | 1.97 Mbit/s | 30.72 Mbit/s | 122.88 Mbit/s |
-
-**27B model, TP (L=32 layers all-reduce):**
-
-| tok/s | Batch=1 | Batch=16 | Batch=64 |
-|-------|---------|----------|----------|
-| 10 | 2.05 Mbit/s | 32.77 Mbit/s | 131.07 Mbit/s |
-| 50 | 10.24 Mbit/s | 163.84 Mbit/s | 655.36 Mbit/s |
-| 100 | 20.48 Mbit/s | 327.68 Mbit/s | 1,310.72 Mbit/s |
-
-### 2.3 Feasibility
-
-Thresholds: **feasible** = under 50% of link capacity, **marginal** = 50–90%,
-**saturated** = over 90%. Cross-checking §2.2 against §2.1: every PP case is
-feasible on every link class — even the heaviest, 27B at batch=64 and 100
-tok/s (81.92 Mbit/s), is feasible on 1 GbE — and every TP case is feasible
-except batch=64:
-
-**27B TP batch=64 at 100 tok/s: 1310.72 Mbit/s > 1 GbE usable (1000 Mbit/s) — link-saturated on 1 GbE**
-
-**27B TP batch=64 at 100 tok/s: 1310.72 Mbit/s ≈ 2.5 GbE usable (2500 Mbit/s) — ~52% saturated**
-
-**Key finding:** PP is feasible across all ethernet classes for all batch sizes up to at least 100 tok/s. TP becomes bottlenecked on 1 GbE at batch=64 and 100 tok/s.
-
----
-
-## 3. Why PP is the Only Realistic Option over Ethernet
-
-### 3.1 Communication cost comparison
-
-| Scheme | Communication per tok per rank | 27B, H=4096, batch=1 | 36B, H=6144, batch=1 |
-|--------|------|---|---|
-| PP (Nₚₚ boundaries) | Nₚₚ × H × 2 | Nₚₚ × 8 KB | Nₚₚ × 12 KB |
-| TP (L layers all-reduce) | L × H × 2 | 32 × 8 KB = 256 KB | 48 × 12 KB = 576 KB |
-
-PP communication per token is **L/Nₚₚ times smaller** than TP communication per token (factor of ~16× for 2 PP stages).
-
-### 3.2 Latency-bound nature of autoregressive generation
-
-During autoregressive generation, each layer computes one token at a time. In TP, every layer requires an all-reduce (2× ring latency or 1× tree latency per layer). For L=32 layers, that's 32 synchronization points per token.
-
-In PP, activations cross the stage boundary only once per layer (or L/Nₚₚ times). The critical path is:
-
-```
-Stage 0 computes → send hidden state across boundary → Stage 1 computes → ...
-```
-
-This is a single bottleneck per token, with latency proportional to network RTT × number of stages, not L.
-
-### 3.3 Bandwidth utilization
-
-With PP, each boundary crossing transfers `H × 2` bytes. For 8 KB at 1 GbE, the transfer completes in ~68 µs. This is negligible compared to compute time (milliseconds per layer on a GPU). The link is underutilized.
-
-With TP, each layer transfers `H × 2` bytes but this happens L times. For 256 KB total at 1 GbE, that's ~2 ms of pure transfer time — potentially comparable to compute time on a fast GPU.
-
-### 3.4 Real-world evidence
-
-- Petals uses PP exclusively over the public internet and achieves usable inference at 1–2 tok/s even with 100 Mbit/s links [1].
-- llama.cpp RPC splits layers across nodes in pipeline order, not tensor sharding [2].
-- exo uses TP but achieves speedup only with RDMA over Thunderbolt (~50 Gbit/s) [3].
-- Petals paper Table 3: inference performance does **not depend much on bandwidth** (1 Gbit/s vs 100 Mbit/s gives nearly identical results) but **degrades significantly with higher latency** [1].
-
-**Conclusion:** For gigabit-class ethernet, PP is the only practical multi-node parallelism strategy for inference. TP can work on 10 GbE for small batches but saturates at 1 GbE with moderate batch sizes.
-
----
-
-## 4. Real-World Multi-Node Inference Benchmarks
-
-### 4.1 Summary table
-
-| System | Model | Hardware | # Nodes | Network | tok/s (batch=1) | Notes |
-|--------|-------|----------|---------|---------|-----------------|-------|
-| Petals (local) | BLOOM-176B | 3× A100 80GB | 3 | 1 Gbit/s, <5 ms RTT | 1.71 | PP, 22 stages, quantized [1] |
-| Petals (local) | BLOOM-176B | 3× A100 80GB | 3 | 100 Mbit/s, <5 ms RTT | 1.66 | ~3% slowdown vs 1 Gbit/s [1] |
-| Petals (local) | BLOOM-176B | 3× A100 80GB | 3 | 100 Mbit/s, 100 ms RTT | 1.23 | Latency matters more than BW [1] |
-| Petals (wan) | BLOOM-176B | 14 consumer GPUs | 14 | Real internet, 100–1000 Mbit/s | 0.83 | Europe+N. America [1] |
-| Petals | Llama 2 70B | consumer GPUs | various | Internet | up to 6 | Interactive chat [4] |
-| Petals | Falcon 180B | consumer GPUs | various | Internet | up to 4 | Interactive chat [4] |
-| llama.cpp RPC | Qwen3-235B | 4× M3 Ultra Mac Studio | 4 | Ethernet (2.5 GbE) | ~20 | Scales down with more nodes [3] |
-| llama.cpp RPC | DeepSeek v3.1 | 4× M3 Ultra Mac Studio | 4 | Thunderbolt 5 | ~25 | Scales down with more nodes [3] |
-| llama.cpp RPC | Kimi K2 (1T) | 4× M3 Ultra Mac Studio | 4 | Thunderbolt 5 | ~30 | A32B sparse, 32B active [3] |
-| Exo (RDMA) | Qwen3-235B | 4× M3 Ultra Mac Studio | 4 | TB5 RDMA (~50 GbE) | 32 | TP, scales UP with more nodes [3] |
-| Exo (RDMA) | DeepSeek v3.1 | 4× M3 Ultra Mac Studio | 4 | TB5 RDMA (~50 GbE) | ~30 | 671B dense TP [3] |
-| vLLM PP | Llama-3-70B | 2× A100 80GB | 2 | NVLink + Ethernet | ~15 | PP across nodes, TP within [5] |
-
-### 4.2 Detailed notes
-
-**Petals [1]:**
-- Uses pipeline parallelism with dynamic 8-bit quantization (halves bandwidth)
-- For BLOOM-176B (H=14336), per-boundary communication is ~14 KB/tok with quantization
-- Key finding: "For inference, performance does not depend much on bandwidth or sequence length but degrades with higher latency" — RTT matters more than throughput
-
-**llama.cpp RPC [2]:**
-- Model weights and KV cache distributed across nodes proportionally to memory
-- Default: pipeline-style distribution of layers (first layers on node A, last on node B)
-- Each node runs `ggml-rpc-server`, main host uses `--rpc` flag
-- RDMA transport supported (RoCEv2), negotiated automatically with libibverbs
-- Over ethernet, performance decreases with more nodes (each extra node adds a hop)
-- Over RDMA/TB5, performance scales up with more nodes (less latency per hop)
-
-**Exo [3]:**
-- Uses tensor parallelism with MLX distributed backend
-- With RDMA over Thunderbolt 5: up to 1.8× speedup on 2 devices, 3.2× on 4 devices
-- Topology-aware auto-parallelism considers latency/bandwidth between links
-- Thunderbolt 5 provides ~50–60 GbE real-world throughput (RDMA)
-
-**Key insight from all systems:** Ethernet (1 Gbit/s) is sufficient for PP inference of 20–40B models because per-token activation traffic is small (8–12 KB/boundary). The bottleneck is **latency** (RTT between nodes), not bandwidth.
-
----
-
-## 5. Overheads
-
-### 5.1 TCP/IP overhead
-
-| Overhead | Amount |
-|----------|--------|
-| Ethernet header + trailer | 18 bytes (14 + 4 FCS) |
-| IPv4 header (no options) | 20 bytes |
-| TCP header (no options) | 20 bytes |
-| Total per packet | ~58 bytes overhead |
-| MTU (jumbo frames 9000) | 9000 bytes payload → 0.6% overhead |
-| MTU (1500 standard) | 1500 bytes payload → 3.9% overhead |
-
-**Practical impact:** For 8 KB transfers (27B PP boundary), use jumbo frames where possible to minimize TCP overhead. For small transfers (< 1.5 KB), a single TCP packet is used and the overhead is 58/8192 = 0.7% per transfer — negligible.
-
-### 5.2 RPC/serialization overhead
-
-- llama.cpp RPC: Protobuf serialization adds ~10–20% overhead for small tensors
-- Petals: PyTorch tensor serialization over hivemind/gRPC, adds serialization time
-- vLLM PP: gRPC with protobuf, similar overhead
-- Serialization scales with tensor size: negligible for 8 KB+ tensors
-
-### 5.3 Pipeline bubble overhead
-
-With Nₚₚ pipeline stages and microbatching, the pipeline bubble fraction is:
-
-```
-bubble_fraction = (Nₚₚ - 1) / (Nₚₚ + microbatches - 1)
-```
-
-For 2 stages, 1 microbatch (single tok): bubble = 1/2 = 50% idle time.
-For 2 stages, 8 microbatches: bubble = 1/9 = 11%.
-
-In autoregressive generation with batch=1 and no microbatching, each stage processes one token sequentially:
-- Stage 0 finishes, sends hidden state, waits for Stage 1
-- Stage 1 processes, returns result, Stage 0 processes next token
-- Effective utilization ≈ 1/Nₚₚ if no pipelining overlap
-
-**Mitigation:** Process multiple tokens before waiting. Send next token to Stage 0 while Stage 1 is computing — pipelining reduces bubble.
-
-### 5.4 Per-hop LAN latency
-
-| Network | Typical RTT | One-way latency |
-|---------|------------|-----------------|
-| 1 GbE (Cat6) | 30–50 µs | 15–25 µs |
-| 2.5 GbE (Cat6) | 30–50 µs | 15–25 µs |
-| 10 GbE (Cat6a) | 20–30 µs | 10–15 µs |
-| RoCEv2 (IB/25GbE) | 1–5 µs | <5 µs |
-| RDMA (Thunderbolt 5) | 10–30 µs round-trip end-to-end | — |
-
-**Impact on PP inference:** With Nₚₚ = 2 stages and batch=1, one token takes:
-```
-t_compute + t_roundtrip + t_compute
-```
-
-At 30 µs RTT and ~2 ms layer compute time per GPU: the network adds ~30 µs, which is 1.5% of total. Negligible at LAN speeds.
-
-With Nₚₚ = 4 and 30 µs per-hop RTT: ~120 µs network, still <6% of compute time.
-
-### 5.5 Quantization overhead
-
-- Petals uses dynamic blockwise 8-bit quantization: halves hidden state bandwidth from 16-bit to 8-bit [1]
-- For 27B: 8 KB → 4 KB per boundary with 8-bit quantization
-- Accuracy impact: <1% on standard benchmarks [1]
-- Quantization/dequantization adds ~1–5 ms per stage on modern GPUs (negligible vs. compute)
-
----
-
-## 6. Conclusions and Recommendations
-
-### For the 27B / 36B models on gigabit-class ethernet:
-
-1. **1 GbE is sufficient for PP inference** up to at least 100 tok/s at batch=64 (81.92 Mbit/s required, well under 1000 Mbit/s capacity). Even with TCP/IP and RPC overheads, margin is >90%.
-
-2. **Pipeline parallelism is the correct choice** over ethernet — PP sends 8–12 KB per boundary vs. 256–576 KB for TP, a 32× advantage for 2 PP stages.
-
-3. **Latency matters more than bandwidth** — the Petals paper demonstrates that dropping from 1 Gbit/s to 100 Mbit/s only reduces throughput by ~3%, while increasing RTT from 5 ms to 100 ms reduces it by ~28%.
-
-4. **With 8-bit quantization**, bandwidth requirements halve, providing additional safety margin.
-
-5. **Jumbo frames recommended** to minimize TCP overhead on larger transfers.
-
-6. **For batch sizes ≥ 16**, 1 GbE remains viable for PP but consider 2.5 GbE for headroom and faster model loading.
-
----
-
-## References
-
-[1] Borzunov et al., "Petals: Collaborative Inference and Fine-tuning of Large Models," ACL 2023. arXiv:2209.01188. Available at: https://arxiv.org/pdf/2209.01188 — Primary benchmark source: Table 3 shows inference throughput at various bandwidths and latencies for BLOOM-176B. Key finding: inference insensitive to bandwidth, sensitive to latency. Recommended 25 Mbit/s bidirectional bandwidth.
-
-[2] ggml-org/llama.cpp RPC documentation. Available at: https://github.com/ggml-org/llama.cpp/blob/master/tools/rpc/README.md — Architecture for distributed LLM inference via RPC. Supports RDMA transport (RoCEv2), automatic negotiation, local tensor caching.
-
-[3] exo-explore/exo README. Available at: https://github.com/exo-explore/exo — Multi-node LLM inference on Apple Silicon with TP and RDMA over Thunderbolt 5. Benchmarks with Qwen3-235B, DeepSeek v3.1, Kimi K2 on 4× M3 Ultra Mac Studio.
-
-[4] Petals chatbot demo and health monitor. Available at: https://chat.petals.dev, https://health.petals.dev — Real-world performance of Llama 2 70B (up to 6 tok/s) and Falcon 180B (up to 4 tok/s) on public swarm.
-
-[5] vLLM pipeline parallel documentation. Available at: https://docs.vllm.ai/en/stable/dev/engine/llm_engine.html — vLLM supports PP across nodes using gRPC with CUDA-graph support for low-latency stage transitions.
-
----
-
-## Appendix: Full bandwidth formulae
-
-### Pipeline and Tensor Parallelism (generation)
-
-`N` is the batch size; `N = 1` is the batch=1 form.
-
-```
-bytes_per_boundary_per_tok = hidden_size × N × 2              [fp16, PP boundary]
-bytes_per_layer_allreduce  = hidden_size × N × 2              [TP, one all-reduce per layer]
-total_bytes_per_tok = num_boundaries × hidden_size × N × 2    [PP]
-total_bytes_per_tok = num_layers      × hidden_size × N × 2   [TP]
-bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
-
-# With 8-bit quantization (like Petals): simply divide all fp16 byte counts by 2
-bytes_per_boundary_per_tok = hidden_size × 1                  [int8]
-```

serverAppConfigs-docs/LLM/Pipeline Parallelism/Expected Performance.md

@@ -1,97 +0,0 @@
----
-type: reference
-title: Expected Performance
-description: "Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on."
-tags:
-  - pipeline-parallelism
-  - performance
-  - estimate
-timestamp: 2026-08-14
----
-
-# Expected Performance
-
-This page estimates the speed of a 25-40B model split across two machines
-over ethernet. It uses the bandwidth math in [[Ethernet Bandwidth Math]]
-and the mixed-vendor measurements in [[x4-pair-experiment]].
-
-## The Bandwidth Ceiling Is Not the Limit
-
-A pipeline boundary moves only the hidden state. For a 27B model (hidden
-5120), that is 10240 bytes per token per boundary in fp16. Even at 64
-tokens per second with batch 64, the traffic is about 262 Mbit/s. 1 GbE can
-carry it. The real limits are latency, RPC overhead, and the slowest node.
-
-The Petals paper measured this directly. Dropping from 1 Gbit/s to
-100 Mbit/s changed throughput by 3 percent. Raising round-trip latency from
-5 ms to 100 ms dropped throughput by 28 percent. Latency matters more than
-bandwidth for pipeline inference.
-
-## Real Benchmark Anchors
-
-| Experiment | Model | Network | Decode (t/s) |
-| ---------- | ----- | ------- | ------------ |
-| Mac Studio + DGX Spark (Metal+CUDA) | Qwen2.5-72B Q4 | 10 GbE | 6 |
-| Mac Studio + DGX Spark (Metal+CUDA) | Qwen2.5-7B Q4 | 10 GbE | 53 |
-| RTX 4070 + MI50 + 3050 (CUDA+ROCm) | Qwen3-VL-30B Q5 | 1 GbE | 51.5 clean, 17.8 at 32k ctx |
-| Two Strix Halos (ROCm RPC) | MiniMax-M2.5-139B Q8 | USB4 | 15.35 |
-| AMD four-node Strix Halo (5 GbE) | Kimi K2.5 (1T) | 5 GbE | 8.30 |
-| Tesla V100 + R9700x2 (Vulkan, one box) | Qwen3.6-35B-A3B Q8 | PCIe | 62 |
-
-Pattern: RPC decode runs about 2-5 times slower than the best single node.
-Prefill benefits from more compute across nodes. The 30B-class model over
-1 GbE with warm cache reached 51.5 t/s on a short context and 17.8 t/s at
-32k context.
-
-## Estimate for Qwen3.6-27B Q8, Two Nodes
-
-Scenario: RTX 3090 (24 GB VRAM, NVIDIA desktop) plus Strix Halo NUC
-(Radeon 8060S, 30 GB physical RAM). The model is the dense 27B at Q8
-(29-32 GB). Each node holds part of the weights and KV cache.
-
-| Link | Decode (t/s) | Prefill (t/s) | Notes |
-| ---- | ------------ | ------------- | ----- |
-| 1 GbE | 8-15 | 200-500 | Latency-bound; per-token RPC round trip |
-| 2.5 GbE | 10-18 | 250-600 | Some headroom; latency still dominant |
-| 10 GbE | 12-20 | 300-800 | Closest to the published 10 GbE runs |
-
-The dense 27B decodes slower per token than the MoE 30B A3B class. Expect
-the lower half of the ranges at full context.
-
-### With Speculative Decoding
-
-Speculative decoding (MTP, DFlash, or ngram-mod) raises effective decode.
-Tightwad measured 1.86x on a slow RPC pool. Estimate the effective rate at
-1.4-1.8x the raw RPC decode:
-
-| Link | Raw decode (t/s) | With spec (t/s) |
-| ---- | ---------------- | --------------- |
-| 1 GbE | 8-15 | 12-24 |
-| 2.5 GbE | 10-18 | 15-28 |
-| 10 GbE | 12-20 | 18-32 |
-
-## Context That Fits
-
-The two real machines give about 51 GiB of usable memory combined
-(24 GiB VRAM plus about 27 GiB on the Strix Halo, which has 30 GiB physical
-RAM; the 96 GiB driver ceiling is not physical). A 29-32 GB model leaves
-about 20-22 GiB for KV cache. The Qwen3.6-27B KV cache costs about 85-100
-KiB per token with q8_0 K and f16 V (measured on the large service at
-602100 tokens = about 50 GB). That budget gives a total context pool of
-roughly 200k-260k tokens across both nodes.
-
-## Caveats
-
-- Every node must run the exact same llama.cpp build. Version skew hangs
-  the handshake.
-- The slowest node sets the pace for every token.
-- Model load crosses the network once per start. Expect minutes, not
-  seconds, for the split load.
-- These are estimates. The exact combination of an RTX 3090 desktop and a
-  Strix Halo NUC has no published benchmark yet.
-
-## See Also
-
-- [[Ethernet Bandwidth Math]]
-- [[Qwen3.8 PP Deployment]]
-- [[Sample Script Set]]

serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.8 PP Deployment.md

@@ -1,313 +0,0 @@
----
-type: reference
-description: "The real split deployment: which node is head and which is worker, the flags, and the verified gotchas."
-tags:
-  - llm
-  - pipeline
-timestamp: 2026-09-03
----
-
-> **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
-> See [frank-redo-2026-08-17.md](../frank-redo-2026-08-17.md) and
-> [deployment.md](../deployment.md) for the active configuration.
->
-> **2026-10-18: nothing pipeline-parallel runs on the fleet.** No RPC listener
-> exists on either host (`ss -ltn` shows no :50052), every artifact named below
-> is gone (§Files), and the worker hardware left the box: the RTX 3090 was
-> removed from Dankserver on 2026-09-15 (it now sits in Dankclaw) and the server
-> runs 2× R9700. The live multi-GPU scheme on Dankserver is tensor parallelism
-> inside one process ([[Dual-R9700-Cooperation]],
-> [[frank-archon]]).
-
-# Qwen3.8-27B Pipeline-Parallel Deployment (Dankclaw + Dankserver)
-
-Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX
-3090 (worker) using llama.cpp RPC. This page is the design record and the
-morning-of runbook. Research background:
-[Qwen3.8 Strix optimizations](../qwen3.8-strix-optimizations.md).
-
-## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68)
-
-This split is the production `frank-dragoon` service. Current
-state, verified against the live journal and output smoke tests
-(2026-08-17, load 02:06):
-
-- Model: `Qwen3.8-27B-Uncensored-Q6_K.gguf` (22.43 GB, imatrix, no
-  bundled MTP head). Draft: `Qwen3.8-27B-Uncensored-draft-Q8_0.gguf`
-  (3.16 GB) via `--spec-draft-model`, running on ROCm0 only (1718 MiB;
-  zero worker impact — measured, not assumed).
-- Split `0.68,0.32` = 45 of 65 blocks (0-44) on the 3090 — 69% of the
-  layer weight. Maximum for this config; see "Split limits" below.
-- KV: K `q8_0` + V `f16` (mismatched), 784 MiB per FA layer: RPC0 8624
-  MiB (11 FA) + ROCm0 3920 MiB (5 FA). Flash Attention ENABLED despite
-  the mismatch (`resolve_fused_ops: Flash Attention enabled`) via the
-  GGML_CUDA_FA_ALL_QUANTS rebuild — see "FA-all-quants rebuild" below.
-- Pool `262144`, `PARALLEL=1`, live `n_ctx_slot = 262144`.
-* MTP: standalone draft head, acceptance 0.83 then 0.75, mean len 4.0-4.3,
-  per-position ~0.86/0.71/0.71/0.71 — stronger at every position than the old bundled
-  head, which measured 0.739/0.554/0.427/0.340.
-- Spec chain as deployed: `--spec-draft-n-max 4` on the standalone draft, with
-  the `--spec-default` ngram-mod fallback at n_match 24 / n_min 48 / n_max 64
-  (the preset that survived the DSpark/DFlash retirements —
-  [[qwen3.8-spec-matrix]]).
-- Batch 8192 / ubatch 256, `--direct-io`, `--fit off`, `--gpu-layers all`. The
-  pi `models.json` declared 262144 (`contextWindow = maxTokens`, the house
-  convention of that era).
-- Head GPU side: **ROCm0** (no `--device` flag; HIP registers before
-  Vulkan).
-- Worker steady state: 23854/24576 MiB used, ~273 MiB free (tight but
-  stable; all allocations are upfront).
-- Unit `frank-dragoon.service`, launched through
-  `serve-pp-head.sh` + `frank-dragoon.env`.
-
-## Service enablement (2026-08-17)
-
-The PP pair is the only inference service enabled for boot. Both units
-(`frank-dragoon` on Dankclaw, `frank-pp-worker` on Dankserver) are
-enabled. The NPU unit (`frank-zealot` on Dankclaw) and all retired
-units are disabled. Restart order: worker first, then head. The Dankserver
-helper `frank-restart-small.sh` is obsolete (it restarts the removed
-`frank-zealot` unit); restart the worker with
-`sudo systemctl restart frank-pp-worker`. The Dankclaw helper
-`frank-restart-large.sh` (sync + cache drop + restart) is correct for the
-head. The "deploy.sh dankserver is broken until retargeted" note is long fixed:
-`./scripts/deploy.sh dankserver --service frank-archon` is the documented
-invocation today (`scripts/deploy.sh:38`, dispatch case
-`dankserver:frank-archon` at `scripts/deploy.sh:149`). See
-[[deployment]] for the full boot-policy table.
-
-## FA-all-quants rebuild (2026-08-17)
-
-Flash Attention with mismatched K/V cache types requires the
-`GGML_CUDA_FA_ALL_QUANTS` compile definition. Without it,
-`ggml/src/ggml-cuda/fattn.cu` gates FA off when `K->type != V->type`
-(`#ifndef GGML_CUDA_FA_ALL_QUANTS`). With it, the K==V check is skipped
-and all quantized-K combos are compiled. The head (HIP gfx1151) and the
-worker (CUDA sm_86) compile the same sources, so the same CMake flag
-covers both:
-
-    cmake -S . -B build -DGGML_CUDA_FA_ALL_QUANTS=ON
-    cmake --build build -j --target llama-server      # dankclaw
-    cmake --build build -j --target ggml-rpc-server   # dankserver
-
-Both hosts at commit 9b05354ec (b10433). The flag was already cached
-ON in the worker build (Aug 16 objects carry it); the head required a
-full ggml-hip rebuild. Keep both binaries on the same commit — version
-skew hangs the RPC handshake.
-
-## Split limits (measured 2026-08-17, Q6_K + q8_0/f16 KV)
-
-KV per FA layer at 262144 = 784 MiB (K q8_0 1088 B/token + V f16 2048
-B/token). Layer pool 20334 MB (GGUF scan, exact), embd on the head side
-(RPC0 buffer 10429.88 MiB at f=0.53 -> the split pool excludes embd).
-
-- f=0.53 -> 35 blocks, 8 FA, KV 6272 MiB, 5721 MiB free. Phase 1.
-- f=0.68 -> 45 blocks (0-44), 11 FA (3..43), KV 8624 MiB, 273 MiB
-  free. PRODUCTION (the max): 46 blocks leaves ~0.3 GiB; 48 blocks
-  (f >= 0.7197, the round(f x 66) boundary) adds FA block 47 -> KV
-  9408 -> OOM by ~0.85 GiB.
-
-The KV-per-device count = round(f x 66) layers, independent of the
-weight boundary (same rule as the Q8_K_XL config). FA blocks at
-3,7,...,63.
-
-## Split limits (measured 2026-08-17, Q8_K_XL + f16/f16 KV) — superseded
-
-The 0.40 split (2026-08-16, 2x200700 pool) was KV-bound at 26 blocks.
-With the pool shrunk to 262144, the KV per full-attention layer drops to
-exactly 1024 MiB, freeing headroom. Measurements:
-
-- 0.50,0.50 -> 32 blocks (0-31), 8 FA on RPC0, KV 8192 MiB, 1477 MiB
-  free. Healthy (2026-08-17 00:29 load).
-- 0.55,0.45 -> 36 blocks (0-35): the 9th FA block, KV 9216 MiB ->
-  cudaMalloc OOM by ~740 MiB.
-- 0.54,0.46 -> weights 35 blocks (0-34) but KV still 9216 MiB -> OOM.
-  The KV-per-device count rounds the split fraction over 66 layers
-  (round(f x 66)), independent of the weight boundary; the 9-FA flip
-  sits at f ~= 0.538, just below the weight threshold for block 35, so
-  the 35-block config is unreachable.
-- 0.53,0.47 -> 34 blocks (0-33), 8 FA, KV 8192 MiB. Safe under every
-  rounding rule (round/floor/ceil of 0.53 x 66 = 34.98). Production.
-
-GGUF block scan (exact sizes): FA blocks at 3,7,...,63 (409.6 MB each),
-linear blocks 431.3 MB; total layer weight 27720 MB. The split pool
-includes the MTP head for weights and token_embd for KV — the two
-boundaries diverge, which is why 0.54 split them apart.
-
-## Cache incident (2026-08-17): garbage output from a full worker disk
-
-Symptom: after a split change, the service loaded "healthy" but every
-completion degenerated to a repeated junk token (e.g. 48 x "/", empty
-content, degenerate logits). Root cause chain:
-
-1. The worker RPC tensor cache keys per session. Each model load writes
-   a fresh full copy (~15 GB); over ~5 sessions of two models it grew to
-   4434 files / 96 GB.
-2. The Dankserver root disk filled to 100%. Cache writes started
-   failing, leaving 0-byte entries. A subsequent reload hit corrupt
-   cache entries and the worker served garbage weights for the
-   newly-needed layers without validation.
-3. Fix: move the cache to .AGENTTRASH (then empty it), restart the
-   worker, free the disk, reload the head (full re-stream ~2.5 min).
-   A clean cache and a healthy disk produced correct output immediately
-   (acceptance 0.89 on the first smoke test).
-
-Guard was `scripts/prune-rpc-cache.sh` on Dankserver (cron, 40 GiB cap). That
-script is gone from the repo and from the host, yet the crontab line still
-runs every 10 minutes against it
-(`*/10 * * * * /home/dank/vLLM/scripts/prune-rpc-cache.sh`, `crontab -l`, read
-2026-10-18 — the target does not exist, so it fails silently). Never let the
-cache directory grow unchecked; a full disk silently corrupts it. Signature to
-recognize: healthy load + degenerate repeated-token output.
-
-## Topology
-
-```
-Dankclaw (head, 192.168.0.143)          Dankserver (worker, 192.168.0.235)
-llama-server b10433                      ggml-rpc-server b10433
-  --rpc 192.168.0.235:50052                 --host 192.168.0.235 --port 50052
-  model file (31.5 GB)                     -d CUDA0 -c (tensor cache)
-  [RPC0, ROCm0|Vulkan0]  layers            RTX 3090: layers 0-44 (45 of 65 blocks)
-  Strix:    layers 45-64 + lm_head
-  hidden state 10 KiB/position/direction (H=5120, GGUF-verified);
-  measured wire bytes ~40 KB/position/direction (serialized graph
-  dominates); 20.5 MB per ubatch both directions. Details:
-  [[Ethernet Bandwidth Math]] section 0.
-```
-
-- **Layer-sequential split** (true pipeline parallelism): layers 0..N run on
-  the worker, the rest on the head, in order. One hidden-state handoff per
-  token step. DECODE is RTT-bound, not bandwidth-bound (network is ~1-2.5%
-  of step time). PREFILL and the prompt-cache KV snapshot ARE
-  bandwidth-bound: the link runs at line rate for the wire burst of every
-  ubatch and ~100% during the snapshot copy. Measured numbers:
-  [[Ethernet Bandwidth Math]] section 0.
-- **Why the head is on Dankclaw:** the model file is already there; the
-  worker streams its tensor share at load (no 31.5 GB copy to the server).
-- **Both sides must run the identical build** (b10433 / 9b05354ec). The
-worker build lived in a worktree `/home/dank/llama-cpp-rpc` on Dankserver so the
-production b10121 build stayed untouched (that path no longer exists, read
-2026-10-18). A version skew hangs the RPC handshake with no error.
-
-## Files
-
-| File | Machine | Purpose |
-|------|---------|---------|
-| `scripts/frank-pp-worker.env` | server | worker tunables (bind, port, device, cache) |
-| `scripts/serve-pp-worker.sh` | server | runs `ggml-rpc-server` from the b10433 worktree build |
-| `scripts/frank-dragoon.env` | claw | head tunables (split, ctx, spec, KV, I/O); was `frank-pp-head.env` before the 2026-08-16 rename |
-| `scripts/serve-pp-head.sh` | claw | runs `llama-server --rpc ...` (see "Gotchas") |
-| `scripts/frank-dragoon.service` / `frank-pp-worker.service` | both | optional systemd units (manual install); the head unit was renamed from `frank-pp-head.service` on 2026-08-16 and is symlinked from the repo into `/etc/systemd/system` |
-| `scripts/pp-test.sh` | any box with ssh to both | test orchestrator (preflight/start/verify/bench/stop-pp/restore) |
-| `scripts/pp-bench.py` | claw | API driver; reports server-side `print_timing` |
-
-
-**Every PP file in that table except `scripts/frank-dragoon.env` no longer
-exists** (checkout read 2026-10-18: `frank-pp-worker.env`,
-`serve-pp-worker.sh`, `serve-pp-head.sh`, `frank-pp-worker.service`,
-`pp-test.sh`, `pp-bench.py` and `prune-rpc-cache.sh` are all absent), and
-`scripts/frank-dragoon.env` is today the halogen-flash-server env, not a PP
-head env ([[frank-dragoon]]).
-
-## Gotchas (all verified in b10433 source / live tests, 2026-08-15)
-
-1. **No `--device` on the head.** Setting `params.devices` makes
-   `llama_prepare_model_devices` use exactly that list; the RPC device is
-   silently dropped and `--rpc` becomes a no-op. Omit it; default selection
-   puts RPC first.
-2. **The iGPU side is ROCm by default.** Backends register in fixed order
-   (HIP before Vulkan); the iGPU dedup rule keeps the first backend's view.
-   Live check showed `[RPC0, ROCm0]`. For a Vulkan side, use the second
-   head build `/home/dank/llama.cpp/build-vulkan-rpc` (HIP off; removed — the
-   path is gone on 2026-10-18) — set `HEAD_BIN` in the env. Trade: ROCm side =
-   faster prefill, ~3% slower decode (measured on this box).
-3. **Pin `--tensor-split 1,1`.** The default split is proportional to
-   *reported free memory*: the Strix reports up to 96 GiB (GTT) vs the
-   3090's 24 GB, so the default lands ~20/80 on the slow device. 1,1 = 50/50 =
-   balanced per-layer latency. The 2026-08-15 4B smoke test that proved it
-   (production untouched): at a 0.2/0.8 split only 4 of 36 layers reached the
-   3090, and at a real split worker VRAM rose ~819 MiB with 95 t/s prefill /
-   58.8 t/s decode on the tiny model; link RTT ~0.3–0.5 ms.
-4. **`-dio` on the head.** Large model + RPC hangs at load without direct
-   I/O.
-5. **RPC is unauthenticated and unencrypted.** LAN only. The worker binds
-   0.0.0.0-equivalent by design (`--host 192.168.0.235`).
-6. **Env var names.** Never `TEMP`, `HOST`, or `GPU_MAX_HEAP_SIZE`
-   (HIP runtime reads them; the 2026-08-13 `TEMP` incident).
-7. **Page-cache state confounds loads.** `pp-test.sh start` drops caches on
-   the head before load (warm-cache loads crash fast; cold loads survive).
-
-## Runbook (morning of 2026-08-16)
-
-Run from any box with ssh access to both (repo path on claw:
-`/home/dank/vllmScripts`, on server: `/home/dank/vLLM` — both still exist as
-stale checkouts; the deployment checkout today is `/home/dank/serverAppConfigs`
-on both hosts).
-
-```
-# 0. one-time: pull the kit on both boxes
-ssh dank@dankclaw   'cd /home/dank/vllmScripts  && git pull'
-ssh dank@dankserver 'cd /home/dank/vLLM && git pull'
-
-# 1. checks (read-only)
-./scripts/pp-test.sh preflight
-
-# 2. full run: stop prod -> start PP -> verify split -> bench -> restore
-FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh all
-
-# or step by step:
-FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh start
-./scripts/pp-test.sh verify          # layer split + worker VRAM
-./scripts/pp-test.sh bench 4 256 2   # ~4k prompt, 256 out, 2 reps
-./scripts/pp-test.sh bench 16 256 1
-FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh stop-pp
-FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh restore
-```
-
-Config variants: edit `scripts/frank-dragoon.env` on the claw (git commit +
-pull if the bench runs from the repo), then re-run `start` + `bench`:
-
-- No spec baseline: `SPEC_NONE=1`
-- Vulkan head side: `HEAD_BIN=/home/dank/llama.cpp/build-vulkan-rpc/bin/llama-server`
-- Q4_0 MTP draft head: add `--spec-draft-model /home/dank/models/Qwen3.8-27B-MTP-Q4_0.gguf` to the serve script's SPEC block (env hook: see `SPEC_DRAFT_MODEL` below)
-- n_max sweep: `SPEC_DRAFT_N_MAX=3|4|5`
-- Plain Q8_0: `MODEL_FILE=Qwen3.8-27B-Q8_0.gguf` (file must be on the claw)
-
-Verification checklist after each `start`:
-
-1. `verify` shows `using device RPC0 ... using device ROCm0|Vulkan0`.
-2. Layer assignment lines (`-lv 10`): worker gets the first ~32 layers at a
-   1,1 split.
-3. `nvidia-smi` on the server showed ~15–16 GB used by the worker
-   (production small service + worker share). No NVIDIA card is in that host
-   today — the equivalent check is `rocm-smi --showmeminfo vram`.
-4. Head GTT usage (rocm-smi) well under 96 GiB; model fully on devices
-   (`--gpu-layers all` offloads everything; CPU holds embeddings/lm_head glue).
-
-## Expected numbers
-
-Per-layer: 3090 ≈ 2–3× faster than the 8060S. 50/50 split → each token
-step ≈ max(32 layers on 3090, 32 layers on Strix) + ~0.5 ms RTT.
-
-- Production reference: 25 t/s @16k (Vulkan, spec, single node).
-- Target: **≥ 30 t/s @4k, ≥ 20 t/s @16k** with spec.
-- Decode floor: with ngram/mod acceptance, spec multiplies the raw rate;
-  raw (no spec) rate should clear ~15–18 t/s @16k for the target to hold.
-
-## Rollback
-
-`pp-test.sh restore` starts both production services. If a PP test wedges a
-box: `systemctl restart <unit>` after killing the stray processes
-(`pkill -x llama-server` / `pkill -x ggml-rpc-server` — never `pkill -f`
-with a pattern matching your own ssh command line).
-
-## Open questions for the test day
-
-1. Does the MTP head (which sits in the last layer group, iGPU side) draft
-   cleanly with the verify pass crossing the RPC boundary? Watch acceptance
-   in the head log (`draft acceptance = X`).
-2. Two parallel slots on the split (PARALLEL=2, CTX 65536): RPC is
-   serialized per compute — expect the 2-slot speedup to be weaker than
-   single-node. Measure, do not assume.
-3. Prefill parity: ROCm-side head should prefill ≥ single-node ROCm; if not,
-   the ubatch crossing the RPC boundary is the suspect.

serverAppConfigs-docs/LLM/Pipeline Parallelism/Sample Script Set.md

@@ -1,261 +0,0 @@
----
-type: reference
-title: Sample Script Set
-description: "An educational two-node RPC split script set, for reading only — never run it or deploy it."
-tags:
-  - pipeline-parallelism
-  - sample
-  - llamacpp
-  - educational
-timestamp: 2026-08-14
----
-
-# Sample Script Set — Two-Node Qwen3.6-27B Q8 Split
-
-> WARNING. This page is for educational reading only. Do not run these
-> scripts. Do not deploy them. They are a template, not a tested product.
-> The RPC backend is a proof of concept that the upstream README calls
-> "fragile and insecure". Running it on a production network is unsafe.
-
-## Goal
-
-Run a Qwen3.6-27B Q8 model (about 29 to 32 GB of weights) across two
-machines using llama.cpp RPC pipeline parallelism. Node A is an NVIDIA
-desktop with an RTX 3090. Node B is a Strix Halo NUC (Ryzen AI MAX+ 395,
-Radeon 8060S). Use the largest context that fits on both nodes together.
-
-## How the Split Works
-
-```mermaid
-graph LR
-    subgraph NodeA[NVIDIA Desktop — HEAD]
-        A1[llama-server<br/>--model 27B-Q8.gguf<br/>--rpc nodeB:50052]
-    end
-    subgraph NodeB[Strix Halo NUC — WORKER]
-        B1[ggml-rpc-server<br/>--host nodeB --port 50052<br/>--device CUDA0? no — ROCm]
-    end
-    A1 <-->|hidden states over ethernet| B1
-```
-
-Node A runs the head `llama-server`. Node B runs `ggml-rpc-server`, which
-exposes its GPU as a remote device. The head splits weights and KV cache
-across its own GPU and the remote device in proportion to free memory.
-Each token generation passes the hidden state once across the boundary.
-
-Key facts that shape the scripts:
-
-- Both nodes must run the exact same llama.cpp build tag. Version skew
-  hangs the handshake.
-- The head build needs its backend plus RPC. The worker build needs its
-  backend plus RPC. CUDA and ROCm do not mix inside one binary; each node
-  uses its own backend.
-- Speculative decoding runs on the head. The draft model loads on the head.
-- The model file must be present on every node with the same bytes.
-
-## File Layout
-
-```
-pipeline-pp/
-  README.md
-  build.sh                  # build llama.cpp with RPC on each node
-  workers/worker-strix.sh   # run ggml-rpc-server on the Strix Halo NUC
-  head/head-nvidia.sh       # run llama-server on the NVIDIA desktop
-  head/frank-27b-pp.env     # tunables for the head
-  head/verify.sh            # health and split check
-```
-
-## Build Script
-
-File: `build.sh`
-
-```bash
-#!/usr/bin/env bash
-# Build llama.cpp with RPC support. Run ONE build per node, with that
-# node's own backend enabled. Educational only. Do not run in production.
-set -euo pipefail
-LLAMA_DIR="$1"          # path to a llama.cpp checkout
-NODE="$2"               # "nvidia" or "strix"
-
-cmake -S "$LLAMA_DIR" -B "$LLAMA_DIR/build-pp" \
-    -DCMAKE_BUILD_TYPE=Release \
-    -DGGML_RPC=ON \
-    $( [ "$NODE" = nvidia ] && echo -DGGML_CUDA=ON \
-      || echo -DGGML_HIP=ON -DGGML_HIP_PLATFORM=amd ) \
-    -DGGML_CUDA_NO_GGVK=ON 2>/dev/null || true
-cmake --build "$LLAMA_DIR/build-pp" --config Release -j
-# Expected binaries: build-pp/bin/ggml-rpc-server, llama-server, llama-cli
-```
-
-Notes: the HEAD does not need a GPU if you offload everything to RPC
-workers. Keep the head on the NVIDIA desktop so it can also use its local
-3090 for part of the split.
-
-## Worker Script (Node B, Strix Halo NUC)
-
-File: `workers/worker-strix.sh`
-
-```bash
-#!/usr/bin/env bash
-# Expose the Strix Halo GPU as an RPC device for a remote head.
-# Educational only. Do not run on an unprotected network.
-set -euo pipefail
-BIN=/home/dank/llama.cpp/build-pp/bin
-# Bind to the LAN address. The default binds 127.0.0.1 only.
-HOST=192.168.0.143     # the Strix Halo NUC address
-PORT=50052
-# -c enables a local model-file cache so the split load runs once, not per boot.
-exec "$BIN/ggml-rpc-server" --host "$HOST" --port "$PORT" --cache
-# Use --device to name the backend device explicitly if auto-detect fails:
-#   ggml-rpc-server --host $HOST --port $PORT --device gfx1151
-```
-
-Security note: `ggml-rpc-server` exposes all memory to any client that can
-reach the port. Keep it on a trusted LAN. Do not bind it to the public
-internet.
-
-## Head Tunables (Node A, NVIDIA Desktop)
-
-File: `head/frank-27b-pp.env`
-
-```env
-# Educational tunables for the two-node split. Do not deploy.
-MODEL_DIR=/home/dank/models
-MODEL_FILE=Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf  # ~32 GB
-# MUCH smaller: use Q8_0 (29.12 GB) if the split must fit more KV.
-# Smaller still: Q6_K ~24 GB leaves several GB more room for KV.
-MODEL="$MODEL_DIR/$MODEL_FILE"
-
-# Worker (Strix Halo NUC). Multiple --rpc entries allowed; one here.
-RPC_SERVERS=192.168.0.143:50052
-
-# Context. 262144 is the native max for this model. Drop to 200000 if
-# memory pressure is too high. This is the single-slot ceiling.
-CTX_SIZE=262144
-PARALLEL=1
-
-# KV cache on GPU. q8_0 K / f16 V saves memory over f16/f16.
-CACHE_TYPE_K=q8_0
-CACHE_TYPE_V=f16
-
-# Generation batching. Decode is latency-bound over RPC; keep batch modest.
-BATCH_SIZE=1024
-UBATCH_SIZE=512
-
-# Compute. Match the physical core count on the head box.
-THREADS=7
-
-# Sampling (HauhauCS thinking-general defaults).
-TEMPERATURE=1.0
-TOP_K=20
-TOP_P=0.95
-MIN_P=0.0
-```
-
-## Head Script (Node A, NVIDIA Desktop)
-
-File: `head/head-nvidia.sh`
-
-```bash
-#!/usr/bin/env bash
-# Run the split model head on the NVIDIA desktop.
-# Educational only. Do not run in production.
-set -euo pipefail
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-# shellcheck disable=SC1091
-source "$SCRIPT_DIR/frank-27b-pp.env"
-
-BIN=/home/dank/llama.cpp/build-pp/bin
-PORT=7770
-
-RPC_ARGS=()
-for s in $RPC_SERVERS; do RPC_ARGS+=(--rpc "$s"); done
-
-# Pick ONE spec variant by uncommenting the matching pair below.
-
-# --- Variant 1: MTP (recommended by research) ---------------------------
-# SPEC="--spec-type draft-mtp --spec-draft-model $MODEL_DIR/Qwen3.6-27B-MTP-ONLY-Q4_K_M.gguf --spec-draft-n-max 3"
-
-# --- Variant 2: DFlash (block-diffusion draft) ---------------------------
-# SPEC="--spec-type draft-dflash --spec-draft-model $MODEL_DIR/Qwen3.6-27B-DFlash-Q8_0.gguf --spec-draft-n-max 8"
-
-# --- Variant 3: ngram-mod (built-in, no draft file) ----------------------
-# SPEC="--spec-type ngram-mod --spec-ngram-mod-n-min 48 --spec-ngram-mod-n-max 64 --spec-ngram-mod-n-match 24"
-
-# --- Variant 4: no speculation --------------------------------------------
-SPEC="--spec-type none"
-
-exec "$BIN/llama-server" \
-    --model "$MODEL" \
-    "${RPC_ARGS[@]}" \
-    ${SPEC} \
-    --ctx-size "$CTX_SIZE" \
-    --parallel "$PARALLEL" \
-    --cache-type-k "$CACHE_TYPE_K" \
-    --cache-type-v "$CACHE_TYPE_V" \
-    --batch-size "$BATCH_SIZE" \
-    --ubatch-size "$UBATCH_SIZE" \
-    --threads "$THREADS" \
-    --host 0.0.0.0 --port "$PORT" \
-    --alias frank-27b-pp
-```
-
-### Spec Variant Summary
-
-| Variant | draft model file | --spec-type | Notes |
-| ------- | ---------------- | ----------- | ----- |
-| MTP | `Qwen3.6-27B-MTP-ONLY-Q4_K_M.gguf` (~3B) | `draft-mtp` | Parallel-safe, best in benchmarks |
-| DFlash | `Qwen3.6-27B-DFlash-Q8_0.gguf` | `draft-dflash` | Known premature-EOS bug |
-| ngram-mod | none (built-in) | `ngram-mod` | Zero VRAM cost |
-| DSpark | vLLM only | (not in llama.cpp) | Use vLLM PP instead; see below |
-
-DSpark does not exist in llama.cpp. It is a vLLM sparse-attention draft. If
-you want DSpark, use vLLM pipeline parallelism, which has its own limits
-(no spec decode with PP as of 0.15.x). The llama.cpp path here covers
-MTP, DFlash, and ngram-mod.
-
-## Verify (Node A)
-
-File: `head/verify.sh`
-
-```bash
-#!/usr/bin/env bash
-# Check the split model is up and that both nodes are in the split.
-set -euo pipefail
-PORT=7770
-# Health must return "ok" after model load (minutes, not seconds).
-curl -sf "http://127.0.0.1:$PORT/health"
-echo
-# Show the device split from the server log.
-curl -sf "http://127.0.0.1:$PORT/props" | head -40
-# Expected: the model and KV cache distributed across CUDA0 (node A)
-# and the remote RPC device (node B).
-```
-
-## Context Sizing Math
-
-The model is about 29 to 32 GB. The two machines give about 51 GiB of
-usable memory. That leaves about 20 to 22 GiB for KV cache. The
-Qwen3.6-27B KV cache costs about 85 to 100 KiB per token at q8_0 K /
-f16 V. The budget therefore gives a pool of roughly 200k to 260k tokens.
-A single 262144-token slot is the practical ceiling for this split.
-
-To run more context, shrink the model quantization. Q6_K (about 24 GB)
-frees several GB for KV. Do not try to run the 96 GiB driver figure on the
-Strix Halo as real memory. It is an overcommit ceiling; the physical RAM
-is 30 GiB.
-
-## What This Does Not Cover
-
-- The remote draft-model-over-RPC path is not confirmed. The draft stays
-  on the head.
-- `--tensor-split` is available to force a manual layer split. The sample
-  uses the automatic proportional split.
-- RDMA (RoCEv2) transport exists but needs a compatible NIC. It adds a few
-  percent over TCP.
-
-## See Also
-
-- [[Qwen3.8 PP Deployment]]
-- [[Expected Performance]]
-- [[Ethernet Bandwidth Math]]
-- [[DFlash DSpark Speculative Decoding]]

serverAppConfigs-docs/LLM/Pipeline Parallelism/index.md

@@ -1,71 +0,0 @@
-# Pipeline Parallelism
-
-This section covers running one large model across two machines. It focuses
-on pipeline parallelism (layer split) over ethernet. The research answers
-what speeds are possible and which tools work. The sample scripts show a
-two-node Qwen3.6-27B Q8 split for reading only.
-
-## Terminology
-
-Pipeline parallelism (PP) splits model layers across machines. Each machine
-computes its layers and passes hidden states to the next machine. Tensor
-parallelism (TP) splits each layer across machines and requires an
-all-reduce per layer. PP is the only realistic option over ethernet.
-
-Speculative decoding is a separate feature. It uses a small draft model (or
-n-grams) to propose tokens that the main model verifies in one pass. It can
-run together with PP in llama.cpp.
-
-## The Two Runtimes
-
-```mermaid
-graph LR
-    subgraph llamaRPC[llama.cpp RPC]
-        A1[ggml-rpc-server on node B] --> A2[llama-server on node A<br/>--rpc nodeB:50052]
-    end
-    subgraph vllmPP[vLLM PP]
-        B1[vllm serve node 0<br/>tp=1 pp=2 rank 0] <-->|hidden states| B2[vllm serve node 1<br/>tp=1 pp=2 rank 1]
-    end
-```
-
-  heterogeneous nodes over ethernet.
-  multiprocessing and does not compose with speculative decoding yet.
-
-## What the Research Found
-
-- [[Ethernet Bandwidth Math]] — Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it.
-  (about 10 KiB per token for a 27B model). 1 GbE supports hundreds of
-  tokens per second on paper; the real limit is latency and RPC overhead.
-  works today via llama.cpp RPC with separate CUDA and ROCm builds. The
-  exact NVIDIA-desktop + Strix-Halo-NUC pair has not been benchmarked with
-  published numbers yet.
-  methods (MTP, DFlash, ngram-mod) work with RPC. DSpark is vLLM-only.
-- [[Expected Performance]] — Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on.
-  across an RTX 3090 and a Strix Halo NUC over 1/2.5/10 GbE.
-
-## Deployment
-
-- [[Qwen3.8 PP Deployment]] — The real split deployment: which node is head and which is worker, the flags, and the verified gotchas.
-  (Strix), worker on Dankserver (RTX 3090), b10433 on both sides, scripts in
-  `scripts/`, morning-of test runbook. Verified gotchas: no `--device` with
-  `--rpc`, default split is proportional to reported free memory (pin 1,1),
-  the iGPU side defaults to the ROCm backend.
-
-## Sample Scripts
-
-- [[Sample Script Set]] — An educational two-node RPC split script set, for reading only — never run it or deploy it.
-  Qwen3.6-27B-Q8_K_P with MTP, DFlash, and ngram-mod variants, sized for
-  the largest context that fits. For educational reading only. Do not run
-  it in production.
-
-## See Also
-
-- [[DFlash DSpark Speculative Decoding]]
-- [[frank-zealot]]
-- [[Dankserver System Specs]]
-- [[Dankclaw System Specs]]
-
-## Pages
-
-| Page | Owns |
-|---|---|

serverAppConfigs-docs/LLM/Quantization Production.md

@@ -83,18 +83,18 @@ Laws that steer everything here:
 ### Ablation / abliteration (directional ablation)
 - Arditi et al. 2024, arXiv 2406.11717: refusal is mediated by a single
   direction; orthogonalizing weights against it removes refusals with a
-  rank-1 edit. Open implementations: `p-e-w/heretic` (automated, above),
+  rank-1 edit. Open implementations: `p-e-w/heretic` (automated, above)
   Artio/abliteration-toolkits, `EleutherAI/repeng`-style rep-overlap
   variants **[verified (tool existence: heretic; rest survey)]**.
 - Hardware-agnostic fp-editing. On a MoE the edit touches attention/norm
   weights, so it composes with any expert quantization.
 - Fleet status: frank-dragoon runs an abliterated base **by Frank's choice**
-  (AGENTS: censored-base refusals were the accepted trade of the 27B era);
+  (AGENTS: censored-base refusals were the accepted trade of the 27B era)
   frank-archon's ukisai Swift1.5 is an abliterated finetune of Qwen3.8.
 
 ### HauhauCS QX_K_P "Perfect" quants
 - Our frank-zealot publisher. Method, from the publisher's own cards
-  **[verified]**: (1) abliterate + finetune ("Aggressive"/"Balanced"),
+  **[verified]**: (1) abliterate + finetune ("Aggressive"/"Balanced")
   (2) quantize with **imatrix calibration on the ABLITERATED weights**,
   (3) K_P = a **per-model, per-tensor mixed-precision profile** —
   "model-specific analysis to selectively preserve quality where it
@@ -204,8 +204,8 @@ u2 draft — all recipe-expressible (see the w4dn8 runbook).
 
 #### Recipe portability: the w4dn8 shape onto other MoEs (2026-10-08 audit)
 The pin 1.0.13 container ships SIX arch plugins
-(`/opt/radiance/share/radiance/architectures/`): `qwen4exp_fp8`,
-**`qwen35moe_fp8`**, `qwen35_fp8`, `qwen35_bf16`, `llama_fp8`,
+(`/opt/radiance/share/radiance/architectures/`): `qwen4exp_fp8`
+**`qwen35moe_fp8`**, `qwen35_fp8`, `qwen35_bf16`, `llama_fp8`
 `llama_dense_fp8` (census via `podman run --rm --entrypoint sh <image> -c`
 — the image has no `ls`/`strings`; grep the `.so` files directly, and
 the image lives in ROOTLESS storage as `docker.io/stilldeadcode/radiance`).
@@ -214,7 +214,7 @@ the image lives in ROOTLESS storage as `docker.io/stilldeadcode/radiance`).
 calibration→GPTQ path exists for the qwen3.5-MoE family; GPTQ itself
 lives in `rad-convert`, not the plugin (0 hits in both `.so`).
 
-Worked example — `Qwen/Qwen3.6-35B-A3B` (config: `qwen3_5_moe_text`,
+Worked example — `Qwen/Qwen3.6-35B-A3B` (config: `qwen3_5_moe_text`
 40 layers, `full_attention_interval=4` DeltaNet hybrid, **256 experts
 top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
 248320, multimodal wrapper with vision tower):
@@ -258,13 +258,13 @@ top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
 - EXL3: github.com/turboderp-org/exllamav3 (QTIP lineage; CUDA; Ampere
   decode incl. 3090 documented in v1.0 performance write-ups).
 - Ampere capability edges: vLLM FP8 docs ("only Hopper and Ada …
-  W8A8; Ampere supported for W8A16 weight-only utilizing Marlin"),
-  vLLM Supported-Hardware matrix (Marlin GPTQ/AWQ on SM 8.0/8.6),
+  W8A8; Ampere supported for W8A16 weight-only utilizing Marlin")
+  vLLM Supported-Hardware matrix (Marlin GPTQ/AWQ on SM 8.0/8.6)
   TensorRT-LLM Support Matrix (FP8 and INT4-AWQ-with-FP8-acts need
   SM≥89; NVFP4 Blackwell), SGLang issue #12887 (Ampere FP8-via-Marlin).
-- Repo: `scripts/convert-swift15-w4dn8.sh`,
-  `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`,
-  `scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`,
+- Repo: `scripts/convert-swift15-w4dn8.sh`
+  `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`
+  `scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`
   `scripts/build-kairic-edge-gfx1151.sh`; endpoints [[frank-archon]],
-  [[frank-dragoon]], [[frank-zealot]]; method [[qwen3.8-strix-optimizations]]
+  , ; method
   and [[GGUF Compatibility]].

serverAppConfigs-docs/LLM/Quantization by Hardware Architecture.md

@@ -132,15 +132,12 @@ quantizer exists for it ([[Quantization Production]]).
 |---------|--------|-----|
 | 1 | `.hgn` (deployed) | the halogen engine's own format; production door is closed |
 | 2 | GGUF K/I quants, K_P blends | the only self-produced option on this card ([[Quantization Production]]) |
-| 3 | INT4 W4A16 (Quark) / MXFP4 (Quark) | the retired Ling-3.0-flash vLLM era, measured under the firmware-VGM frame buffer: [[ling-3.0-flash-trial]] |
+| 3 | INT4 W4A16 (Quark) / MXFP4 (Quark) | the retired Ling-3.0-flash vLLM era, measured under the firmware-VGM frame buffer:  |
 | 4 | FP8 (vLLM) | software emulation; memory savings only; no compute win |
 
 ## See Also
 
 - [[DFlash DSpark Speculative Decoding]]
-- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
-- [[Running Qwen3.6 27B FP8 on Dankclaw]]
 - [[Dankclaw System Specs]]
 - [[Dankserver System Specs]]
-- [[VLLM Overview]]
 - [[Shared Memory Architecture Dankclaw]]

serverAppConfigs-docs/LLM/RAM CPU Offloading in VLLM.md

@@ -1,206 +0,0 @@
----
-type: reference
-description: "vLLM uses CPU RAM and swap space to extend GPU memory capacity."
-tags:
-  - llm
-  - vllm
-timestamp: 2026-08-28
----
-
-# RAM CPU Offloading in vLLM
-
-## Overview
-
-Stock vLLM offloads **KV pages**, not weights: the scheduler moves KV cache between GPU
-and CPU. That is the upstream behaviour and it is NOT the whole story on our hosts — the
-tcclaviger/R4D fork ran a real **weight** offload (`--enable-expert-offload`, tiered
-expert placement, a pinned host pool per rank), which is how a 147 GiB MoE served from a
-246 GiB box: [[RDNA4 MoE Expert Offload VRAM Budget]] and the tcclaviger era row of
-[[frank-archon]]. First-hand in-repo evidence outranks a generic upstream claim about our
-own hosts ([[Model Swap and Rollback]] §10).
-
-> **Verified 2026-10-19 against `vllm/config/cache.py` on `main`:** there is no
-> `swap_space` field — `--swap-space` is gone from current engine args. CPU KV offload is
-> `--kv-offloading-size`, and it activates **only** together with `--kv-offloading-backend`
-> (default `native`); both fields live in that one file. This page is the fleet's owner of
-> that knob contract — other pages link here instead of restating it.
-
-## How CPU Offloading Works
-
-VLLM manages KV cache as pages. Each page holds attention key and value tensors for a set of tokens. When GPU memory is full, VLLM can move pages to CPU memory or disk swap space.
-
-The KV cache swap mechanism works as follows:
-
-1. VLLM allocates GPU blocks for active KV cache pages
-2. When GPU blocks are exhausted, VLLM swaps inactive pages to CPU
-3. Swapped pages are stored in a CPU memory buffer
-4. When a swapped page is needed, VLLM loads it back to GPU
-5. If CPU buffer is full, pages are written to disk swap space
-
-## Key Parameters
-
-### gpu_memory_utilization
-
-This parameter controls the fraction of GPU memory used for KV cache.
-
-| Value | Description |
-|-------|-------------|
-| 0.7 | Uses 70% of GPU memory for KV cache |
-| 0.8 | Uses 80% of GPU memory for KV cache |
-| 0.92 | Uses 92% of GPU memory for KV cache (default) |
-| 0.95 | Uses 95% of GPU memory for KV cache |
-
-Lower values leave more GPU memory for model weights. Higher values allow more concurrent requests but increase swap frequency.
-
-### swap_space (removed) / kv-offloading-size + kv-offloading-backend (current)
-
-`--swap-space` no longer exists in current engine args (verified above); the live pair is
-`--kv-offloading-size` with `--kv-offloading-backend`, and offload stays off until the size
-is set. The table below is the historical `--swap-space` sizing guidance, kept as a sizing
-reference only.
-
-| Value | Description |
-|-------|-------------|
-| 0 | No CPU swap space |
-| 4 | 4 GiB swap space (historical default) |
-| 8 | 8 GiB swap space |
-| 16 | 16 GiB swap space |
-| 32 | 32 GiB swap space |
-| 64 | 64 GiB swap space |
-
-The swap space acts as an extension of GPU memory. When GPU blocks are full, VLLM moves KV cache pages to swap space.
-
-### max_model_len
-
-This parameter sets the maximum context length.
-
-| Value | Description |
-|-------|-------------|
-| auto | VLLM calculates optimal length |
-| 2048 | 2048 tokens |
-| 4096 | 4096 tokens |
-| 8192 | 8192 tokens |
-| 32768 | 32768 tokens |
-
-Smaller values reduce memory usage but limit context length.
-
-### max_num_batched_tokens
-
-This parameter sets the maximum number of tokens per batch.
-
-| Value | Description |
-|-------|-------------|
-| 2048 | Default for online serving |
-| 4096 | Moderate batch size |
-| 8192 | Large batch size |
-| 16384 | Very large batch size |
-
-### max_num_seqs
-
-This parameter sets the maximum number of sequences per batch.
-
-| Value | Description |
-|-------|-------------|
-| 128 | Default for online serving |
-| 256 | Large batch |
-
-### kv_cache_dtype
-
-This parameter sets the KV cache data type.
-
-| Value | Description |
-|-------|-------------|
-| auto | Uses model default dtype |
-| fp8 | Uses FP8 for KV cache (reduces memory by 50%) |
-| fp8_e5m2 | Uses FP8 E5M2 format for KV cache |
-
-## Memory Configuration Examples
-
-All three examples carry the retired `--swap-space` syntax and are kept as sizing
-references; the live knob pair is named above.
-
-### Small Model on Single GPU
-
-```
-vllm serve Qwen/Qwen2.5-1.5B-Instruct \
-    --gpu-memory-utilization 0.9 \
-    --swap-space 4 \
-    --max-num-batched-tokens 4096 \
-    --max-num-seqs 128
-```
-
-### Large Model with CPU Offload
-
-```
-vllm serve meta-llama/Llama-3.1-70B-Instruct \
-    --tensor-parallel-size 4 \
-    --gpu-memory-utilization 0.8 \
-    --swap-space 16 \
-    --max-model-len 8192
-```
-
-### Memory-Constrained GPU
-
-```
-vllm serve meta-llama/Llama-3.1-70B-Instruct \
-    --tensor-parallel-size 4 \
-    --gpu-memory-utilization 0.7 \
-    --swap-space 32 \
-    --max-model-len 4096 \
-    --max-num-batched-tokens 2048
-```
-
-## Preemption Modes
-
-VLLM uses two preemption modes when GPU memory is insufficient:
-
-### Recompute Mode (Default in V1)
-
-VLLM recomputes the prefill for preempted requests. This uses GPU compute instead of memory. This is the default mode in VLLM V1.
-
-### Swap Mode
-
-VLLM swaps KV cache pages to CPU and reloads them later. This uses CPU memory instead of compute.
-
-## Monitoring
-
-Watch for preemption warnings:
-
-```
-WARNING scheduler.py: Sequence group 0 is preempted by PreemptionMode.RECOMPUTE mode because there is not enough KV cache space.
-```
-
-If you see frequent preemptions:
-
-1. Increase `gpu_memory_utilization`
-2. Raise the KV offload budget (`--kv-offloading-size`, with `--kv-offloading-backend`)
-3. Decrease `max_num_seqs`
-4. Increase `tensor_parallel_size`
-5. Increase `max_model_len` to match actual usage
-
-## Memory Conservation Tips
-
-- Use quantization (FP8, INT8) to reduce model size
-- Reduce `max_model_len` to match actual needs
-- Reduce `max_num_seqs` to limit concurrent requests
-- Disable CUDA graphs with `--enforce-eager` to save memory
-- Set `compilation_config.cudagraph_capture_sizes` to smaller values
-- Set `--kv-cache-memory-bytes` to pin the KV budget instead of letting the profiler guess
-  (the flag our own `scripts/patches/gpu_worker-kv-cap.patch` reads)
-- Limit multi-modal inputs with `limit_mm_per_prompt`
-
-## Environment Variables
-
-| Variable | Description | Default |
-|----------|-------------|---------|
-| vLLM_CPU_KVCACHE_SPACE | KV cache space in GiB for CPU backend | 4 |
-| vLLM_CPU_OMP_THREADS_BIND | CPU thread binding for OpenMP | auto |
-| vLLM_CPU_NUM_OF_RESERVED_CPU | Reserved CPU cores per rank | 0-1 |
-| vLLM_CACHE_ROOT | torch.compile cache directory | ~/.cache/vllm |
-
-## See Also
-
-- [[VLLM Overview]]
-- [[VLLM Index]]
-- [[Hybrid CPU GPU Processing Dankserver]]
-- [[Shared Memory Architecture Dankclaw]]

serverAppConfigs-docs/LLM/RDNA4 Container Serving.md

@@ -1,258 +0,0 @@
----
-type: reference
-title: RDNA4 Container Serving
-description: "Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage."
-tags: [rdna4, gfx1201, r9700, vllm, expert-offload, podman, rocm]
-timestamp: 2026-10-05T00:00:00-07:00
----
-
-# RDNA4 Container Serving
-
-Bring-up path for Dankserver (2x Radeon AI PRO R9700, gfx1201 — card count,
-per-card VRAM and host RAM are [[Dankserver System Specs]]'s) with
-`docker.io/tcclaviger/vllm` (vLLM 0.29.0.dev, GLM/Qwen-specialised RDNA4 build
-with native MoE quant kernels and a first-class expert offload planner). The
-tuning history and current deployed shape live in
-[[frank-archon]]; this page is the bring-up law.
-
-## 0. Know the box — and its phantom third GPU
-
-`torch.cuda.device_count()` returns **3**; summing `total_memory` overstates
-the box as 186.7 GiB. The third device is the 7700X's integrated Vega
-(gfx1036): torch reports ~123 GiB for it, `rocm-smi` reports 0.50 GiB,
-allocating on it moves system `MemAvailable` 1:1 (it is GTT over DDR5, not
-dedicated VRAM), and it dies with `hipErrorInvalidKernelFile` when the image
-ships no code for that arch. **Never sum device `total_memory`; never count
-the iGPU as headroom or a workload.**
-
-## 1. Preflight: is the quant class EXECUTABLE, not merely present?
-
-Expert offload is mandatory when weights exceed VRAM, and it requires the
-checkpoint's experts class to have BOTH a compiled kernel AND an
-`offload_recipe` ("a pinned expert has no host copy, so there is no
-read-through fallback" — startup aborts otherwise). Check BEFORE downloading
-anything:
-
-```bash
-podman run --rm --device /dev/kfd --device /dev/dri --device /dev/mem \
-  -e HIP_VISIBLE_DEVICES=0,1 --entrypoint bash $IMAGE -c '
-  nm -D /app/r4dhip/r4d.so | grep -oE "gemm_moe_[a-z0-9_]+" | sort -u
-  grep -rl offload_recipe /opt/vllm/lib/python*/site-packages/vllm/model_executor/layers/fused_moe/'
-```
-
-| quant | kernel in `r4d.so` | experts class | verdict |
-|---|---|---|---|
-| FP8 | yes | `Fp8HipMoEExperts` | **serves** |
-| MXFP4 / DSFP4 | yes (`gemm_moe_mxfp4a8_*` / `dsfp4a8_*`) | `R4dMxfp4MoEExperts` / `R4dDsfp4MoEExperts` | **serves** |
-| W4A16 | **no** `gemm_moe_w4a16_*` | `R4dW4a16MoEExperts` exists | silently falls back to generic `TritonWNA16Experts`, no recipe → startup halts |
-| RFA | none | no class at all | out |
-
-Also confirm the architecture routes natively with NO overlay
-(`ModelRegistry.get_supported_archs()` for `Qwen4Exp`/`Glm5`). The stock image
-natively carries Qwen3.8/Qwen4Exp and GLM-5.x including MTP heads. **Do not
-apply vendor overlays** against a fast-moving pre-release image — the breakage
-misleads (a missing symbol in the overlay's OWN copy of a module masks the
-real cause, then moves to another symbol each patch;
-`cannot import name 'checkpoint_has_lm_head'` = overlay clobbered
-`transformers_utils/config.py`). The image ships quantizers for
-`rfa|rfi|mxfp4|mxfp4ocp|paro5` under `/app/tools/quantize.py`
-(`--fp8-attn --quant-only --dry-run` first) if a re-quant is needed — note it
-can run MXFP4/DSFP4/FP8 but only PRODUCE rfa/rfi/mxfp4/paro.
-
-## 2. Fetch with verification, always
-
-Gated repos: token never in argv (`pgrep -a curl` prints command lines).
-Build a mode-600 curl config once:
-
-```bash
-{ printf 'header = "Authorization: Bearer '; cat ~/.cache/huggingface/token; printf '"\n'; } \
-  > ~/.cache/huggingface/curl.conf && chmod 600 ~/.cache/huggingface/curl.conf
-curl -K ~/.cache/huggingface/curl.conf -f -sS -L -C - -o "$out" "$url"
-```
-
-`--fail` is MANDATORY — `curl -o FILE URL` exits 0 on HTTP 401/403 and writes
-the error body as a "successful artifact" (a gated repo once yielded three
-153-byte shards with identical sha256s and a cheerful DONE banner). `-C -`
-resumes torn shards; HTTP/2 cancels leave silently truncated files, so
-re-verify EVERY shard's byte count against the Hub manifest (and the LFS
-sha256) before serving. The AGENTS.md Secret Rules carry the token-hygiene
-law; [[frank-medium-model-swap]] has the same detail for GGUFs and
-`.partNofM` fragments.
-
-## 3. Serve — the known-good invocation and its traps
-
-```bash
-sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'
-podman run -d --name <ctr> --ipc=host --ulimit memlock=-1:-1 \
-  --cap-add SYS_PTRACE \
-  --device /dev/kfd --device /dev/dri --device /dev/mem \
-  -e HIP_VISIBLE_DEVICES=0,1 -e VLLM_ROCM_USE_AITER=0 \
-  -v /home/dank/models/<model>:/models:ro \
-  $IMAGE /models --served-model-name <name> --port 7770 \
-  --tensor-parallel-size 2 --max-model-len 65536 \
-  --kv-cache-memory-bytes 2147483648 \
-  --max-num-seqs 4 --gpu-memory-utilization 0.95 \
-  --enable-prefix-caching --trust-remote-code
-```
-
-- **Entrypoint already execs `vllm serve "$@"`** — pass the model path as the
-  first argument and do NOT pass `serve` (podman then tries to pull `serve`
-  as an image name). `-e VAR=1` must come BEFORE the image name.
-- **Mount the whole `/dev/dri`, never individual render nodes** (§4).
-- `--ipc=host` and `--shm-size` are mutually exclusive in podman.
-- `podman run -d` without `-a` sends container stdout nowhere — read
-  `podman logs <ctr>`; and vLLM writes `Application startup complete` to
-  **stderr**, so a `podman logs X 2>/dev/null | grep -q` readiness gate never
-  fires.
-- Start at 65536, not the native ceiling; prove the plan, then scale.
-- Load takes ~5 min with `/health` refusing the whole time — do not kill the
-  process (deploy.sh waits 20 min).
-
-## 4. Container GPU-init triage (ordered)
-
-1. **Enumerate-vs-context trap (most common):** individual
-   `renderD*` nodes give HIP the render nodes WITHOUT the primary `card*`
-   nodes — it enumerates agents (`device_count()` returns N) and then refuses
-   every context: `RuntimeError: No CUDA GPUs are available` on the first
-   real op. A/B-confirmed 2026-09-19. Rule: `--device /dev/kfd --device
-   /dev/dri --device /dev/mem`; restrict GPUs with `HIP_VISIBLE_DEVICES`,
-   never with device nodes.
-2. **`device_count()` is not evidence.** Prove usability with a real op
-   (`(a@a).sum().item()` on a fresh `torch.randn`), then repeat via
-   `mp.spawn`, `mp.fork`, and `subprocess.run([sys.executable, ...])`, and
-   cross-check `rocm-smi --showmeminfo vram`.
-3. **memlock rlimit → a MISLEADING CUDA OOM.** Host default 8192 KB; a pin
-   larger than the limit fails as
-   `torch.AcceleratorError: CUDA error: out of memory` raised from
-   `pin_memory()` — it names the accelerator but the fault is host pinned
-   memory. It cannot be raised from inside a container or even from root's
-   own shell (`Operation not permitted`); only PAM applies it at login, so
-   fixes only affect NEW sessions:
-   `/etc/security/limits.d/99-memlock.conf` with
-   `* - memlock unlimited` + `root - memlock unlimited`; the container needs
-   `--ulimit memlock=-1:-1` to inherit. Discriminating test after the fix:
-   the container reports `unlimited` AND small pins succeed; if pins still
-   fail, memlock was never the cause.
-4. **`/proc/meminfo` TAB trap:** the key/value separator is a TAB, so
-   `grep MemAvailable /proc/meminfo | tr -s " " | cut -d" " -f2` returns
-   EMPTY and a following awk prints `0.0` — a false "host out of memory" on a
-   host with hundreds of GiB free, which invites aborting something healthy.
-   Tab-safe: `awk -v k='MemAvailable:' '$1==k {printf "%.1f", $2/1048576;
-   exit}' /proc/meminfo`, cross-check `<= MemTotal`, refuse to print a zero.
-   Use `scripts/dankstatus.sh` instead of hand-rolling.
-5. **Silent-parse traps:** `grep -c PATTERN file || echo 0` yields TWO lines
-   (grep prints 0 AND exits 1) — `[: 0\n0: integer expected`; use
-   `| head -1` or `|| true`. `systemctl is-enabled` can emit multiple lines.
-   `rocm-smi --showmeminfo vram --csv` column order is `device, VRAM Total
-   Memory (B), VRAM Total Used Memory (B)` — reading $3 as total/$4 as used
-   yields 0.00; filter to `^card[0-9]+$`.
-6. **Guard the box:** long loads with co-tenants can drive RAM to 0 — abort
-   before the OOM killer (`MemAvailable < 4 GiB → podman rm -f`, exit 1).
-7. **Drop page cache before a large load** (`echo 3 > /proc/sys/vm/drop_caches`)
-   and confirm the drop was actually in effect for the run before treating a
-   failure as a capacity wall — a stale "it doesn't fit" from a contaminated
-   run is expensive to unwind.
-
-## 5. Memory budgeting: the levers are not where they look
-
-Every figure in this section is a tcclaviger-era boot on the port that is
-[[frank-archon]]'s today; the laws they teach are general. The planner builds
-`budget = total × util`, subtracts a KV reserve, and gives
-the REMAINDER to the expert arena. Consequences, all measured:
-
-- **`--gpu-memory-utilization` grows the ARENA, not the KV pool** (KV stayed
-  0.64 GiB at util 0.95 and 0.98). The arena is the only elastic term.
-- **Lowering `--max-model-len` is the WRONG lever and backfires:** a smaller
-  context shrinks the reserve, handing the space to the arena — measured 65536
-  → 0.64 GiB KV, 32768 → 0.34 GiB KV.
-- **Hybrid-attention undercount:** the planner's per-token KV figure for
-  Qwen4Exp is an undercount — the printed figure and the true one are both on
-  [[frank-archon]] — because it misses the large fixed per-sequence Mamba state
-  (~2x), so the reserve is always short and the arena silently covers it.
-  Fix by pinning the pool: `--kv-cache-memory-bytes <BYTES>` (era reading: a
-  2 GiB pin = 113,595 tokens ≈ 3.47x concurrency at 32k — that pair is this
-  page's record of the tcclaviger boots, not [[frank-archon]]'s). For long
-  context,
-  `--kv-offloading-size` (GiB) spills KV to host.
-- **`--expert-offload-mem` caps the HOST side, not the VRAM arena**
-  (`cap = int(x * GIB)` against `host_memory_available()`). Do not reach for
-  it to fix a VRAM problem; to grow the arena, shrink the KV pin or raise
-  util (see [[frank-archon]] for the
-  util=1.0 unbootability law).
-
-**Read the plan — it is the real acceptance test:**
-
-```
-expert offload: AUTO-ENABLED, weights 62.54 GiB per rank exceed the 30.27 GiB budget (TP2)
-  budget (total x util)      30.27 GiB
-  non-expert weights          6.28 GiB
-  KV reserve                  0.36 GiB   <- sanity-check against the model's own per-token figure
-  CUDA graphs                 0.66 GiB
-  runtime overhead            7.60 GiB
-  expert cache space         15.36 GiB   6712 slots of 2.344 MiB
-  host-backed experts/rank   52.74 GiB   23040 experts
-  host: 105.48 GiB across 2 ranks of 160.89 GiB usable
-```
-
-Host fits when `host_total < usable`. The PLE n-gram table rides memfd and is
-SHARED across TP ranks.
-
-**Loading mechanism** (for reasoning about peak RAM): weights never materialise
-in VRAM — expert Parameters start as CPU staging tensors; each layer's
-`weight_loader` counts shard events and, on layer completion,
-`ExpertTopology.split_layer` moves that layer's experts into the VRAM arena
-and compacts the rest into exact-size pinned host tensors behind UVA. Peak
-host RAM = `host-backed set + layers still loading`, not the whole model. At
-runtime `apply_hot_cold` sends arena-resident experts down a hot GEMM and the
-rest straight over PCIe via UVA views — no staging copy in the critical path.
-
-**Emulation check:** a dequantize-to-compute path doubles real host RAM — the
-measured pair (offload request vs host footprint) is on [[frank-archon]] — and
-the doubling is invisible in the weight count. Read the log for `Using
-'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit".
-
-## 6. Performance profile and honest measurement
-
-Decode on this pair is **PCIe-gather-bound, context-independent**: the rate is
-flat from a few hundred to sixteen thousand context tokens, so KV attention is
-not the cost (the flat figure and its MTP/gather deltas are
-[[Tcclaviger Fork Ops]]'s profile of the same shape). With ~29 % of experts
-arena-resident, under a gigabyte crosses PCIe per token at the engine's
-effective read rate; cold-vs-warm differing ~10 % confirms the LRU works;
-growth is linear in hit rate, and only speculative decode cuts sequential STEPS
-rather than making each cheaper.
-
-**Benchmark traps that produced fake numbers here:**
-
-- `max_tokens=1` is not TTFT (the server can satisfy it without doing the
-  work — prefill looked flat at 1.26 s for 16k AND 30k prompts). A 64-token
-  non-streamed reply is time-to-64-tokens (decode-bound, flat ~9.6 s at every
-  length). Correct TTFT: stream and time the FIRST content-bearing chunk.
-- A small repeating word loop is prefix-cacheable AND n-gram-predictable — a
-  43-word cycle reported 22,897 tok/s prefill, almost entirely cache hit. Use
-  a large vocabulary in shuffled order and print `cached_tokens`.
-- ~1.2 s fixed per-request overhead swamps prefill below ~16k tokens; compare
-  prefill at 30k+ only.
-- Size prompts with `/v1/messages/count_tokens` (a chat call for counting
-  pollutes the metric); guessing words-per-token silently overran the window
-  (HTTP 400).
-- Always run a canary: plant a unique token at the END of a long prompt and
-  ask the model to quote it — proves prompts reach the model at all.
-- Measure warm AND long (`podman exec` + urllib at 127.0.0.1:7770): a 33-token
-  reply read 14.6 cold / 20.3 warm; a 768-token generation hit 25.7; two
-  concurrent streams scaled to 47 aggregate.
-
-**Levers (all opt-in):** `--expert-prefetch N` takes an INT depth, not a
-boolean (default 0; at depth 1 only ~10 % cold-vs-warm difference — the cache
-already works); `CLAV_TUNABLEOP_SWEEP=1` autotunes GEMM shapes (silently
-skipped under expert offload unless `CLAV_TUNABLEOP_WITH_OFFLOAD=1` too —
-see [[frank-archon]] for why it was rejected);
-`--expert-precise-memory` swaps the fitted overhead figure in.
-
-## Related
-
-- [[frank-archon]] — deployed shape, plan law,
-  measured campaigns
-- [[PCIe Link and Atomics]] — why decode is gather-bound and what cannot fix it
-- [[RDNA4 MoE Expert Offload VRAM Budget]] — arena/plan arithmetic deep-dive
-- [[frank-dragoon]] and [[frank-zealot]] — window discipline on live hosts

serverAppConfigs-docs/LLM/RDNA4 MoE Expert Offload VRAM Budget.md

@@ -14,7 +14,6 @@ figure and rate below is from [[frank-archon]]'s tcclaviger era (2026-09-29 →
 to host RAM) — that page owns the endpoint's shape, tuning history and era
 record; this page owns only the plan arithmetic, the ladder method and the audit
 procedure. Boot-failure capture lives in
-[[vLLM Boot Forensics and Config Interrogation]].
 
 ## 1. The plan law
 
@@ -164,7 +163,7 @@ spec maximum, never hand arithmetic.
    `Tried to allocate 10.28 GiB … 9.46 GiB free` died in KV ALLOCATION.
 6. **Clean between rungs:** stop the transient unit, `podman rm -f`, kill
    orphans by explicit PID
-   (`ps aux --sort=-rss | awk '/vllm\/bin\/python3/ && $6 > 50000000 {print $2}'`),
+   (`ps aux --sort=-rss | awk '/vllm\/bin\/python3/ && $6 > 50000000 {print $2}'`)
    wait for `MemAvailable` to recover.
 
 **The hazard that cost production:** a leftover ladder script whose teardown
@@ -215,7 +214,7 @@ at graph capture; 6 seqs fp8 → 10.70 GiB ask with 9.18 GiB free (fails) —
 
 ## Related
 
-- [[RDNA4 Container Serving]] — the bring-up path and container traps
-- [[vLLM Boot Forensics and Config Interrogation]] — capture and contracts
+-  the bring-up path and container traps
+-  capture and contracts
 - [[frank-archon]] — measured history
 - [[Llama Cpp Serving Sizing]] — the llama.cpp equivalent

serverAppConfigs-docs/LLM/Running Qwen3.6 27B FP8 on Dankclaw.md

@@ -1,246 +0,0 @@
----
-type: reference
-description: "Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate."
-tags:
-  - llm
-timestamp: 2026-09-03
----
-
-# Running Qwen3.6 27B FP8 on Dankclaw
-> **RETIRED - vLLM era (before the 2026-08-17 redo).** Nothing on this page serves
-> today: no vLLM or ROCm process runs on Dankclaw, and the iGPU endpoint at
-> `192.168.0.143:7770` is [[frank-dragoon]] (halogen flash-server). The page stays as
-> the era record of the Qwen3.6-27B-FP8 vLLM bring-up and the Strix Halo ROCm
-> recipes ([[frank-redo-2026-08-17]]). The hardware table below carries the era's
-> host figures; live ones are [[Dankclaw System Specs]].
-
-## Model Information
-
-Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. It uses 27B parameters with gated DeltaNet and attention layers.
-
-| Property | Value |
-|----------|-------|
-| HuggingFace ID | `Qwen/Qwen3.6-27B-FP8` |
-| Base Model | `Qwen/Qwen3.6-27B` |
-| Architecture | Qwen3_5ForConditionalGeneration |
-| Total Parameters | 27B |
-| Hidden Size | 5120 |
-| Layers | 64 (16 blocks of 3 DeltaNet + 1 Attention) |
-| Attention Heads | 24 Q, 4 KV (head dim 256) |
-| DeltaNet Heads | 48 V, 16 QK (head dim 128) |
-| Intermediate Dim | 17,408 |
-| Vocab Size | 248,320 (padded) |
-| Context Length | 262,144 (natively extensible to 1,010,000) |
-| Quantization | FP8 (block size 128) |
-| Vision Encoder | Yes (image-text-to-text) |
-
-## Hardware: Dankclaw
-*Era figures (2026-08). The host has changed since - [[Dankclaw System Specs]].*
-
-| Property | Value |
-|----------|-------|
-| CPU | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S (Strix Halo) |
-| GPU | Integrated Radeon 8060S (unified memory) |
-| Physical RAM | 64 GiB (61 GiB visible; corrected 2026-08-17) |
-| GPU-Visible (ROCm) | 96 GiB (overcommitted) |
-| Swap | 8 GiB |
-| ROCm | 7.2.4 |
-| Python | 3.12.13 (vLLM venv) |
-
-## Memory Calculation
-
-| Component | BF16 | FP8 |
-|-----------|------|-----|
-| Model weights (27B) | 54 GB | 13.5 GB |
-| Vision encoder | ~1 GB | ~0.5 GB |
-| **Total weights** | ~55 GB | ~14 GB |
-| KV cache (64K ctx, 1 seq) | ~1 GB | ~0.5 GB |
-| Overhead | ~2 GB | ~2 GB |
-| **Total (approximate)** | ~58 GB | ~16.5 GB |
-
-The FP8 variant requires approximately 16-17 GB. ThinkingCap-FP8 requires ~29 GB. Both fit within the 61 GiB physical RAM, but ROCm reports 96 GiB to vLLM. Set `--gpu-memory-utilization` conservatively (0.80) to avoid OOM.
-
-## Prerequisites
-
-> **Era note (2026-08, superseded twice):** at writing time Dankclaw's primary
-> service was the vLLM-ROCm Ling-3.0-flash CIRU INT4 trial ([[ling-3.0-flash-trial]],
-> retired 2026-08-17), and this page documented the alternative vLLM FP8 deployment
-> of Qwen3.6-27B. Both paths are dead; `scripts/frank-dragoon.service` today is the
-> halogen flash-server unit ([[frank-dragoon]]).
-
-Install vLLM for ROCm. Version 0.24.0+ is required for Qwen3.6 with speculative decoding:
-
-```
-uv venv --python 3.12 --seed
-source .venv/bin/activate
-uv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/
-```
-
-Installed: vLLM 0.26.0 in Python 3.12.13 venv at `/home/dank/vllm-rocm-venv`.
-
-### MPI Dependency
-
-Torch requires `libmpi_cxx.so.40`. If missing, install `openmpi-bin`:
-
-```
-sudo apt install -y openmpi-bin
-```
-
-### Environment Variables
-
-Set these for ROCm on Strix Halo:
-
-```
-export HSA_OVERRIDE_GFX_VERSION=11.5.1
-export HF_HOME=/home/dank/.cache/huggingface
-```
-
-## vLLM Serve Commands
-
-### Text-Only (Recommended for Memory Savings)
-
-Skip the vision encoder to reduce memory:
-
-```
-vllm serve Qwen/Qwen3.6-27B-FP8 \
-    --port 8000 \
-    --kv-cache-dtype fp8 \
-    --gpu-memory-utilization 0.85 \
-    --swap-space 8 \
-    --max-model-len 131072 \
-    --max-num-batched-tokens 2048 \
-    --max-num-seqs 32 \
-    --reasoning-parser qwen3 \
-    --language-model-only \
-    --dtype auto
-```
-
-### With Vision Encoder
-
-```
-vllm serve Qwen/Qwen3.6-27B-FP8 \
-    --port 8000 \
-    --kv-cache-dtype fp8 \
-    --gpu-memory-utilization 0.85 \
-    --swap-space 8 \
-    --max-model-len 131072 \
-    --max-num-batched-tokens 2048 \
-    --max-num-seqs 32 \
-    --reasoning-parser qwen3 \
-    --dtype auto
-```
-
-### Parameters Explained
-
-| Parameter | Value | Reason |
-|-----------|-------|--------|
-| --kv-cache-dtype fp8 | FP8 KV cache | Cuts KV cache memory by 50% |
-| --gpu-memory-utilization 0.85 | 85% of unified memory | Leaves 15% for CPU/OS |
-| --swap-space 8 | 8 GB CPU swap | Matches system swap size |
-| --max-model-len 131072 | 128K context | Full 262K is large but possible with 61 GB |
-| --language-model-only | Skip vision | Saves ~1 GB for KV cache |
-| --reasoning-parser qwen3 | Qwen3 parser | Required for thinking mode |
-
-## Sampling Parameters
-
-| Mode | Temperature | Top P | Top K | Min P | Presence Penalty |
-|------|-------------|-------|-------|-------|-----------------|
-| Thinking (general) | 1.0 | 0.95 | 20 | 0.0 | 0.0 |
-| Thinking (coding) | 0.6 | 0.95 | 20 | 0.0 | 0.0 |
-| Non-thinking | 0.7 | 0.80 | 20 | 0.0 | 1.5 |
-
-Set `max_tokens` to 32768 for general use. Set to 81920 for complex reasoning.
-
-## API Examples
-
-### Chat with Thinking
-
-```
-curl http://localhost:8000/v1/chat/completions \
-    -H "Content-Type: application/json" \
-    -d '{
-        "model": "Qwen/Qwen3.6-27B-FP8",
-        "messages": [{"role": "user", "content": "Solve: 2+2"}],
-        "max_tokens": 32768,
-        "temperature": 1.0,
-        "top_p": 0.95,
-        "top_k": 20
-    }'
-```
-
-### Non-Thinking Mode
-
-```
-curl http://localhost:8000/v1/chat/completions \
-    -H "Content-Type: application/json" \
-    -d '{
-        "model": "Qwen/Qwen3.6-27B-FP8",
-        "messages": [{"role": "user", "content": "Hello"}],
-        "max_tokens": 2048,
-        "temperature": 0.7,
-        "top_p": 0.8,
-        "top_k": 20,
-        "presence_penalty": 1.5,
-        "extra_body": {
-            "chat_template_kwargs": {"enable_thinking": false}
-        }
-    }'
-```
-
-### Tool Calling
-
-```
-vllm serve Qwen/Qwen3.6-27B-FP8 \
-    --port 8000 \
-    --reasoning-parser qwen3 \
-    --enable-auto-tool-choice \
-    --tool-call-parser qwen3_coder
-```
-
-## Ultra-Long Context (YaRN)
-
-Extend context beyond 262,144 tokens:
-
-```
-VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve Qwen/Qwen3.6-27B-FP8 \
-    --hf-overrides '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}' \
-    --max-model-len 524288
-```
-
-## ROCm Notes
-
-- Strix Halo (gfx1151) is listed as a supported AMD GPU for vLLM
-- FP8 uses **software emulation** on RDNA architecture (gfx1151 has no native FP8 tensor cores). This gives memory savings, not full compute speedup. Compare against a BF16 test first
-- Attention backends: try `ROCM_AITER_UNIFIED_ATTN` if the default backend underperforms
-- Context length on ROCm may be limited for some models. Start with 131072 and increase gradually
-
-## Troubleshooting
-
-### Out of Memory
-
-1. Use `--language-model-only` to skip vision encoder
-2. Reduce `--gpu-memory-utilization` to 0.75
-3. Reduce `--max-model-len` to 65536
-4. Reduce `--max-num-batched-tokens` to 1024
-
-### FP8 Errors
-
-1. Try BF16 first: `vllm serve Qwen/Qwen3.6-27B --dtype bfloat16`
-2. Verify ROCm version supports FP8
-3. Set `HSA_OVERRIDE_GFX_VERSION=11.5.1`
-4. Use `--kv-cache-dtype auto` instead of fp8
-
-### Slow Inference
-
-1. Check GPU utilization with `rocm-smi`
-2. Verify attention backend is optimal
-3. Increase `--max-num-batched-tokens` if memory allows
-4. Ensure CPU cores are not throttled
-
-## See Also
-
-- [[VLLM Overview]]
-- [[Dankclaw System Specs]]
-- [[Shared Memory Architecture Dankclaw]]
-- [[RAM CPU Offloading in VLLM]]
-- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]

serverAppConfigs-docs/LLM/Running Qwen3.6 35B A3B FP8 on Dankserver.md

@@ -1,210 +0,0 @@
----
-type: reference
-description: "Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape."
-tags:
-  - llm
-timestamp: 2026-09-03
----
-
-# Running Qwen3.6 35B A3B FP8 on Dankserver
-
-> **RETIRED.** This page documents the FP8 vLLM path for Qwen3.6-35B-A3B on Dankserver. The llama.cpp 35B service is retired (2026-08-16), its unit file was removed (2026-08-17), and the FP8 vLLM unit (`vllm-qwen36`) never shipped and is disabled. The content is kept as research history. The RTX 3090 now runs the PP worker: see [[Qwen3.8 PP Deployment]].
-
-## Model Information
-
-Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series. It uses 35B total parameters with only 3B activated per token.
-
-| Property | Value |
-|----------|-------|
-| HuggingFace ID | `Qwen/Qwen3.6-35B-A3B-FP8` |
-| Base Model | `Qwen/Qwen3.6-35B-A3B` |
-| Architecture | Qwen3_5MoeForConditionalGeneration |
-| Total Parameters | 35B |
-| Activated Parameters | 3B |
-| Experts | 256 (8 routed + 1 shared per token) |
-| Layers | 40 (10 blocks of 3 DeltaNet + 1 Attention) |
-| Attention Heads | 16 Q, 2 KV (head dim 256) |
-| DeltaNet Heads | 32 V, 16 QK (head dim 128) |
-| Expert Intermediate Dim | 512 |
-| Vocab Size | 248,320 (padded) |
-| Context Length | 262,144 (natively extensible to 1,010,000) |
-| Quantization | FP8 (block size 128) |
-| Vision Encoder | Yes (image-text-to-text) |
-
-## Hardware: Dankserver
-
-| Property | Value |
-|----------|-------|
-| GPU | NVIDIA GeForce RTX 3090 |
-| VRAM | 24 GB |
-| System RAM | 62 GB |
-| Swap | 127 GB |
-
-## Memory Calculation
-
-| Component | BF16 | FP8 |
-|-----------|------|-----|
-| Activated weights (3B) | 6 GB | 1.5 GB |
-| Inactive experts (32B) | 64 GB | 16 GB |
-| Vision encoder | ~1 GB | ~0.5 GB |
-| **Total weights** | ~71 GB | ~18 GB |
-
-The FP8 variant requires approximately 18 GB for weights. This leaves approximately 6 GB of the RTX 3090's 24 GB VRAM for KV cache and overhead. Use aggressive memory settings.
-
-## Prerequisites
-
-Install vLLM. Version 0.19.0 or later is required for Qwen3.6:
-
-```
-uv pip install vllm --torch-backend=auto
-```
-
-## Hardware Quantization Notes (RTX 3090 / Ampere)
-
-The RTX 3090 is an Ampere GPU (compute capability 8.6). It has **no native FP8 tensor cores**. FP8 W8A8 compute runs through a fallback/dequantization path, so the FP8 model gives memory savings but not full compute speedup. vLLM loads these models and runs them; verify quality against a BF16 run. `--kv-cache-dtype fp8` is still valid on CUDA 12 for an FP8 KV cache.
-
-> Do not run on production without a BF16 comparison. The FP8 precision path may differ from native-FP8 hardware.
-
-## vLLM Serve Commands
-
-### Text-Only (Recommended for Memory Savings)
-
-Skip the vision encoder to free VRAM for KV cache:
-
-```
-vllm serve Qwen/Qwen3.6-35B-A3B-FP8 \
-    --port 8000 \
-    --kv-cache-dtype fp8 \
-    --gpu-memory-utilization 0.75 \
-    --swap-space 32 \
-    --max-model-len 131072 \
-    --max-num-batched-tokens 2048 \
-    --max-num-seqs 32 \
-    --reasoning-parser qwen3 \
-    --dtype auto
-```
-
-### With Vision Encoder
-
-If you need multimodal input, expect higher VRAM usage:
-
-```
-vllm serve Qwen/Qwen3.6-35B-A3B-FP8 \
-    --port 8000 \
-    --kv-cache-dtype fp8 \
-    --gpu-memory-utilization 0.7 \
-    --swap-space 32 \
-    --max-model-len 131072 \
-    --max-num-batched-tokens 1024 \
-    --max-num-seqs 16 \
-    --reasoning-parser qwen3 \
-    --dtype auto
-```
-
-### Parameters Explained
-
-| Parameter | Value | Reason |
-|-----------|-------|--------|
-| --kv-cache-dtype fp8 | FP8 KV cache | Cuts KV cache memory by 50% |
-| --gpu-memory-utilization 0.75 | 75% of VRAM | Leaves room for all MoE weights |
-| --swap-space 32 (removed) | 32 GB CPU overflow | Modern vLLM removed swap-space; use --kv-offloading-size |
-| --max-model-len 131072 | 128K context | Full 262K context is too large for 24 GB |
-| --max-num-batched-tokens 2048 | Conservative batch | Prevents OOM with MoE |
-| --reasoning-parser qwen3 | Qwen3 parser | Required for thinking mode output |
-
-### Model Storage
-
-Root disk has only 20 GB free. Store models on /mnt/entropy:
-
-```
-export HF_HOME=/mnt/entropy/huggingface
-```
-
-### Active Service Conflict
-
-The retired `frank-zealot` service (llama.cpp) used about 22 GB of the 24 GB VRAM. The current PP worker holds ~23.8 GiB. The `vllm-awq-qwen36` (Podman) and `vllm-qwen36` units are disabled. Before starting any research service, verify that the GPU is free:
-
-```
-nvidia-smi  # verify GPU is free
-```
-
-## Sampling Parameters
-
-| Mode | Temperature | Top P | Top K | Min P | Presence Penalty |
-|------|-------------|-------|-------|-------|-----------------|
-| Thinking (general) | 1.0 | 0.95 | 20 | 0.0 | 0.0 |
-| Thinking (coding) | 0.6 | 0.95 | 20 | 0.0 | 0.0 |
-| Non-thinking | 0.7 | 0.80 | 20 | 0.0 | 1.5 |
-
-Set `max_tokens` to 32768 for general use. Set to 81920 for complex reasoning.
-
-## API Examples
-
-### Chat with Thinking
-
-```
-curl http://localhost:8000/v1/chat/completions \
-    -H "Content-Type: application/json" \
-    -d '{
-        "model": "Qwen/Qwen3.6-35B-A3B-FP8",
-        "messages": [{"role": "user", "content": "Solve: 2+2"}],
-        "max_tokens": 32768,
-        "temperature": 1.0,
-        "top_p": 0.95,
-        "top_k": 20
-    }'
-```
-
-### Non-Thinking Mode
-
-```
-curl http://localhost:8000/v1/chat/completions \
-    -H "Content-Type: application/json" \
-    -d '{
-        "model": "Qwen/Qwen3.6-35B-A3B-FP8",
-        "messages": [{"role": "user", "content": "Hello"}],
-        "max_tokens": 2048,
-        "temperature": 0.7,
-        "top_p": 0.8,
-        "top_k": 20,
-        "presence_penalty": 1.5,
-        "extra_body": {
-            "chat_template_kwargs": {"enable_thinking": false}
-        }
-    }'
-```
-
-## Ultra-Long Context (YaRN)
-
-Extend context beyond 262,144 tokens. Use this environment variable and override:
-
-```
-VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve Qwen/Qwen3.6-35B-A3B-FP8 \
-    --hf-overrides '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}' \
-    --max-model-len 524288
-```
-
-## Troubleshooting
-
-### Out of Memory
-
-1. Kill llama-server to free 20 GB VRAM
-2. Use `--language-model-only` to skip vision encoder
-3. Reduce `--max-model-len` to 65536
-4. Reduce `--max-num-batched-tokens` to 512
-5. Reduce `--max-num-seqs` to 8
-
-### Slow Inference
-
-1. Verify GPU is exclusive to vLLM
-2. Check temperature with `nvidia-smi`
-3. Ensure `--kv-cache-dtype fp8` is set
-4. Increase `--max-num-batched-tokens` if memory allows
-
-## See Also
-
-- [[VLLM Overview]]
-- [[Dankserver System Specs]]
-- [[Hybrid CPU GPU Processing Dankserver]]
-- [[RAM CPU Offloading in VLLM]]
-- [[Running Qwen3.6 27B FP8 on Dankclaw]]

serverAppConfigs-docs/LLM/Shared Memory Architecture Dankclaw.md

@@ -85,7 +85,7 @@ Available backends for ROCm:
 
 All seven names exist on current vLLM `main` (`AttentionBackendEnum` in
 `vllm/v1/attention/backends/registry.py` —
-[source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/v1/attention/backends/registry.py),
+[source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/v1/attention/backends/registry.py)
 read 2026-10-18), and `--attention-backend`, `--numa-bind`, `--numa-bind-nodes`
 are real `vllm serve` flags (`vllm/engine/arg_utils.py:1098,1171-1176`).
 
@@ -103,7 +103,7 @@ vllm serve model \
 
 - `gpu_memory_utilization`: Set lower (0.8-0.85) to leave room for CPU operations
 - `max_model_len`: Match actual context needs
-- `--swap-space` is retired from current vLLM; the CPU KV-offload knob pair, with the upstream `CacheConfig` verification, is owned by [[RAM CPU Offloading in VLLM]]
+- `--swap-space` is retired from current vLLM; the CPU KV-offload knob pair, with the upstream `CacheConfig` verification, is owned by
 
 ### NUMA Binding
 
@@ -213,7 +213,7 @@ Unified memory limits concurrent requests:
 
 The two "No" rows (Async Output, Multi-step Decoding) were transcribed from a
 vLLM feature-support snapshot and not re-checked against current upstream
-*(unverified 2026-10-18)*.
+- (unverified 2026-10-18)*.
 
 ### Context Length
 
@@ -226,7 +226,7 @@ FP8 quantization depends on GPU architecture:
 - MI300 series supports native FP8
 - Radeon RX 7900 supports FP8 via software emulation
 - Strix Halo (gfx1151) uses **software emulation** for FP8 matmuls — RDNA 3.5 has
-  no FP8 tensor instructions ([FP8-ROCM](https://github.com/wabibito/FP8-ROCM));
+  no FP8 tensor instructions ([FP8-ROCM](https://github.com/wabibito/FP8-ROCM))
   expect memory savings, not full compute speedup. (Conflicting vendor-side
   evidence exists for an FP8/INT8 WMMA path used by FSR4 on RDNA 3.5, so treat
   the matmul claim as the settled half.)
@@ -294,8 +294,3 @@ rocm-smi --showtemperature
 
 ## See Also
 
-- [[VLLM Overview]]
-- [[VLLM Index]]
-- [[RAM CPU Offloading in VLLM]]
-- [[Hybrid CPU GPU Processing Dankserver]]
-- [[Running Qwen3.6 27B FP8 on Dankclaw]]

serverAppConfigs-docs/LLM/Tcclaviger Fork Ops.md

@@ -1,202 +0,0 @@
----
-type: reference
-title: Tcclaviger Fork Ops
-description: "Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence."
-tags: [tcclaviger, vllm, dankserver, expert-offload, sweeps, cache]
-timestamp: 2026-10-05T00:00:00-07:00
----
-
-# Tcclaviger Fork Ops
-
-The tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`,
-vLLM 0.29.0.dev, GLM-5.3/Qwen-specialised with native MoE quant kernels and a
-first-class expert-offload planner) served `frank-archon` at shape G until
-the 2026-10-05 FreeToken cutover; it remains the rollback target
-(`git tag frank-archon-vllm-shapeG-20261003`). Full plan law and VRAM
-arithmetic: [[RDNA4 MoE Expert Offload VRAM Budget]]; boot forensics:
-[[vLLM Boot Forensics and Config Interrogation]]; window discipline:
-[[LLM Bench Windows and Measurement]].
-
-## The ad-hoc sweep harness (dankserver)
-
-Serve script `/home/dank/serve_qwen_fp8.sh` with env hooks: `SPEC` (MTP
-depth), `TUNABLE` (TunableOp), `PREFETCH`, `UTIL`, `MAXLEN`, `KV_BYTES`,
-`MNBT`, `UVA_TOK`, `GATHER_C/GATHER_L` (R4D_LRU gather), `TELEM`
-(R4D_LRU_TELEMETRY), `PROFDIR` (torch profiler bind-mount + --profiler-config
-JSON). The script re-copies the harness (`mtp_measure.py`, `bench_iter.py`)
-into every fresh container — `podman cp` files die on container recreate.
-
-- Measure ONLY via usage counters: `mtp_measure.py` (3 depths) or
-  `mtp_measure_short.py` (166 ctx). `decode = completion_tokens/(wall −
-  ttft)`; chunk counting is INVALID under spec decode.
-- Jobs: `bash /home/dank/job.sh start NAME CMD…` / `wait NAME SECS 40` /
-  `out NAME` ([[Remote Job Runner]]).
-- **Validity check on ANY speed result:** `completion_tokens == max_tokens`
-  (else repetition-halt or early EOS), finish_reason, and inspect the text —
-  `bench_iter` prompts are repeated word-lists + `!` padding, loop-prone. A
-  loop makes LRU hits ~100% → PCIe traffic vanishes → fake 3–4× decode.
-  Acceptance swings 48–61% run-to-run — noise; decode is the metric.
-- Each arm = full reload: ~8.5 min cold, ~12 min with measurement.
-- Readiness gate: `podman logs <c> 2>&1` (stdout+stderr merged) for
-  `Application startup complete` — it is on STDERR; an unmerged grep can
-  never match and burns the whole wait timeout.
-- GPU util ~0% during load/compile is NORMAL (weights + CUDA-graph capture
-  are CPU-side); VRAM climbing is the load indicator.
-- Config edit discipline: the serve script is patched by scp'd python
-  scripts with asserts (anchor must exist, addition must not); `bash -n`
-  after every patch.
-
-## Measured knob verdicts (2026-09-29 campaign)
-
-Incumbent: `SPEC=4 PREFETCH=1 UTIL=0.98 MAXLEN=32768 KV_BYTES=2147483648
-TUNABLE=0`, stock gather 8×16 → 26.80/25.15/26.81 tok/s @ 166/1087/4064 ctx.
-
-| Knob | Verdict | Evidence |
-|---|---|---|
-| Gather geometry `R4D_LRU_GATHER_{CHUNKS,LANES}` | **RULED OUT** | 16×32 worse (~24); 32×64 tie (26.6). The 9.5 vs 22.7 GB/s read-path gap is per-transfer structure (~105 µs/slab vs 78 ideal), not a tunable knob |
-| MTP depth | **RULED OUT** | SPEC=3 ~9% worse; SPEC=6 collapses (13.5–19 tok/s, acceptance 17.4%) — also proves UVA read-through (crosses at step tokens > `R4D_LRU_UVA_TOKENS=5`) is SLOWER than copy-in |
-| TunableOp | **REJECTED** | `CLAV_TUNABLEOP_SWEEP=1` silently SKIPPED under expert offload unless `CLAV_TUNABLEOP_WITH_OFFLOAD=1`; adds ~7 min/load, persists nothing (container-local), and CORRUPTS NUMERICS (loops, repetition-halt, fake 79.8 tok/s @ 96.5% acceptance on degenerate completions). Any "3× win" with early-EOS + absurd acceptance = the loop artifact |
-| MNBT | neutral at 8192 vs 2048 | small values re-chunk prefill only (hurts TTFT, never decode — decode step = max-num-seqs × (spec+1) tokens). No ubatch concept in vLLM |
-
-Decode on this stack is PCIe-gather-bound (~30% GPU duty cycle), so
-batch-shaped compute knobs move ~nothing.
-Measured with no spec: **12.11 tok/s FLAT from 304 to 16,095 context tokens** — KV
-attention is not the cost, so a context-slope hunt on this stack is null by construction.
-
-The `R4D_*/CLAV_*` knob family (QSA_*, MHC, ROUTE, ACT_QUANT, AR_QUANT,
-SILU_QUANT) are kill-switches defaulting ON, not perf dials; real levers are
-the R4D_LRU_* family, TunableOp (rejected), R4D_LRU_TELEMETRY, and spec
-depth.
-
-Telemetry gotchas: `--profiler-config` takes a JSON blob, not separate
-flags; `torch_profiler_with_stack` defaults TRUE — a full measure inside a
-profile window OOM-kills the engine worker (~40 GB trace buffer; kernel
-kills python3 at ~95 GB anon-rss). Keep windows to ~15 s / one depth
-(`mtp_measure_short.py`). `telemetry_report` (`expert_cache_rank<N>.json`)
-fires ONLY from the worker's profiler-stop path; worker-side stop can hang
->5 min on ROCm/kineto. No alternative trigger exists. The LRU hit rate
-measured 82.45% — NOT the ~27% the arena-fraction arithmetic implies;
-temporal reuse dominates, so never infer miss rates from resident-slot
-fractions.
-
-## Sweep hygiene (fork-specific preflight)
-
-1. **Knob landed:** for each `-e` env knob, confirm in-container
-   `podman exec … env` (or the engine log's confirmed-values line). A knob
-   the fork silently ignores produces a "parity" arm that is really the
-   default — the v1 gather sweep lost all three arms to exactly this
-   (GATHER_C/GATHER_L silently dropped, all arms ran stock 8×16).
-2. **Prefetch uniformity:** every arm in a sweep MUST carry the same
-   expert-prefetch setting. The 2026-09-29 campaign measured losing arms
-   with prefetch ON and incumbents OFF; untaxed, its verdicts inverted
-   (no-spec 12.11→41.47; gather 32×64 parity→+18.7%). Never re-read a taxed
-   verdict.
-3. **Spec arms must prove engagement:** engine log
-   `SpecDecoding metrics: Mean acceptance length …` and
-   `curl :7770/metrics | grep vllm:spec_decode_`. Zero such lines = MTP off.
-4. Boot-fail fast-detection: poll `podman ps` INSIDE the health loop; a
-   refused boot dies in one poll instead of 20 min of health waits. Capture
-   >8 lines of `podman logs` on failure — the root cause (e.g. a SEQS init
-   refusal) lives higher up.
-5. Report per-depth tables, never one number: g32×64's win grows with depth
-   (54@1k, 39.5@short) — a single 4064 median hides the short-ctx cost. TTFT
-   rows are noisy ±0.15 s and non-monotonic across depths; compare same-ctx
-   rows across arms only.
-6. Generate ≥256 tokens per sample (at 8 tokens inter_token is first-token
-   noise).
-
-Boot-log facts survive measure failures: KV pool size (`GPU KV cache size`),
-arena slots (`expert cache: … -> N slots`), and the spec-reservation ladder
-(MTP3 766,666 / MTP4 729,545 / MTP6 665,263 tokens) print at BOOT — a
-measure-stage bug costs decode tables but not boot-log findings.
-
-**MTP depth-brittleness law (2026-10-02, all arms rc=0):** MTP3 on this fork
-is depth-brittle — 47.01 t/s at 166 tokens (47.88 at temp 0.7) but 23.09 at
-4064, vs no-spec baseline 37.02 / 33.99. MTP3 + SEQS 16 + MNBT 4096 + util
-0.95 + ladder [4..32] = best overall 37.04 / 38.61 t0.7 and keeps 33.15 at
-4064. `--expert-precise-memory` is load-bearing for MTP (same arm with it
-OFF = worst arm, 19.81 at 4064). Every spec arm costs KV (398,658 → 313,002
-tokens). Acceptance is real: mean length 2.4–2.8, per-position
-0.76/0.55/0.33, avg 52–57%. Next lever named by the engine: MNBT 4096 is
-exactly SEQS×(n+1) (`suboptimal` warning) → try 8192.
-
-## Serving a model larger than VRAM (dankserver recipe)
-
-Dankserver's card count, per-card VRAM and host RAM are in [[Dankserver System Specs]];
-`torch.cuda.device_count()` returns 3 on this box and summing `total_memory` overstates it
-badly (the Raphael iGPU is in the list — reading discipline: [[Llama Cpp Serving Sizing]]).
-
-Everything else about running this image — the quant-kernel gate that decides
-which checkpoints can serve at all, the entrypoint contract, the known-good
-invocation, the `/dev/dri` and memlock traps, the util and max-model-len arena
-traps, the `--ipc=host` / `-d` stdout traps and the gated-fetch discipline — is
-written once on [[RDNA4 Container Serving]], and the plan arithmetic behind the
-arena/KV traps is [[RDNA4 MoE Expert Offload VRAM Budget]]'s. Three findings
-stay here because this harness produced them:
-
-- **The KV reserve undercounts hybrid-attention models ~2×** (the planner's
-  B/token figure against ~18,000 real: the Mamba state is a fixed per-sequence
-  cost). Fix by pinning the pool: `--kv-cache-memory-bytes`.
-- **`--expert-offload-mem` caps the HOST side**, not the VRAM arena —
-  structurally orthogonal to `--expert-precise-memory` (verified in-container
-  2026-10-05: precise-memory only swaps the GPU overhead term,
-  expert_offload_mem only sets the host bound; they rejoin only inside
-  `place_experts`, and the plan rewrites `expert_offload_mem` back to the cap
-  the plan met, so plan and allocation cannot drift).
-- **Check whether the stack supports the model natively before porting an
-  overlay.** The image already registered `Glm5NextForConditionalGeneration`,
-  `Qwen4ExpForCausalLM` and both MTP variants — chasing four missing symbols
-  through two image tags to bring an overlay up was entirely wasted work.
-
-The checkpoint this recipe was verified on is
-`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (gated; 172.8 GiB, 131 shards,
-`Qwen4ExpForConditionalGeneration`, 48 layers, 512 experts top-10, multimodal,
-native 262,144) — an [[frank-archon]] era payload.
-
-## Compile-cache persistence (survives `podman rm -f`)
-
-Serve scripts that recreate the container discard the writable layer, so
-every boot re-pays compile/JIT (frank-archon: `torch.compile took 181.20 s`
-cold, of which 173 s was one graph).
-
-1. **Prove it and locate it:** fresh container vs live
-   (`podman run --rm --entrypoint bash IMG -c "ls -la /root/.cache"` vs
-   `podman exec LIVE bash -c "du -sb /root/.cache/*; find /root/.cache -type
-   f | wc -l"`). vLLM cache root: `VLLM_CACHE_ROOT` (default
-   `/root/.cache/vllm`, `torch_compile_cache/`); a HIP runtime adds
-   `/root/.cache/comgr`.
-2. **Mount the PARENT, not the leaf** — one bind on `/root/.cache` persists
-   both trees, safe whenever the image ships no such directory (step 1
-   proves it; otherwise you would be hiding baked state). Wire as an opt-in
-   tunable: `mkdir -p "$CACHE_DIR"` BEFORE the mount, then
-   `${CACHE_DIR:+-v "$CACHE_DIR":/root/.cache}`. Host path outside the
-   checkout.
-3. **Verify with two consecutive boots** — cold `Compiling a graph … takes
-   N s`, warm `Directly load the compiled graph(s) for compile range (…)
-   from the cache` with `torch.compile took` in single-digit seconds.
-4. Traps: the cache key includes the compile RANGE — an MNBT change costs
-   ONE recompile, not a permanent tax; do not claim a proportional boot cut
-   (time to `/health` is dominated by weight loading: 490 s warm vs 188 s
-   cold compile); stale entries are misses, not wrong answers (version +
-   config hashed); if the serve script is seam-gated, assert the mkdir and
-   the exact mount target.
-
-## Plan-line verification checklist (after any fork deploy)
-
-The journal one-liner that reads the plan lines (digit-anchored `grep -oE`,
-because plain grep truncates those columns to `17.`) and the post-deploy
-verification order — plan vs captured graph memory, `GPU KV cache size` vs the
-ask, per-card `rocm-smi` high-water, `NRestarts` = 0, `VLLM_CUDAGRAPH_*` present
-in the container env, then a real completion from a PEER because
-`curl localhost:7770/health` inside an ssh session on the host can return `000`
-and look like an outage — are step 3 of
-[[RDNA4 MoE Expert Offload VRAM Budget]] §7. Run that; do not re-derive it
-here.
-
-## Related
-
-- [[RDNA4 MoE Expert Offload VRAM Budget]] — the plan law this harness tunes
-- [[vLLM Boot Forensics and Config Interrogation]] — journal forensics for these boots
-- [[LLM Bench Windows and Measurement]] — window discipline and bash laws
-- [[Model Swap and Rollback]] — where the fork sits in the swap history
-- [[Remote Job Runner]] — the job.sh harness referenced above

serverAppConfigs-docs/LLM/VLLM Index.md

@@ -1,104 +0,0 @@
----
-type: reference
-description: "Locator for every vLLM page in this vault."
-tags:
-  - llm
-  - vllm
-timestamp: 2026-08-28
----
-
-# vLLM Index
-
-> **RETIRED (2026-08-18).** The vLLM service on Dankclaw was replaced by the CIRU ROCmFPX llama.cpp runtime (v3). These pages are historical reference. See [[frank-dragoon]].
-
-## Documentation Overview
-
-This index contains all documentation for vLLM deployment on Dankserver and Dankclaw.
-
-## Hardware Guides
-
-- [[Dankserver System Specs]] — NVIDIA GPU server specifications
-- [[Dankclaw System Specs]] — AMD GPU server specifications
-
-## vLLM Architecture
-
-- [[VLLM Overview]] — Complete vLLM documentation including installation, CLI, API, and key arguments
-- [[RAM CPU Offloading in VLLM]] — CPU RAM offloading mechanisms, swap space, and memory configuration
-- [[Hybrid CPU GPU Processing Dankserver]] — NVIDIA GPU with system RAM hybrid processing
-- [[Shared Memory Architecture Dankclaw]] — AMD ROCm unified memory and APU configuration
-
-## Model Deployment Guides
-
-- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B FP8 deployment on NVIDIA GPU
-- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B FP8 deployment on AMD GPU
-
-## Quick Reference
-
-### Installation Commands
-
-```
-# NVIDIA GPU
-uv pip install vllm --torch-backend=auto
-
-# AMD ROCm
-uv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/
-
-# CPU
-uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly/cpu/
-```
-
-### Server Start Commands
-
-```
-# Basic server
-vllm serve Qwen/Qwen2.5-1.5B-Instruct
-
-# With tensor parallelism
-vllm serve meta-llama/Llama-3.1-70B-Instruct --tensor-parallel-size 4
-
-# With quantization
-vllm serve Qwen/Qwen2.5-1.5B-Instruct --quantization fp8
-
-# With GPU memory control
-vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --kv-offloading-size 8 --kv-offloading-backend native
-```
-
-### Common API Endpoints
-
-| Endpoint | Method | Description |
-|----------|--------|-------------|
-| /v1/models | GET | List available models |
-| /v1/completions | POST | Text completion |
-| /v1/chat/completions | POST | Chat completion |
-| /health | GET | Health check |
-| /metrics | GET | Prometheus metrics |
-
-### Key Parameters
-
-| Parameter | Description | Typical Value |
-|-----------|-------------|---------------|
-| tensor_parallel_size | Number of GPUs for model sharding | 1-8 |
-| gpu_memory_utilization | Fraction of GPU memory for KV cache (default 0.92) | 0.7-0.95 |
-| swap_space | Retired; live knob pair in [[RAM CPU Offloading in VLLM]] | n/a |
-| max_model_len | Maximum context length | auto |
-| max_num_batched_tokens | Max tokens per batch | 2048-8192 |
-| max_num_seqs | Max sequences per batch | 128-512 |
-
-### Quantization Methods
-
-| Method | GPU Support | Description |
-|--------|-------------|-------------|
-| FP8 | Hopper, Ada (native); Ampere W8A16 via Marlin; RDNA via emulation | FP8 E4M3 inference |
-| INT8 | Most GPUs | Weight-only INT8 |
-| GPTQ | Most GPUs (NVIDIA; limited on ROCm) | Post-training GPTQ |
-| AWQ | Turing+ (NVIDIA) | Activation-aware weight quantization |
-| GGUF | CPU, GPU | General GPU unification format |
-
-## See Also
-
-- [[VLLM Overview]]
-- [[RAM CPU Offloading in VLLM]]
-- [[Hybrid CPU GPU Processing Dankserver]]
-- [[Shared Memory Architecture Dankclaw]]
-- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
-- [[Running Qwen3.6 27B FP8 on Dankclaw]]

serverAppConfigs-docs/LLM/VLLM Overview.md

@@ -1,287 +0,0 @@
----
-type: reference
-description: "vLLM is a fast and easy library for large language model inference and serving."
-tags:
-  - llm
-  - vllm
-timestamp: 2026-08-28
----
-
-# vLLM Overview
-
-## What Is vLLM
-
-vLLM is a fast and easy library for large language model inference and serving. You can use vLLM to run models on NVIDIA GPUs, AMD GPUs, Intel GPUs, and CPUs. vLLM supports over 200 model architectures from Hugging Face.
-
-vLLM is fast with these features:
-
-- PagedAttention for efficient key and value memory management
-- Continuous batching of incoming requests
-- Chunked prefill and prefix caching
-- FP8, INT8, INT4, GPTQ, AWQ, and GGUF quantization
-- Tensor, pipeline, data, expert, and context parallelism
-- Speculative decoding with n-gram, suffix, EAGLE/EAGLE3, DFlash, DSpark, and MTP
-- Automatic kernel generation with torch.compile
-- Disaggregated prefill, decode, and encode
-
-vLLM is flexible with these features:
-
-- Seamless integration with Hugging Face models
-- High-throughput serving with parallel sampling and beam search
-- Streaming outputs
-- Structured outputs with xgrammar or guidance
-- Tool calling and reasoning parsers
-- OpenAI-compatible API server
-- Anthropic Messages API and gRPC support
-- Efficient multi-LoRA support
-
-## Installation
-
-You can install vLLM with uv (recommended) or pip:
-
-```
-uv pip install vllm
-```
-
-Or with pip for NVIDIA GPUs:
-
-```
-pip install vllm --extra-index-url https://download.pytorch.org/whl/cu129
-```
-
-For AMD GPUs with ROCm:
-
-```
-uv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/
-```
-
-For CPU:
-
-```
-uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly/cpu/
-```
-
-## Quick Start
-
-### Offline Batched Inference
-
-```python
-from vllm import LLM, SamplingParams
-
-llm = LLM(model="Qwen/Qwen2.5-1.5B-Instruct")
-
-prompts = [
-    "Hello, my name is",
-    "The capital of France is",
-]
-
-sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
-
-outputs = llm.generate(prompts, sampling_params)
-
-for output in outputs:
-    generated_text = output.outputs[0].text
-    print(generated_text)
-```
-
-### Online Serving
-
-Start the OpenAI-compatible API server:
-
-```
-vllm serve Qwen/Qwen2.5-1.5B-Instruct
-```
-
-Query the server:
-
-```
-curl http://localhost:8000/v1/completions \
-    -H "Content-Type: application/json" \
-    -d '{
-        "model": "Qwen/Qwen2.5-1.5B-Instruct",
-        "prompt": "San Francisco is a",
-        "max_tokens": 7,
-        "temperature": 0
-    }'
-```
-
-Chat completions:
-
-```
-curl http://localhost:8000/v1/chat/completions \
-    -H "Content-Type: application/json" \
-    -d '{
-        "model": "Qwen/Qwen2.5-1.5B-Instruct",
-        "messages": [
-            {"role": "user", "content": "Hello!"}
-        ]
-    }'
-```
-
-## Key Arguments
-
-### Model Arguments
-
-| Argument | Default | Description |
-|----------|---------|-------------|
-| `model` | required | Model name or path |
-| `tokenizer` | same as model | Tokenizer name or path |
-| `tokenizer_mode` | auto | Tokenizer mode |
-| `trust_remote_code` | false | Trust remote code from Hugging Face |
-| `dtype` | auto | Data type (auto, float16, bfloat16, float32) |
-| `max_model_len` | auto | Maximum context length |
-| `quantization` | null | Quantization method (awq, gptq, fp8, compressed-tensors) |
-| `load_format` | auto | Weight loading format |
-
-### GPU Arguments
-
-| Argument | Default | Description |
-|----------|---------|-------------|
-| `tensor_parallel_size` | 1 | Number of GPUs for tensor parallelism |
-| `pipeline_parallel_size` | 1 | Number of GPUs for pipeline parallelism |
-| `gpu_memory_utilization` | 0.92 | Fraction of GPU memory for KV cache |
-| `swap_space` | — | retired; the live CPU-KV-offload knob pair is in [[RAM CPU Offloading in VLLM]] |
-| `enforce_eager` | false | Use eager mode (no CUDA graphs) |
-| `max_num_batched_tokens` | 2048 | Max tokens per batch |
-| `max_num_seqs` | 128 | Max sequences per batch |
-| `block_size` | 16 | Token block size |
-| `num_gpu_blocks_override` | null | Override number of GPU blocks |
-| `max_model_len_per_seq` | null | Max length per sequence |
-
-### Optimization Arguments
-
-| Argument | Default | Description |
-|----------|---------|-------------|
-| `disable_log_stats` | false | Disable logging |
-| `disable_log_requests` | false | Disable request logging |
-| `max_log_len` | null | Max log length |
-| `disable_custom_all_reduce` | false | Disable custom all-reduce |
-| `tokenizer_pool_size` | 0 | Tokenizer pool size |
-| `tokenizer_pool_type` | none | Tokenizer pool type |
-| `enable_chunked_prefill` | auto | Enable chunked prefill |
-| `max_num_batched_tokens` | 2048 | Max tokens per batch |
-| `attention_backend` | auto | Attention backend (flash_attn, flashinfer) |
-
-### Memory Arguments
-
-| Argument | Default | Description |
-|----------|---------|-------------|
-| `gpu_memory_utilization` | 0.92 | Fraction of GPU memory for KV cache |
-| `swap_space` | — | retired; the live CPU-KV-offload knob pair is in [[RAM CPU Offloading in VLLM]] |
-| `kv_cache_dtype` | auto | KV cache data type (auto, fp8, fp8_e5m2) |
-| `max_model_len` | auto | Maximum context length |
-| `num_gpu_blocks` | auto | Number of GPU blocks |
-| `num_cpu_blocks` | auto | Number of CPU blocks |
-
-### Device Arguments
-
-| Argument | Default | Description |
-|----------|---------|-------------|
-| `device` | auto | Device type (cuda, rocm, xpu, cpu, metal) |
-| `distributed_executor_backend` | auto | Distributed executor backend |
-| `huggingface_tokenizer_kwargs` | null | Hugging Face tokenizer kwargs |
-
-## Architecture
-
-vLLM uses a multi-process architecture:
-
-- API server process handles HTTP requests, tokenization, and input processing
-- Engine core process runs the scheduler and coordinates GPU workers
-- One GPU worker process per GPU executes model forward passes
-
-vLLM uses PagedAttention for memory management. PagedAttention treats KV cache as a set of memory pages. This allows efficient memory allocation and reduces fragmentation.
-
-vLLM supports multiple attention backends:
-
-- FlashAttention for NVIDIA GPUs
-- FlashInfer for NVIDIA GPUs
-- Triton attention for AMD ROCm
-- ROCm attention for AMD ROCm
-
-## Supported Models
-
-vLLM supports over 200 model architectures including:
-
-- Decoder-only LLMs (Llama, Qwen, Gemma)
-- Mixture-of-Expert LLMs (Mixtral, DeepSeek-V3, Qwen-MoE)
-- Hybrid attention and state-space models (Mamba, Qwen3.5)
-- Multi-modal models (LLaVA, Qwen-VL, Pixtral)
-- Embedding and retrieval models (E5-Mistral, GTE, ColBERT)
-- Reward and classification models (Qwen-Math)
-
-## Quantization
-
-vLLM supports these quantization methods:
-
-- FP8 for Hopper and Ada GPUs (native). Ampere runs FP8 W8A16 via Marlin; FP8 W8A8 computes via fallback
-- MXFP8/MXFP4 for Hopper GPUs
-- NVFP4 for Hopper and Blackwell GPUs
-- INT8 for most GPUs
-- INT4 for most GPUs
-- GPTQ for most GPUs
-- AWQ for Turing and newer GPUs (NVIDIA; native support varies on ROCm)
-- GGUF for CPU and GPU
-- Compressed-tensors for custom quantization
-- ModelOpt for NVIDIA Model Optimizer
-- TorchAO for PyTorch AO
-
-AMD RDNA 3 / RDNA 3.5 (gfx11, Strix Halo) run FP8 via software emulation, not native tensor cores. AWQ and GPTQ support in vLLM on AMD ROCm is limited; use GGUF or AMD Quark.
-
-## Parallelism
-
-vLLM supports multiple parallelism strategies:
-
-- Tensor Parallelism (TP) shards model parameters across GPUs
-- Pipeline Parallelism (PP) distributes model layers across GPUs
-- Data Parallelism (DP) replicates the model across GPU sets
-- Expert Parallelism (EP) distributes expert networks in MoE models
-- Context Parallelism (CP) splits context across GPUs
-
-
-## Chunked Prefill — the flag surface, and what it actually does
-
-**YES, the runtime exposes chunked-prefill options, and the running shape
-already uses them.** The tcclaviger fork is a vLLM fork, so the standard
-scheduler flags are present in the shipped image (verified by reading the
-`vllm.engine.arg_utils` and `vllm.config.scheduler` source in the live
-container `frank-archon-qwen`, not from a stub):
-
-| Flag | Default | What it controls |
-|---|---|---|
-| `--enable-chunked-prefill` | `True` in `SchedulerConfig`; the CLI arg is an auto-flag (`"default": None`) | whether prefill may be split into chunks |
-| `--max-num-batched-tokens` (MNBT) | 2048 (`DEFAULT_MAX_NUM_BATCHED_TOKENS`) | tokens processed per prefill iteration |
-| `--prefill-schedule-interval` | 0 | gap between prefill iterations |
-
-The running shape's argv carries `--max-num-batched-tokens 2048` alongside
-`--max-model-len 200704`, so chunked prefill is **ON and already active**: a
-200704-token prompt is prefilled in ~98 iterations of 2048 tokens, not in one
-shot.
-
-**It is a TTFT lever, not a throughput lever.** MEASURED 2026-09-29 on the
-ad hoc qwen-fp8 container (the same fork, same payload, same TP=2):
-
-- MNBT 8192 vs the default 2048: **neutral** on decode at every depth.
-- Small values re-chunk prefill only. They hurt TTFT (each chunk is its own
-  iteration, so the first token waits on more iterations) and they never touch
-  decode: a decode step is `max_num_seqs x (spec+1)` tokens, independent of
-  MNBT.
-
-**There is no `--ubatch-size` equivalent in vLLM.** `--ubatch-size` is a
-llama.cpp compute-buffer concept (`--batch-size`/`--ubatch-size`); the tcclaviger
-fork has no ubatch knob at all. The llama.cpp compute-buffer pair that frank-zealot
-was doubled on (2026-10-05, +4.6 % prefill at 8,928 tokens for 616 MiB) has no
-counterpart here, and the two backends should not be reasoned about by analogy.
-
-**MNBT is not a lever on this shape either.** The deployed 2048 is the fork's
-default, so a change is a deliberate departure from the shipped value, not a
-tune. The one thing MNBT does cost: every change re-enters the compile RANGE
-path, so `FRANK_ARCHON_CACHE_DIR` pays one recompilation.
-
-## See Also
-
-- [[VLLM Index]]
-- [[RAM CPU Offloading in VLLM]]
-- [[Hybrid CPU GPU Processing Dankserver]]
-- [[Shared Memory Architecture Dankclaw]]
-- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
-- [[Running Qwen3.6 27B FP8 on Dankclaw]]

serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md

@@ -19,7 +19,7 @@ timestamp: 2026-09-11
 **Current state (since 2026-10-18 01:24): Dankclaw boots `iommu=pt`** — identity
 DMA, IOMMU armed, XDNA2 NPU bound and usable. Everything below is the record of
 the 2026-09-11 decision that booted it with **`amd_iommu=off`** on a vendor-claimed
-13–16 % prefill figure, what that cost, and the measured flip back: see
+13–16 % prefill figure, what that cost, and the measured flip back:
 § *2026-10-18 execution* at the bottom for the numbers and the revert path.
 
 ## What changed
@@ -54,7 +54,7 @@ takes the NPU with it.
 
 1. **The NPU endpoint dies.** With IOMMU off there is no PASID/SVA state and
    `/dev/accel/accel0` cannot be opened — `err=-19`, the exact failure
-   documented in [[frank-npu]]. Audio input left the fleet with it; image input
+   documented in . Audio input left the fleet with it; image input
    survives on `frank-dragoon` through halogen's vision tower.
 2. **Posture change machine-wide.** Dankclaw also serves Caddy (the TLS edge
    and the Anubis/Authentik guard chain), Authentik, Pi-hole, Gitea, and the
@@ -135,7 +135,7 @@ mode**". So `pt` is most of the recovery and PerfOpt is the last part of it.
 
 **The NPU is compatible with identity mode.** `amdxdna` binds through
 `iommu_sva_bind_device()` + PASID + GCR3 into its own `IOPT_TYPE_SVA` domain,
-**independent of the default domain type** (`drivers/accel/amdxdna/aie2_pci.c`),
+**independent of the default domain type** (`drivers/accel/amdxdna/aie2_pci.c`)
 and PerfOpt is not on that path at all. It only needs the IOMMU *enabled* — that
 is the whole `err=-19` story. Kernel pick notes: the Ubuntu 6.17 `-95` SVA
 failures were a GCR3 regression fixed before our kernel, but **6.18–6.18.7
@@ -204,7 +204,7 @@ depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafan
 answered 200. (Later the same day the pool DOUBLING to 401,408 positions for
 slot parity was COMMITTED (`2b12716`) and **deployed the same evening**: the
 boot now reserves `401408 positions (about 11.0 GiB)` and the engine-side total
-is `79.6 GiB in all` (62.9 weights + 11.0 KV + 5.7 working),
+is `79.6 GiB in all` (62.9 weights + 11.0 KV + 5.7 working)
 see [[frank-dragoon]].)
 
 **The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
@@ -212,7 +212,7 @@ Krackan/Strix Halo NPU rev 11), firmware `amdnpu/17f0_11/npu_7.sbin`, `Initializ
 amdxdna_accel_driver 0.7.0 … on minor 0`, IOMMU group 26 `identity`, PASID cap width
 10 (`Exec- Priv-`). `/dev/accel/accel0` is `crw-rw---- root:render 261,0` and `dank`
 is in `render`, so a quadlet needs `AddDevice=/dev/accel/accel0` +
-`GroupAdd=keep-groups`. **No NPU endpoint is deployed** — [[frank-npu]] stays the
+`GroupAdd=keep-groups`. **No NPU endpoint is deployed** —  stays the
 history of the retired endpoint; what changed is only its precondition.
 
 **Deliberately deferred:** the `pp_dpm_fclk` pin (level 7, 2000 MHz) from the issue's
@@ -228,7 +228,7 @@ lives in `~/.local/bin`.
 
 ## Related
 
-- [[frank-npu]] — the endpoint this retired, including the err=-19 root cause
+-  the endpoint this retired, including the err=-19 root cause
 - [[frank-dragoon]] — the runtime bought by the prefill figure
 - [[frank-dragoon]] — the runtime it replaced (rollback target)
 

serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md

@@ -1,227 +0,0 @@
----
-type: reference
-title: Frontier MoE on Dankserver — what fits, on which runtime
-description: "The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded."
-tags:
-  - dankserver
-  - r9700
-  - gfx1201
-  - moe
-  - deepseek-v4.1
-  - deepseek-v4
-  - mimo
-  - glm
-  - affinity
-  - lucebox
-  - feasibility
-timestamp: 2026-09-28
----
-
-# Frontier MoE on Dankserver — what fits, on which runtime
-
-> **Question this page answers.** Given Dankserver's hardware, which of
-> DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash runs best, on
-> which runtime, and can Affinity / vllm-radiance / FreeToken-AMD be modified
-> to run them fast?
->
-> **Answer in one line.** V4.1-Flash does not fit this box at any
-> configuration; the engine that does run it (Lucebox) measures
-> 0.8–9.1 tok/s on R9700 because decode is bound by PCIe expert streaming
-> rather than by the GPUs. The 284B **V4-Flash** on **Affinity** is the only
-> shape that clears 35 tok/s, at 110.
-
-Everything below was measured or read from source on 2026-09-28. Claims
-carried over from the engine authors are marked as theirs.
-
-## Hardware (measured live, 2026-09-28)
-
-| Property | Value |
-|---|---|
-| GPUs | 2x AMD Radeon AI PRO R9700 (Navi 48, **gfx1201**), 32,624 MiB each = **63.7 GiB** total |
-| Interconnect | PCIe 5.0 **x8** each, CPU root ports `00:01.1` / `00:01.3`, ~31.5 GB/s per card |
-| PCIe atomics | `CAP_EXP+28.w = 0x1440` (ReqEn+) on both endpoints — RCCL collectives are legal |
-| CPU | Ryzen 7 7700X — **8 cores / 16 threads** (small; CPU expert compute is expensive) |
-| RAM | **246 GiB** + 63 GiB swap |
-| Disk | 1.8 TB NVMe. `/mnt/entropy` is a separate 91 TB ext4 at **100%** — do not plan payload there |
-
-The knowledge base was wrong about RAM: [[Dankserver System Specs]] said
-60 GiB until 2026-09-28 and has been corrected. The Lucebox runbook's 246 GiB
-was right.
-
-## The three candidates
-
-| Model | Total / active | Routed experts | Attention & KV | Working gfx1201 runtime |
-|---|---|---|---|---|
-| **DeepSeek-V4.1-Flash** | 552B / 8B prefill, 16B decode | 384 top-6, 40 layers, `moe_intermediate_size` 2304, hidden 5120 | CED 20 enc + 20 dec; CSA2 Full/Reindex/Reuse; **FP4 main KV, 890 B/token**; Engram 196B at layers 1 & 14 | Lucebox `feat/ds41-engine` only (unmerged) |
-| **MiMo-V2.6-Flash-MOPD** | 309B / 15B | 256 top-8, no shared expert, 48 layers (39 SWA / 9 GA) | hybrid SWA (window 128) + GA; 5-layer MTP | vLLM / SGLang, NVIDIA-first. No AMD path |
-| **GLM-5.3-Flash** | 320B / 18B | MoE | KDA (linear) + DSA (sparse) hybrid; mHC | **tcclaviger vLLM fork — plausible native path (2026-09-29 source reading: `glm5next` registered with an `amd/` backend dir + DSA indexer + mHC path; r4d.so carries native mxfp4/dsfp4 MoE entries). Gate: a compressed-tensors MXFP4 payload (~165 GiB, none exists publicly) + a HIP boot probe.** llama.cpp GGUF (community R9700 guide) remains the verified-but-slower path |
-
-## Sizing — the arithmetic that decides it
-
-| Quantity | Value | Note |
-|---|---:|---|
-| DS4.1 official checkpoint | **475.27 GiB** | 48 shards, summed from the HF API |
-|   of which Engram | **189.13 GiB** | exactly two 94.56 GiB shards — the layer 1 and 14 lookup tables |
-| DS4.1 routed-expert params | 543.6 B | 40 x 384 x 3 x 5120 x 2304 |
-|   shipped as | 269 GiB (fp4) | DeepSeek's own `expert_dtype: fp4` |
-| Lucebox `ROCMFP23.gguf` | **377.51 GiB** | 179 GiB experts (2.5/3.5 bpw) + 189 GiB Engram + ~9.5 GiB dense |
-| V4-Flash official checkpoint | 148.67 GiB | **3.2x smaller than V4.1** |
-| Affinity container for V4-Flash | 101.1 GiB | 92.9 GiB experts @ 2.875 bpw + 6.2 dense + 2.0 embed |
-
-The Engram tables are **never mapped** by Lucebox, so they do not evict the
-expert page cache — that is the one piece of good news in the 377 GiB file.
-
-### The decode wall
-
-A V4.1 token selects `6 x 40 = 240` routed experts at ~11 MB each. At the
-residency the hot stack can hold (37 of 384 per layer, 9.6%), **~2.4 GB of
-expert weights cross PCIe per token**. PCIe 5.0 x8 does ~55 GB/s real:
-
-```
-2.4 GB / 55 GB/s = 44 ms  ->  ~23 tok/s absolute ceiling
-```
-
-That is **below the 35 tok/s target before any attention, Engram or dense work
-is counted.** Raising residency is the only lever, and VRAM is what bounds it:
-63.7 GiB / 11 MB = 5,800 of 15,360 experts = 37.8% maximum, and that assumes
-VRAM holds nothing else.
-
-The engine author measured the same wall on an R9700 with 61 GB of RAM
-(DS41.md, "Measured"): 0.8 tok/s cold / 1.8 tok/s warm at a 6 GB hot budget,
-0.9 / 1.9 at 16 GB, and 9.1 tok/s for the best tuned three-tier case. They
-name the cause: *"90 to 96% of the 240 routed experts a token selects are still
-cold, and decode moves roughly 2.4 GB of expert weights per token through the
-synchronous stream path. That, not the GPU, bounds decode today."*
-
-## Affinity — a DeepSeek-V4-Flash engine, and what a V4.1 port costs
-
-[Affinity](https://codeberg.org/StillDeadcode/affinity) is *"an inference
-engine for DeepSeek-V4-Flash on one or two RDNA4 GPUs"*, C++/HIP, no framework.
-**It builds clean on this host**: CMake 3.24+, `hipcc` 23, `HIP enabled:
-gfx1201`, zero errors, producing `affinity`, `aff-quantize`, `aff-info` and
-`aff-gpucheck`. Its README claims **~110 tok/s on two R9700s** against 8 on
-one. That is 3x Frank's 35 tok/s goal, for the 284B model.
-
-The 110 works because the 101 GiB container leaves the cold expert set in a
-**pinned host pool in RAM** — the SSD tier that Affinity calls "a cliff" is
-never reached. V4.1 would need 182 GiB of experts at the same 2.875 bpw, plus
-189 GiB of Engram, against the same 246 GiB of RAM.
-
-### What a V4.1 port would reuse
-
-The config parser is **fully data-driven** — every dimension is read from
-`config.json`, so nothing is compiled in at the config layer. And the expert
-tensor naming is *identical* between the two checkpoints:
-
-```
-layers.N.ffn.experts.E.w1.weight   (11,008 in V4  = 43 x 256)
-layers.N.ffn.experts.E.w1.weight   (15,360 in V4.1 = 40 x 384)
-```
-
-So the quantiser, the `.aff` container, the expert slab, the host pool and the
-heat/placement engine — **the part that produces the 110 tok/s** — all carry
-over. Hyper-connections are the same family (`hc_mult 4`, Sinkhorn 20).
-
-### What a V4.1 port would have to write
-
-Verified against both checkpoints' real `model.safetensors.index.json` on
-2026-09-28 — not inferred from the config prose.
-
-**Tensor presence.** `aff-quantize` requires 32 per-layer tensors. V4.1
-supplies 26 plus every global (`embed`, `head`, `norm`, `hc_head_*`) and the
-whole expert set. Six are missing:
-
-| Missing in V4.1 | Replaced in V4.1 by |
-|---|---|
-| `ffn.gate.tid2eid` | the Engram tables |
-| `attn.compressor.ape` | nothing — ratio-2 is *pooled, no APE* |
-| `attn.indexer.compressor.{wkv,wgate,ape,norm}.weight` | `attn.indexer.wk` + `attn.indexer.k_norm` |
-
-**Config keys.** Of the 35 Affinity reads, V4.1 supplies **34**. The one it
-lacks is the dangerous one:
-
-| Gap | Detail |
-|---|---|
-| **Engram — fails silently** | `num_hash_layers` (V4: 3) is absent, so `cfg_int` returns its default and `model.h:36`'s `n_hash_layer = 3` survives. It would route layers 0–2 through a table that does not exist: fluent output, wrong numbers, no error. This is the only silent failure; the other four gaps fail loudly. |
-| **Ratio vocabulary is disjoint** | V4 uses `{0, 4, 128}`; V4.1 uses `{0, 1, 2}` — **no overlap in the non-zero case**. Affinity computes `coff = (ratio == 4) ? 2 : 1`, so V4.1's 18 ratio-2 layers get the ratio-1 width; `comp_ape` is requested 38 times for a tensor that does not exist; and the `ratio == 4` indexer branch fires on **0** of the 8 layers that need one (`index_source_layer_ids` = 2, 8, 14, 20, 24, 28, 32, 36) — those 8 would run dense. |
-| **CED** | A 20-layer encoder feeding a 20-layer decoder whose global KV is projected from the final encoder hidden state, through an `aligner`. Affinity's `model.cpp` is a flat layer stack. |
-| **DSpark** | V4.1's draft carries **its own** 128-expert MoE at top-3 (`dspark_n_routed_experts`, `dspark_num_experts_per_tok`), 3 stages rather than 1. Affinity's draft shares the target's slab and routing. |
-| Vision | V4.1 is multimodal; Affinity has no vision path. Text-only is an acceptable loss. |
-
-V4.1 also adds 31 config keys Affinity never reads, including the whole
-`engram_*` block, `kv_source_layer_ids`, `index_source_layer_ids` and the
-candidate block pre-selection.
-
-**Verdict.** The reusable half is the *right* half — the expert quantiser,
-the `.aff` container, the slab, the host pool and the heat/placement engine
-are what produce the 110 tok/s. The rest is 6 tensor bindings, one new ratio
-vocabulary, the CED aligner, a new draft geometry and Engram: weeks of work
-in a third-party codebase, with no cheap way to prove correctness. And it
-meets the same capacity wall regardless, because 182 GiB of experts plus
-189 GiB of Engram against 246 GiB of RAM is exactly the problem Lucebox has.
-Not advisable.
-
-## The other runtimes
-
-| Runtime | gfx1201 | Architectures | Note |
-|---|---|---|---|
-| **Affinity** | yes, built here | **V4-Flash only** — "the geometry is compiled in; there is no architecture abstraction" | 110 tok/s for 284B. Best engine for this box by an order of magnitude, wrong model |
-| **tcclaviger vLLM fork** | **yes — proven 2026-09-29** | qwen4exp measured 37.4-40.6 tok/s decode (see [[frank-archon]]); **`glm5next` registered** (`Glm5NextForCausalLM`/`MTP`, backend-split `amd/`/`common/`/`nvidia/`, `sparse_indexer.py` for DSA); r4d.so has **native mxfp4 + dsfp4 MoE entries** (`has_dsfp4() == True`); W4A16 MoE class (`R4dW4a16MoEExperts`) wired into the same tiered offload; TunableOp silently skipped under offload (needs `CLAV_TUNABLEOP_WITH_OFFLOAD=1`, and then corrupts numerics — do not use) | the ad hoc qwen-fp8 container's runtime; **the old "MXFP4 = emulation/missing kernels" verdict predates this fork and is dead** — the GLM-5.3-Flash gate is now the payload format + a boot probe, not kernels |
-| **Lucebox** | yes, image on host | Qwen 3.8, DS4; **V4.1 on the unmerged `feat/ds41-engine` branch** | The only V4.1 path. **Refuses layer split and layer-major prefill for V4.1** |
-| **radiance-vLLM** (drwolfen) | yes — served frank-archon 2026-09-27 → 09-30 | welded to `qwen4_exp`; PLE / QSA / packed-HC live inside that model class | 12–20 person-days per architecture |
-| **FreeToken-ROCm** | partial | block-FP8 safetensors only (modelopt); known `CpuMoeExecutor` issue under ROCm | cannot load a GGUF |
-| **llama.cpp** | yes | GLM-5.3 confirmed by a community R9700 guide; DS-V4.1 CED unverified | no expert LRU; **8 CPU cores** make CPU expert placement the wrong trade |
-| **TokenSpeed** | claimed | GLM-5.x incl. a 5.3-Flash recipe | NVIDIA-first, unverified on gfx1201 |
-| **KTransformers** | no | GLM-5.3 tutorial exists | built for Intel AMX + NVIDIA |
-| **radiance** (StillDeadcode — the standalone C++/HIP engine, NOT the vLLM fork of the same name) | image `stilldeadcode/radiance` prebuilt for gfx1201 | qwen35 dense/MoE at fp8, qwen4exp quantised-at-convert, llama-shaped | see the probe below: our FP8 payload is REFUSED; tiers, MTP and vision native |
-
-### radiance (standalone) — 2026-10-05 probe
-
-`codeberg.org/StillDeadcode/radiance` — Deadcode's from-scratch engine
-(v1.0.13): same author as the `vllm-radiance` fork we ran in September
-([[frank-medium-radiance-mxfp4]]), an unrelated codebase — C++/HIP, no
-Python, kernels/architectures/quantisers as `.so` plugins, flags written
-for two 32 GB R9700s, native Qwen3.8-Flash-Next with `--placement
-expert_tiered`, MTP and vision.
-
-Probe with the DEPLOYED payload (`rad-convert --plan-only`, which stops
-at declare — no GPU touched, no bytes written):
-
-```
-E no architecture plugin claims 'qwen4exp' at quantisation 'fp8'.
-  'qwen4exp' is served at: (unquantised).
-```
-
-So the 173 G orcarouter FP8 file cannot enter this engine at all: the
-qwen4exp plugin takes bf16/f16/f32/GGUF in and every quantisation happens
-at `rad-convert` time by recipe. Not a lock-in to their published quant —
-recipes are per-weight-class rules (`rtn`/`gptq`/`awq`/`autoround`/
-`paroquant`/`ggml`, grids from fp8 128×128 to i4/w4nl/w5nl codebooks,
-custom `--quant` rules go ahead of the file's), and libr4d lists routed
-experts-as-fp8 among the encodings it reads — so today's quantization is
-reproducible **from a bf16 source**, not from our file. Uncensored-family
-bf16 source candidates: `windowsxp811203/Qwen3.8-Flash-Next-Abliterated-GGUF`
-(ships the bf16 GGUF) and `mradermacher/Qwen3.8-Flash-Next-Uncensored-GGUF`
-(i1 quants only). 622 G free fits source + container. A trial is:
-download → `rad-convert` (CPU-side) → stop+mask frank-archon window to
-serve. Caveats on record: `--tp-wire wht6` is a lossy int8 all-reduce
-wire; the recurrent GDN state is f32 by default (144 MiB/sequence) with
-an opt-in f16 that changes generated text; every pool is budgeted
-explicitly — no profiling pass, no utilisation fraction.
-
-The two-R9700 three-owner split (primary + secondary + stream) is **not
-available on this host**: the retired quadlet recorded that
-`LUCE_DS4_MOE_TP_INPROC` is qualified for gfx1151+gfx1201 pairs, *not*
-gfx1201+gfx1201. hip:0 takes dense weights and the hot expert stack;
-`LUCE_EXPERT_STREAM_DEVICE=1` gives hip:1 the streamed-expert slot cache.
-
-## Measured on Dankserver
-
-<!-- MEASUREMENT -->
-
-## See Also
-
-- [[frank-archon]] — the retired Lucebox frank-archon runbook
-- [[Dual-R9700-Cooperation]] — the PCIe/atomics facts behind the board swap
-- [[Dankserver System Specs]] — the host (RAM corrected 2026-09-28)
-- [[frank-archon]] — the ACTIVE frank-archon

serverAppConfigs-docs/LLM/dankserver-vram-growth.md

@@ -1,144 +0,0 @@
----
-type: reference
-title: Dankserver VRAM and RAM Growth Under Load
-description: Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next.
-tags:
-  - dankserver
-  - llamacpp
-  - vram
-  - ram
-  - ctx-checkpoints
-  - crash
-timestamp: 2026-09-03
----
-
-> **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
-> See [[frank-redo-2026-08-17]] and [[deployment]] for the active configuration.
-> The `frank-zealot` below is the 2026-08 service: llama.cpp on the RTX 3090
-> **while that card sat in Dankserver**. Today's [[frank-zealot]] is a different host
-> (Dankclaw), payload and runtime build, though the same card class - this crash
-> signature is not a live fault.
-
-# Dankserver VRAM and RAM Growth Under Load
-
-This page records the investigation of the recurring cuBLAS crash on
-`frank-zealot` (llama.cpp, RTX 3090 24 GiB). The service aborts
-with `CUDA error: an unsupported value or parameter was passed to the
-function` in `cublasSgemm_v2` at `ggml-cuda.cu:1541` during
-`update_slots` -> `decode` -> `process_ubatch`. It crashed on 2026-08-13
-at 19:45 and 19:55 and on 2026-08-14 at 12:00:43.
-
-## What Was Ruled Out
-
-### ctx-checkpoints do not grow VRAM
-
-`--ctx-checkpoints 4` stores context checkpoints in host RAM. The log
-records every checkpoint at exactly 62.813 MiB. The size never grows.
-The reasoning:
-
-- `common_prompt_checkpoint` (common/common.h) stores the serialized KV
-  state in `std::vector<uint8_t>` fields. These live on the host heap.
-- `update_tgt()` and `update_dft()` call `llama_state_seq_get_data_ext`
-  with `LLAMA_STATE_SEQ_FLAGS_PARTIAL_ONLY`. The serialized state moves
-  from VRAM to RAM for the snapshot and back on restore.
-- With 4 slots and `--ctx-checkpoints 4`, the maximum checkpoint RAM is
-  16 x 62.813 MiB, about 1.0 GiB. That is fixed and small.
-- `data_spec` (spec-decode state) is empty for ngram-mod. The ngram-mod
-  implementation does not override `get_state()`.
-
-### ngram-mod does not grow VRAM
-
-ngram-mod allocates one fixed 64 MiB CPU hash table
-(`common_ngram_mod::entries`, 16M int32 entries). It never resizes. The
-matcher keeps about 12 bytes per sequence. Draft generation runs on the
-CPU. The speculative path allocates zero GPU memory.
-
-### The CUDA graph cache has eviction in this build
-
-`ggml_backend_cuda_context::cuda_graph()` sweeps every 5 seconds and
-evicts graphs unused for 10 seconds. Upstream issue #25082 describes an
-unbounded graph cache, but build b10108-13-g555881ebc includes the
-eviction sweep. This build does not exhibit that leak.
-
-## What Grows
-
-### RAM growth is suspected, not yet confirmed
-
-systemd reported MemoryPeak 46.9 GiB for the crashed process and 45.9
-GiB for the replacement. RSS is flat at 32.3 GiB, which matches the
-documented idle baseline of about 32 GiB. The difference is file-backed
-page cache from the `--no-mmap` model read; cgroup memory.current counts
-file pages attributed to the process. That gap is not proof of a leak by
-itself.
-
-The upstream pattern to watch is issue #26216: the prompt cache and
-checkpoint accumulation exceed the `--cache-ram` cap and do not plateau.
-The environment file does not set `--cache-ram`, so the llama.cpp
-default of 8192 MiB applies. The monitor logs RSS, which separates
-anonymous memory from file cache. If RSS climbs under load while VRAM is
-flat, the RAM-side accumulation is real.
-
-### VRAM suspects under load
-
-VRAM is flat at idle: 21,268 MiB across every monitor sample. The crash
-correlates with load density and slot context size (143K tokens on
-slot 2), not with wall-clock time. The two mechanisms worth testing:
-
-1. Flash-attention F16 K conversion workspace. The prefill kernel on
-   Ampere is `MMA_F16`. It converts the quantized K cache (q8_0) to F16
-   for the attention math. The workspace size scales with the attended
-   context length (`ggml_nelements(K) x 2`). It is allocated into the
-   graph buffer via `ggml_cuda_flash_attn_ext_get_alloc_size`.
-2. The CUDA VMM pool grows monotonically. `ggml_cuda_pool_vmm::alloc`
-   only reserves more physical memory; `free` only releases the
-   reservation slot. The pool never shrinks during the process
-   lifetime. Any new peak allocation permanently expands VRAM usage.
-
-Both match the observed pattern: growth under prefill bursts at deep
-context, flat at idle, crash when a burst exceeds the ~2.8 GiB headroom.
-
-## Monitoring
-
-The monitor runs on Dankserver:
-
-```
-/home/dank/vram-monitor.sh
-/home/dank/vram-log.csv
-```
-
-It samples GPU VRAM and process RAM every 10 seconds and appends a CSV
-row. The columns are timestamp, uptime, GPU VRAM MiB, GPU utilization,
-process RSS, process VSZ, and systemd MemoryPeak.
-
-## Resolution: dual-f16 KV Cache (2026-08-14)
-
-The fix chose the first mechanism over the second. Both K and V moved to f16 on 2026-08-14. When K is already f16, the flash-attention MMA_F16 prefill kernel needs no F16 conversion copy, so the context-proportional workspace disappears. The pool becomes static: 20,480 bytes per cell (10 full-attention layers x 2 kv heads x 256 dims x 2 bytes for K plus 2 bytes for V).
-
-The pool at 800768 cells cost 15,640 MiB and did not fit. The load-time warmup graph (1472x1472 image) failed cudaMalloc by 248 MiB, and the service idled at 23,966 of 24,576 MiB. The context pool was refitted to 700416 cells (4 slots x 175104): KV pool 13,680 MiB, idle 22,110 MiB, about 2.0 GiB free after warmup. Parallel stays 4.
-
-Verified 2026-08-14 after deploy: flash attention enabled, warmup clean, decode about 42-44 t/s, cached prompt tokens working. The vram monitor log shows the restart arc (23,966 -> 5,210 during load -> 22,110 idle).
-
-## Mitigation Options
-
-The documentation for the benchmark ([[frank-zealot]])
-prescribes the first option for this crash class. None of these change
-the model.
-
-| Option | Effect | Cost |
-| ------ | ------ | ---- |
-| `GGML_CUDA_FORCE_MMQ=1` | Bypasses the cuBLAS path in the crashing function | Slight decode slowdown |
-| `--cache-ram 2048` | Bounds the confirmed RAM growth | Cache misses on prompt reuse |
-| `--ctx-checkpoints 2` | Cuts checkpoint RAM and save cost in half | More prefill work after eviction |
-| Apply PR #24649 | Clears slot checkpoints before the prompt-cache save | Needs a llama.cpp rebuild |
-
-Upstream references: issue #26216 (prompt cache growth), issue #25082
-(graph cache, not applicable to this build), issue #23635 (quantized KV
-with speculative decoding), PR #24649 (checkpoint-to-cache duplication),
-PR #25649 (slot checkpoints freed on prompt clear, merged), issue #26119
-(same cuBLAS signature, confirmed on V100 with speculative decoding).
-
-## See Also
-
-- [[frank-zealot]]
-- [[Dankserver System Specs]]
-- [[deployment]]

serverAppConfigs-docs/LLM/dealign-ornith-crack.md

@@ -1,127 +0,0 @@
----
-type: reference
-title: dealign Ornith-1.5-35B-A3B CRACK on Dankserver
-description: "Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card."
-tags:
-  - llamacpp
-  - cuda
-  - dankserver
-  - moe
-  - ornith
-  - mtp
-  - retired-endpoint
-timestamp: 2026-08-24
----
-
-# dealign Ornith-1.5-35B-A3B CRACK on Dankserver
-
-> **Active 2026-08-25.** `frank-zealot` serves this model. It
-> replaced the CRACK Q8_0 file on the same upstream llama.cpp binary (b10121).
-> The Q6_K file was downloaded from the model repository and validated with
-> `llama-gguf` before installation.
-
-`frank-zealot` (port 7770) serves
-`Ornith-1.5-35B-A3B-CRACK-Q6_K.gguf` from `dealignai` on Hugging Face. It is
-a **MoE** model (256 experts, 8 active) with a hybrid GatedDeltaNet (SSM) +
-attention architecture, abliterated (uncensored). The GGUF also contains a
-bundled MTP head, but this deployment uses ngram-mod instead.
-
-## Model Files
-
-| File | Size |
-|------|------|
-| `Ornith-1.5-35B-A3B-CRACK-Q6_K.gguf` | 27.2 GiB |
-| `mmproj-Ornith-1.5-35B-A3B-f16.gguf` (vision) | 0.84 GiB |
-
-Store the files under `/home/dank/models/Ornith-1.5-35B-A3B-CRACK/`.
-
-## Layer Audit and Expert Offload
-
-The Q6_K file contains 41 transformer blocks and 753 tensors. Each block
-contains 630 MiB of expert tensors. Blocks 3, 7, 11, 15, 19, 23, 27, 31,
-35, and 39 contain full attention. Block 40 contains the bundled MTP head.
-
-The non-expert tensors total 2.02 GiB. The layer totals are 660.96 MiB for
-ordinary SSM blocks, 655.81 MiB for full-attention blocks, and 662.40 MiB for
-block 40. The expert portion is 630 MiB in every block.
-
-The selected configuration keeps all non-expert tensors on CUDA. It also
-keeps expert tensors from every full-attention block, plus every block from 35
-through 40, on CUDA. The unique GPU expert set is
-`3,7,11,15,19,23,27,31,35,36,37,38,39,40`. Expert tensors from all other
-blocks move to CPU through this explicit override:
-
-```
-OVERRIDE_EXPERT='blk\.(0|1|2|4|5|6|8|9|10|12|13|14|16|17|18|20|21|22|24|25|26|28|29|30|32|33|34)\.ffn_(up|down|gate|gate_up)_(ch|)exps'
-OVERRIDE_BUFT=CPU
-```
-
-The 16K benchmark found the hybrid policy fastest among the three tested
-placements. A later no-spec/ubatch sweep established that `ubatch=1024` is a
-stable low-memory choice. The current service uses ngram-mod only; MTP is
-intentionally disabled. The full-attention expert set and final blocks are
-kept on CUDA.
-
-To list the buffer type names on the host, run a bogus override:
-
-```
-llama-server --model <ANY> -c 64 -ot foo=BADTYPE
-```
-
-On Dankserver the available buffer types are `CPU` and `CUDA0`.
-
-## Speculative Decoding
-
-The GGUF contains a bundled MTP head, but this deployment deliberately does
-not use it. The current service enables only `ngram-mod`, with a 48-token
-minimum, 64-token maximum, and 24-token lookup match:
-
-```
---spec-type ngram-mod
---spec-ngram-mod-n-min 48
---spec-ngram-mod-n-max 64
---spec-ngram-mod-n-match 24
-```
-
-In the controlled 8K test, no speculation produced approximately 53.7–53.9
-generation tokens per second. MTP was slower for that workload, so it remains
-disabled.
-
-## Launch Excerpt
-
-```
-llama-server \
-  -m Ornith-1.5-35B-A3B-CRACK-Q6_K.gguf \
-  --mmproj mmproj-Ornith-1.5-35B-A3B-f16.gguf \
-  --chat-template-file chat-template.jinja \
-  --spec-type ngram-mod \
-  --spec-ngram-mod-n-min 48 \
-  --spec-ngram-mod-n-max 64 \
-  --spec-ngram-mod-n-match 24 \
-  --override-tensor "blk\\.(0|1|2|3|4|5|6|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|28|29|30|32)\\.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU" \
-  -ngl all -c 491520 -np 3 \
-  --ubatch-size 1024 \
-  --cache-type-k f16 --cache-type-v f16 --no-mmap \
-  --temp 1.0 --top-p 0.95 --top-k 20
-```
-
-## Known Issues
-
-1. The 27.2 GiB Q6_K file and 0.84 GiB vision projector fit the 62 GiB RAM
-   box with the selected expert offload.
-2. The context pool contains 491520 cells. Three slots receive 163840 cells
-   each, which is 96K below the model context per stream.
-3. `--n-cpu-moe` remains supported. This service prefers the explicit tensor
-   override because the 16K benchmark found the hybrid placement fastest. It
-   preserves every full-attention expert block and blocks 35–40 on CUDA.
-4. MTP is bundled in the GGUF but intentionally disabled; ngram-mod is the
-   only active speculative method.
-4. The service uses the Qwen-Sharp template from
-   `peculiar-ragdoll/Qwen-Sharp-Chat-Templates` through
-   `scripts/chat-template.jinja`.
-
-## See Also
-
-- [[deployment]] — deploy and update process
-- [[Dankserver System Specs]] — the box
-- [[frank-redo-2026-08-17]] — fleet topology histor

serverAppConfigs-docs/LLM/deployment.md

@@ -1,178 +0,0 @@
----
-type: howto
-title: LLM Deployment
-description: Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot).
-tags:
-  - llm
-  - deployment
-  - systemd
-timestamp: 2026-09-03
----
-
-# Deployment and Update Process
-
-This page describes how to deploy and update the LLM servers on Dankserver and
-Dankclaw. The process uses Git as the single source of truth. A change to
-SearXNG, Caddy, or another container service does **not** restart inference.
-
-## Fleet
-
-| Target | Host | Service | Port |
-|---|---|---|---:|
-| Dankserver | `dank@dankserver` | `frank-archon` | 7770 |
-| Dankclaw | `dank@dankclaw` | `frank-dragoon` | 7770 |
-| Dankclaw | `dank@dankclaw` | `frank-zealot` | 7771 |
-
-## Scoped workflow
-
-```mermaid
-graph LR
-    A[Edit LLM files] --> B[Run tests]
-    B --> C[Commit main]
-    C --> D[deploy.sh selects matching service]
-    D --> E[Target pulls ff-only]
-    E --> F[Relink one unit]
-    F --> G[Restart one inference service]
-    G --> H[Wait for that service health]
-```
-
-Run the unified deployer from the repository root:
-
-```bash
-./scripts/deploy.sh dankserver --service frank-archon
-./scripts/deploy.sh dankclaw --service frank-dragoon
-./scripts/deploy.sh dankclaw --dry-run
-```
-
-Without `--service`, `scripts/deploy-plan.py` maps changed files to the service
-on the selected host. The following files select the corresponding service:
-
-- `scripts/frank-archon.env`, `frank-archon.service`, and
-  `serve-frank-archon.sh` select `frank-archon` on Dankserver
-  (`scripts/deploy-plan.py:422-428`). The quadlet-era directory
-  `dankserver/frank-medium/` left the tree on 2026-09-26 and plans nothing
-  (`scripts/deploy-plan.py:374-377`).
-- `scripts/qwen-sharp-v22.4.0.jinja` selects `frank-dragoon` — a planner rule
-  left from the llama.cpp era when dragoon loaded it via
-  `--chat-template-file` (`scripts/deploy-plan.py:415-421`).
-- `scripts/frank-dragoon.env`, `frank-dragoon.service`, and
-  `serve-frank-dragoon.sh` select `frank-dragoon` on Dankclaw.
-- `scripts/frank-zealot.env`, `frank-zealot.service`, and
-  `serve-frank-zealot.sh` select `frank-zealot` on Dankclaw.
-- Documentation, tests, deployment tooling, and unrelated container changes
-  do not select any inference service.
-
-The target-side `scripts/deploy-remote.sh` installs the selected service.
-All three LLM services are **root units** whose repo unit file is relinked
-into `/etc/systemd/system/` (`relink_unit`), daemon-reloaded, enabled and
-restarted (`deploy-remote.sh:1158` `deploy_llm_dragoon`, `:1254`
-`deploy_llm_zealot`, `:1315` `deploy_llm_archon`). The dragoon and archon
-handlers wait for port 7770 to be released before the replacement starts
-(`wait_port_free`; the 2026-09-16/26 cutover lessons). The archon handler
-additionally hard-gates on the env file, the digest-pinned image ref, and the
-payload before touching the unit (refuse rather than restart-loop). Each
-handler then waits on that service's health endpoint —
-`http://127.0.0.1:7770/health` for frank-dragoon (budget 1200 s) and
-frank-archon (1800 s), `http://127.0.0.1:7771/health` for frank-zealot
-(900 s) — and refuses a false green: the deploy is only green when the unit
-itself reports a live MainPID behind the health answer
-(`deploy-remote.sh:1395-1403`, `:1285`). A failed handler does not advance
-its per-service state.
-
-## Unit and artifact paths
-
-The root units point into the stable target checkout:
-```text
-/home/dank/serverAppConfigs/scripts/frank-archon.service
-/home/dank/serverAppConfigs/scripts/frank-dragoon.service
-/home/dank/serverAppConfigs/scripts/frank-zealot.service
-```
-All three services keep their unit, env and serve script as this flat trio
-under `scripts/` — there is no LLM Quadlet left (the `frank-medium`
-quadlet-era directory left the tree 2026-09-26).
-
-The unit files and serve scripts validate their model/runtime artifacts before
-starting. Keep `REMOTE_DIR` stable; moving the checkout requires updating both
-the unit absolute paths and the deployment tests in the same change.
-
-## Retired endpoints
-
-`frank-npu` (the FastFlowLM XDNA2 NPU endpoint) was retired on 2026-09-11
-with the `amd_iommu=off` decision and is no longer a deployable service — the
-planner has no `frank-npu` entry. Its runbook is [[frank-npu]].
-
-## Isolation guarantee
-
-The remote runner has one explicit service dispatch. It does not run a
-host-wide LLM loop, `podman-compose up` for unrelated services, or a blanket
-systemd restart. Each successful service is recorded under the target user's
-state directory outside Git:
-
-```text
-~/.local/state/serverAppConfigs/deploy-state/
-```
-
-On a later ordinary deployment, a service remains pending only while its
-recorded revision differs from the checkout and the planner still finds a
-change in that service's inputs.
-
-## Locally built images
-
-`build_compose_image` in `scripts/deploy-remote.sh` builds the images that have
-no registry digest — `comfyui-rocm`, `franksdank/loki`, `franksdank/alloy` and
-the voice pair `franksdank/frank-tts` / `franksdank/frank-tts-engine` — and
-**skips** the build when a `cfhash` label on the installed image matches a hash
-it computes from the checkout now. That skip is what keeps the engine's
-ten-minute `git clone --recursive` off every re-deploy, so the rule has to be
-honest about what the image bakes:
-
-- The hash covers the `Containerfile` **plus the files its `COPY` lines pull
-  in**, named as `extra=(…)` in each case arm. Hashing the `Containerfile`
-  alone was wrong in practice: editing `dankclaw/voiceservice/app.py` or
-  `tts-engine-warm.sh` left the label matching, `podman build` never ran, and
-  the redeployed container still carried the previous code while `/health`
-  answered — invisible to every gate. Corrected 2026-10-18;
-  `tests/frank_tts_engine.bats` drives the function against a stub `podman`
-  (first deploy builds, unchanged tree does not, a `COPY`'d file rebuilds).
-- A base-image assumption is part of the image's identity too. The engine's
-  runtime stage creates its unprivileged user with `useradd`, not `adduser`,
-  because the pinned `ubuntu:24.04` base ships no `adduser` — checked by
-  running the command against the digest on 2026-10-18. That failure mode is a
-  build that dies at the `RUN` line, so it is loud, unlike the stale-hash one.
-
-Both engines keep serving during a rebuild: `build_compose_image` runs after
-`install_quadlet` only for the LLM-style handlers, and the voice deploy gates on
-recorded health rather than a fixed sleep — see [[Frank TTS Engine]].
-
-## Manual recovery
-
-If the LLM handler fails after the target pull:
-
-1. Inspect `journalctl -u frank-dragoon -n 200`, `journalctl -u frank-zealot
-   -n 200` or `journalctl -u frank-archon -n 200` (all three are root units);
-   frank-dragoon's engine boot ledger is in `/home/dank/logs/frank_dragoon.log`,
-   frank-archon's in `podman logs frank-archon` — not in the journal.
-2. Confirm the model files, the runtime binary and (for frank-archon) the
-   digest-pinned container image are present and readable by `dank`.
-3. Retry only the affected service:
-
-   ```bash
-   ./scripts/deploy.sh dankserver --service frank-archon
-   ./scripts/deploy.sh dankclaw --service frank-dragoon
-   ./scripts/deploy.sh dankclaw --service frank-zealot
-   ```
-4. Verify the matching health endpoint before declaring recovery.
-
-Do not stop the other host's inference service during recovery. The two
-Dankclaw endpoints are independent processes on different devices: restarting
-frank-dragoon does not disturb frank-zealot, and the reverse holds too — with one
-exception worth knowing, that the deployer waits for port 7770 to be released
-before it starts the replacement container. For benchmark work, follow the
-separate stop/benchmark/restart procedure in [[Testing Policy]] and the
-repository rules.
-
-## See Also
-
-- [Deployment Flow](../Architecture/Deployment%20Flow.md)
-- [[index]]
-- [Dankclaw deployment architecture](../Architecture/Network%20Architecture.md)

serverAppConfigs-docs/LLM/frank-archon.md

@@ -35,7 +35,7 @@ header carries the measured arithmetic beside each number).
 `qwen4exp-w4nl64-i8-hc8m`): routed experts as 4-bit codes in a 16-level non-uniform codebook
 (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent
 sweeps, E4M3 scale per 64 weights); **10 whole bf16-protected experts per layer**; int8 W8A8
-trunk + lm_head; fp8 HC mixtures; PLE n-gram fp8 at the checkpoint's fixed scale (47.7 GiB);
+trunk + lm_head; fp8 HC mixtures; PLE n-gram fp8 at the checkpoint's fixed scale (47.7 GiB)
 MTP head and the bf16 27-layer vision tower **in the container**. Card quality vs bf16: mean
 KL 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %. The abliterated family lives on
 [[frank-dragoon]]. Fetched by `scripts/download-frank-archon-iq4r.sh --fetch` (hfget resume +
@@ -53,7 +53,7 @@ command (the 2026-09-30 no-pipe/grep-race lesson) — so a retagged look-alike o
 `latest` cannot satisfy it. 1.2.0 is a rejected serve pin (Laws); the
 `--vram-weights-mib`/`--vram-kv-mib` pair is accepted by both pins, which is the rollback symmetry.
 
-**Trio** — `scripts/frank-archon.env` (every number, with its arithmetic in the header),
+**Trio** — `scripts/frank-archon.env` (every number, with its arithmetic in the header)
 `scripts/serve-frank-archon.sh`, `scripts/frank-archon.service`. Gates run in order: image
 digest present → `.rad` present and ≥ `RAD_MIN_MIB` (a truncated convert would otherwise buy a
 guaranteed long-timeout boot instead of this fast failure) → template present → `/dev/kfd` +
@@ -64,10 +64,10 @@ the name registered, and every one of those identities holds this GPU. Container
 (publisher-vetted, `docs/DOCKER.md`): `--network host`, `--init` (so the engine releases the
 cards on SIGTERM), `--security-opt seccomp=unconfined` (the default profile refuses io_uring —
 the PLE table's reads — and the NUMA calls the ROCm runtime makes for host memory), `--ulimit
-memlock=-1:-1`, `/models` ro + `/data` rw + the tune cache at `/opt/radiance/share/radiance/tune`,
+memlock=-1:-1`, `/models` ro + `/data` rw + the tune cache at `/opt/radiance/share/radiance/tune`
 template as a single-file mount, then `exec podman wait`: the script's lifetime IS the
 container's, so a crash is a nonzero `ExecStart` and the restart policy is real.
-Unit: `After=network-online.target`, `WantedBy=multi-user.target`, `User=dank`,
+Unit: `After=network-online.target`, `WantedBy=multi-user.target`, `User=dank`
 `LimitMEMLOCK=infinity`, `StartLimitIntervalSec=0`, `OOMScoreAdjust=-500` (the last process the
 OOM killer should pick), `Restart=always` **not** `on-failure` — a clean API-server exit after a
 worker fault reads as success to systemd (the 2026-10-01 vLLM lesson) — and
@@ -84,7 +84,7 @@ on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_p
 | VRAM pair | `VRAM_WEIGHTS_MIB=25600` + `VRAM_KV_MIB=5120` = 30,720 of the 30,969 ceiling; `GPU_HEADROOM_MIB=96` |
 | Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=44928` **per rank** |
 | PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
-| Spec / batch | `SPEC_TOKENS=2` — the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); the publisher's card ships depth 3 as its peak and this fleet's decode ladder was measured at 2. `MAX_BATCHED_TOKENS=2048` |
+| Spec / batch | — |
 | Prefix cache | `PREFIX_HOST_MIB=8192` + `PREFIX_DISK_MIB=65536` under `STATE_DIR=/home/dank/radiance-state` (mounted rw at `/data`) |
 | Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
 | TTM gate | `TTM_MIN_MIB=150000`, `TTM_TARGET_MIB=204800` |
@@ -107,9 +107,9 @@ one string).
 **Harness contract** (`/home/blank/.omp/agent/models.yml`) — provider `llamacpp (dankserver)`
 → `http://192.168.0.235:7770/v1`, `api: openai-completions`, `auth: none`, `id: frank-archon`
 (= `FRANK_ARCHON_SERVED_NAME`; the provider *label* is legacy and renaming it breaks
-`config.yml` references). `contextWindow: 200000` must equal `MAX_MODEL_LEN`;
+`config.yml` references). `contextWindow: 200000` must equal `MAX_MODEL_LEN`
 `maxTokens: 16384` (clamp `min(maxTokens, window − estimate − 4096)`; compaction fires at
-85 % → 170,000 est × 1.03 + 16,384 ≤ 200,000); `input: [text, image]`;
+85 % → 170,000 est × 1.03 + 16,384 ≤ 200,000); `input: [text, image]`
 `compat.reasoningContentField: reasoning_content`; `reasoningEffortMap` minimal/low→low,
 medium→medium, high/max→xhigh; `extraBody` temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0 /
 presence_penalty 0 / repetition_penalty 1.0 + `chat_template_kwargs {enable_thinking,
@@ -125,23 +125,21 @@ host-side unit drift is invisible to it.
 ## Eras
 
 One row per retired era on this port. The **Rollback** cell is the load-bearing one: anything
-marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `StillDeadcode`,
+marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `StillDeadcode`
 `rad` and `tpl.jinja`.
 
 | Window | Payload | Runtime | Shape | Replaced because | Rollback |
 |---|---|---|---|---|---|
 | 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
-| 2026-10-05 → 10-08 | `qwen4exp-fp8-native.rad`, 177,109 MiB (173.0 GiB) — native modelopt block-FP8, no requantisation | radiance 1.0.7 line, same digest | TP=2 wht6, 200,000 × **2** slots (3 later), slab 26,071 + fp8 KV pool 3,543, host pool 40,960/rank, `--expert-vs-cache-ratio 0.82`, spec 3, MNBT 2048; that pair measured 30.08 t/s decode on the 2026-10-06 43.6k greedy probe and 4,379 t/s A-cold prefill on 2026-10-11 | native FP8 experts (56.4 GiB/rank) ate a 40 GiB pool and left KV at 3,543 MiB = 386,904 tokens, **1.93 sessions** — context economics, not quality; and the host pool could not be trimmed to pay for it (36,864 refused the boot) | rebuild-only: `scripts/download-frank-archon.sh` (official `Qwen/Qwen3.8-Flash-Next-FP8`, 131 shards, 172.8 GB, ungated) + `scripts/build-frank-archon-rad.sh` (no-recipe convert under the descriptor law); raw **and** `.rad` were `rm`'d 2026-10-15 (Frank's call, to free the volume for the iq4r download) |
-| 2026-10-05 (same day) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` safetensors, 131 shards | FreeToken PR #587 head `77a7201` (`rocm714_torch214`), image `localhost/frank-primary-freetoken:pr587-77a7201` | ONE R9700; KV sized in tokens `299,968` (a multiple of 64 — the `qsa_sparse` backend pins page 64), 2 running requests, `--moe-cache-rate 0.10`; 17.1 t/s decode | TP=2 cannot boot on ROCm: the per-rank `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201, so the second card idled | git tag `frank-archon-freetoken-pr587-20261005` (trio + `scripts/docker/frank-primary-freetoken.Dockerfile`, which exists only there); image still in rootless storage; payload needs a re-fetch, and that repo 403s until the gate is accepted on the account |
+| 2026-10-05 → 10-08 | `qwen4exp-fp8-native.rad`, 177,109 MiB (173.0 GiB) — native modelopt block-FP8, no requantisation | radiance 1.0.7 line, same digest | — | native FP8 experts (56.4 GiB/rank) ate a 40 GiB pool and left KV at 3,543 MiB = 386,904 tokens, **1.93 sessions** — context economics, not quality; and the host pool could not be trimmed to pay for it (36,864 refused the boot) | rebuild-only: `scripts/download-frank-archon.sh` (official `Qwen/Qwen3.8-Flash-Next-FP8`, 131 shards, 172.8 GB, ungated) + `scripts/build-frank-archon-rad.sh` (no-recipe convert under the descriptor law); raw **and** `.rad` were `rm`'d 2026-10-15 (Frank's call, to free the volume for the iq4r download) |
+| 2026-10-05 (same day) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` safetensors, 131 shards | FreeToken PR #587 head `77a7201` (`rocm714_torch214`), image `localhost/frank-primary-freetoken:pr587-77a7201` | — | TP=2 cannot boot on ROCm: the per-rank `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201, so the second card idled | git tag `frank-archon-freetoken-pr587-20261005` (trio + `scripts/docker/frank-primary-freetoken.Dockerfile`, which exists only there); image still in rootless storage; payload needs a re-fetch, and that repo 403s until the gate is accepted on the account |
 | 2026-09-29 → 10-05 (interrupted 09-30 → 10-01 by GLM) | same orcarouter FP8 safetensors | tcclaviger vLLM fork, `docker.io/tcclaviger/vllm@sha256:da402966…` | shape G: 3 × 200,704 fp8, KV 3,166,500,000 B/rank → 401,408 tokens, arena 17.77 GiB / 7,762 slots, util 0.982, MNBT 2048, no spec, no prefetch, prefix caching ON | radiance gave real TP=2 (no RCCL) and a 2-3 min boot, and the fork never fixed its long-context class | git tag `frank-archon-vllm-shapeG-20261003` (trio + the 362-line env lore); image still in dank's rootless storage; payload needs a re-fetch. The 2026-10-01 fp8+MTP4 rung is tag `frank-archon-fp8-mtp-20261001` |
-| 2026-09-30 → 10-01 | `Intel/GLM-5.3-Flash-W4A16-AutoRound` ~170 GiB (sym int4 g128, `auto_gptq`) | same tcclaviger digest; `R4dW4a16MoEMethod` → libr4d grouped W4A16 | TP=2 + `--enable-expert-offload --expert-precise-memory`, 2 × 131,072 bf16 KV, util 0.98, MNBT 2048, 2,383 slots + 73.64 GiB pinned per rank; 9.9-10.0 t/s decode / 918 t/s prefill | the 35 t/s target is unreachable as the fork ships: 44 % of the ~93 ms/token budget is pointwise glue in an **uncompiled** GLM trunk — the expert gather is 1.05 % of device time | payload (170 GiB) + drafter (2.2 GiB) went to `~/.AGENTTRASH/20261001-glm-purge/` and were emptied; the cutover was reversed by `d91379c`, restoring the `2339fae` trio; `scripts/patches/glm53-*.patch` survive in-tree but none applies byte-exact since the 2026-10-10 rename rewrote their context lines |
-| 2026-09-27 → 09-30 | unsloth `Qwen3.8-Flash-Next-GGUF` Q8_0, 175.30 GiB / 6 shards + blockfp8 MTP4 head + mmproj; PLE derived 54,400,261,120 B | R9V radiance-vLLM fork `drwolfen/radiance-vllm-qwen4exp`, docker bundle `2dac17a2`, behind the docker→podman shim | `MAX_MODEL_LEN=131072`, 80-slot tiered cache (the policy max), MTP4, PLE **pinned**, prefix caching ON from 09-28 | the tcclaviger fork took the port for the GLM W4A16 experiment; Q8_0 had halved expert residency and a 128K envelope | host state survives: `/home/dank/r9v-state/q8xl/setup.json`, the fork checkout `/home/dank/GitPackages/radiance-vllm-qwen4exp` (local-only commits `363d606`/`30d4a32`/`2de0357`) and `.podman-shim/docker`; payload re-fetch first |
+| 2026-09-30 → 10-01 | `Intel/GLM-5.3-Flash-W4A16-AutoRound` ~170 GiB (sym int4 g128, `auto_gptq`) | same tcclaviger digest; `R4dW4a16MoEMethod` → libr4d grouped W4A16 | — | — | payload (170 GiB) + drafter (2.2 GiB) went to `~/.AGENTTRASH/20261001-glm-purge/` and were emptied; the cutover was reversed by `d91379c`, restoring the `2339fae` trio; `scripts/patches/glm53-*.patch` survive in-tree but none applies byte-exact since the 2026-10-10 rename rewrote their context lines |
 | 2026-09-27 (same day) | `Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS`, 92.39 GiB + PLE 26.82 GiB (`34fa36f8…`) | same R9V bundle + shim | `MAX_MODEL_LEN=196608`, offload budgets 146.25 / 78.75 GiB, PLE residency `ssd` mmap, tool parser `qwen3_coder`, thinking on the `reasoning` (vLLM) channel | same-day flip back to the reference quant with the MTP4 head, at Frank's direction | `/home/dank/r9v-state/uncensored/` (`setup.json`, `restore-config-20260927.env`) is still on disk; the IQ4_XS payload was deleted in the 2026-10-17 cleanup → re-fetch, point `R9V_STATE_FILE` back, drop the two q8-only keys and the template keys |
-| 2026-09-26 → 09-27 | unsloth Q8_0, 175.30 GiB / 6 parts + shared-Q8_0 MTP 2,786,568,256 B + `mmproj-BF16.gguf` 907,542,944 B | unsloth llama.cpp b11030-mix prebuilt (commit `6ba30d05b`), ROCm gfx120X | `-ts 1,1` layer split, routed experts **and** the PLE table computed in host RAM (`-ot exps=CPU,per_layer_token_embd.weight=CPU`), 262,144 × 1 slot, then the same-day reshape to 4 × 262,144 with V `q8_0` and hot expert blocks 5/3 | the R9V radiance-vLLM fork took the port at Frank's direction (true TP=2 plus MTP4) | trio at commit `4992e5a`+; the GGUF/MTP/mmproj set is no longer on disk → re-fetch, then redeploy |
-| 2026-09-26 (same day) | `Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF`, 377.2 GiB, sha256 `2daea6ca…` | `localhost/lucebox-rocm:gfx1201`, built from `Dockerfile.rocm` + 3 local patches | ONE R9700, `--max-ctx 131072`, `--chunk 8192`, q4_0 K+V, `--ds4-prefill sparse`, routed experts on CPU from the page cache | 377 GiB fits neither 63.7 GiB of VRAM nor 246 GiB of RAM → NVMe page-cache thrash; Qwen Flash-Next Q8 gave better quality, vision and 262k at comparable speed | image still in dank's rootless storage; payload gone; the quadlet and its entrypoint were deleted by `8500600` — `git show 8500600^:dankserver/frank-primary/quadlet/frank-primary.container` (and `…:dankserver/frank-primary/frank-primary-entrypoint.sh`), stop the root unit, start the user quadlet |
+| ≤ 2026-09 — 4 earlier eras: payloads, shapes and every measured number destroyed 2026-10-19 as obsolete | — | — | — | — | —
 
 Before 2026-09-26 the port belonged to the **`frank-medium`** service, and its era
-pages are **not** superseded here: [[frank-medium]] is the locator for that
+pages are **not** superseded here:  is the locator for that
 endpoint's whole shape history, including the unresolved end date of the
 vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
 
@@ -195,7 +193,7 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   buffers + arena onto the weights side (~29.6 GiB/card effective).
 - **Idle KV VRAM is the expert cache's spillover — the price is paid twice.** `mover: 2275
   slots can be lent out of the KV cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <=
-  256 tokens`; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 today),
+  256 tokens`; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 today)
   so a bigger KV pool steals weight residency from the slab AND from the lendable cache. Priors
   on this card/model: stealing 2.14 GiB of weights to buy `MAX_MODEL_LEN 262144` cost 2.5 %
   decode; the other direction (`--expert-vs-cache-ratio` 0.82 → 0.84, +2.5 GiB of weights) cost
@@ -252,7 +250,7 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   EMULATION backend an offload request of **118 GiB** landed as **~238 GiB** of real host RAM —
   the dequantized copy rides on top of the pinned weights, and the doubling is invisible in the
   weight count. Read the engine log for `Using 'EMULATION' Mxfp4 MoE backend` before concluding
-  "doesn't fit"; the container-side procedure and the kernel gate are [[RDNA4 Container Serving]].
+  "doesn't fit"; the container-side procedure and the kernel gate are .
 - **One encoding per expert table** (`rad_arch_declare`): a whole expert kept bf16 is the only
   thing legal beside the table, so a split tier (down=fp8, gate_up=w4) is unservable.
 - **Descriptor-selection law.** The plugin is picked by the exact `(arch_id, quant descriptor)`
@@ -268,7 +266,7 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   write temp + `os.replace`.
 - **Convert is card-less** (`rad-convert` declares on CPU, GPTQ solves on CPU BLAS) and
   `--plan-only` goes first: a plan that resolves every op will boot. `rad-tune` must be
-  declared at the shapes you serve and its cache lives in the *image* (`$RADIANCE_HOME/tune`),
+  declared at the shapes you serve and its cache lives in the *image* (`$RADIANCE_HOME/tune`)
   so it only survives on the `$FRANK_ARCHON_STATE_DIR/tune` bind-mount the serve script adds.
 - **Quant gates are scored at the production shape** (MNBT 2048, not the recording's 32768)
   against the BF16 container: KL ≤ 0.03, top-1 ≥ 95 %, and the verdict is chunk-size invariant
@@ -333,7 +331,7 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
 - **TP=2 here** is legal since the 2026-09-20 ProArt B850 swap (x8/x8 on CPU root ports,
   `Routing+`/`ReqEn+` end to end) only because radiance's wire is `libr4d`'s own one-shot
   all-reduce; RCCL needs atomics on the whole chain and fails `hipErrorIllegalState` without
-  them ([[Dual-R9700-Cooperation]]). `wht6` buys +5.6-6.3 % prefill for ~1.3 % of 8-stream
+  them (). `wht6` buys +5.6-6.3 % prefill for ~1.3 % of 8-stream
   decode, `exact` is the bit-exact alternative.
 - `--prefix-cache-dir` creation is **not recursive** — the serve script `mkdir -p`s the path or
   the disk tier silently never opens. The tier does survive a payload swap when arch and
@@ -341,7 +339,7 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
 
 ### Dead ends — do not re-try
 
-- **radiance 1.2.0 as the serve pin** (`sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa`,
+- **radiance 1.2.0 as the serve pin** (`sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa`
   commit `e95d397`), benched 2026-10-06 on this exact shape with a 43.6k greedy probe: decode
   29.21 t/s and draft acceptance 0.7123 vs the 1.0.13 control's 30.08 / 0.7549, and the control
   reproduced the recorded 0.755 exactly — greedy acceptance is deterministic, so the drop is a
@@ -352,7 +350,7 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   both pins accept. Re-test 1.2.0+ once the rotation change is untangled.
 - **FreeToken TP=2 on ROCm:** the `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201 —
   an untested edge of that PR, not a board fault.
-- **tcclaviger forks:** `--max-num-seqs 2` (`ConstraintViolationError` on `query_start_loc`);
+- **tcclaviger forks:** `--max-num-seqs 2` (`ConstraintViolationError` on `query_start_loc`)
   `--gpu-memory-utilization 1.00` unbootable, 0.982 is the law; expert prefetch is poison (36 %
   precision — deleting it was +40-51 % decode); TunableOp-with-offload corrupts numerics
   (repetition loops, early EOS) — autotuning that changes text is broken tuning; ngram-only
@@ -372,12 +370,12 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   `-ot` granularity the lever is COUNT.
 - **Never mount a diagnostic hook on `PYTHONPATH` without a marker guard:** `sitecustomize` is
   imported by every interpreter, a child inherits the hook and spawns again — a 19-level python
-  fork bomb inside the container. Kill with `podman rm -f` (the cgroup owns every generation),
+  fork bomb inside the container. Kill with `podman rm -f` (the cgroup owns every generation)
   never pattern-kill an interpreter name on this host; use `TORCH_LOGS` for dynamo questions.
 
 ## Contract (the wire)
 
-- **Live surface:** `/health` (== `/ping`), `/v1/models`, `/v1/chat/completions`,
+- **Live surface:** `/health` (== `/ping`), `/v1/models`, `/v1/chat/completions`
   `/v1/completions`, `/metrics`, `/stats`, `/sessions`, `/dashboard`. `/health` answers 200
   JSON (model id, `in_flight`, `queue_depth`) while serving and **503 only while stopping**
   (`core/server/admin.cpp:385`), and the listener binds *after* the full load (`main.cpp`:
@@ -414,12 +412,12 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
 - **Observability:** `/metrics` carries vLLM-compatible names plus `radiance:*`, scraped by the
   Dankclaw Prometheus job at `192.168.0.235:7770` over plain LAN routing (the netavark
   published-port quirk is same-host only). The boot ledger — the referee for every shape number
-  above — is in the container log (`podman logs frank-archon`);
+  above — is in the container log (`podman logs frank-archon`)
   `/home/dank/logs/frank_archon.log` is where the serve script redirects `podman run`.
 
 ## Files
 
-- Shape and law owner: `scripts/frank-archon.env` (measured arithmetic in its header),
+- Shape and law owner: `scripts/frank-archon.env` (measured arithmetic in its header)
   `scripts/serve-frank-archon.sh` (gates image / `.rad` size / template / GPU nodes, retires
   stale containers, `podman wait`), `scripts/frank-archon.service`, `scripts/ttm-preflight.sh`.
 - Payload doors: `scripts/download-frank-archon-iq4r.sh` (live), `scripts/download-frank-archon.sh`
@@ -427,22 +425,19 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   rebuild), `scripts/download-frank-archon-swift15.sh` (the BF16 source a w4dn8 re-convert
   needs), `scripts/hfget.sh` / `scripts/hf-fetch.sh`; template
   `scripts/qwen38-froggeric-v22.5.jinja`.
-- Convert campaign (dormant, kept for a future quant): `scripts/convert-swift15-w4dn8.sh`,
-  `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`, `scripts/run-swift15-bf16.sh`,
-  `scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`, `scripts/calib-push.py`,
-  `scripts/make-calib-corpus.py`, `scripts/derive-protected-experts.py`,
+- Convert campaign (dormant, kept for a future quant): `scripts/convert-swift15-w4dn8.sh`
+  `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`, `scripts/run-swift15-bf16.sh`
+  `scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`, `scripts/calib-push.py`
+  `scripts/make-calib-corpus.py`, `scripts/derive-protected-experts.py`
   `scripts/apply-protected-cast.sh`, `scripts/verify-swift15-ple.py`.
 - Probes: `scripts/probe-frank-archon-kv.py` (reads slots/window/dtype from the deployed env so
-  it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py`,
+  it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py`
   `scripts/autoresearch-frank-archon.sh` + `scripts/autoresearch/frank-archon-measure.py`.
-- Deploy and tests: `scripts/deploy.sh` / `scripts/deploy-remote.sh` (`deploy_llm_archon`),
+- Deploy and tests: `scripts/deploy.sh` / `scripts/deploy-remote.sh` (`deploy_llm_archon`)
   `tests/frank_archon_unit.bats` (the seams gating the KV-pool arithmetic, the VRAM pair and the
   ttm gate), `tests/changed_tests.bats`, `tests/deployment_scoping.bats`.
-- Retired-era leftovers in-tree: `scripts/patches/glm53-*.patch`,
+- Retired-era leftovers in-tree: `scripts/patches/glm53-*.patch`
   `scripts/patches/gpu_worker-kv-cap.patch` (fork archaeology; none applies byte-exact today).
 
 Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]],
-[[Dual-R9700-Cooperation]], [[deployment]], [[swift15-campaign-worklog]],
-[[dankserver-frontier-moe-feasibility]], [[x4-pair-experiment]], [[RDNA4 Container Serving]],
-[[frank-medium-radiance-mxfp4]], [[LLM/index]].
 ---

serverAppConfigs-docs/LLM/frank-dragoon.md

@@ -54,7 +54,7 @@ unauthenticated token port 8730 stays on container loopback. Engine stdout →
   ~20 GiB above the 20 GiB reserve, against 10–15 GiB of baseline co-tenancy (frank-zealot, ComfyUI,
   Caddy/observability).
 - Harness (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw)`): `contextWindow: 155648`
-  (= ceiling − 45k per the 400-zone law, never higher), `maxTokens: 45056`, `input: [text, image]`,
+  (= ceiling − 45k per the 400-zone law, never higher), `maxTokens: 45056`, `input: [text, image]`
   `reasoningContentField: "reasoning_content"`, `high→xhigh` fold, `extraBody` 1.0/0.95. Compaction
   rides [[frank-zealot]].
 - Deploy `./scripts/deploy.sh dankclaw --service frank-dragoon`; gate = boot-ledger greps (weights,
@@ -70,28 +70,19 @@ rollback plan — recoverable only until the trash is purged.
 
 | Window | Payload | Runtime | Shape | Replaced because | Rollback |
 |---|---|---|---|---|---|
-| 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | 2026-10-17 A/B at that shape: NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls + red-PNG probe identical, needle hit at both depths of a 21.5k haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s, PPL 3.968 vs 3.877 (finetune's edge gone: +2.3 %), KL vs the swift15 logits mean 0.155 / median 0.057 with top-1 agreement 85.1 % — two sound models disagreeing on content, not a corrupt forward pass, thinking 1,476 vs 784 mean, 7.4 GiB RAM back | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` |
+| 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | — | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` |
 | 2026-10-05 → 10-06 | peonist `halogen-qwen3.8-27b` `qwen3.8-27b-p1w4d-d2.hgn` 35,865,565,184 B `274c3fc7…` | halogen 0.1.4 `sha256:dc0a39a0…` | 1 slot × 200,700, DFlash2 drafter, `HALOGEN_CACHE_MB=16384`, text-only, censored base | flash-server v2 restored vision, abliteration and MTP | `/home/dank/models/halogen-qwen3.8-27b/` + `scripts/download-frank-dragoon-halogen.sh`; 0.1.4 digest still in podman storage |
-| 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | halogen card claimed 31.7 t/s vs 27.6–28.1 measured | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` |
+| 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | — | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` |
 | 2026-10-02 → 10-03 | aligned `Q8_0_ROCMFPX_AGENT.gguf` 28,214,425,952 B `f3ef2bdb…` | same fork pin | 1 × 262,144 f16 KV, `draft-mtp` 4 (MTP +123 % decode) | the uncensored file was faster *and* uncensored | `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` + `download-frank-dragoon-kingjones.sh` |
-| 2026-10-01 (one day) | MiMo-V2.6-Distill-Qwen-9B BF16 17.53 GiB + cross-model MTP from `Qwen3.5-9B` | vLLM 0.30 TheRock `kyuz0/vllm-therock-gfx1151` `sha256:98c1f3ac…` | 4 seqs × 262,144 on a scheduler-shared pool, 2 draft tokens | 6.19 t/s with MTP / 9.01 without vs gufo ~15, and no ≥6-bit vLLM quant path boots on gfx1151 | staged `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` until purged; fetch trio in `old/` |
-| 2026-09-28 → 10-01, again 10-01 → 10-02 | Swift-1.5 Uncensored `UD-Q8_K_XL` 31,457,990,272 B `9076f99a…`, then `UD-Q4_K_XL`; z-lab DFlash2 Q4_K_M | gufo v0.1.1 (`b0f8673`), source build | 1 × 262,144, `--draft-policy adaptive --draft-tokens 7`, no KV knob, no Jinja | ROCmFPX gave 8.39 bpw + MTP at 17 t/s | payloads **deleted 2026-10-03** → re-download `download-frank-dragoon-swift.sh` (+ `download-frank-dragoon.sh` for the drafter); tag `frank-dragoon-gufo-q4kxl-20261001`; `scripts/build-gufo.sh` |
+| 2026-10-01 (one day) | MiMo-V2.6-Distill-Qwen-9B BF16 17.53 GiB + cross-model MTP from `Qwen3.5-9B` | vLLM 0.30 TheRock `kyuz0/vllm-therock-gfx1151` `sha256:98c1f3ac…` | 4 seqs × 262,144 on a scheduler-shared pool, 2 draft tokens | — | staged `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` until purged; fetch trio in `old/` |
+| 2026-09-28 → 10-01, again 10-01 → 10-02 | Swift-1.5 Uncensored `UD-Q8_K_XL` 31,457,990,272 B `9076f99a…`, then `UD-Q4_K_XL`; z-lab DFlash2 Q4_K_M | gufo v0.1.1 (`b0f8673`), source build | 1 × 262,144, `--draft-policy adaptive --draft-tokens 7`, no KV knob, no Jinja | — | payloads **deleted 2026-10-03** → re-download `download-frank-dragoon-swift.sh` (+ `download-frank-dragoon.sh` for the drafter); tag `frank-dragoon-gufo-q4kxl-20261001`; `scripts/build-gufo.sh` |
 | 2026-09-21 → 09-28 | unsloth `Qwen3.8-27B-GGUF:UD-Q8_K_XL` 31.46 GiB + DFlash2 Q4_K_M 1.14 GiB | unsloth llama.cpp `b11030` prebuilt | 1 × 262,144, q8_0 KV, Qwen-Sharp v22.4.0 template | gufo: +37 % prefill shallow, +80 % at 131k, 4.5× faster load | `/home/dank/GitPackages/unsloth-llama-b11030/` on disk; payload deleted 2026-10-03 → `download-frank-dragoon-unsloth.sh` |
-| 2026-09-21 (same day) | unsloth `Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS` 87.3 GiB in 3 split parts | same prebuilt | 1 × 200,000, MTP depth 2 via explicit `-md` shared Q8_0 head | swapped to the 27B dense hours later | payload + prebuilt on disk at retirement; the parts ARE the payload (concatenating → `invalid split file name`) |
-| 2026-09-21 (same day) | `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` + 3 `.pfs` = 26.05 GiB | ciru-ai/ROCmFPX `kairic-edge-qwen38-27b-v1.2` @ `205a3e5` | 1 × 262,144, native MTP 4 + n-gram | same-day swap to the 27B dense | payload + runtime intact on disk at retirement |
-| 2026-09-19 → 09-21 | `jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca` | CIRU v4.4.1 (QSA window hotfix) | never written down — that runbook is a frontmatter stub | the stock llama.cpp prebuilt won | `scripts/download-frank-dragoon-orca.sh` |
 | 2026-09-11 → 09-15, again 09-16 → 09-19 | `peonist-ai/halogen-qwen3.8-flash-next` w4b 115.55 GiB + in-place-versioned `overlay.hgn` | halogen-flash-server 0.5.8, rootless Quadlet | 2 × 200,192 over a 400,384 pool, `MAX_TOK 24576`, `HOST_RESERVE 20`, vision OFF | displaced by CIRU Orca; the shape starved co-tenants | re-tag 0.5.8 + first-start `HALOGEN_DOWNLOAD` re-fetch into `/home/dank/models/halogen`; the Quadlet left the repo 2026-10-05, so the unit must be rewritten |
-| 2026-09-15 → 09-16 (one day) | `mradermacher/Qwen3.8-Flash-Next-Uncensored` i1-Q4_K_M 111 GiB + f16 mmproj | ik_llama.cpp `d85a1d7`, 3090 + CPU hybrid | 1 × 262,144, `-ot exps=CPU`, `GGML_CUDA_NO_PINNED_WEIGHTS=1` | 3.8–4.0 t/s: ~100 CPU↔GPU graph splits per token, GPU at 5 % SM | `/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M`, `/home/dank/GitPackages/ik_llama.cpp/build`, trio `scripts/.AGENTTRASH/frank-dragoon.{env,service}.ik-llama.cpp` + `serve-frank-dragoon.sh.ik-llama.cpp` |
-| 2026-08-30 → 09-11 | kingjones STRIX `Q4_0_ROCMFP4-STRIX.gguf` 113.47 GiB (51.2 B PLE via mmap) | ROCmFPX @ `36e9acd` | 2 × 200,704 over one 401,408 f16 window, spec off, f16 mmproj | halogen-flash-server ≈ 4× prefill | payload dir + `/home/dank/GitPackages/kingjones-ROCmFPX/build`; trio `.AGENTTRASH/frank-dragoon.{env,service}.llama.cpp` + `serve-frank-dragoon.sh.llama.cpp` |
-| 2026-08-25 → 08-29 (restored 08-29 → 08-30 after the Q8_0 AGENT file crashed before it ever served) | `kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP`: `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` 23.15 GiB (private ftype 114) + a **SEPARATE** MTP head `mtp-Qwen3.8-27B-Q4_0.gguf` 1.56 GiB via `--model-draft` + `mmproj-Qwen3.8-27B-Q8_0.gguf` 0.59 GiB | CIRU ROCmFPX llama.cpp (`build-promptforge`) | 1 slot at `-c 196608`, `-b 8192 -ub 8192 -ctxcp 32`, `-fa on --no-mmap`, f16 K+V, `draft-mtp` n-max 4 with the head on ROCm1 at f16 KV, QwenSharp template (`enable_thinking`, `preserve_thinking`) behind the `peg-native` + `deepseek` chat/reasoning pairing the Pi agent expected | the Q8_0 AGENT experiment, then the STRIX Q4_0 tier | `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/` (six card files + projector), runtime `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` |
-| 2026-08-29 → 08-30 (one day) | agentionai FAST-imatrix **`v2/` joined-PLE** file 87.06 GiB | charlie12345/ROCmFPX `build-qwen38` + `scripts/patches/qwen4exp-qsa-f32-mask.patch` | 2 × 200,704 / 401,408 f16 KV, no MTP head loads | kingjones STRIX on its own pinned fork | `/home/dank/GitPackages/ROCmFPX/build-qwen38` + the agentionai payload, both on disk |
-| 2026-08-18 → 08-25 | caf-eemin `Q4_0_ROCMI4.gguf` 13.50 GiB `ad51ecbe…` with embedded NextN/MTP | charlie12345/ROCmFPX, ROCmI4 W4A4 | one window, MTP `n4 / p0.60` | the 27B STRIX tier took the slot | re-fetch into `/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP/` |
 | through 2026-08-24 | CIRU ActiveFPX PromptForge v3: GGUF 15.44 GiB + 3 `.pfs` + mmproj ≈ 36.7 GiB | CIRU ROCmFPX `build-promptforge` | 1 × 262,144, native MTP depth 4, ngram off | eemin ROCmI4 | `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` + `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`; scripts `git revert 4c43839 7aee1df 40d50fe` or checkout `132e147` |
-| 2026-08-22 (same day) | `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` 15.48 GiB + 3 `.pfs` | ciru-ai/ROCmFPX tag `kairic-edge-qwen38-27b-v1` @ `e97b324` | one window, IU4 W4A4 lane, ubatch 512 | prefill 1.5–2.3× faster but prose decode 15–26 % slower → back to v3 the same day | `/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/` + `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server` |
-| 2026-08-21 (same day) | unsloth `UD-Q8_K_XL` 31.4 GiB + z-lab DFlash2 Q4_K_M 2.0 GiB | upstream llama.cpp PR #27342, `GGML_VULKAN=ON` | Vulkan0 — HIP has no `TOP_K`, so DFlash could not run there | slower than the CIRU runtime it replaced | both files in `/home/dank/models/`; the PR build must be rebuilt |
+| ≤ 2026-09 — 11 earlier eras: payloads, shapes and every measured number destroyed 2026-10-19 as obsolete | — | — | — | — | —
 
 Port note: this slot settled on **:7770** on 2026-08-22 (52625 before it); the 2026-08-17 fleet redo
-created it — earlier topologies in [[frank-redo-2026-08-17]].
+created it — earlier topologies in .
 
 ## Laws learned here
 
@@ -134,7 +125,7 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
   85–180 s until `HALOGEN_CACHE_MB=16384` was pinned. What the pool takes is what the table loses, so
   re-measure the warm ratio after any pool change.
 - **400-zone law:** the harness window MUST sit ~45k under the engine ceiling (`contextWindow:
-  155648` = 200,704 − 45,056) because the harness clamps `max_tokens` to `contextWindow − est − 4096`,
+  155648` = 200,704 − 45,056) because the harness clamps `max_tokens` to `contextWindow − est − 4096`
   its chars/4 `est` under-counts code/agent traffic by ~3 %, and compaction only fires at 85 %. The
   3 % is measured, not theory: 2026-10-07 at the old 200,192 ceiling, est 152,938 vs server 157,388 —
   over by 354, a 400. At the live shape compaction fires at 132,301 est ≈ 136k measured (the 180k
@@ -180,9 +171,9 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
   Gated-DeltaNet layers it also disables MTP's rollback path, −7…−15 %); DFlash across parallel slots
   (shared draft ctx spuriously emits EOG); `-ot`/`-ncmoe` on a hybrid split (disables mmap, page-locks
   ~108 GiB and stranded the host until a power cycle); `--no-mmap` on any PLE-table file (silent cgroup
-  OOM-kill); forcing PLE to CPU (23 → 13.4 t/s); `HALOGEN_FA_TQ=256` (no decode win, −9.5 % prefill);
+  OOM-kill); forcing PLE to CPU (23 → 13.4 t/s); `HALOGEN_FA_TQ=256` (no decode win, −9.5 % prefill)
   big ubatches as a decode knob (prefill-only, ub 512 fastest); Flash-Next MTP heads on the qwen4exp
-  loader (only `blk.48`, never the full per-layer norm set); mradermacher i1 GGUFs (NextN stripped);
+  loader (only `blk.48`, never the full per-layer norm set); mradermacher i1 GGUFs (NextN stripped)
   vLLM quant paths on gfx1151; a custom Jinja template on gufo (no Jinja engine — send the `terse` text
   as a client system message instead).
 - Build lore if a fork is rebuilt: purge Ubuntu `librocwmma-dev` (shadows AMD's headers, `-I` cannot beat
@@ -191,7 +182,7 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
 
 ## Contract (wire)
 
-- Live: `/v1/models`, `/health`, `/metrics`, POST `/v1/chat/completions`; `404`: `/ping`, `/props`,
+- Live: `/v1/models`, `/health`, `/metrics`, POST `/v1/chat/completions`; `404`: `/ping`, `/props`
   `/completion`, `/tokenize`. **No `Server-Timing` header and no `/completion` timing object** — measure
   wall-clock plus `usage.prompt_tokens`.
 - Thinking lands in `reasoning_content` and **counts against `max_tokens`**: a budget that dies
@@ -200,7 +191,7 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
   thinking tokens mean on the 2026-10-17 battery, against 784 for the retired Swift1.5 finetune (×1.9)
   and 905 for the censored 27B.
 - `reasoning_effort` accepts only `xhigh|medium|low` (anything else is a template error — the
-  `models.yml` fold stays) and there is **no `"none"`**: disable thinking with `enable_thinking:false`,
+  `models.yml` fold stays) and there is **no `"none"`**: disable thinking with `enable_thinking:false`
   which is also what the llama.cpp-era QwenSharp template wanted (`reasoning_effort:"none"` threw
   there too). `preserve_reasoning` is a 400 on this engine; the llama.cpp-era key was
   `chat_template_kwargs.preserve_thinking`, which decided whether `<think>` blocks stayed in recorded
@@ -214,7 +205,7 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
   model answers "video", ~2,040 tokens per image, `http(s)` image URLs refused on purpose.
 - The first request after a boot is a warmup artifact and may sit silent 45–60 s with 503
   `engine_unresponsive` then self-recover — do not restart over it; `/health` 503 during load is loading.
-  Flags read once at startup, all bitwise except `HALOGEN_CK_OVERLAY`, `HALOGEN_MATMUL_TUNING_FILE`,
+  Flags read once at startup, all bitwise except `HALOGEN_CK_OVERLAY`, `HALOGEN_MATMUL_TUNING_FILE`
   `HALOGEN_INDEXER_BUDGET`, `HALOGEN_COMPOSABLE_CONTEXT`.
 - Deploy waits for :7770 to be free: a rootless container binds it at CREATE and exits 126, and a
   leftover Quadlet beside the root unit guarantees the race.
@@ -225,14 +216,12 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
   `podman run`, ledger to `/home/dank/logs/frank_dragoon.log`) · `scripts/frank-dragoon.service`
   (`User=dank`, `LimitMEMLOCK=infinity`, `Restart=always`, `OOMScoreAdjust=-500`, `ExecStop=podman stop
   -t 60`) → `deploy_llm_dragoon` in `scripts/deploy-remote.sh`.
-- Fetchers: `download-frank-dragoon-v2abl.sh` (live); era rollbacks `-flashnext`, `-halogen`,
-  `-kingjones`, `-kingjones-uncensored`, `-swift`, `-unsloth`, `-q4`, `-orca`,
+- Fetchers: `download-frank-dragoon-v2abl.sh` (live); era rollbacks `-flashnext`, `-halogen`
+  `-kingjones`, `-kingjones-uncensored`, `-swift`, `-unsloth`, `-q4`, `-orca`
   `download-frank-dragoon.sh`; runtime pins `build-rocmfpx.sh`, `build-gufo.sh`.
 - Verification: `tests/frank_dragoon_unit.bats` (payload gates, deployer/serve parity, spec block, KV
-  band), `tests/llm_units.bats`, `tests/llm_secrets.bats`, `scripts/census-frank-dragoon.sh`,
-  `scripts/battery-frank-dragoon.py`, `scripts/stage-frank-dragoon-eval.sh`,
+  band), `tests/llm_units.bats`, `tests/llm_secrets.bats`, `scripts/census-frank-dragoon.sh`
+  `scripts/battery-frank-dragoon.py`, `scripts/stage-frank-dragoon-eval.sh`
   `scripts/trial-frank-dragoon.sh`.
-- Related: [[deployment]] · [[Model Swap and Rollback]] · [[frank-zealot]] · [[frank-archon]] ·
-  [[dankclaw-iommu-translation-off]] · [[frank-redo-2026-08-17]] · [[Dankclaw System Specs]] ·
-  [[DFlash DSpark Speculative Decoding]] · [[qwen3.8-spec-matrix]] · [[qwen3.8-strix-optimizations]].
+-Related:  ·  ·  ·  ·
 ---

serverAppConfigs-docs/LLM/frank-medium-batch-ubatch.md

@@ -1,227 +0,0 @@
----
-timestamp: 2026-09-17
-type: reference
-title: frank-medium — Batch and Ubatch Sweep
-description: "The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced."
-tags:
-  - benchmark
-  - llamacpp
-  - ubatch
-  - frank-medium
-  - dankserver
-  - layer-split
----
-  - retired-endpoint
-
-# frank-medium — Batch and Ubatch Sweep
-
-Run date: 2026-09-16. Harness: `scripts/bench-batch-ubatch.sh` (one
-`llama-bench` invocation per cell, a VRAM sampler per cell, the service stopped
-for the duration and restarted on exit). Tested at the deployed shape: 262144
-ctx, two slots, `--split-mode layer`, f16 KV, flash attention on, DFlash2
-loaded by the service but absent from the bench.
-
-The question was how large the logical batch and the micro-batch may be, and
-what each buys. The answer is that `--ubatch-size` has an **optimum** on this
-stack, that the logical batch does not matter at all, and that the optimum is
-much cheaper than what was deployed — which then moved the layer split.
-
-The deployment this sweep tuned (`frank-medium`, 2x R9700, llama.cpp layer
-split) is retired; its shape and boot ledger are
-[[frank-medium-llamacpp-dflash2]]'s, and the endpoint lineage continues on
-[[frank-archon]]. What this page owns is the sweep, the harness traps and the
-coupling law between batch and split.
-
-## The shape of the measurement
-
-A cell is a **64k prefill measured with 64k of KV already resident**, plus a
-128-token decode at that same depth:
-
-```
-llama-bench -m <UD-Q8_K_XL> -p 65536 -n 128 -d 65536 -b <b> -ub <ub> \
-            -ngl 999 -dev ROCm0/ROCm1 -sm layer -ts 36/29 \
-            -fa on -ctk f16 -ctv f16 -r 3 --no-warmup
-```
-
-`-d 65536` is what puts depth in front of the measurement: a `tg` instance runs
-with `n_prompt = 0`, and an unfilled `-d` would measure a cold cache. The depth
-fill itself is excluded from the timing (and cached across reps), so the number
-is "prefill a 64k prompt into a 64k conversation", which is the agent-loop shape
-rather than a cold start.
-
-Three `llama-bench` properties shaped the harness, all of them traps:
-
-- **No `-c`**: the context is `n_prompt + n_gen + n_depth`, so a bench can never
-  decide whether a configuration's buffers fit beside the *deployed* KV.
-- **A test that cannot allocate calls `exit(1)`**, taking the process with it —
-  hence one invocation per cell.
-- **`/` separates devices within a configuration and `,` separates sweep
-  values**, so `-ts 36,29` would have benched each card on its own.
-
-## Results — prefill, 64k depth (tok/s)
-
-| `--batch-size` | ub 256 | ub 384 | **ub 512** | ub 768 | ub 1024 | ub 1280 | ub 1536 | ub 1792 | ub 2048 |
-|---|---|---|---|---|---|---|---|---|---|
-| 2048 | — | — | **1193.9** | 1147.9 | 1151.9 | 1141.7 | 1170.2 | 1128.8 | 1077.5 |
-| 4096 | — | — | **1193.3** | 1129.6 | 1151.8 | 1143.7 | 1161.9 | 1151.9 | 1077.4 |
-| 6144 | 1046.0 | 1068.0 | **1192.9** | 1135.0 | 1151.9 | 1151.4 | 1165.0 | 1147.7 | 1077.6 |
-| 8192 | — | — | **1193.2** | — | — | — | — | — | — |
-
-Per-cell standard deviations are 0.5–2.3 tok/s, i.e. under 0.2 %. **Decode is
-15.87–15.89 tok/s on every single cell**, and the VRAM columns depend only on
-the ubatch (2048 / 4096 / 6144 at a fixed ubatch report identical free space).
-
-## Results — the same shape with no depth in front of it
-
-`N_DEPTH=0`, same 64k prompt, two points for contrast:
-
-| `--batch-size` | ubatch | prefill (tok/s) | decode (tok/s) |
-|---|---|---|---|
-| 6144 | 256 | 1543.5 ± 13.4 | 17.80 |
-| 6144 | 1536 | 1722.7 ± 6.2 | 17.80 |
-
-The **ubatch curve has the same shape at both depths** (below 512 is slower),
-and the two tables differ by a uniform ~32 %: that is what attending over 64k of
-resident KV costs. Depth is a tax on every configuration, not something the
-batch knobs can trade against.
-
-## What the data says
-
-- **`--ubatch-size` peaks at 512 and loses on both sides.** Below it the SMs are
-  not saturated (256 → 1046, 384 → 1068); at the top end 2048 is 10 % off the
-  peak (1077). The middle is a shallow basin: 768–1792 sit at 1130–1170.
-- **The curve is deterministic, not noisy.** 512 came out 1192.88 / 1193.23 /
-  1193.28 / 1193.88 across four different batch values, 1024 came out 1151.77 /
-  1151.91 / 1151.94, and 2048 came out 1077.38 / 1077.54 / 1077.59. Three
-  independent batch columns agreeing to 0.4 tok/s is what makes the 2.4 % gap
-  between 512 and 1536 real rather than run-to-run drift.
-- **The logical `--batch-size` is irrelevant here.** At a fixed ubatch, 2048,
-  4096, 6144 and 8192 land within 2 % of each other — and use *identical* VRAM,
-  because the compute buffers are sized by the ubatch alone.
-- **Decode does not move at all.** 15.88 tok/s at 64k depth on every cell and
-  17.80 at depth 0, to ±0.05. Whatever this knob does, it does it to prefill.
-
-## What it costs at the deployed shape
-
-Prefill rates do not decide a deployment; VRAM does. The same shape
-re-measured at 262144 over two slots (load, poll `/health`, read `rocm-smi`,
-kill — no requests):
-
-| Split | ubatch | GPU 0 free | GPU 1 free | gap |
-|---|---|---|---|---|
-| 36,29 — deployed before | 1536 | 4.71 GiB | 4.91 GiB | 0.20 GiB |
-| 36,29 | **512** | 6.01 GiB | 7.32 GiB | 1.31 GiB |
-| **35,30 — deployed after** | **512** | **6.44 GiB** | **6.90 GiB** | **0.46 GiB** |
-
-Two things follow.
-
-**The ubatch move is free money.** 1536 → 512 gives back 1.30 GiB on ROCm0 and
-2.41 GiB on ROCm1 — 3.71 GiB in total — and the mean throughput goes *up*, not
-down. The asymmetry is the interesting part: ROCm1 pays nearly twice what ROCm0
-does per unit of ubatch, because it holds the output layers and the drafter's
-graphs.
-
-**And it moves the balance, so the split had to follow.** Those uneven
-give-backs left ROCm1 1.31 GiB lighter, and one layer is one `-ts` step: 36:29
-became 35:30, which lands at 0.46 GiB — a *single DeltaNet layer's* worth of
-asymmetry, i.e. the floor on how well any split can balance this stack (0.43 GiB
-per DeltaNet layer, 1.43 per attention layer). The 0.20 GiB gap of the old
-configuration is not reachable at the new batch; 0.46 GiB is.
-
-## The decision
-
-```ini
-BATCH_SIZE=6144      # throughput-neutral and VRAM-neutral; left at the old value
-UBATCH_SIZE=512      # the measured optimum: fastest AND cheapest
-TENSOR_SPLIT=35,30   # rebalanced for the new ubatch, measured not interpolated
-```
-
-Verified live after the deploy: `--tensor-split 35,30 --ubatch-size 512`, two
-slots of 131072, `6.44 / 6.90 GiB` free — matching the pre-deploy measurement to
-0.01 GiB — a tool call and two concurrent requests answered, draft acceptance on
-both slots (0.63 / 0.48 on slot 1, 0.39 on slot 0), and no ROCm errors.
-
-## The split: re-measure it at the deployed shape
-
-The batch and the tensor split are **coupled**: compute buffers are not
-symmetric across the two cards, so any batch change moves the balance point.
-Order of work — sweep batch with llama-bench (above), THEN re-measure the
-split at the DEPLOYED shape (262144, two slots, drafter loaded) by loading
-the real service with a per-config env override and reading `rocm-smi`. A
-llama-bench number cannot decide the split: it has no `--parallel`, sizes
-its own context as `n_prompt + n_gen + n_depth`, and cannot load the drafter.
-
-The settle gate is mandatory (above); wait for BOTH cards under ~1 GiB AND
-the port to be free (`ss -ltn | grep :7770` until gone) before the next
-load, or the transient server dies with `couldn't bind HTTP server socket`.
-Free bytes follow `card = weights_share + KV_share + constant + ubatch_term`,
-where the ubatch term costs the drafter-carrying card roughly **1.85×** the
-other (measured 2.41 GiB vs 1.30 GiB going 1536 → 512). Balance by
-measurement only: one `-ts` step is one layer, worth ~0.43 GiB on a DeltaNet
-boundary and ~1.43 GiB on an attention one, so the reachable minimum gap is
-about one DeltaNet layer, never zero. Reference: 36,29 @1536 → 4.71/4.91;
-36,29 @512 → 6.01/7.32; **35,30 @512 → 6.44/6.90 (the shape that deployed)**.
-
-Re-fit both ubatch coefficients in `tests/frank_medium_unit.bats` at the new
-deployed point and re-anchor the per-card constants; check neighbouring
-splits still FAIL the balance gate or the seam has stopped discriminating
-(keep the 1.5 GiB/card floor and the one-DeltaNet-layer slack). Deploy,
-then verify live: flags in `/proc/<pid>/cmdline`, `/slots`, VRAM matching
-the pre-deploy measurement, a tool call, two concurrent requests,
-`draft acceptance` lines on BOTH slots, 0 ROCm errors.
-
-**Per-card utilization (who is hot, and the split lever)** — a different
-question from batch/prefill. Measure decode tok/s AND per-card
-`rocm-smi --showuse` at 2–3 neighbouring splits in ONE session. Pitfalls
-(each cost a re-run): run the transient unit as ROOT (a `pkill -9
-llama-server` as dank auto-restarts the service and `systemctl` as dank
-fails "Access denied"); warm up 3 throwaway requests before the timed one
-(first 384-token decode after load measured 1.77 tok/s against a true ~32);
-`rocm-smi --showmeminfo vram --csv` columns are `$2`=Total, `$3`=**Used** —
-parsing the wrong column spins the settle gate the full 600 s; the util% is
-sampled post-hoc (which card finished last), so treat it as "which card is
-the pipeline tail", not a duty cycle.
-
-**What the split does and does not buy (measured 2026-09-17, 393216 /
-MTP n=3):** the split is NOT a decode-throughput lever — 34:31, 35:30, 36:29
-all sit in the same ~32 tok/s band. The x1 card runs hot (~91–97 %) at
-35:30 because the MTP drafter + output layers + heavier compute buffers are
-pinned to the card holding the LAST layers — not because PCIe is the
-bottleneck (~5.2 MB of boundary activation crosses per 512-token ubatch).
-36:29 makes the x16 card the utilized worker (42 %/3 %) at no throughput
-cost but spends headroom (0.45 GiB free, under the floor) — Frank's standing
-preference is that the card with the FASTEST PCIe lane should be the more
-utilized one; deploying it requires re-fitting the seam's slack/floor and
-re-checking that 35:30 and 37:28 still fail so the gate keeps discriminating.
-
-## Limitations
-
-- **`llama-bench` cannot run the drafter, the slots or the deployed context.**
-  Its decode figure is the *drafterless* baseline (15.9 at 64k depth against the
-  service's ~44 with DFlash2 + ngram-mod), and it never exercises `--parallel` or
-  the 16 GiB KV of a real 262144 deployment. The throughput comparison between
-  batch values is sound; the absolute decode numbers are not the service's.
-- **Five cells of the intended grid were not run** (batch 8192 above ubatch 512).
-  Batch 8192 at ubatch 512 matches the other batch columns to 0.4 tok/s, and the
-  sweep was stopped deliberately once the curve was established.
-- **A totals model cannot see contiguity.** The earlier DSpark-era failure at
-  8192/2048 was a 2020 MiB pp buffer that found no room while every total looked
-  fine. Nothing here rules that in or out; it only shows there is no reason to
-  want a batch that large.
-- **A measurement like this needs a settle gate.** Re-loading the model
-  immediately after a kill fails with `unable to allocate ROCm0 buffer` — the
-  previous instance's ~26 GiB is not returned instantly — which reads exactly
-  like a configuration that does not fit. Wait for `rocm-smi` to show both cards
-  near-empty before the next load.
-
-## See Also
-
-- [[frank-medium-llamacpp-dflash2]] — the deployment this tuned, and its split
-  history
-- [[frank-zealot]] — the same sweep on a CUDA 3090, where a 16k prompt
-  at depth 0 made prefill climb all the way to ubatch 2048. Depth is the
-  difference: with no KV to attend over, per-pass overhead dominates and a
-  bigger micro-batch wins; at 64k of resident KV it does not.
-- [[Dual-R9700-Cooperation]] — why the cards are layer-split at all
-- [[Dankserver System Specs]] — hardware inventory---
-timestamp: 2026-09-17

serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md

@@ -1,734 +0,0 @@
----
-type: reference
-title: frank-medium llama.cpp Runbook (DFlash2 drafter)
-description: "Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on."
-tags:
-  - frank-medium
-  - dankserver
-  - llama-cpp
-  - dflash2
-  - dspark
-  - qwen35
-  - gfx1201
-  - r9700
-  - layer-split
-  - retired-endpoint
-timestamp: 2026-09-20
----
-
-> **SUPERSEDED 2026-09-20.** Replaced by [[frank-medium-radiance-mxfp4]]
-> (radiance TP=2, measured faster on every axis). The layer-split shape below is
-> the rollback target, and the llama.cpp measurements in this page stay valid for
-> it — but nothing here is the deployed shape any more.
-
-# frank-medium — llama.cpp (DFlash2 drafter)
-
-The frank-medium deployment from 2026-09-16 to 2026-09-20 (model swapped to
-orcarouter Q8_0 on 2026-09-19): **mainline llama.cpp**
-serving `orcarouter/Qwen3.8-27B-Uncensored-GGUF` at `Q8_0`
-with the **z-lab DFlash2** `Q4_K_M` block-diffusion drafter, as **one process across both
-Radeon AI PRO R9700 cards**, layer-split at a balanced 32:33. The predecessor
-model (ajgazin Swift-Qwen3.8-27B UD-Q8_K_XL, 35:30 skew, 393216 pool) is the
-calibration history in the sizing tables below. DSpark was tried in the drafter's
-place on 2026-09-16 and reverted — see *The DSpark excursion*.
-It replaced the vllm-radiance MXFP4 replica pair + routing proxy
-([[frank-medium-radiance-mxfp4]] holds that record, and the PCIe-atomics
-evidence) the same day the pair went live — and was itself replaced by
-radiance TP=2 on 2026-09-20, once the board swap made a collective legal.
-During its window it was the box's **only** LLM runtime: the replicas, the
-proxy and their unit files were retired.
-
-## Why one process and layer split
-
-The endpoint's whole history is one constraint: **the two cards could not take
-part in an RCCL collective.** RCCL's kernels declare a hidden hostcall buffer,
-and ROCr refuses to supply it to a device without PCIe atomics; on the previous
-board the second R9700 hung off the B650 chipset (`AtomicOpsCap: Routing-`,
-read-only), so every collective that touched it died with
-`hipErrorIllegalState`. That ruled out tensor parallelism — and it ruled out
-**vLLM's pipeline parallelism too**, whose stage handoff is a `send`/`recv` of
-the hidden state on the device backend
-(`vllm/distributed/parallel_state.py`), the very primitive that failed. The
-full matrix is in [[Dual-R9700-Cooperation]].
-
-**That gate cleared on 2026-09-20** with the ProArt B850-Creator swap: both
-cards are on CPU root ports, every bridge reads `Routing+`, both endpoints read
-`ReqEn+`. Nothing has dispatched a collective since, so tensor parallelism is
-*unblocked, not proven* — and this service still runs the shape below, which
-needs no collective either way.
-
-llama.cpp's `--split-mode layer` needs **no collective at all**: layers are
-distributed across the cards in order and the boundary activation is a plain
-per-backend copy, once per forward pass. Measured on this host with the
-predecessor runtime (llama-bench, pp4096 / tg128, f16 KV, flash-attn on):
-
-| Split | pp4096 | tg128 |
-|---|---:|---:|
-| one R9700 (`-sm none`) | 1160.9 | 19.42 |
-| **two cards, layer (`-sm layer`)** | **1991.2** | **18.76** |
-| two cards, tensor (`-sm tensor`, no RCCL: butterfly fallback) | 271.8 | 26.26 |
-
-Tensor split buys ~40 % decode and loses 7× prefill. The workload here is
-prefill-heavy (agentic, long prompts, reasoning traces), so layer split is the
-shape — prefill nearly doubles and one model serves from both cards.
-
-## Hardware
-
-| Property | Value |
-|---|---|
-| GPUs | 2x AMD Radeon AI PRO R9700 (DID 0x7551), gfx1201 (Navi 48, RDNA4) |
-| VRAM | 34,208,743,424 B (31.86 GiB) each, as `rocm-smi` reports it |
-| Link | PCIe 5.0 **x8** each on CPU GPP ports (x8/x8 bifurcation, ~31.5 GB/s per card), no XGMI; atomic routing enabled on both chains since the 2026-09-20 board swap — see [[Dankserver System Specs]] |
-| iGPU | Raphael (gfx1036) enumerates as HIP device 2 — hidden via `HIP_VISIBLE_DEVICES=0,1` |
-| CPU / RAM | Ryzen 7 7700X (8c/16t), 60 GiB RAM, 63 GiB swap |
-
-## Runtime
-
-Mainline llama.cpp at **`fb27a525`** (2026-09-16), built for **gfx1201 only** by
-`scripts/build-llama-rdna4.sh` against a private ROCm `7.14.0a20260624`
-toolchain (TheRock nightly) that is **bundled next to the binary** —
-`RPATH=$ORIGIN/../lib`, no `LD_LIBRARY_PATH`, nothing from the host's ROCm 7.1
-packages.
-
-Two features pin that ref, and neither is in the ROCmFPX fork the endpoint used
-earlier the same day (its tip, `c49ebdb`, predates both):
-
-- **DFlash2** (`ggml-org/llama.cpp#27342`): grouped dynamic depthwise
-  convolution plus the candidate selector, detected from the draft GGUF
-  (`selector_top_k > 0`) and served through the ordinary
-  `--spec-type draft-dflash` path.
-- **qwen35 MTP**: the `draft-mtp` implementation covers this architecture's
-  single trained head.
-
-Build with:
-
-```bash
-bash scripts/build-llama-rdna4.sh     # ~25 min on this box, JOBS=$(nproc)
-```
-
-> **Bundling trap, measured.** rocBLAS and hipBLASLt look for their device code
-> at a fixed depth under the binary's lib dir (`../lib/rocblas/library`,
-> `../lib/hipblaslt/library/<arch>`). `cp -r SRC DST/` with a missing `DST`
-> creates `DST` as a copy of `SRC` and **flattens it one level**, and the
-> failure is invisible at startup: the engine loads 29 GiB of weights, answers
-> `/health`, and dies on the **first request** with
-> `rocBLAS error: Cannot read .../lib/rocblas/library/TensileLibrary.dat: No
-> such file or directory for GPU arch : gfx1201`. The build script now declares
-> the destination and copies contents.
-
-## Model and drafter
-
-| File | Bytes | Role |
-|---|---:|---|
-| `Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf` | 31,457,990,112 | target, 29.29 GiB |
-| `Qwen3.8-27B-DFlash2-Q8_0.gguf` | 2,056,414,816 | DFlash2 drafter, 1.92 GiB |
-
-Fetched by `scripts/download-frank-medium-gguf.sh` (resumable, byte-verified —
-a truncated 31 GB GGUF is otherwise a load-time crash).
-
-The target is Unsloth Dynamic 3.0 (per-tensor types from Unsloth's own quant of
-the same size, on Unsloth's importance matrix) of an abliterated Swift
-fine-tune: orcarouter's single refusal direction projected out of 131 tensors,
-vision tower included. Vision has been ON since 2026-09-17
-(`--mmproj`); this is one of the fleet's two image endpoints.
-
-**The file also carries the model's MTP head** (`blk.64.nextn.*`), and since
-2026-09-17 that is the deployed drafter: `--spec-type draft-mtp` builds the
-draft context against the main model's own weights (`shares_model`), so there
-is no second copy of the model and no `--model-draft` argument — the cost is
-one nextn layer's KV (4096 B/position) plus small buffers on the card that
-holds the last layers (ROCm1). The DFlash2 sidecar stays on disk and in the
-fetch script: `SPEC_TYPE=draft-dflash` (with the sidecar back on the command
-line) brings the measured-faster chain back, at the price of ~1.1 GiB on ROCm1.
-
-## Deployment
-
-| File | Role |
-|---|---|
-| `scripts/frank-medium.env` | every tunable, with the sizing arithmetic in comments |
-| `scripts/serve-frank-medium.sh` | env-driven llama-server launcher; no arguments |
-| `scripts/frank-medium.service` | ROOT systemd unit, `User=dank`, `WantedBy=multi-user.target` |
-
-```bash
-./scripts/deploy.sh dankserver --service frank-medium
-```
-
-The handler verifies the runtime and payload exist (and refuses otherwise),
-retires the three vLLM quadlets into `~/.AGENTTRASH/20260916-vllm-radiance-retire/`,
-waits for **port 7770 to be released** (the retired proxy held it on the host
-network), relinks and restarts the unit, and gates on
-`http://127.0.0.1:7770/health`.
-
-Deployed flags:
-
-| Flag | Value | Why |
-|---|---|---|
-| `--split-mode` / `--device` / `--tensor-split` | `layer` / `ROCm0,ROCm1` / `32,33` | both cards, one model, no collective. Balanced (operator call 2026-09-19): 65 layers split as evenly as they can be, ROCm1 (the last-layers card that also carries the drafter and output layers) holding the extra one. The predecessor's 35:30 skew (2026-09-17, part 3) leaned work onto the x16 card because ROCm1's Gen4 x1 chipset path made it the 98 % bottleneck; the balanced point trades that lean for symmetry. Split granularity is one LAYER — 0.43 GiB on a DeltaNet boundary, 1.43 on an attention one |
-| `--gpu-layers` / `--fit` | `999` / `off` | everything on the cards; auto-fit must not resize a context we set |
-| `--ctx-size` / `--parallel` | `327680` / `2` | the pool over two slots of 163840; `--parallel` **divides** the context. 163840 is inside the checkpoint's trained 262144 window, so the whole pool is trained territory — no rope extrapolation (unlike the predecessor's 393216) |
-| `--batch-size` / `--ubatch-size` | `6144` / `512` | measured: ubatch 512 is the prefill optimum *and* the cheapest point on the curve; the logical batch is throughput- and VRAM-neutral. See [[frank-medium-batch-ubatch]] |
-| `--ctx-checkpoints` / `--cache-ram` | `0` / `16384` | no per-slot KV snapshots (the build's default is **32** — this is a deliberate deviation, not an inherited default); the RAM-side prompt cache is capped at 16 GiB, which is what lets a full 163840-token slot state be cached at all (2026-09-18, see below) |
-| `--flash-attn` / cache types | `on` / `f16` K and V | an agentic checkpoint is the wrong place to spend KV precision |
-| `--spec-type` / `--spec-draft-n-max` | `draft-dflash` / `7` | the z-lab DFlash2 `Q4_K_M` sidecar (1.06 GiB), the measured throughput winner (43.5 tok/s vs the model's own MTP head at 32.4); n_max 7 is the DFlash2 optimum. The target file also carries its own MTP head (`nextn_predict_layers=1`), so `SPEC_TYPE=draft-mtp` is one change away — the drafter file stays on disk |
-| `--spec-draft-device` | `ROCm1` | the drafter rides the card that ends up holding the target's last layers |
-| `--temp` / `--top-p` / `--top-k` | `0.7` / `0.95` / `20` | the values the vLLM predecessor served on this endpoint |
-| `--reasoning-format` / `--reasoning-budget` | `deepseek` / `-1` | trace into `reasoning_content`, unlimited |
-| `--mmproj` | `mmproj-Qwen3.8-27B-Uncensored-f16.gguf` (beside the target) | vision enabled; this build has NO autoload, so the flag is explicit and the serve script refuses to start without the file |
-
-## The DSpark excursion (2026-09-16, tried and reverted)
-
-The endpoint went live on **z-lab DFlash2**, was switched to
-**magnitudedev/Qwen3.8-27B-DSpark** for a few hours the same day, and was put
-back. Both are 5-layer block-diffusion sidecars, and their differences are what
-decided it:
-
-| | DFlash2 (deployed) | DSpark (tried, reverted) |
-|---|---|---|
-| file | 2,056,414,816 B — 1.92 GiB | 1,455,376,576 B — 1.36 GiB |
-| draft depth | `block_size 8` -> `n_max 7` | `block_size 7` -> `n_max 6` (llama.cpp asserts `n_max <= block_size - 1`) |
-| attention | `sliding_window 2048`, `causal false`, candidate selector (rank 256, top-k 16) | **no sliding window**; a Markov head in place of the selector |
-| draft KV at 262144 | ~40 MiB — the window bounds it | **5.0 GiB** — the whole context |
-
-> **A fetch-script bug this threw up.** The DSpark file's size was recorded as
-> 1,359,284,737 when it was fetched — 96 MB short of the Hub's 1,455,376,576 —
-> and that wrong number became the fetch script's declared size, so
-> `--verify-only` failed on a payload that was in fact byte-correct. Both
-> drafters' sha256 have since been checked against the Hub's LFS oids and match.
-
-**The draft context is sized from the target's.** `cparams.n_ctx =
-llama_n_ctx(ctx_tgt)` in `common/speculative.cpp`, with no flag to cap it
-independently, so a drafter that declares no sliding window caches the entire
-target window on whichever device `--spec-draft-device` names. DSpark's KV is
-5 layers x 2 x 8 kv heads x 128 dim x 2 B = 20 KiB per position.
-
-That cost has to be paid somewhere, and the failure was instructive: at a 1:1
-split the model **loaded fine, answered `/health`, and then died on the first
-draft decode** with `ROCm error: out of memory` inside
-`common_speculative_impl_draft_dflash::process` — ROCm1 sat at 33.9 GB of
-34.2 GB. A unit that crash-loops with an allocation failure *after* a
-successful load is a sizing failure, not a broken drafter.
-
-Read `sliding_window` out of a drafter's GGUF header before sizing a
-spec-decode deployment — 40 lines of `struct` parsing is enough, and the
-flag-level configuration looks identical either way.
-
-### The split moved twice: once for the batch, once for the drafter
-
-**Under DSpark.** Tripling the batch (2048/512 -> 6144/1536) made the two cards
-diverge: at 262144 the bigger `--ubatch-size` cost **ROCm1 ~3.3 GiB against
-ROCm0's ~1.7 GiB** — ROCm1 holds the output layers *and* the drafter's graphs —
-which left the draft-carrying card at 0.08 GiB free against ROCm0's 3.78 under
-the 38:27 split that had been balanced at the smaller batch:
-
-| `-ts` | GPU 0 free | GPU 1 free |
-|---|---:|---:|
-| 38,27 | 3.78 GiB | 0.08 GiB |
-| 41,24 | 1.47 GiB | 2.38 GiB |
-| 40,25 | 1.88 GiB | 1.88 GiB |
-
-**Then DSpark came off**, which frees ~4.3 GiB on ROCm1 (5.0 GiB of draft KV
-out, 0.65 GiB of heavier DFlash2 weights in) and moves the balance the other
-way. Re-measured at the same batch, the same two slots and the same window:
-
-| `-ts` | GPU 0 free | GPU 1 free | gap |
-|---|---:|---:|---:|
-| 39,26 | 2.42 GiB | 7.20 GiB | 4.78 |
-| 38,27 | 3.85 GiB | 5.77 GiB | 1.92 |
-| 37,28 | 4.28 GiB | 5.34 GiB | 1.06 |
-| **36,29 — deployed** | **4.71 GiB** | **4.91 GiB** | **0.20** |
-| 35,30 | 5.14 GiB | 4.48 GiB | 0.66 |
-| 34,31 | 6.57 GiB | 3.05 GiB | 3.52 |
-
-**One `-ts` step is one layer, and this stack's layers are not the same size.**
-qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is worth
-**~0.43 GiB** on a DeltaNet boundary and **~1.43 GiB** on a full-attention one
-(0.45 GiB of weights plus ~1.0 GiB of its own f16 KV at 262144). Both fell out
-of that table — 37,28 -> 38,27 moves a DeltaNet layer, 38,27 -> 39,26 an
-attention layer. Balance by measurement, never by interpolating.
-
-**Then the batch moved, and the split followed it** (2026-09-16, the same day
-— see [[frank-medium-batch-ubatch]]). 36:29 was the balance at ubatch **1536**;
-the sweep showed 512 is both faster and 3.71 GiB cheaper, and — because the
-compute buffers cost ROCm1 nearly twice what they cost ROCm0 — shrinking them
-handed ROCm1 1.31 GiB more than it handed ROCm0:
-
-| `-ts` | ubatch | GPU 0 free | GPU 1 free | gap |
-|---|---:|---:|---:|---:|
-| 36,29 | 1536 | 4.71 GiB | 4.91 GiB | 0.20 |
-| 36,29 | 512 | 6.01 GiB | 7.32 GiB | 1.31 |
-| **35,30 — deployed** | **512** | **6.44 GiB** | **6.90 GiB** | **0.46** |
-
-35:30's 0.46 GiB gap is one DeltaNet layer's worth, which is the floor here: no
-split can be tuned finer than a layer, so the 0.20 GiB of the old configuration
-is not reachable at this batch. It clears the operator's 1.5 GiB-per-card floor
-by ~5 GiB, so no headroom is bought out of `CTX_SIZE` and the model's full
-262144-token window stays.
-
-A batch of 8192/2048 failed here under DSpark — its 2020 MiB pp buffer could not
-be allocated on ROCm1, which had 0.08 GiB free. That was a *contiguity* failure,
-not a total-bytes one, and the sweep has since removed the reason to care: with
-the batch geometry measured, there is nothing to gain by making either knob
-larger, so the question is closed rather than pending.
-
-> **A fault to watch.** A 39:26 split, at 1x batch and one slot, died on its
-> eighth request: the amdgpu driver logged SDMA0 UTCL2 page faults and the
-> engine surfaced `ROCm error: an illegal memory access was encountered` ->
-> `ggml_abort` — a fault in the cross-card boundary copy, which is this layer
-> split's hot path. It has not reproduced (36:29 served the whole verification
-> battery and the e2e gate, and 35:30 at ubatch 512 is the deployed shape now;
-> 39:26 itself was only *loaded* at the deployed batch, not exercised), and it
-> is **not** an OOM: no amount of headroom would prevent it.
-
-## 2026-09-17: the pool grows to 327680, the split moves to 36:29, vision comes on
-
-The window grew from 262144 to a **327680-position pool** (163840 per
-conversation over the two slots), the split moved one layer to ROCm0
-(35:30 → 36:29), and the **BF16 mmproj** (888 MiB) came on.
-
-**Why 36:29, not the balance.** At this context the fit model puts the balance
-point at 34:31 (0.11 GiB gap). The operator moved a layer to ROCm0 to buy
-headroom on ROCm1 — the card that carries the drafter, its KV, the output
-layers and the heavier compute buffers — accepting a 3.15 GiB gap by
-construction:
-
-| Card | Fit | Measured at load | Headroom |
-|---|---:|---:|---:|
-| ROCm0 (36 layers + mmproj) | 29.19 GiB | 28.40 GiB | 3.46 GiB |
-| ROCm1 (29 layers + drafter) | 26.04 GiB | 26.66 GiB | 5.20 GiB |
-
-Both clear the 1.5 GiB floor with room; the fit lands within 0.8 GiB on both
-cards, which is the accuracy it has always had (at the 35:30 point it
-reproduced ROCm1 to 0.00 and ROCm0 to 0.86). The seam's slack is the 3.15 GiB
-gap rounded up, with the trade written in its comment.
-
-**The rope caveat.** The GGUF carries no `context_length` and no rope-scaling
-keys (freq_base 1e7, `dimension_sections [11,11,10,0]`); the checkpoint's
-trained window is 262144. Positions beyond that — the top 65k of the pool —
-extrapolate on the base rope. It runs, and 1.25x is a modest ask of a rope,
-but it is untrained territory, and the seam's ctx cap encodes 327680 with
-that note.
-
-**Vision: this build has no mmproj autoload.** The first 327680/36:29 deploy
-came up healthy and *text-only*: image requests answered 500 "image input is
-not supported — hint: … provide the mmproj" and the load log had no mmproj
-line, so `llama-server` fb27a525 does not pick up `mmproj*.gguf` from the
-model directory. The serve script now passes `--mmproj` explicitly (the
-frank-zealot pattern), refuses to start without the file when vision is on, and
-the deploy handler's preflight checks for it. Verified end-to-end after the
-fix: a generated 64x32 test image was described from its pixels — with
-thinking on the model reasoned about "the three horizontal bars" (its reading
-of the crude glyph) and with thinking off answered the letter question in one
-token.
-
-The e2e crawl spec flaked once on the first 327680 deploy (1 failed / 31
-passed) and passed 32/32 on the redeploy; the service itself was healthy
-throughout (health gate green, `NRestarts=0`).
-
-## 2026-09-17 (part 2): the drafter switches to MTP, the pool grows to 393216
-
-The operator asked for **2x196k contexts**, which the DFlash2 sidecar's
-~1.1 GiB on ROCm1 was eating. The drafter switched to the model's own
-**MTP head** (`SPEC_TYPE=draft-mtp`, `n_max 3` — the measured MTP winner from
-the same-day A/B: 32.4 tok/s at n_max 3 vs 26.7 at n_max 7, against DFlash2's
-43.5), the pool grew 327680 → **393216** (196608 per conversation), and the
-split moved one layer back to **34:31**, the balance point at the new pool.
-
-| Card | Fit | Measured at load | Headroom |
-|---|---:|---:|---:|
-| ROCm0 (34 layers + mmproj) | 29.77 GiB | 28.91 GiB | 2.95 GiB |
-| ROCm1 (31 layers + MTP buffers) | 28.32 GiB | 30.78 GiB | 1.08 GiB |
-
-The fit predicted 28.32 on ROCm1; the load came in **2.46 GiB hotter** than
-predicted. The MTP charge is one nextn layer's KV (4096 B/position — the
-draft context is sized from the target, same law as the DFlash2 one) plus
-draft buffers the fit only estimated. The seam was recalibrated to the
-measured value: the MTP buffer constant is now 2,901,231,913 B (the 4.20 GiB
-total MTP charge on card 1), and the floor dropped 1.5 → **1.0 GiB** for this
-shape — ROCm1's 1.08 GiB of headroom is real. 35:30 failed card 0's floor
-only at the OLD 1.5 GiB bar (it clears 1.0 with 1.27 GiB to spare); 36:29
-fails both gates. Under the recalibrated 1.0 GiB floor 35:30 became
-reachable — and the split went there the same day (part 3).
-
-The rope caveat deepens: the top 131k of a conversation (196608 vs the
-trained 262144… i.e. 393216/2 = 196608 per slot, so the top 131,072 positions
-of a FULL conversation) extrapolates on the base rope. 1.5x is a bigger ask
-than 1.25x; the seam's ctx cap encodes 393216 with that note.
-
-Verified after deploy: `adding speculative implementation 'draft-mtp'` with
-`shares_model` in the load log, no `--spec-draft-model` flag, `--mmproj`
-present; 207 drafts / 108 accepted in a text run (52 % acceptance, mean len
-~1.7); vision, tools and `reasoning_effort` all pass; e2e 32/32; `NRestarts=0`.
-
-## 2026-09-17 (part 3): the PCIe topology, and the split leans to 35:30
-
-`lspci -vv` on dankserver answers why the split now leans toward ROCm0:
-the two cards do not sit on equal roads.
-
-| Card | Path | Bottleneck link |
-|---|---|---|
-| ROCm0 `03:00.0` | CPU root port `00:01.1` (Gen4 x16) -> switch -> card | Gen4 **x16** (~31.5 GB/s) |
-| ROCm1 `0b:00.0` | CPU root port `00:02.1` (Gen4 x4) -> chipset -> `06:09.0` -> `09:00.0` -> card | Gen4 **x1** (~1.97 GB/s) |
-
-With the MTP drafter, the output layers and the heavier compute buffers
-all pinned to the last-layers card, live `rocm-smi` under traffic read
-**98% busy on the x1 card vs 40% on the x16 card**. The boundary crossing
-itself is cheap (~5.2 MB of activation per 512-token ubatch rides the
-wire — a few ms at 1.97 GB/s against a ~430 ms ubatch), but the
-saturated card should not be the one behind the narrowest link. The
-operator moved one layer to ROCm0: **34:31 -> 35:30**. The moved layer
-is 34 — the 4k+2 position of block 8, a DeltaNet — which is what keeps
-the shift cheap (~0.43 GiB).
-
-Fit at the new shape: card 0 30.59 GiB (free 1.27), card 1 30.01 GiB
-(free 1.85), gap 0.58 — inside the 1.0 GiB floor and the 1.483 GiB
-slack. The next step, 36:29, crosses the full-attention layer at 35 and
-fails BOTH gates (card 0 free 0.45 GiB, gap 2.22 GiB), so the
-layer-toward-ROCm0 lever is spent: further rebalancing means a ctx cut,
-not a layer. The seam's balance test re-derives all of this from the env
-file on every run.
-
-Verified after deploy: `rocm-smi` reads **30.84 GiB used on ROCm0
-(headroom 1.02) and 28.86 GiB on ROCm1 (headroom 3.00)** — the shift
-landed 1.80 / 1.97 GiB, more than the fit's 0.58 GiB gap predicted, and
-the x1 card's headroom tripled. Utilization under live traffic: 60% on
-the x16 card (was 40%), 98% on the x1 card — the drafter, the output
-layers and the heavier compute buffers are structurally pinned to the
-last-layers card, so one DeltaNet layer was all the split lever had left
-to give. A chat completion answers in 1.4 s end-to-end; MTP acceptance
-runs 0.51–0.91 (mean len 2.7–3.7) on the live traffic.
-
-
-## 2026-09-18: the prompt cache doubles to 16 GiB
-
-`--cache-ram` is the one cache tunable on this service that is **not VRAM**: it
-caps the total bytes of saved prompt states in host RAM (`server_prompt_cache`,
-`tools/server/server-task.cpp`). `update()` evicts oldest-first past the cap,
-and `alloc()` **refuses outright** any single state larger than the whole cap —
-with one `WRN` line and nothing else:
-
-```
- - prompt state size 11388.990 MiB exceeds cache size limit 8192.000 MiB, skipping
-```
-
-That line appeared **137 times** in the two days from 2026-09-16: at 8192 MiB
-every conversation past ~170k tokens lost its state on the next slot
-reassignment and had to be re-prefilled in full. It matters more here than the
-cap alone suggests — this is a hybrid model (16 x 3 DeltaNet + 1 gated
-attention), so a slot's recurrent cell sits at the end of its own prompt and
-`--ctx-checkpoints 0` leaves no checkpoint to roll back to; llama.cpp then
-takes the "forcing full prompt re-processing due to lack of cache data" path.
-The RAM-side cache is the only mechanism that can carry a state *across* slots,
-and a refused state is simply gone. The cap is a token budget in disguise,
-because a state is the serialized KV of a whole sequence plus its
-constant-size DeltaNet state plus the MTP draft context's own KV:
-
-| Component | Per token |
-|---|---|
-| 16 full-attention layers x 4 kv heads x 256 dim x 2 B x 2 (K,V) | 64 KiB |
-| MTP draft context (one nextn layer, same arithmetic) | 4096 B |
-| **state law** | **68 KiB** |
-
-| Cap | Tokens storable | Fits one 196608-token slot (~12.8 GiB)? |
-|---|---|---|
-| 8192 MiB (until 2026-09-18) | ~120k | no — refused, then re-prefilled |
-| **16384 MiB (deployed)** | ~240k | **yes**, with ~3 GiB of budget spare |
-
-The cost is host RAM, allocated on demand and never pinned, so no VRAM figure
-above moves. dankserver has 62 GiB beside the media-stack co-tenants, which is
-why the cache is bounded rather than set to `-1` (unlimited): the seam test in
-`tests/frank_medium_unit.bats` gates both ends — the cap must clear one full
-slot's state, and it may not claim more than half the host's RAM.
-
-## Verified serving behaviour (2026-09-16)
-
-- Load: `model loaded` ~25 s after start with a warm page cache, then
-  `listening on http://0.0.0.0:7770`. The drafter adds ~2.5 s.
-- Slot shape: `n_slots = 2, n_ctx_slot = 196608`; `/props` reports
-  `modalities: {vision: true}` and the sampler defaults above.
-- The drafter is engaged — the init lines are the proof, not the flags.
-  Current (MTP, since 2026-09-17 part 2):
-
-  ```
-  common_speculative_impl_draft_mtp: adding speculative implementation 'draft-mtp'
-  common_speculative_impl_draft_mtp: - n_max=3, n_min=0, p_min=0.00
-  ... (draft context shares the main model's weights: `shares_model`)
-  ```
-
-  Under the DFlash2 era (2026-09-16 through 2026-09-17 part 1) the block was:
-
-  ```
-  common_speculative_init_result: loading draft model '/home/dank/models/Qwen3.8-27B-DFlash2-GGUF/Qwen3.8-27B-DFlash2-Q8_0.gguf'
-  common_speculative_impl_draft_dflash: adding speculative implementation 'draft-dflash'
-  common_speculative_impl_draft_dflash: - n_max=7, n_min=0, p_min=0.00
-  common_speculative_impl_draft_dflash: - block_size=8, mask_token_id=248070, n_extract=5, sample_from_anchor=true
-  ```
-
-  `block_size=8` and `n_extract=5` are DFlash2's own metadata; the candidate
-  selector is what distinguishes it from DFlash 1. DSpark logs under the same
-  `draft_dflash` symbol with `block_size=7` and a Markov head, so read the
-  numbers, not the implementation name.
-- Both cards hold the model. The measurement below is the **34:31-era**
-  record of the part 2 cutover; the deployed 35:30 shape is measured in
-  the part 3 section (ROCm0 30.84 GiB / ROCm1 28.86 GiB, headroom
-  1.02 / 3.00 GiB, both above the 1.0 GiB floor):
-  at 393216 with the deployed batch and the 34:31 split, `rocm-smi`
-  reported **28.91 GiB used on ROCm0 and 30.78 GiB on ROCm1** of 31.86
-  GiB each — the fit predicted 29.77 / 28.32, so ROCm1 ran 2.46 GiB
-  hotter than predicted and the seam's MTP constant was recalibrated to
-  the measured value (see the 2026-09-17 part 2 section). ROCm1's
-  1.08 GiB of headroom cleared the 1.0 GiB floor that shape required.
-- Drafting is live on **both slots** (the 2026-09-16 DFlash2-era numbers below
-  are the record; MTP-era acceptance is in the part 2 section), which is what
-  a 2-slot deployment has to
-  prove: `slot print_timing` reported `draft acceptance = 0.62857 (44 accepted /
-  70 generated), mean len = 5.40` and `0.48214 (27 / 56)` on slot 1, and
-  `0.38776 (57 / 147), mean len = 3.71` on slot 0, from one concurrent pair.
-  Acceptance moves with the text; the point is that neither slot is idle.
-- **140,069 prompt tokens prefilled in 134 s** (1043 tok/s) was measured
-  through a *single* slot, at the previous configuration: the raised window is
-  real rather than declared, and it is 22 % beyond the 114688 the vLLM
-  predecessor could offer. The window is now split over two slots.
-
-- **Nothing is replicated.** The loader does not print a per-device ledger at
-  this verbosity, so this is measured instead of read: raising CTX_SIZE from
-  131072 to 262144 added **4.83 GB to GPU 0 and 4.84 GB to GPU 1** — both cards
-  carry the KV for their own attention layers, about half each — and neither
-  card's *total* usage (24.94 / 29.42 GB at 262144) could hold the 31.46 GB
-  file, so no card carries a full copy of the weights. Compare the shape this
-  replaced: two vLLM replicas, i.e. two complete copies of the model, one per
-  card, which is exactly why a 29.8 GB FP8 checkpoint could not be served there
-  at any usable context.
-
-### Measured throughput (2026-09-16, deployed service, `temperature 0`)
-
-Same filler corpus for every shape, so these are shapes rather than a
-benchmark: a large prompt with a tiny budget isolates prefill, a small prompt
-with a long budget isolates decode. `draft`/`acc` are per request; acceptance
-moves with the text (filler prose accepts far worse than code or a real agent
-loop).
-
-| Shape | Prefill | Decode | Acceptance |
-|---|---|---|---|
-| 4k prompt / 32 out | 3994 tok in 3.25 s — 1230 tok/s | 59.8 tok/s | 71 % |
-| 1k prompt / 512 out | 998 tok in 1.41 s — 707 tok/s | **43.8 tok/s** | 41 % |
-| 16k prompt / 16 out | 12,528 tok in 8.94 s — **1402 tok/s** | — | 100 % (13/13) |
-| 64k prompt / 16 out | 48,508 tok in 39.8 s — 1219 tok/s | — | 53 % |
-| 140k prompt / 16 out | 140,069 tok in 134.3 s — 1043 tok/s | — | — |
-| same 1k prompt again | **4 tokens prefilled** | 43.8 tok/s | 41 % |
-
-The last row is the prefix cache doing its job: the second request of the same
-conversation re-prefilled 4 of 998 tokens. Prefill holds ~1.0-1.4k tok/s from
-4k to 140k, and decode sits at ~44 tok/s on a long generation — against the
-vLLM predecessor's 42.8 tok/s single-stream (two replicas) and the ROCmFPX
-llama.cpp measured here at 35.7-46.2 tok/s. One process, one model, both cards,
-no proxy.
-
-### What the drafter is worth — and why MTP is not the alternative
-
-Measured 2026-09-16 with `scripts/bench-spec-ab.py` on the **DFlash2** drafter —
-the one deployed now: four configurations of the same binary, identical prompts
-at `temperature 0`, unit stopped between runs (one host, one copy of a 29 GiB
-model). 1k-token prompt, 512-token budget. These are the numbers that selected a
-trained multi-token drafter over the model's own MTP head.
-
-| Configuration | Decode | Drafting | VRAM (card 1) |
-|---|---:|---|---:|
-| `none` — autoregressive | 17.7 tok/s | — | 26.07 GB |
-| `draft-mtp` n_max 3 (the model's own head) | 32.4 tok/s | 58 % of 72 | 29.35 GB |
-| `draft-mtp` n_max 7 | 26.7 tok/s | 33 % of 147 | 29.65 GB |
-| **`draft-dflash` n_max 7 — deployed** | **43.5 tok/s** | 41 % of 119 | 29.42 GB |
-
-- **DFlash2 is worth 2.46× over autoregressive and 34 % over the best MTP.**
-  Deeper MTP drafting costs more than it returns: at depth 7 only a third of
-  the proposals survive, at depth 3 more than half do.
-- **All four produced identical greedy text** (`2, 3, 5, 7, 11`): speculation
-  buys speed, never a different answer. The harness warns loudly if a future
-  configuration ever breaks that.
-- **MTP is not the cheaper option.** Selecting it loads `blk.64.nextn.*` from
-  the target and adds ~3.3 GB to the card that holds it, against 2.06 GB for
-  the drafter's file — which is also why the load log says
-  `unused tensor blk.64.nextn... ignoring` under `draft-dflash`: correct
-  behaviour, not a truncated download.
-- **They cannot be chained.** `--spec-type draft-mtp,draft-dflash` dies at load
-  with `context type MTP requested but model doesn't contain MTP layers`. A
-  chain has ONE draft context, and MTP builds against whatever model that
-  context holds — here the DFlash2 file. MTP's head is loaded only when MTP *is*
-  the draft model (`creating MTP draft context against the target model ...`).
-  That is structural, not a tuning question.
-
-### Chaining the draftless n-gram matcher (deployed)
-
-`ngram-mod` **can** ride beside a context-backed drafter, and it is enabled.
-The reason is in `common/speculative.cpp`: the n-gram types are added to the config vector
-unconditionally while every model-backed drafter is gated on
-`ctx_dft != nullptr` — so n-gram needs no draft context and cannot collide with
-the drafter over one. (MTP is the opposite: it *is* context-backed, which is why
-`draft-mtp,draft-dflash` cannot load.) The vector's fixed order — ngram types
-first, then the drafters — also means CLI order is irrelevant.
-
-The measurements below are DFlash2's — the deployed configuration. (During the
-DSpark excursion the chain was `draft-dspark,ngram-mod`; the matcher's behaviour
-is a property of the matcher, not of the drafter beside it.)
-
-Measured with `scripts/bench-spec-ab.py`, decode tok/s, three shapes that a
-proposer can and cannot match:
-
-| Configuration | prose 1k/512 | echo 1.5k/400 (copy 0.94) | structured/400 | identity |
-|---|---:|---:|---:|---:|
-| `draft-dflash@7` | 43.77 | 76.82 | 91.08 | 70.37 |
-| `ngram-mod` alone | 17.72 *(0 drafts)* | 102.50 | 17.80 *(0 drafts)* | 17.77 |
-| **`draft-dflash,ngram-mod@7`** | **43.65** | **200.58** | 91.22 | 70.40 |
-
-- **On prompt-echo traffic the chain is 2.6× the drafter alone** (200.6 vs 76.8)
-  and twice `ngram-mod` by itself, at 100 % acceptance.
-- **On prose it is indistinguishable from DFlash2 alone** — the same 119 drafts
-  and 49 accepts, the same 43.7 tok/s. That is the property that makes it safe
-  as a default: where the matcher cannot match, it costs nothing.
-- `ngram-mod` alone is a *negative control*: zero drafts on prose and identity
-  (17.7 tok/s = the autoregressive baseline), 100 % acceptance on echo. Nothing
-  else could have proposed those tokens, which is how its drafts are identified
-  at all — see the trap below.
-- VRAM is unchanged (29.42 GB on card 1 with or without it): the matcher is a
-  64 MiB host-RAM hash table.
-- `structured` (25 records with unique ids) rewards no proposer: its lines
-  differ, so there is no long span to match. It is a negative control for
-  *self-repetition*, not evidence about it.
-
-**The window matters, and shorter is not better.** The chain at three windows:
-
-| `SPEC_NGRAM_N_MIN`/`N_MAX` | prose 1k/512 | echo 1.5k/400 |
-|---|---:|---:|
-| **48/64 — deployed** | **43.9** | **200.6** |
-| 16/32 | 36.0 | 161.0 |
-| 8/16 | 36.2 | 113.8 |
-
-A short window makes the matcher propose on ordinary prose fragments, where the
-proposals are rejected and the verification is still paid — the "ngram hurts"
-result this fleet measured twice before on other payloads. At 48/64 the chain
-keeps DFlash2's exact numbers on prose and wins on echo.
-
-> **Operational trap: `ngram-mod` prints no startup banner in this build.**
-> `journalctl -u frank-medium` shows `adding speculative implementation
-> 'draft-dflash'` and nothing for the matcher — the only evidence it is running
-> is behavioural (`draft_n` > 0 where text repeats) or the per-task line
-> `slot print_timing: ... draft acceptance = 1.00000 (256 accepted / 256
-> generated), mean len = 65.00`. Do not conclude from the load log that the
-> chain is not engaged.
-
-### The agent-loop shape (fleet harness, 2 families × 2 turns)
-
-`scripts/bench-turn-latency.py http://127.0.0.1:7770 2 2 LAYERSPLIT` — two
-agent loops re-sending a ~30k-token document concurrently:
-
-| | turn 1 | turn 2 |
-|---|---:|---:|
-| family 1 | 24,799 ms | 30,697 ms |
-| family 0 | 51,424 ms | 8,015 ms |
-
-Read that against the prefill table, not as a latency regression: with **one**
-slot the second family's first turn waited for the first (25 s of cold 30k
-prefill + the wait ≈ 51 s), and the last number — 8.0 s — was a re-prefill the
-cache could only partly serve. That was the 1-slot configuration. The
-deployment now runs **two slots of 131072**, so a second agent loop has a slot
-of its own instead of queueing, at the cost of halving the window each one
-gets. `--parallel` splits the context and the KV between slots; it does not add
-throughput, so the trade is concurrency against conversation length.
-
-### Benchmarking this endpoint
-
-`scripts/bench-batch-ubatch.sh` sweeps `--batch-size` against `--ubatch-size`
-with `llama-bench` and records, per cell, the prefill rate, the decode rate and
-each card's **peak** VRAM. Run it from a transient unit rather than a shell:
-
-```bash
-sudo systemd-run --unit=fm-bench --collect --no-block --property=Type=oneshot \
-     --property=LimitMEMLOCK=infinity --property=User=dank \
-     scripts/bench-batch-ubatch.sh
-```
-
-Properties of `llama-bench` shape that harness, and three of them have already
-produced wrong numbers on this fleet:
-
-- **It separates devices differently from `llama-server`.** `/` splits devices
-  *within one configuration*, `,` splits *sweep values* — so this service's
-  `--tensor-split 36,29` is `-ts 36/29` to llama-bench, and the comma form
-  silently benches each card on its own instead. Likewise `--device
-  ROCm0,ROCm1` → `-dev ROCm0/ROCm1`.
-- **There is no `-c`.** The context is a consequence of the test shape:
-  `n_ctx = n_prompt + n_gen + n_depth`. No bench can therefore prove that a
-  configuration's activation buffers fit beside the *deployed* KV — that needs
-  the live unit at 327680 across two slots.
-- **A test that cannot allocate calls `exit(1)`**, taking the whole process with
-  it, so the harness runs one invocation per cell and a failure costs one TSV
-  line instead of the rest of the sweep.
-- **A shell's memlock limit is not the unit's.** The unit sets
-  `LimitMEMLOCK=infinity` for the ROCm staging buffers; an SSH shell here gets
-  8 MiB, which fails the load for a reason that has nothing to do with the
-  model being benched.
-- **No `--spec-*` and no `--parallel`.** The drafter cannot take part, so a
-  `llama-bench` decode figure is the *drafterless* baseline — 15.9 tok/s at 64k
-  depth against the deployed 43.8 — and slots are never exercised. `-d <n>` is
-  what puts KV in front of the measurement at all: a `tg` instance runs with
-  `n_prompt = 0`, i.e. at ~zero depth unless `-d` says otherwise.
-
-## Observability
-
-The endpoint exports Prometheus metrics under the **`llamacpp:`** namespace
-(`--metrics`), which is what the fleet's scrape and dashboard read — the
-`vllm:` series the predecessor exported no longer exist:
-
-| Need | Series |
-|---|---|
-| Throughput | `llamacpp:prompt_tokens_seconds`, `llamacpp:predicted_tokens_seconds` |
-| Queue | `llamacpp:requests_processing`, `llamacpp:requests_deferred` |
-| Cache reuse | `llamacpp:prompt_tokens_cached_total` vs `llamacpp:prompt_tokens_total` |
-| **Speculative decoding is live** | `llamacpp:spec_decode_num_drafts_total` and `llamacpp:spec_decode_num_accepted_tokens_total` (their ratio is the acceptance rate) |
-
-`spec_decode_num_drafts_total` staying at 0 is the one-line answer to "is the
-drafter actually running": the flags can all be right while the draft model
-failed to load.
-
-## Client contract
-
-OpenAI-compatible `/v1` on **7770**, alias **`frank-medium`**, streaming and
-`/health`, `/props`, `/metrics` as llama.cpp provides them. Reasoning traces
-are projected into `reasoning_content`. A request that sends no sampler fields
-gets the server defaults above; a client that sends its own wins. Images ride
-the BF16 projector — `/props` reports `modalities: {vision: true, video: true,
-audio: false}` (read live 2026-09-18), and there is no audio anywhere in the
-fleet.
-
-## Troubleshooting
-
-- **Loads, then dies on the first request with a rocBLAS Tensile error** — the
-  bundled device-library depth (see the bundling trap above); check
-  `build-rdna4-rocm714/lib/rocblas/library/` exists and is populated.
-- **`hipErrorNoDevice` / a card missing** — `HIP_VISIBLE_DEVICES` must be
-  `0,1`; the iGPU enumerates as device 2 and must stay hidden.
-- **Drafting not engaged** — read the journal for
-  `common_speculative_impl_draft_dflash` (DFlash2 and DSpark both log under that
-  symbol); a missing drafter file makes the serve script exit before
-  llama-server starts, and a drafter whose GGUF the build does not recognise
-  fails the load instead.
-- **Why is `blk.64.*` "unused"?** That is the model's own MTP head and it is
-  correct for `draft-dflash`. It is not a truncated download:
-  `bash scripts/download-frank-medium-gguf.sh --verify-only` proves the bytes.
-- **A rebuild segfaults at load** — the build is gfx1201-only on purpose; a
-  gfx1200 sibling binary links, loads the model and then segfaults on these
-  cards.
-
-## See Also
-
-- [[frank-medium-radiance-mxfp4]] — the retired vllm-radiance MXFP4 deployment
-  (replicas + proxy) and the PCIe-atomics evidence
-- [[frank-medium-rocmfpx]] — the llama.cpp/ROCmFPX deployment earlier on
-  2026-09-15, whose `--split-mode layer` measurements this service reuses
-- [[Dual-R9700-Cooperation]] — why the two cards cannot take part in a
-  collective, and every software escape that is closed
-- [[frank-zealot]] — the fleet's other llama.cpp endpoint (RTX
-  3090, MTP, the image path)
-- [[Dankserver System Specs]] — hardware inventory
-- [[frank-medium-batch-ubatch]] — the batch/ubatch sweep that set the deployed
-  geometry and moved this split from 36:29 to 35:30: the ubatch optimum at 512,
-  why the logical batch is free, and the harness traps
----