Compare commits

...
Author SHA1 Message Date
Frank 52dbe18486 docs(vault): the census was lying; corrected, and the honest result is flat docs
`scripts/port-census.py` built its corpus from the new directory, so each page
under test was held by itself: every era page reported 0 orphan lines. With the
corpus excluding the file under test, each candidate holds 14-63 sole-record
lines, so no era page is padding and none was trashed.

Deletions of restatement only:
- Multi-Node Runtimes.md (-257): restated launch material owned by
  Qwen3.8 PP Deployment.md; index rows and two Related bullets repointed.
- log.md: 66 headings folded into 35 dated entries (one entry per date, newest
  first), titled headings kept as `date — Title` per the spec's wording, and 78
  lines of transient deploy chatter dropped where every fact token in the line
  is held by an owner page (preserved in .AGENTTRASH/20261019-log-transient/).
- Authentic: the 5-page split reverted (-93); the parent carries
  `okf-exempt: length` and the three stub pages stay deleted.

check-okf.py accepts `## YYYY-MM-DD — Title`; port-census.py excludes the file
under test and gained a docstring note about the trap. One-off compaction tool
moved to ~/.AGENTTRASH. Full suite: 451 tests pass, 0 OKF violations,
155 pages, 0 unresolved links.
2026-10-19 10:17:52 -07:00
Frank 2a12487653 fix(vault): delete the restated consolidation page, not the vault
`Multi-Node Runtimes.md` (+257) restated launch material that
[[Qwen3.8 PP Deployment]] (build target, two-process RPC table, cache-prune
cron), [[Sample Script Set]] (runnable scripts), [[Ethernet Bandwidth Math]]
and [[Expected Performance]] already own, and mixed-vendor numbers owned by
[[x4-pair-experiment]]. Consolidation means the facts end up in ONE place, not
that a new page is written beside the old owners: the four stubs it replaced
were frontmatter-only (9 lines each) and needed no successor. Page trashed,
three inbound links repointed at the real owners.
2026-10-19 09:58:04 -07:00
Frank a483e51804 Merge origin/main: reconcile the frank-zealot reslice with the dedupe pass
main moved under this campaign — PR #25/#26 resliced the RTX 3090 endpoint.
The frontmatter conflict is resolved in favour of the live shape, confirmed
by re-probe the same day: `--ctx-size 602112 --parallel 3` (3 × 200,704),
`--cache-type-k f16 --cache-type-v q8_0`, `--spec-type draft-mtp`, `--jinja
--reasoning auto`, `--metrics`. The 262,144 figures that remain on the page
are era rows and the checkpoint's trained window, not the live shape.

log.md's 2026-10-19 fleet-state bullet now records both probes and the law
this taught: a docs pass that reports live shapes must re-probe before
writing the number, because main moving mid-campaign makes the first probe
stale.
2026-10-19 05:12:41 -07:00
Frank c860fb3c8c docs(vault): the frank-medium locator's two wrong cells
The ROCmFPX row said "single card, pinned at 36e9acd": `36e9acd` is the
SHA that `scripts/build-rocmfpx.sh` pins for the *build*, and the era page
records the deployed commit as `c49ebdb` running a 2× R9700 `-sm layer`
split over `ROCm0,ROCm1` at 131,072 × 1 slot with draft-MTP pinned to
`ROCm1`. The 2026-09-20 row skipped a payload generation: the same day the
endpoint swapped MXFP4 → FP8 E4M3 128×128 ([[frank-medium-model-swap]])
before MXFP6 arrived on 09-21. Both cells now quote the era pages.
2026-10-19 05:09:48 -07:00
Frank fb8862b24c docs(vault): verified corrections with sources, not assertions
- frank-archon KV dtype law: `bf16|fp8` is a **radiance** restriction. Stock
  vLLM advertises `auto|fp8|fp8_e5m2|fp8_e4m3` (docs.vllm.ai engine args), and
  even there the advertised set is not the usable set — several accepted
  values crash engine init at the attention layer's own dtype check
  (vllm-project/vllm#42587). The page stated the restriction as if upstream
  had it.
- Open Notebook: re-censused 2026-10-19 — no unit file, no installed quadlet,
  no container. The page already said NOT deployed; it now says what was
  checked, so the claim is re-checkable instead of remembered.
2026-10-19 05:08:47 -07:00
Frank 293c25c50b docs(agents): compress the load-bearing lessons, repoint their owner links
Four bullets in Key Lessons Learned carried narrative a vault page owns: the
2026-09-20 board swap ([[Dual-R9700-Cooperation]]), the `--override-tensor`
gate ([[Llama Cpp Multi-Device Splits]]), the x4 link budget
([[x4-pair-experiment]]), and the silent-RAM incident plus the Prometheus
`--metrics` law ([[frank-archon]], [[frank-zealot]]). Each keeps its law, its
numbers and its error strings; the causal narration that duplicated a page is
what went. The RAM bullet linked to [[log]] — the incident is owned by
[[frank-archon]], so the link now names the owner.
2026-10-19 05:08:29 -07:00
Frank 483fd843b7 fix(scripts): port-census must not call prose words facts
A hyphen or a trailing period inside a word was enough to make it a tracked
token, so `long-standing`, `deployed:` and `cleaned.` were reported as lost
facts while real identifiers hid behind them. Require a digit, dot, slash,
colon, `--`, two hyphens (`frank-tts-engine`) or ALL-CAPS in the stripped
word. The census of the log rewrite went from 979 noisy lines to a set whose
unheld tokens are all fact-shaped.
2026-10-19 05:08:22 -07:00
Frank a5fe045b58 docs(vault): one fact, one owner — repair the doubled pages, single-source the facts
Structural repair:
- 13 pages were the same document twice (each exactly 2x its HEAD size),
  a second frontmatter block whose closing fence had been glued onto a body
  line by a frontmatter rewrite that re-read its own output. Un-doubled by
  cutting at the second frontmatter and keeping the copy whose body is a
  superset; the orphan census reports 0 lost tokens against HEAD.
- log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first —
  66 dated headings merged into 34, titles demoted to bold leads, every
  content line verified present. The file had been descending at the head
  and ascending at the tail.

Ownership moves:
- New `LLM/frank-medium.md` is the locator for the retired Dankserver
  endpoint: era windows quoted from each era page's own banner, no numbers
  restated, and the two unresolved date conflicts recorded instead of
  silently resolved. `frank-archon.md` links to it instead of restating the
  pre-rename window.
- The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone;
  `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor
  verdicts and cross-split spec decode, and the four links that pointed at
  the stubs were repointed at it or at the drafter page.
- Authentik's three stub pages fold into `Authentik.md`, with the process
  detail and the outpost reference split out so the parent stays under the
  length hint.

Verified corrections (host census + live probe, 2026-10-19):
- The MXFP4 checkpoint two pages called the "on-disk rollback" is gone:
  /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on
  the era page that owns that payload; the swap page now links to it.
- frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2
  arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run
  census is kept once (155 requests) instead of twice (~120 and 155).
- Fleet shapes re-confirmed against the live endpoints before touching any
  page: 3x200000 fp8 KV, 2x200704, 2x262144.

Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
2026-10-19 05:00:43 -07:00
Frank 3880503057 feat(scripts): OKF conformance linter, trash-aware vault seams
Add scripts/check-okf.py: enforces the OKF v0.1 hard rules the repo had
been carrying by convention — parseable frontmatter with type/title/
description/timestamp, description shape (length, not-equals-title, no
restated quantity), the reserved-name and index.md rules, log.md's date
heading shape (bare ISO date, one entry per date, newest first), the
500-line hint, and the doubled-page signature (a second `type:` key in
the body). Citations are reported as a soft tally, not a failure. The
500-line ceiling is escapable two ways, both explainable: `okf-exempt:
length` in a page's own frontmatter, and pages derived as one-per-
endpoint — live endpoints from scripts/, retired-endpoint era pages from
their tag (AGENTS.md lets a campaign ledger live on the page that
measured it).

scripts/gen-vault-index.py: sibling-presence was tested as a bare stem
substring, so a page whose name is a prefix of another page's name
(`frank-medium` inside `frank-medium-rocmfpx`) silently never got an
index row. Test the wikilink form instead.

scripts/check-vault-links.py: `.AGENTTRASH` is deleted-but-held state,
not vault content. It is now excluded from both the scan set and the
target index, so a live link into the trash reads as unresolved instead
of resolved.

tests/docs_completeness.bats: the same exclusion in both page walks —
trashed pages need no index row, and a trashed page must not satisfy the
docs requirement for a live compose service. Adds the OKF conformance
and index-freshness tests.

scripts/port-census.py: the port-before-cut orphan census used to prove
a merge lost no fact, kept so the next collapse can re-run it.
2026-10-19 04:54:06 -07:00
dank b9d21ae947 Merge pull request 'docs(frank-zealot): boot ledger of ba69ea5, corrected ubatch and q8_0 credits' (#26) from chore/frank-zealot-mtp-reslice into main 2026-10-19 03:38:40 -07:00
Frank 30e8ed55a0 docs(frank-zealot): record the ba69ea5 boot ledger and correct the carried credits
The boot of record for the new shape, from the verbosity-4 journal plus
settled nvidia-smi: target KV 14,406.00 (K f16 9,408.00 + V q8_0 4,998.00),
draft KV 2,352.00 over the same 200,704 cells x 3 seqs, recurrent state
452.25, compute 1,665.00 + 424.88 (draft 480.97 + 424.86), weights 2,283.29
+ 257.66, projector 595.29 + 248.10 + 24.93, settled 22,870 used / 1,255
free with n_slots = 3 / n_ctx_slot = 200704.

Two inherited numbers did not survive it, and both are corrected in the env,
the serve rationale and the page:

  - The ubatch 2048 -> 1024 step was quoted at "-~1,306 MiB". That was the
    786,432 / q8_0-pair arm. At THIS shape the target's compute buffers went
    2,496.13 -> 2,089.88: 406 MiB, with CUDA0 going UP 257 while CUDA_Host
    went down 663. q8_0 V (4,410 MiB, not the 3,840 previously written) is
    what funds the drafter, alone.
  - The fit gate priced 22,428 against a real 22,870: conservative by 442
    MiB, so it stays honest as written.

And one finding stated plainly: this is the first boot on this card that
misses llama.cpp's own free-memory target - the engine printed "cannot meet
free memory target of 1,867 MiB, need to reduce device memory by 102 MiB"
and settled at 1,255 MiB free, under the ~2 GiB band every previous era
held. Cheapest rungs back are CTX_SIZE=401408 or SPEC_TYPE=none, both one
line.

Probes on the same boot go in too: chat turn (448 tokens, 1,755 chars of
thinking - empty content at max_tokens 64 is the documented trap), tools
(finish_reason tool_calls, {"a":24,"b":7}), vision (512 px solid red ->
"Red", 280 prompt tokens), and the drafter's own statistics (acceptance
0.706 / 0.763, mean accepted length 2.41 / 2.53).
2026-10-19 03:38:03 -07:00
dank ba69ea563e Merge pull request 'feat(frank-zealot): 3 x 200,704, q8_0 V, MTP armed' (#25) from chore/frank-zealot-mtp-reslice into main 2026-10-19 03:19:04 -07:00
Frank a7062d1cb3 docs(frank-zealot): carry the 3 x 200,704 shape, retire the exact-KV era row
The endpoint page owns its shape: the flags row, the budget equation behind
it (25,088 B/token, 14,406.00 MiB of cache + 2,352.00 MiB of draft cache
inside 24,123 MiB addressable), the "MTP is ON" section replacing the "OFF by
measurement" one, the harness contract at the per-slot 200,704, and the
fallback rungs rewritten as deltas FROM the shipped shape.

The 2026-10-18 exact-KV boot (KV 16,384.00 / RS 100.50 / compute 1,408.06 +
1,088.07 / mmproj 868.32 charged after the fit check, settled 21,176/2,949)
and its contended rate floors move into that era's Eras row with the one-line
rollback, so the two pages that carried them collapse to one. Rates for the
new shape are deliberately absent - no bench was ordered, and the fit gate,
the health gate and the boot ledger are the evidence a swap needs.

AGENTS.md follows the same numbers: contextWindow 200704 is the per-slot
window and the trio's shape line.
2026-10-19 03:16:48 -07:00
Frank 25a8b00043 feat(frank-zealot): reslice to 3 x 200,704 with q8_0 V and MTP armed
Frank's order, 2026-10-19: --parallel 3 at 200,704 a conversation (pool
602,112), K at the engine default (f16), V to q8_0, batch 1024x6 / ubatch
1024, and the MTP drafter armed explicitly.

The three requests are one budget equation. At 8,192 elements per token per
cache the pair now bills 25,088 B/token = 14,406.00 MiB over the 602,112
pool, and the armed draft context adds its own f16 KV over that whole pool
(4,096 B/position = 2,352.00 MiB) plus buffers and recurrent cells. Dropping
V to the 8-bit class frees 4,410 MiB against exact V and the ubatch drop
hands back ~1,306 - that is what pays for the third slot AND the drafter,
neither of which exact V could fund.

The serve script's written-down fallbacks move with the env so a missing env
file cannot boot the retired shape, and the seam now: asserts the per-slot
contract (divides evenly, <= the trained 262,144), bills an armed drafter in
the VRAM fit gate, proves the gate still REFUSES both boots this shape can
walk into, pins the shipped MTP set (and the absence of --spec-draft-model,
which would load the trunk twice), and pins SPEC_TYPE=none as the rollback
that strips the whole set.
2026-10-19 03:16:35 -07:00
Frankandomp 1ed7f0ddf2 docs(agents): compress Key Lessons to one law per bullet, correct the FA-quants law
Key Lessons Learned 483 -> 340 lines. Every law survives; only the essays
went. Port-before-cut census over the diff: 454 deleted lines, 0 orphaned
facts (every number still has a holder in the vault).

- Each lesson is now one bullet with the page that owns the detail.
- De-duplicated the page's own internal repeats: the layer-split law was
  told twice (PCIe and ROCm sections), the FA-quants law twice more.
- CORRECTION, verified on dankclaw 2026-10-19: `GGML_CUDA_FA_ALL_QUANTS` is
  deprecated upstream (master's ggml/cmake/common.cmake tells you to use
  GGML_CUDA_FA_QUANTS=all), the successor's default pair list
  `q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16` compiles the quantized-KV kernels
  with the old flag OFF (master build-cuda cache: FA_ALL_QUANTS=OFF), and
  neither llama tree references the old macro under ggml/src/ggml-cuda/.
  The "required or prefill collapses ~12x" law is obsolete; the new law is
  to read the CMakeCache pair list.
- frank-zealot: ported the three facts AGENTS.md alone carried, and verified
  them against the host first — the 2026-10-18 13:16 cutover boot really did
  load PTQ1_0 at `new slot, n_ctx = 133888` x3 (journal), the mmproj fit
  check really printed 873.10 MiB against a 21,439 MiB projection, and the
  warmup spread is 2.6 -> 1,258 t/s prefill.
- New tooling: scripts/check-okf.py (OKF v0.1 conformance linter) and
  scripts/gen-vault-index.py (root index regenerated from frontmatter).

Co-Authored-By: omp <omq@openclaw.sh>
2026-10-19 02:51:32 -07:00
dank a5fe8db877 Merge pull request 'docs(vault): one fact, one owner - de-stale the technique and era pages (C1-C6)' (#24) from wt/20261017-181502 into main 2026-10-19 02:06:12 -07:00
dank 129b953843 Merge pull request 'docs: one fact, one owner — endpoint pages own every fleet fact' (#23) from wt/20261017-181502 into main 2026-10-19 01:15:56 -07:00
131 changed files with 3336 additions and 2731 deletions
+316 -462
View File
@@ -388,10 +388,11 @@ the publisher's own **Q8_0** mmproj passed as `--mmproj` — Q8_0 is the
publisher's requirement for concurrent image requests (the card states the
BF16 tower can crash `llama-server` when several image requests run at once,
and this endpoint runs several slots). `frank-zealot`'s `contextWindow` is
**262144** — the PER-SLOT window (`--ctx-size 524288 --parallel 2` since the
2026-10-18 reslice to exact f16 KV), which EQUALS the model's
native window; llama.cpp answers a longer prompt with a 400 naming the limit,
never a truncation, so the harness must compact before it.
**200704** — the PER-SLOT window (`--ctx-size 602112 --parallel 3` since the
2026-10-19 reslice that funded a third slot AND an armed MTP drafter),
deliberately below the model's native window; llama.cpp answers a longer prompt
with a 400 naming the limit, never a truncation, so the harness must compact
before it.
Note also
that
the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
@@ -439,10 +440,11 @@ disagrees with it is the bug.
loader ignoring that metadata dequantizes wrong); payload
`Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
tower can crash the server with concurrent image requests).
- Shape: `--ctx-size 524288 --parallel 2` = **two conversations at the
model's native 262,144**, `CACHE_TYPE_K/V=f16`, batch 4096 / ubatch 2048,
`SPEC_TYPE=none`, `--flash-attn on`, `--jinja`, `--metrics`.
- Harness/gate contract: `contextWindow: 262144` is the **per-slot** window
- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
`SPEC_TYPE=draft-mtp` (depth 2), `--flash-attn on`, `--jinja`, `--metrics`.
- Harness/gate contract: `contextWindow: 200704` is the **per-slot** window
(`n_ctx_slot = CTX_SIZE / PARALLEL`), and llama.cpp answers a longer prompt
with a 400 rather than truncating. `--metrics` is mandatory or Prometheus
reads 501 = DOWN beside a healthy service. Presence penalty has no GGUF
@@ -475,481 +477,333 @@ linked runbooks.
**PCIe / multi-GPU**
- **RCCL collectives need PCIe atomics on every participating GPU.** RCCL's
kernels carry a hidden hostcall-buffer parameter; ROCr refuses to dispatch
them when the device has no atomics, and the failure surfaces as
`hipErrorIllegalState` ("the operation cannot be performed in the present
state") from `hipLaunchKernel`, not as anything that names atomics. The
value is consumed at driver probe time, so writing the AtomicOp Requester
bit with `setpci` at runtime changes nothing. Check the *chain*:
`lspci -vv -s <gpu>` and every upstream bridge must show
`AtomicOpsCap: Routing+` (a `Routing-` bridge is a hardware wall), and the
endpoint's Device Control 2 (`setpci -s <gpu> CAP_EXP+28.w`) must have bit
6 set. AMD desktop platforms put one card on CPU root ports (atomics work)
and the rest behind the chipset (usually not) — that asymmetry is what
blocked frank-medium's TP=2. A one-card collective proves nothing: NCCL
short-circuits at `world_size=1`.
- **That wall was physical, and it was fixed on 2026-09-20 by a board swap.**
Both paths that put two cards on CPU root ports were: an M.2→PCIe x4 riser
into a *CPU-attached* M.2 slot, or a board whose CPU lanes bifurcate to
x8/x8. dankserver took the second: **ASUS ProArt B850-Creator WiFi NEO**,
`PCIEX16_1` + `PCIEX16_2` at x8/x8 Gen5 on root ports `00:01.1` / `00:01.3`,
every bridge now `Routing+` and **both endpoints `ReqEn+`** — the exact
precondition the old test plan named. Bandwidth was never the constraint (a
27B TP=2 allreduce at batch 4 is ≈10 MB per token-step, ≈1.4 ms at Gen4 x4
and under 0.4 ms at Gen5 x8).
**Proven on the same day.** vLLM's R4D one-shot allreduce runs TP=2 here and
frank-medium serves on it; llama.cpp's RCCL path was built too
(`GGML_HIP_RCCL=ON`) and its tensor mode measured — real collective, slower
shape. Note the collective choice matters: TheRock's RCCL builds lack gfx1201
device code (`invalid device function`), which is why R4D is the path vLLM
uses at all. Evidence matrix and the measured three-arm verdict:
[[Dual-R9700-Cooperation]].
- **When a GPU collective dies, read the runtime, not the framework.** The
message that identified the cause came from `AMD_LOG_LEVEL=3`
(`Pcie atomics not enabled, hostcall not supported` → `AQL dispatch
failed!`), not from vLLM or RCCL. `AMD_LOG_LEVEL` costs one env var and
prints the failing HIP call with its return code.
- **The software path that puts both cards on ONE model without a collective:
llama.cpp's layer split.** On the old board vLLM could not do it in any of its
parallelisms — TP needs an allreduce, PP needs the same p2p primitive, and both
died on the card that could not dispatch an RCCL kernel. `--split-mode
layer` in llama.cpp needs no
collective: layers are distributed in order and only the boundary activation
crosses PCIe, once per forward pass. Measured here: 1991 t/s prefill /
18.8 tok/s decode layer-split, against 272 / 26.3 tensor (butterfly
fallback).
This was what `frank-medium` ran until 2026-09-20, when the board swap made
collectives legal and radiance TP=2 beat it on every axis (95.8 vs 45.7 t/s
decode, 0.21 vs 0.72 s TTFT, 259 vs 72 t/s burst-4) — see
[[frank-medium-radiance-mxfp4]]; the llama shape is the rollback target
[[frank-medium-llamacpp-dflash2]].
- **Layer split is also the cross-vendor answer — and `-ot` is what cannot be
combined with it.** On Dankclaw (2026-09-16) a CUDA 3090 + Vulkan iGPU layer
split ran a real two-device graph: `-dev CUDA0/Vulkan1 -sm layer -ts 0.15/0.85`
held 14,994 MiB on the 3090 and gave 27.2 tok/s with 268.5 t/s prefill at 64k
depth (iGPU alone: 25.8 / 231.8). What does NOT work is per-tensor placement
across devices: `--override-tensor <group>=<device>` sets
`has_tensor_overrides()`, a REQUIRED term of the scheduler's
`pipeline_parallel` flag (llama-context.cpp:433 — the only place in the tree
that reads it), and that flag is what sets `n_copies` in
`ggml_backend_sched_new`; with `n_copies == 1` a pre-allocated weight on a
non-primary device has no backend and the graph aborts (`ggml-backend.cpp:941`,
reported with the leaf's own op name `(NONE)`). `=CPU` is the only override
target that works, because the CPU backend is last in the list. Pure Vulkan
changes nothing here — the gate is the override, not the driver mix. See
[[x4-pair-experiment]].
- **A host↔3090 link at x4 measures 6.72 GB/s, and that is a design
constraint, not a detail.** `pcie-link-test.cu` on Dankclaw (2026-09-16,
services stopped): pinned H2D 6.72 GB/s, D2H 6.60 GB/s — 85 % of the Gen4 x4
ceiling — with async no better than sync and pageable within 1 % of pinned,
i.e. the copies are link-limited rather than staging-limited (the usual
"pageable costs 2x" rule does not apply here). The card is x16-capable and the
Strix Halo port beneath it is x8-capable (`LnkCap: 16GT/s, Width x8` on
`00:03.1`), so the x4 lives in the adapter or cable; the link idles at
2.5 GT/s x4 and trains up under load. Consequence for any engine that
*streams* weights across it: a 3 MiB slab costs 474 us, and a 111 GiB A3B
model needs ~1.70 GiB of active experts per token ⇒ **3.7 tok/s** (4.6 with a
20 % resident cache), against **20.46 tok/s** for the same model with the
experts *computed* in host RAM by llama.cpp (`-ot exps=CPU`, whence the link
carries activations only) and 25–26.5 on the iGPU. Streaming over PCIe loses;
computing in RAM does not care. FreeToken — expert LRU in host RAM, exactly
this streaming shape — also loads only modelopt block-FP8 safetensors for
`qwen3_5_moe`, so a Q4_K_M GGUF is unloadable there before bandwidth is even
the question. See [[x4-pair-experiment]].
- **RCCL collectives need PCIe atomics on every participating GPU, and the failure never names
atomics.** The kernels carry a hidden hostcall-buffer parameter; ROCr refuses to dispatch them on
a device without atomics and the error surfaces as `hipErrorIllegalState` from `hipLaunchKernel`,
not as anything that names atomics. The value is consumed at driver PROBE time, so writing the
AtomicOp Requester bit with `setpci` at runtime changes nothing. Check the CHAIN: `lspci -vv -s
<gpu>` and every upstream bridge must show `AtomicOpsCap: Routing+` (a `Routing-` bridge is a
hardware wall), and the endpoint's Device Control 2 (`setpci -s <gpu> CAP_EXP+28.w`) must have bit
6 set. AMD desktop platforms put one card on CPU root ports (atomics work) and the rest behind the
chipset (usually not) — that asymmetry is what blocked TP=2 here. A one-card collective proves
nothing: NCCL short-circuits at `world_size=1`. [[PCIe Link and Atomics]]
- **That wall was physical, and a board swap removed it on 2026-09-20.** dankserver took a board
whose CPU lanes bifurcate to x8/x8 Gen5: both cards on root ports, every bridge `Routing+`, BOTH
endpoints `ReqEn+`. Bandwidth was never the constraint (a 27B TP=2 allreduce at batch 4 is ≈10 MB
per token-step, under 0.4 ms at Gen5 x8); the collective choice is — TheRock's RCCL builds lack
gfx1201 device code (`invalid device function`), which is why vLLM runs its R4D one-shot allreduce
here at all. Evidence matrix and the three-arm verdict: [[Dual-R9700-Cooperation]].
- **When a GPU collective dies, read the runtime, not the framework.** The message that identified
the cause came from `AMD_LOG_LEVEL=3` (`Pcie atomics not enabled, hostcall not supported` →
`AQL dispatch failed!`), never from vLLM or RCCL — the env var costs one line and prints the
failing HIP call with its return code. [[PCIe Link and Atomics]]
- **Layer split is the no-collective path, on one vendor's cards and across two — and
`--override-tensor` is what cannot ride with it.** Layers go in order and only the boundary
activation crosses PCIe once per forward pass, so it works wherever a collective cannot be
dispatched (on an RCCL-less fork it beats tensor mode, which falls back to a butterfly allreduce),
and it is the cross-vendor answer too (a CUDA + Vulkan split held a real two-device graph). But
`--override-tensor <group>=<device>` sets `has_tensor_overrides()`, a REQUIRED term of the
scheduler's `pipeline_parallel` flag, and that flag sets `n_copies`: with `n_copies == 1` a
pre-allocated weight on a non-primary device has no backend and the graph aborts (reported with
the leaf's op name `(NONE)`). `=CPU` is the only override target that works — the CPU backend is
last in the list — and pure Vulkan changes nothing: the gate is the override, not the driver mix.
Pin the spec-decode drafter to one device; on a split its home is the card holding the target's
last layers. [[Llama Cpp Multi-Device Splits]] [[x4-pair-experiment]]
- **A host↔3090 link at x4 measures 6.72 GB/s, and that is a design constraint, not a detail.**
Pinned H2D 6.72 / D2H 6.60 GB/s (≈85 % of the Gen4 x4 ceiling), pageable within 1 % of pinned —
the copies are LINK-limited, so the usual "pageable costs 2×" rule does not apply; the x4 lives in
the adapter or cable (card and port are both wider). For any engine that STREAMS weights across
it: a 3 MiB slab costs 474 µs, so a 111 GiB A3B model needing ~1.70 GiB of active experts per
token lands at 3.7 tok/s against 20.46 tok/s with the experts COMPUTED in host RAM
(`-ot exps=CPU`, where the link carries activations only). Streaming loses; RAM does not care.
[[x4-pair-experiment]]
**ROCm / GPU**
- **Layer split needs no collective** — layers go in order and only the boundary
activation crosses PCIe, so it is the fallback wherever a collective cannot be
dispatched (and on an RCCL-less fork it beats tensor mode, which falls back to
a butterfly allreduce). Pin the spec-decode drafter to one device: an unpinned
draft under tensor mode aborts the KV build, and on a split its home is the
card holding the target's last layers. Deployed history:
[[frank-medium-llamacpp-dflash2]].
- **Bundling a ROCm runtime next to a llama.cpp binary: `cp -r SRC DST/` is a
trap.** With a missing `DST`, cp creates `DST` as a copy of `SRC` and drops
one directory level. rocBLAS and hipBLASLt look for their device code at a
fixed depth (`../lib/rocblas/library`, `../lib/hipblaslt/library/<arch>`),
and the flattened copy fails at *request* time, not load time: the engine
reads the weights, answers `/health` and dies on the first completion with
`Cannot read .../TensileLibrary.dat: No such file or directory for GPU arch
: gfx1201`. Declare the destination, then copy the contents.
- **mmap must stay on** for the big-PLE models: `--no-mmap` forces the
~50 GiB PLE table into anonymous memory → silent cgroup kill, dmesg is the
only evidence. mmap pages ARE page cache. Never pin the PLE tensor with
`-ot` (23 → 13.4 t/s).
- **HIP reads bare env vars** `TEMP`, `HOST`, `GPU_MAX_HEAP_SIZE`. Serve
scripts export env files with `set -a`, so a tunable named `TEMP` leaks
into HIP (`TEMP=1.0` sent the JIT cache to path `1.0` and segfaulted every
load, fixed 2026-08-13 by renaming to `TEMPERATURE`). Never name a tunable
these; check with
- **Bundling a ROCm runtime next to a llama.cpp binary: `cp -r SRC DST/` is a trap.** With a missing
`DST`, cp creates `DST` as a copy of `SRC` and drops one directory level. rocBLAS and hipBLASLt
look for their device code at a FIXED depth (`../lib/rocblas/library`,
`../lib/hipblaslt/library/<arch>`), so the flattened copy fails at REQUEST time, not load time:
the engine reads the weights, answers `/health` and dies on the first completion with
`Cannot read .../TensileLibrary.dat: No such file or directory for GPU arch : gfx1201`. Declare
the destination, then copy the contents.
- **mmap must stay on** for the big-PLE models: `--no-mmap` forces the ~50 GiB PLE table into
anonymous memory → silent cgroup kill, with dmesg as the only evidence. mmap pages ARE page cache,
and pinning the PLE tensor with `-ot` cost 23 → 13.4 t/s. [[Llama Cpp Serving Sizing]]
- **HIP reads bare env vars** `TEMP`, `HOST`, `GPU_MAX_HEAP_SIZE`. Serve scripts export env files
with `set -a`, so a tunable named `TEMP` leaks into HIP (`TEMP=1.0` sent the JIT cache to path
`1.0` and segfaulted every load, fixed 2026-08-13 by renaming to `TEMPERATURE`). Never name a
tunable these; check with
`strings /opt/rocm-*/lib/libamdhip64.so.* | grep -xE "TEMP|HOST|GPU_MAX_HEAP_SIZE"`.
- **`-DGGML_CUDA_FA_ALL_QUANTS=ON` is required** for q8_0 KV + flash
attention on CUDA builds; without it prefill collapses ~12x (95 vs 1300+
t/s, measured 2026-08-17).
- **Flash-Next MTP heads do not load on any current loader** — they need full
per-layer `blk.<il>.hc_attn_norm.weight` and ship only `blk.48` (verified
3x). Don't re-attempt without new loader support.
- **QSA flash-attn assert:** `GGML_ASSERT(a-type == GGML_TYPE_F32)` on
qwen4exp needs `scripts/patches/qwen4exp-qsa-f32-mask.patch` — but only on
the agentionai FAST layer set; the kingjones STRIX set works unpatched.
- **GPU budget on Dankclaw is the GTT aperture (~114 GiB), not the
rocm-smi "dedicated" figure.** The Radeon 8060S (gfx1151) reports
116,732 MiB (`llama-server --list-devices`;
`gtt_total=122402963456` in `/sys/class/drm/card1/device/`). The older
"64 GiB, SVM-backed" lesson was already stale — the CIRU Orca predecessor
held 92.6 GiB of weights on the iGPU. `free`/`dmidecode` show the
CPU-side carve-out — intentional, do not change.
- **`GGML_CUDA_FA_ALL_QUANTS` is dead; do not build for it.** Verified on this host 2026-10-19:
master's `ggml/cmake/common.cmake` warns it is deprecated in favour of `GGML_CUDA_FA_QUANTS`, the
successor's DEFAULT pair list (`q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16`) already compiles the
quantized-KV kernels with the old flag OFF, and neither llama tree on this host references the old
macro anywhere under `ggml/src/ggml-cuda/`. The old "~12× prefill collapse" belongs to the older
source line. Prove a build's KV ability from its `CMakeCache.txt`. [[frank-zealot]]
- **Flash-Next MTP heads do not load on any current loader** — they need full per-layer
`blk.<il>.hc_attn_norm.weight` and ship only `blk.48` (verified 3×). Do not re-attempt without new
loader support.
- **QSA flash-attn assert:** `GGML_ASSERT(a-type == GGML_TYPE_F32)` on qwen4exp needs
`scripts/patches/qwen4exp-qsa-f32-mask.patch` — but only on the agentionai FAST layer set; the
kingjones STRIX set works unpatched. [[Llama Cpp Serving Sizing]]
- **The GPU budget on Dankclaw is the GTT aperture, not the rocm-smi "dedicated" figure.** The
Radeon 8060S (gfx1151) reports 116,732 MiB (`llama-server --list-devices`; `gtt_total` in
`/sys/class/drm/card1/device/`), and the older "64 GiB, SVM-backed" lesson was already stale.
`free`/`dmidecode` show the CPU-side carve-out — intentional, do not change. [[Dankclaw System Specs]]
**Load / crash diagnosis**
- Model load is slow (minutes cold); health returns 503 while loading; do
not kill the process. `deploy.sh` waits up to 20 minutes (`LLM_WAIT`). On
halogen, the FIRST request after a (re)start can go silent ~45–60 s
(health `engine_unresponsive`, 503s) and then self-recovers — do not
restart over it.
- Crash-loop procedure: (1) `journalctl -u <svc> -n 100` + kernel line via
`journalctl -k` (segfault at address 8 in `libamdhip64` = NULL unref in
HIP); (2) run serve script, then raw binary — different outcome isolates
the env file; (3) `gdb -batch -ex run -ex "bt 25" --args bash
scripts/serve-<svc>.sh` — llama-server buffers stdout and loses the log
tail on SIGSEGV; trust gdb, not the log; (4) `amdgpu: SVM mapping failed`
spam is a symptom, not a root cause.
- Journal signatures: client abort logs `W srv stop: cancel task, id_task =
N`; natural stop logs only `slot release` with `truncated = 0`. The server
never preempts a running generation; excess tasks queue when slots are
busy.
- Model load is slow (minutes cold) and health returns 503 while it runs: do NOT kill the process,
`deploy.sh` waits up to 20 minutes (`LLM_WAIT`). On halogen the FIRST request after a (re)start
can go silent for ~45–60 s (`engine_unresponsive`, 503s) and then self-recovers — do not restart
over it. [[frank-dragoon]]
- Crash-loop procedure: (1) `journalctl -u <svc> -n 100` AND the kernel line via `journalctl -k` (a
segfault at address 8 in `libamdhip64` is a NULL unref in HIP); (2) run the serve script, then the
raw binary — a different outcome isolates the env file; (3) `gdb -batch -ex run -ex "bt 25"
--args bash scripts/serve-<svc>.sh`, because llama-server buffers stdout and loses the log tail on
SIGSEGV: trust gdb, not the log; (4) `amdgpu: SVM mapping failed` spam is a symptom, not a root
cause. [[LLM Performance Diagnosis]]
- Journal signatures: a client abort logs `W srv stop: cancel task, id_task = N` while a natural stop
logs only `slot release` with `truncated = 0`. The server never preempts a running generation;
excess tasks queue when slots are busy.
**Host memory / swap**
- Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff`
of the old one, or live pages must land in RAM that `MemAvailable`
overstates. Dankclaw swap (64 GiB at `/swap.img`, fstab-driven) protects
the co-tenants, not frank-dragoon — pinned weights don't swap, and paging
the KV pool is worse than the OOM it prevents.
- Host swap state is invisible to `deploy.sh` (no repo file) — check
`swapon --show` before reasoning about headroom.
- **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults
to half of RAM, so the manual raise to 200 GiB (2026-10-05) vanished with the
2026-10-12 23:17 POWER FAILURE on dankserver: the cap fell to 125,988 MiB,
frank-archon's pinned-ledger preflight (floor 150,000 MiB) refused the start,
and `Restart=always` re-ran that refusal **6,919 times in ~20 h** — a unit in
`activating (auto-restart)` that exits 1 in 50 ms with no container and no
engine log is a PREFLIGHT refusal, not an engine fault. Repo law now:
`scripts/ttm-preflight.sh` (sourced by `serve-frank-archon.sh`) raises the knob
to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only
after a write the kernel rejected. Never prescribe a manual sysfs write as the
recovery procedure for a service that can boot before a human reaches it.
- **Silent RAM corruption outranks every software theory.** frank-archon's
~40k-token output corruption (2026-10-04/05, repetition/garbage in long
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and
a short in-band pattern check (60 s, 8 GiB of 246 GiB) is not a stability
proof — combined, those two instruments said "ruled out" while the RAM
was the cause. "4800 is the JEDEC floor on 5600-rated DIMMs" is a rating,
not a margin guarantee. When the fault is data-shaped (NaN states,
garbage tokens) and a software revert doesn't move it, price in the RAM
before the runtime — and remember a revert tested under the same marginal
hardware is itself confounded. See the runbook's 2026-10-05 section.
- **Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff` of the old one**, or
live pages must land in RAM that `MemAvailable` overstates. Dankclaw's 64 GiB `/swap.img`
(fstab-driven) protects the co-tenants, not an endpoint with pinned weights — paging the KV pool
is worse than the OOM it prevents. Host swap state is invisible to `deploy.sh` (no repo file):
`swapon --show` before reasoning about headroom. [[Swap Configuration]]
- **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults to half of RAM, so
a manual raise vanished with a power failure, frank-archon's pinned-ledger preflight refused the
start, and `Restart=always` re-ran that refusal 6,919 times in ~20 h — **a unit in
`activating (auto-restart)` that exits 1 in 50 ms with no container and no engine log is a
PREFLIGHT refusal, not an engine fault.** Repo law now: `scripts/ttm-preflight.sh` raises the knob
to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only after a write the
kernel rejected. Never prescribe a manual sysfs write as the recovery procedure for a service that
can boot before a human reaches it. [[frank-archon]]
- **Silent RAM corruption outranks every software theory.** frank-archon's long-conversation
repetition/garbage was marginal DDR5 at 4800 MT/s on four DIMMs, fixed by dropping to 4600.
Consumer non-ECC raises NO MCE for a silent flip, and a short in-band pattern check (60 s over
8 GiB of 246 GiB) is not a stability proof — both instruments said "ruled out" while the RAM was
the cause, and "4800 is the JEDEC floor" is a rating, not a margin guarantee. When the fault is
data-shaped (NaN states, garbage tokens) and a software revert does not move it, price in the RAM
before the runtime; a revert tested under the same marginal hardware is itself confounded.
[[frank-archon]]
**Runtime-specific guards**
- **Rootless podman cannot RAISE rlimits, and `--ulimit nofile=1048576`
dies at container create, not at runtime.** runc reports
`error setting rlimit type 7: operation not permitted` (type 7 =
RLIMIT_NOFILE) when the requested hard limit exceeds the host process's
hard limit — on Dankclaw that ceiling is 524288. Inherited-equal limits
are fine, so `memlock=-1:-1` works iff the unit grants
`LimitMEMLOCK=infinity` (systemd is privileged; the value flows through
podman to runc without any raise). Upstream halogen's own run lines pass
NO `--ulimit` at all — copy vendor run lines before adding flags
(measured 2026-10-05 on the frank-dragoon cutover).
- **Kairic Edge** (if ever re-deployed): the pinned CK commit needs
`third_party/composable_kernel/include/ck/config.h` generated manually
(untracked; `git clean` removes it), and its greedy-argmax fastpath rejects
non-greedy requests and reasoning budgets. Re-test chat before production.
- **FastMTP sidecar**: build must be commit `4df29be4` + the HauhauCS patch;
stock binaries fail the draft load with a tensor-shape mismatch.
- **A GGUF's custom tensor type picks the runtime — read the histogram before
promising a file to a loader.** `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` is
ftype 115 with 311 tensors of type 103 (`GGML_TYPE_Q8_0_ROCMFPX`); stock
llama.cpp and gufo refuse it at load, so ROCmFPX was not a preference but
the only door. Counting tensor types out of the GGUF KV section takes one
python read and settles a runtime argument in minutes. The build identity
matters as much as the fork: the 2026-08-25 ROCmFPX builds SIGSEGV in
`ggml_compute_forward_set_rows` on this same file, so
`scripts/build-rocmfpx.sh` pins commit `36e9acd` by SHA.
- **Speculative-decode flags move as a SET.** Handing the ROCmFPX fork
`--model-draft` with no `--spec-type` (the naive way to build a "no MTP"
bench arm) segfaulted at load, core dumped, no diagnostic (2026-10-02). A
baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND
`--spec-type`, or it measures a crash instead of the absence of drafting.
- **An orphaned scratch server taxes the whole host.** A bench child left
alive by a client-side exception kept ~26 GiB of weights resident on a
121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
Record the child pid and kill it in a `finally`, then prove `free` came back
before believing a number. Kill by pid — never `pkill -f llama-server`,
which also hits frank-zealot.
- **Rootless podman cannot RAISE rlimits, so `--ulimit nofile=1048576` dies at container CREATE**,
not at runtime: runc reports `error setting rlimit type 7: operation not permitted` (type 7 =
RLIMIT_NOFILE) when the requested hard limit exceeds the host process's ceiling (524288 on
Dankclaw). Inherited-equal limits are fine, so `memlock=-1:-1` works iff the unit grants
`LimitMEMLOCK=infinity` — systemd is privileged and the value flows through podman to runc without
any raise. Upstream halogen's own run lines pass NO `--ulimit` at all: copy vendor run lines
before adding flags. [[frank-dragoon]]
- **A GGUF's custom tensor type picks the runtime — read the histogram before promising a file to a
loader.** `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` is ftype 115 over 311 tensors of type 103
(`GGML_TYPE_Q8_0_ROCMFPX`), so stock llama.cpp and gufo refuse it at load and the fork was not a
preference but the only door. Counting tensor types out of the GGUF KV section takes one python
read and settles a runtime argument in minutes. Build identity matters as much as the fork: the
2026-08-25 ROCmFPX builds SIGSEGV in `ggml_compute_forward_set_rows` on this same file, so
`scripts/build-rocmfpx.sh` pins commit `36e9acd` by SHA. [[GGUF Compatibility]]
- **Speculative-decode flags move as a SET.** Handing the fork `--model-draft` with no `--spec-type`
(the naive way to build a "no MTP" bench arm) segfaulted at load, core dumped, no diagnostic. A
baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND `--spec-type`, or it
measures a crash instead of the absence of drafting. [[frank-zealot]]
- **An orphaned scratch server taxes the whole host.** A bench child left alive by a client-side
exception kept ~26 GiB of weights resident on a 121 GiB box and pushed `available` to 3 GiB, so
every later rate was suspect. Record the child pid, kill it in a `finally`, then prove `free` came
back before believing a number — and kill by pid, never `pkill -f llama-server`.
[[LLM Bench Windows and Measurement]]
- Retired-runtime traps that can come back: **Kairic Edge** needs
`third_party/composable_kernel/include/ck/config.h` generated manually (untracked, so `git clean`
removes it) and its greedy-argmax fastpath rejects non-greedy requests and reasoning budgets, so
re-test chat before production; the **FastMTP sidecar** must be commit `4df29be4` + the HauhauCS
patch, since stock binaries fail the draft load with a tensor-shape mismatch.
**llama.cpp serving**
- **MTP is a property of the FILE, not of the flags.** The `-mtp` GGUF build
carries the head as `blk.<n>.nextn.*` (`qwen35.nextn_predict_layers = 1`);
`--spec-type draft-mtp` then sets `load_mtp`, and the draft context runs on
the MAIN model's weights (`shares_model`), so there is no second copy of a
12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
tensors at all, and pointing `--model-draft` at the file would load it twice.
- **What can be chained, what cannot, and what a chain costs on a hybrid.**
`--spec-type a,b` builds every enabled implementation and chains drafting
through them, in a fixed internal order (ngram types first, then the
drafters — CLI order is irrelevant). The gate is `ctx_dft`: **draftless**
types (the ngram family) are added unconditionally, while `draft-mtp`,
`draft-dflash`, `draft-dspark` and `draft-eagle3` all require a draft
CONTEXT. So a chain may hold one context-backed drafter plus any number of
ngram types — `draft-dflash,ngram-mod` is legal and measured (2.6x the
drafter alone on prompt-echo traffic, byte-identical to it on prose at the
fleet's 48/64 window) — but `draft-mtp,draft-dflash` dies at load: *"context
type MTP requested but model doesn't contain MTP layers"*, because MTP
builds its context against whatever model the draft context holds. Two
context-backed types cannot share one process, and MTP's head only loads
when MTP **is** the draft model. **A LEGAL chain can still lose on a
recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35 hybrid
measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer
logs `recurrent state rollback is not compatible with 'ngram-mod' -
disabling rollback support`, and MTP verification depends on that rollback,
so acceptance drops on ALL traffic even where the matcher fires. Measure
chains per architecture, never by analogy.
- **A drafter's KV scales with the TARGET's context, and its window policy
decides the price.** The draft context is built at `llama_n_ctx(ctx_tgt)` with
no flag to cap it, so a drafter declaring NO sliding window caches the whole
window on `--spec-draft-device` and can OOM the first draft graph capture
AFTER `/health` has answered. Read `sliding_window` out of the drafter's GGUF
header before sizing, and prefer a skewed `-ts` over a context cut. Worked
example: [[frank-medium-llamacpp-dflash2]].
- **A draft context bills its own f16 KV over the WHOLE POOL, and on a
single-card endpoint that squeeze can cost more than the drafter wins.** With
`--parallel N` the drafter pays the pool, not the slot, plus its own compute
buffers. So the MTP question is never "does the file carry the head" but "what
do draft KV + buffers cost against what the batch knobs leave for compute" —
a per-shape answer. Measure the arm, never the flag; the booted arm table is
on [[frank-zealot]].
- **A quant's scale overhead belongs in its byte rate; doubling a measured
figure to "get the f16 number" invents 6.25 %.** q8_0 is 1.0625 B/element
(32 codes + one 2-byte scale per 32-block), f16 is exactly 2.0 ⇒ the ratio is
**1.882×**, not 2×. Derive byte rates from element count × block layout, then
check them against the boot ledger — this repo carried an invented 2× number
for weeks. Current KV math: [[frank-zealot]].
- **`--ubatch-size` has an OPTIMUM, not a plateau, and the curve is
BACKEND-SPECIFIC — never carry it across hosts.** Its compute buffers land
about TWICE as heavy on the card that holds the output layers and the drafter,
so a split balanced at one batch is unbalanced at another: re-measure the
balance after any batch change. The logical `--batch-size` is nearly free.
A large-batch allocation can fail on BUFFER CONTIGUITY while totals still fit
— no totals model can see that. Ladders: [[frank-medium-llamacpp-dflash2]]
(R9700 split) and [[frank-zealot]] (CUDA 3090).
- **On the CUDA 3090 that same knob has no interior optimum up to 2048.** A
prompt shorter than the ubatch cannot see the knob at all — that is the
control that proves the knob, not the traffic, moved the number. When a bench
IS ordered, A/B it by restarting the SERVICE through a transient
`FRANK_ZEALOT_ENV_FILE` drop-in, never a scratch port: one card holds one
server. Numbers: [[frank-zealot]].
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
size** (a DeltaNet boundary and a full-attention boundary can differ by ~1 GiB
each with its own KV), so balance by measurement. An affine fit reproduces the
deployed point and overstates the gaps in between.
[[frank-medium-llamacpp-dflash2]].
- **ngram-mod prints no startup banner.** The load log shows the drafter's
`adding speculative implementation 'draft-dflash'` and nothing at all for the
matcher, so a chain can look unengaged while it is working. Identify it
behaviourally (`draft_n` > 0 only where text repeats) or by the per-task
`draft acceptance = ... mean len = ...` line. It costs no VRAM (a 64 MiB
host-RAM hash table) and its window decides which traffic it can see: too
short and it proposes on ordinary prose fragments, which costs verification
without acceptance.
- **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the
per-slot number is the contract — not `ctx-size`.** `n_ctx_slot =
CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT with
a 400 even when the pool has room, so `models.yml` `contextWindow` must follow
every reslice. Both slot directions cost or buy window, never pool; shorter
slots decode slightly faster (less attention to scan, one fewer recurrent
state). Measured reslices: [[frank-zealot]].
- **Price a VRAM budget from two boots, never from a formula — and an mmproj
costs ~1.5× its file size** (the encoder's activation and compute buffers ride
with it). Subtracting two booted rows isolates it exactly; extrapolating
weights+KV+"a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7.
Then solve `KV ≤ card − non-KV − headroom` and round to a multiple of
`PARALLEL × 4096`. The two-boot table: [[frank-zealot]].
- **`images` and `image_url` are NOT interchangeable: only the content part
reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
model still invents an answer if the client used the legacy top-level
`images` field on `/v1/chat/completions` — frank-zealot answered "Brown." for
a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
failure is silent because a hallucinated color is a plausible answer. Probe
vision with a solid color and expect an exact word.
- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
`500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
then served frank-dragoon: every tool-using agent on that endpoint was broken
and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults
`--jinja` on and the serve script pins it; halogen also accepts `tools` and
returns `tool_calls`. Check tool support before routing a role at an endpoint.
- **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
option and the same ~12x prefill collapse described in the ROCm note above.
f16 KV needs nothing; frank-zealot ran f16 for that reason until
2026-09-17, when `build-cuda` was rebuilt with the option ON so the
endpoint could take q8_0 KV.
- **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both
`-dev` and `-ts` follow this: `-dev CUDA0,Vulkan1` is two *single-device*
configurations, never one two-device run, and `-ts 0.85,0.15` is two sweep
values each becoming `[0.85, 0, …]` / `[0.15, 0, …]`, i.e. device 0 takes
everything. Write `-dev A/B -ts x/y`. This artifact produced a wrong fleet
conclusion on 2026-09-16 — "CUDA + Vulkan cannot serve one model", with two
bogus OOMs that were really "all 111 GiB on the 24 GiB card" — and the bench
table's own `dev` column was printing the truth the whole time. Any
placement-dependent number needs an independent check: sample the other
- **MTP is a property of the FILE, not of the flags.** The `-mtp` export carries the head as
`blk.<n>.nextn.*` (`qwen35.nextn_predict_layers = 1`); `--spec-type draft-mtp` sets
`load_mtp` and the draft context runs on the MAIN model's weights (`shares_model`), so there
is no second copy of a 12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
tensors at all, and pointing `--model-draft` at that file loads it twice. [[frank-zealot]]
- **What can be chained: ONE context-backed drafter plus any number of draftless ngram types.**
`--spec-type a,b` chains in a fixed internal order (CLI order irrelevant) and the gate is
`ctx_dft`: the ngram family is added unconditionally while `draft-mtp`, `draft-dflash`,
`draft-dspark` and `draft-eagle3` all need a draft CONTEXT. So `draft-dflash,ngram-mod` is
legal (measured 2.6x the drafter alone on prompt-echo traffic) but `draft-mtp,draft-dflash`
dies at load — *"context type MTP requested but model doesn't contain MTP layers"*. **A LEGAL
chain can still lose on a recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35
hybrid measured −15 % prose / −7 % block-copy, because adding the ngram layer logs
`recurrent state rollback is not compatible with 'ngram-mod' - disabling rollback support` and
MTP verification depends on that rollback. Measure chains per architecture, never by analogy.
[[frank-dragoon]]
- **A drafter's KV scales with the TARGET's context, and its window policy decides the price.**
The draft context is built at `llama_n_ctx(ctx_tgt)` with no flag to cap it, so a drafter
declaring NO sliding window caches the whole window on `--spec-draft-device` and can OOM the
first draft graph capture AFTER `/health` has answered. Read `sliding_window` out of the
drafter's GGUF header before sizing, and prefer a skewed `-ts` over a context cut.
[[Llama Cpp Multi-Device Splits]]
- **A draft context bills its own f16 KV over the WHOLE POOL, not the slot**, plus its own
compute buffers, so on a single-card endpoint that squeeze can cost more than the drafter
wins. The MTP question is never "does the file carry the head" but what draft KV + buffers
cost against what the batch knobs leave for compute. Measure the arm, never the flag.
[[frank-zealot]]
- **A quant's scale overhead belongs in its byte rate; doubling a measured figure to "get the
f16 number" invents 6.25 %.** q8_0 is 1.0625 B/element (32 codes + one 2-byte scale per
32-block), f16 is exactly 2.0 ⇒ the ratio is **1.882×**, not 2×. Derive byte rates from element
count × block layout, then check them against the boot ledger — this repo carried an invented
2× number for weeks. Current KV math: [[frank-zealot]]
- **`--ubatch-size` has an OPTIMUM, not a plateau, and the curve is BACKEND-SPECIFIC — never
carry it across hosts.** Its compute buffers land about TWICE as heavy on the card that holds
the output layers and the drafter, so a split balanced at one batch is unbalanced at another:
re-measure the balance after any batch change. The logical `--batch-size` is nearly free; a
large-batch allocation can fail on BUFFER CONTIGUITY while totals still fit, which no totals
model can see. A prompt shorter than the ubatch cannot see the knob at all — that is the
control that proves the knob, not the traffic, moved the number. The 2× R9700 split peaks at
512; the CUDA 3090 has no interior optimum up to 2048. When a bench IS ordered, A/B it by
restarting the SERVICE through a transient `FRANK_<SVC>_ENV_FILE` drop-in, never a scratch
port: one card holds one server. Ladders: [[frank-medium-llamacpp-dflash2]] and
[[frank-zealot]].
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same size** (a DeltaNet
boundary and a full-attention boundary can differ by ~1 GiB each with its own KV), so balance
by measurement. An affine fit reproduces the deployed point and overstates the gaps in between.
[[Llama Cpp Multi-Device Splits]]
- **`ngram-mod` prints no startup banner.** The load log names `draft-dflash` and nothing for the
matcher, so a chain can look unengaged while it works. Identify it behaviourally (`draft_n` > 0
only where text repeats) or by the per-task `draft acceptance = ... mean len = ...` line. It
costs no VRAM (a 64 MiB host-RAM hash table) and its window decides which traffic it can see:
too short and it proposes on ordinary prose fragments, costing verification without acceptance.
[[Llama Cpp Multi-Device Splits]]
- **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the per-slot number is
the contract — not `ctx-size`.** `n_ctx_slot = CTX_SIZE / PARALLEL`, and llama.cpp refuses a
prompt longer than the SLOT with a 400 even when the pool has room, so `models.yml`
`contextWindow` MUST follow every reslice. Both slot directions cost or buy window, never
pool; shorter slots decode slightly faster. Measured reslices: [[frank-zealot]].
- **Price a VRAM budget from two boots, never from a formula — and an mmproj costs ~1.5× its file
size** (the encoder's activation and compute buffers ride with it). Subtracting two booted rows
isolates it exactly; extrapolating weights+KV+"a bit for buffers" is what leaves 1.1 GiB free
when you meant 2.7. Then solve `KV ≤ card − non-KV − headroom` and round to a multiple of
`PARALLEL × 4096`. **The projector is NOT billed in llama.cpp's own fit check**: the engine
prints its worst case and fits WITHOUT it, so price a shape from settled `nvidia-smi` after a
real boot — the projector, and any drafter context, lands after `/health` would already have
answered (same class as the DSpark drafter OOM). Two-boot table: [[frank-zealot]].
- **`images` and `image_url` are NOT interchangeable: only the content part reaches the model.**
`--mmproj` loads, `usage.prompt_tokens` rises, and the model still invents an answer if the
client used the legacy top-level `images` field on `/v1/chat/completions` — frank-zealot
answered "Brown." for a solid-red PNG that way and "Red." through the OpenAI `image_url` part.
The failure is silent because a hallucinated color is a plausible answer; probe with a solid
color and expect an exact word. [[frank-zealot]]
- **`--jinja` is what `tools` needs.** A subagent died with `500 tools param requires --jinja
flag` on the ik_llama.cpp runtime that served frank-dragoon: every tool-using agent on that
endpoint was broken and the failure read like a model fault. Mainline llama.cpp defaults
`--jinja` on and the serve scripts pin it; halogen also accepts `tools` and returns
`tool_calls`. Check tool support before routing a role at an endpoint.
- **A model's own template can REJECT the harness's thinking level.** Qwen3.8's template accepts
only `xhigh`, `medium` and `low` for `reasoning_effort` and raises on anything else — HTTP 500,
not a clamp — so the harness's levels must be folded onto that set in `models.yml`
(`compat.reasoningEffortMap`). A role whose level is unmapped fails on its FIRST request and
the failure names Jinja, not the model: every subagent on `frank-zealot` died with
`Unexpected reasoning effort high` until `high: xhigh` was added. Halogen accepts the wider
set, so the two entries legitimately differ.
- **Thinking tokens count against `max_tokens`, so a short-budget vision probe reads as
blindness.** A probe at `max_tokens: 64` returned `content: ""` with `finish_reason: "length"`
and half a thought in `reasoning_content` while the image HAD reached the model (322 prompt
tokens vs 42 for the text alone). Budget ≥512 for any thinking model you probe for vision.
- **The FIRST request after a boot is a warmup artifact**, and the 3× spread on one shape makes
graph-capture cost read as a model fault. Benchmark the second request or later.
[[frank-zealot]]
- **A fork build hides its own dtype laws.** Vendor builds carry their own compile options, so
prove a build's KV behaviour from the KV buffer line in the boot ledger and its
`CMakeCache.txt` — never from the build flags you remember passing. [[frank-zealot]]
- **A llama.cpp endpoint is invisible to Prometheus unless `--metrics` is passed**: without it
the engine answers **501 Not Implemented** on `/metrics`, so the scrape fails while `/health`
answers 200 and `LlmEndpointDown` fires beside a healthy service. halogen serves `/metrics`
natively, so the asymmetry reads as "only the 3090 is broken". Diagnose with `lastError` from
`/api/v1/targets`, not the alert list, and note Alertmanager has no persistence: a reboot
re-stamps `startsAt` on an alert that has fired for weeks. [[Observability]] [[frank-zealot]]
- **A leftover bench drop-in silently hijacks the next deploy, and no health gate sees it.** The
sanctioned bench-window method (a transient `Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env`
drop-in sourcing the repo env) outlives the bench unless retired: a leftover
`frank-zealot.service.d/ptq1.conf` made the cutover deploy boot a **different model at a
different shape** while `/health` answered 200, `/v1/models` echoed the alias (a flag, not a
fact) and the Playwright gate passed 33/33. After deploying any recently-benched LLM service:
`systemctl show <svc> -p DropInPaths` (must be empty), then
`tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot ledger — and retire drop-ins
to `~/.AGENTTRASH/<date>-<topic>/` + `daemon-reload`. [[frank-zealot]]
- **A bench verdict that never reaches the repo is indistinguishable from a rogue arm.** That
same arm served production for 24 minutes with real numbers (~48 t/s decode, 801 t/s prefill at
141k) that lived only in `/tmp` env files and journald, so the next agent read the surviving
drop-in as an unfinished experiment and reported a hijack. Write the measurement into the
endpoint page and the arm script's header in the session that takes it — `/tmp` and the journal
are not records.
- **`llama-bench` separates a LIST with `/` and SWEEP values with `,`** — for both `-dev` and
`-ts`. `-dev CUDA0,Vulkan1` is two *single-device* configurations, never one two-device run,
and `-ts 0.85,0.15` is two sweep values each putting everything on device 0. Write
`-dev A/B -ts x/y`. This artifact produced a wrong fleet conclusion ("CUDA + Vulkan cannot serve
one model") from two bogus OOMs, and the bench table's own `dev` column was printing the truth
the whole time. Any placement-dependent number needs an independent check: sample the other
device's memory (`nvidia-smi`) while the run is loaded.
- **A model's own template can REJECT the harness's thinking level.** Qwen3.8's
template accepts only `xhigh`, `medium` and `low` for `reasoning_effort` and
raises on anything else — HTTP 500, not a clamp — so the harness's levels
must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`).
A role whose level is unmapped fails on its FIRST request, and the failure
names Jinja, not the model: every subagent on `frank-zealot` died with
`Unexpected reasoning effort high` until `high: xhigh` was added on
2026-09-16. Halogen accepts the wider set, so the two entries legitimately
differ.
- **The vision projector is NOT billed in llama.cpp's own fit check.** The
engine prints `[mtmd] estimated worst-case memory usage of mmproj is
873.10 MiB` and then fits WITHOUT it: frank-zealot's Bonsai-2 boot projected
21,439 MiB and settled at 22,664 (2026-10-17). Price a 3090-class shape from
settled `nvidia-smi` after a real boot, never from
`model + context + compute` — the projector (and any drafter context) lands
after `/health` would already have answered. Same failure class as the
DSpark drafter OOM.
- **The FIRST request after a boot is a warmup artifact.** Measured on
Bonsai-2/3090: 2.6 t/s prefill and 37.7 t/s decode on request #1, then
1,258 t/s prefill and 54.7 t/s (per stream, two concurrent) on the next — a
3× spread on the same shape. Benchmark the second request or later, or the
graph-capture cost reads as a model fault.
- **Thinking tokens count against `max_tokens`, so a short-budget vision probe
reads as blindness.** The Bonsai-2/`image_url` probe at `max_tokens: 64`
returned `content: ""` with `finish_reason: "length"` and half a thought in
`reasoning_content` — the image HAD reached the model (322 prompt tokens vs
42 for the text alone). At 512 tokens the same image answers "Red". Budget
≥512 for any thinking model you probe for vision.
- **Quantized KV is not the only dtype law a fork build can hide.** Vendor
builds carry their own compile options: frank-zealot's PrismML prebuilt
served `--cache-type-v q8_0` under flash attention at the exact 34,816
B/token (the `GGML_CUDA_FA_ALL_QUANTS` door), so a build decision must be
proven by the KV buffer line in the boot ledger, not by the build flags you
remember passing.
- **A llama.cpp endpoint is invisible to Prometheus unless `--metrics` is passed.**
Without it the engine answers **501 Not Implemented** on `/metrics`, so the scrape
fails while `/health` answers 200 and `LlmEndpointDown` fires beside a healthy
service — that is how `frank-zealot` sat red across four payload generations
(the oldest retained sample of `up{job="frank-zealot"}`, 2026-10-11, is already
0). halogen serves `/metrics` natively, so the asymmetry reads as "only the 3090
is broken". Diagnose with `lastError` from `/api/v1/targets`, not the alert list,
and note Alertmanager has no persistence: a reboot re-stamps `startsAt` on an
alert that has been firing for weeks.
- **A leftover bench drop-in silently hijacks the next deploy, and no health
gate sees it.** The sanctioned bench-window method — a transient
`Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env` drop-in, the arm env
`source`ing the repo env and then overriding it — outlives the bench unless
it is retired. On 2026-10-18 a leftover `frank-zealot.service.d/ptq1.conf` —
the arm Frank had ordered at 02:36 and measured from 03:14 — made the cutover
deploy boot a **different model at a different shape** (3 slots of 133,888,
not 3 × 262,144) while `/health` answered 200, `/v1/models` echoed the alias
(a flag, not a fact) and the Playwright gate passed 33/33. After deploying any
recently-benched LLM service: `systemctl show <svc> -p DropInPaths` (must be
empty), then `tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot
ledger — and retire drop-ins to `~/.AGENTTRASH/<date>-<topic>/` +
`daemon-reload`.
- **A bench verdict that never reaches the repo is indistinguishable from a
rogue arm.** Same window: PTQ1_0 served production for 24 minutes with real
numbers (~48 t/s decode, 801 t/s prefill at 141k), but the figures lived only
in `/tmp` env files and journald, so the next agent on the host read the
surviving drop-in as an unfinished experiment and reported a hijack. Write the
measurement into the runbook and the arm script's header in the session that
takes it — `/tmp` and the journal are not records.
**Ops**
- Never `pkill -f` a pattern that appears in your own command line — it kills
your ssh session. Use `pkill -x <exact-name>`.
- Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in
committed scripts (no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts
without it.
- ComfyUI on Dankclaw runs with **GPU access** (`/dev/kfd` + `/dev/dri` via
`AddDevice`, `GroupAdd=keep-groups`, plus the udev rule that makes those nodes
world-writable — user-unit spawns carry no supplementary groups). It must NOT
pass `--cpu`: the gfx1151 ROCm wheels exist and the unit's `--cpu` era ended
2026-09-16, when a stale device-less container was found restarted 9,146
times (`RuntimeError: No HIP GPUs are available`).
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
ports off the gluetun service.
- Dankclaw boots **`iommu=pt`** since 2026-10-18. The boot line is tracked in
`dankclaw/host/grub` and moves ONLY through `scripts/set-dankclaw-bootline.sh`
(diff, install, `update-grub`, verify `grub.cfg`, `--reboot`) — never by
hand-editing `/etc/default/grub`. It replaced `amd_iommu=off` (2026-09-11),
adopted on the vendor's 13–16 % prefill figure and never measured here: the
local A/B on byte-identical traffic says identity mode costs **nothing**
(+2.0 %, 1,616 t/s cold prefill), so the XDNA2 NPU returned for free —
`amdxdna` binds `c6:00.1` (PASID width 10, identity group 26) and the old
`err=-19` only ever asked for an enabled IOMMU. **AMD PerfOpt** (iommu tree
2026-09-24, Linux 7.4) still skips the page walk outright for the privileged
iGPU, but it arms only in the identity domain and 26.04 caps at `7.0.0-38`.
- Rootless Quadlet containers run with `--replace --rm`: `podman logs` covers
only the CURRENT instance; history across recreations lives in
`journalctl --user -u <name>.service`. Read rootless storage WITHOUT sudo
(per-user storage — `sudo podman logs` finds nothing).
- **A dead rootless podman pause process breaks ALL container management on the
host, and `podman system migrate` panics in 5.7.0.** Symptoms: every `podman`
command fails with `invalid internal status … could not find any running
process`, every container start fails binding podman's bridge addresses
(`Error starting server failed to bind udp listener on 10.89.x.1:53: Cannot
assign requested address`), and every `Restart=always` quadlet loops. Container
storage is intact; what is gone is `/run/user/<uid>/libpod/tmp/alive` (a UUID
token, not a pid) and the rootless netns holding the bridges — deleting the
stale token is not enough, and `podman system migrate` segfaults in
`teardownNetworkBackend`. Repair: **reboot** (`/run/user` clears and all 28
quadlets return on their own — check `Linger=yes` and `WantedBy=default.target`
first). Seen 2026-09-16, hours into heavy GPU work on the same host — but the
trigger is NOT established (`journalctl -k` recorded no OOM kill; the kernel
had no other word on it). Treat the pause process as host state that can die
for its own reasons, and reach for the reboot when it does.
- **A container cannot reach a `Network=host` service at the host's LAN IP.**
netavark DNATs *published* ports only: from a bridge container
`192.168.0.143:<published>` answers while `192.168.0.143:<host bind>` and the
gateway `10.89.1.1` are both refused. Use `host.containers.internal`
(169.254.1.2). This is what 502'd `hermes.franksdank.cloud` until the Caddyfile
upstream was changed on 2026-09-16 (its vhost comment had claimed the LAN IP
was open from that bridge — it never was).
- **A `*.gguf` filter misses mradermacher's payloads: large quants ship as
`.partNofM` fragments.** `mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF`
carries Q4_K_M (37.00 + 37.00 + 36.97 = 110.97 GiB) and Q4_K_S (35.00 + 35.00
+ 34.09 = 104.09 GiB) as parts, while the smaller rungs (Q2_K, Q3_K_*, IQ3_*,
IQ4_XS) are single files. Fetch the parts, verify each against the Hub's LFS
sha256, concatenate in numeric order — that is how the deployed 111 GiB file
was assembled. Listing the Hub tree and filtering `endswith(".gguf")` makes
Q4_K_S look like it does not exist.
- **A fetch script must `-f` and verify, or it reports success on an access
error.** `curl -o FILE URL` exits 0 on HTTP 401/403 and writes the error body
as the "artifact": a gated repo yielded three 153-byte shards with identical
sha256 values and a cheerful `DOWNLOAD_DONE` in 20 seconds for 104 GiB. Use
`-f`/`--fail-with-body`, compare each file's byte count to the Hub's declared
size, and check the LFS sha256 — `scripts/download-frank-medium-gguf.sh`
already does all three; ad-hoc fetches tend not to. Gated repos (e.g.
`orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF`, same 104.09 GiB Q4_K_S)
return 403 even with the host's HF token until that account accepts the gate.
- **A scout verdict is a lead, never a finding.** In the 2026-10-19 vault de-dup
campaign, four read-only scouts worked the same pages: one classified era pages as
stale *because* they document retired payloads (that is an era page's content), one
reported files that do not exist, one reported duplications that grep disproved
(`457.9 t/s` and the ping-RTT table are solo-held by `Ethernet Bandwidth Math`),
and one cited a line number past the end of the file it judged. Re-verify every
claimed duplicate and every claimed staleness against the file and today's fleet
state before editing; a scout yield buys no evidence.
- Never `pkill -f` a pattern that appears in your own command line — it kills your ssh session.
Use `pkill -x <exact-name>`.
- Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in committed scripts
(no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts without it.
- **ComfyUI on Dankclaw runs with GPU access** (`/dev/kfd` + `/dev/dri` via `AddDevice`,
`GroupAdd=keep-groups`, plus the udev rule that makes those nodes world-writable — user-unit
spawns carry no supplementary groups) and must NOT pass `--cpu`: the gfx1151 ROCm wheels exist
and a stale device-less container was found restarted 9,146 times. [[ComfyUI]]
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its ports off the gluetun
service. [[SearXNG]] [[Gluetun]]
- **Dankclaw's kernel command line is a repo file.** It boots **`iommu=pt`** since 2026-10-18,
tracked in `dankclaw/host/grub`, and moves ONLY through `scripts/set-dankclaw-bootline.sh`
(diff, install, `update-grub`, verify `grub.cfg`, `--reboot`) — never by hand-editing
`/etc/default/grub`. It replaced `amd_iommu=off`, and the local A/B on byte-identical traffic
says identity mode costs nothing, so the XDNA2 NPU returned for free.
[[dankclaw-iommu-translation-off]]
- **Rootless Quadlet containers run with `--replace --rm`**: `podman logs` covers only the CURRENT
instance; history across recreations lives in `journalctl --user -u <name>.service`. Read
rootless storage WITHOUT sudo — it is per-user storage, so `sudo podman logs` finds nothing.
[[Quadlet Lifecycle]]
- **A dead rootless podman pause process breaks ALL container management on the host, and
`podman system migrate` panics in 5.7.0.** Every `podman` command fails with
`invalid internal status`, every start fails binding the bridges
(`failed to bind udp listener on 10.89.x.1:53`), every `Restart=always` quadlet loops — while
container storage stays intact. What is gone is `/run/user/<uid>/libpod/tmp/alive` (a UUID
token) and the rootless netns; deleting the stale token is not enough. Repair: **reboot**
(`/run/user` clears and the quadlets return on their own — check `Linger=yes` and
`WantedBy=default.target` first). The 2026-09-16 occurrence had no OOM in `journalctl -k`:
treat the pause process as host state that can die for its own reasons. [[Quadlet Lifecycle]]
- **A container cannot reach a `Network=host` service at the host's LAN IP**: netavark DNATs
*published* ports only, so the host bind and the `10.89.1.1` gateway are both refused from a
bridge container while the published port answers. Use `host.containers.internal`
(169.254.1.2). This 502'd `hermes.franksdank.cloud` until its Caddyfile upstream changed.
[[Container Host Reachability]]
- **A `*.gguf` filter misses mradermacher's payloads: large quants ship as `.partNofM`
fragments** while the smaller rungs are single files, so filtering `endswith(".gguf")` makes a
whole quant class look absent. Fetch the parts, verify each against the Hub's LFS sha256,
concatenate in numeric order. [[Model Swap and Rollback]]
- **A fetch script must `-f` and verify, or it reports success on an access error.**
`curl -o FILE URL` exits 0 on HTTP 401/403 and writes the error body as the "artifact": a gated
repo yielded three 153-byte shards with identical sha256 and a cheerful `DOWNLOAD_DONE` in 20
seconds for 104 GiB. Use `-f`/`--fail-with-body`, compare each file's byte count to the Hub's
declared size, and check the LFS sha256 (`scripts/download-frank-medium-gguf.sh` does all three;
ad-hoc fetches tend not to). Gated repos 403 even with the host's HF token until that account
accepts the gate. [[Model Swap and Rollback]]
- **A scout verdict is a lead, never a finding.** In the 2026-10-19 vault de-dup campaign six
read-only scouts worked the same pages: one classified era pages as stale *because* they
document retired payloads (that is an era page's content), one reported files that do not exist,
one reported duplications that grep disproved (`457.9 t/s` and the ping-RTT table are solo-held
by `Ethernet Bandwidth Math`), one cited a line number past the end of the file it judged, and
one drafted an Eras table with invented windows, payloads and rollback paths. Re-verify every
claimed duplicate, every claimed staleness and every quoted holder against the file and today's
fleet state before editing; a scout yield buys no evidence.
## Runbook Index
+293
View File
@@ -0,0 +1,293 @@
#!/usr/bin/env python3
"""OKF v0.1 conformance linter for a docs bundle (default serverAppConfigs-docs).
Checks the hard rules of the Open Knowledge Format spec plus the maintenance
guidance, so "conforms to OKF" is a test and not a claim:
FRONTMATTER every non-reserved .md has parseable YAML frontmatter with a
non-empty `type` from the spec's value set
TYPE the value is one of reference / howto / tutorial / explanation
DESC `description` is present, non-blank, not the title, <= 120 chars,
and does not restate a fleet quantity
TIMESTAMP `timestamp` present, ISO 8601, not in the future
LENGTH lines between 1 and 500 inclusive
INDEX-FM no index.md may carry frontmatter (index.md is reserved)
LOG-DATE log.md date headings are `## YYYY-MM-DD` or `## YYYY-MM-DD — Title`,
one entry per date, newest first
INDEX-COVER every index.md links each of its sibling pages; the root index
links every page in the bundle
INDEX-NUM an index table row restates a fleet quantity — a digit run of
three or more, a digits-plus-unit figure, a port, or a
multiplier. Names (Qwen3.6, RDNA4, RTX 3090, PICO-8, P4) are
not quantities and are allowed
CITATIONS informational: pages with factual claims and no Citations
section are counted, never failed
LENGTH lines between 1 and 500 inclusive. A page may opt out of the
ceiling with `okf-exempt: length` in its frontmatter: AGENTS.md
makes one page per endpoint permanent, and an endpoint page with
a full Eras table cannot split without creating a second owner
of the same shape (tests/docs_completeness.bats enforces that)
Exit 0 when there are no violations.
"""
import datetime
import re
import sys
from pathlib import Path
TYPES = {"reference", "howto", "tutorial", "explanation"}
RESERVED = {"index.md", "log.md", "AGENTS.md"}
# A quantity, as opposed to a name: a comma-grouped figure, a run of 3+ digits,
# a unit-bearing figure, a port (":7770"), or a multiplier ("2x", "3 *").
QUANTITY = re.compile(
r"\d{1,3}(?:,\d{3})+" # 262,144
r"|\d{3,}" # 200000, 7770, 3090
r"|\d+(?:\.\d+)?\s*(?:GiB|MiB|GB|MB|TB|tok/s|t/s|x\b|%)"
r"|[:×]\s*\d"
r"|\d+\s*[x×]\s*\d"
)
# Product / model / standard names that legitimately contain digits.
NAMES = re.compile(
r"Qwen[\d.]*[-\w]*|Gemma\s?\d[\w.]*|MiMo[-\w.\d.]*|Ornith[-\w.\d]*"
r"|Bonsai[-\w.\d]*|Ling[-\w.\d.]*|DeepSeek[-\w.\d]*|GLM[-\w.\d]*"
r"|PICO-\d+|RDNA\s?\d+|R\d{4}|RTX\s?\d+|XDNA\d|gfx\d+|CUDA\s?[\d.]+"
r"|\d+\s?GbE|Wi-?Fi\s?\d|v?\d+\.\d+(?:\.\d+)?|b\d+|P\d\b|A\d+\b"
r"|frank-\w*2026-\d\d-\d\d|2026-\d\d-\d\d",
re.I,
)
DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}")
FM_KEY = re.compile(r"^([A-Za-z][\w-]*):\s*(.*)$")
def split_frontmatter(text):
if not text.startswith("---"):
return None, text
end = text.find("\n---", 3)
if end < 0:
return None, text
body_start = text.find("\n", end + 1)
return text[3:end], text[body_start + 1 :] if body_start >= 0 else ""
def fields(raw):
out = {}
if not raw:
return out
key = None
for line in raw.splitlines():
if not line.strip():
continue
m = FM_KEY.match(line)
if m:
key, val = m.group(1), m.group(2).strip().strip("\"'")
out[key] = val
elif line.startswith(" ") and key:
out[key] = (out[key] + " " + line.strip()).strip()
return out
def strip_names(value):
return NAMES.sub(" ", value)
def check_file(path: Path, root: Path, length_exempt=frozenset()):
v = []
rel = str(path.relative_to(root))
text = path.read_text(errors="replace")
lines = text.splitlines()
name = path.name
raw, body = split_frontmatter(text)
fm = fields(raw)
if name not in RESERVED:
if raw is None:
v.append(("FRONTMATTER", rel, "no parseable frontmatter"))
else:
t = fm.get("type", "")
if not t:
v.append(("TYPE", rel, "missing type"))
elif t not in TYPES:
v.append(("TYPE", rel, f"type '{t}' not in {sorted(TYPES)}"))
d = fm.get("description", "")
if not d:
v.append(("DESC", rel, "missing description"))
else:
if len(d) > 120:
v.append(("DESC", rel, f"description is {len(d)} chars (max 120)"))
if d.strip("# ").lower() == fm.get("title", "\0").strip("# ").lower():
v.append(("DESC", rel, "description equals title"))
if QUANTITY.search(strip_names(d)):
v.append(("DESC", rel, "description restates a quantity"))
ts = fm.get("timestamp", "")
if not ts:
v.append(("TIMESTAMP", rel, "missing timestamp"))
else:
try:
stamp = datetime.date.fromisoformat(ts[:10])
if stamp > datetime.date.today():
v.append(("TIMESTAMP", rel, f"timestamp {ts} is in the future"))
except ValueError:
v.append(("TIMESTAMP", rel, f"unparseable timestamp {ts!r}"))
if name == "index.md" and raw is not None:
v.append(("INDEX-FM", rel, "index.md must not carry frontmatter"))
if name == "log.md":
dates = []
for line in lines:
m = re.match(r"^##\s+(\S.*?)\s*$", line)
if not m:
continue
head = m.group(1)
if not DATE_RE.match(head):
continue
dates.append(head[:10])
# OKF requires the heading to USE the date format; a title after an
# em-dash is allowed and costs zero lines, so `## 2026-08-17 — Why the
# pair was rebuilt` is conformant and `## 2026-08-17Rebuilt` is not.
if head != head[:10] and not re.match(r"^\d{4}-\d{2}-\d{2}\s+—\s+\S", head):
v.append(("LOG-DATE", rel,
f"date heading must be the date alone or `date — Title`: '## {head}'"))
if len(dates) != len(set(dates)):
v.append(("LOG-DATE", rel, "duplicate date heading"))
if dates != sorted(dates, reverse=True):
v.append(("LOG-DATE", rel, "date headings not newest first"))
if raw is not None and name not in RESERVED:
n = len(lines)
body_lines = len([ln for ln in body.splitlines() if ln.strip()])
opted_out = "length" in fm.get("okf-exempt", "").lower()
if n > 500:
if rel in length_exempt:
v.append(("LENGTH-OK", rel,
f"{n} lines — over the hint, pinned by the "
"one-page-per-endpoint law"))
elif opted_out:
v.append(("LENGTH-OK", rel,
f"{n} lines — over the hint, opted out by the page"))
else:
v.append(("LENGTH", rel, f"{n} lines (max 500)"))
elif body_lines < 15:
v.append(("LENGTH", rel, "stub: fewer than 15 body lines"))
# A second `type:` key inside the body means the page carries two copies of
# itself with a frontmatter block in the middle — the shape a naive
# frontmatter rewrite produces. It doubles the length, splits the wikilinks
# and makes the page's facts ambiguous, so it is a hard failure.
if raw is not None and name not in RESERVED:
dupes = [i + len((raw or "").splitlines()) + 2
for i, ln in enumerate(body.splitlines())
if re.match(r"^(type|title|description):", ln)]
if dupes:
v.append(("DOUBLED", rel,
f"frontmatter keys inside the body at line(s) "
f"{', '.join(str(d) for d in dupes[:5])} — the page "
"contains a duplicated copy of itself"))
return v
def index_rows(text):
rows = []
for line in text.splitlines():
if line.startswith("|") and "[[" in line:
rows.append(line)
return rows
def check_links(root: Path):
v = []
pages = [
p for p in sorted(root.rglob("*.md"))
if ".AGENTTRASH" not in str(p) and ".obsidian" not in str(p)
]
for index in [p for p in pages if p.name == "index.md"]:
text = index.read_text(errors="replace")
links = set(re.findall(r"\[\[([^\]|#]+)", text))
if index.parent == root:
targets = [p for p in pages if p.name not in RESERVED]
else:
targets = [p for p in index.parent.glob("*.md") if p.name != "index.md"]
for target in targets:
if target.stem not in links and target.name not in links:
v.append(("INDEX-COVER", str(index.relative_to(root)),
f"does not link sibling {target.name}"))
for index in [p for p in pages if p.name == "index.md"]:
for row in index_rows(index.read_text(errors="replace")):
if QUANTITY.search(strip_names(row)):
v.append(("INDEX-NUM", str(index.relative_to(root)),
"row restates a quantity: " + row[:80]))
return v
def live_endpoints(root: Path):
"""Pages pinned by AGENTS.md's one-page-per-endpoint law.
Derived, never hardcoded — the same derivation as
tests/docs_completeness.bats: an endpoint is live when scripts/ holds both
its env and its unit. Those pages carry the Eras table for their whole
history, so the OKF 500-line split hint cannot apply to them without
creating a second owner of the same shape.
"""
scripts = root.parent / "scripts"
out = set()
if scripts.is_dir():
for envf in scripts.glob("frank-*.env"):
if (scripts / f"{envf.stem}.service").exists():
out.add(f"LLM/{envf.stem}.md")
return out
def retired_era_pages(root: Path):
"""Era pages of a retired endpoint: campaign ledgers, one owner each.
AGENTS.md lets a campaign ledger that exists nowhere else live on the page
that measured it, and a retired endpoint keeps exactly one page per era.
Splitting such a ledger to satisfy the 500-line hint would fragment one
campaign across two owners, so the tag opts the page out.
"""
out = set()
for path in root.rglob("*.md"):
s = str(path)
if ".AGENTTRASH" in s or ".obsidian" in s:
continue
raw, _ = split_frontmatter(path.read_text(errors="replace"))
if raw is not None and "retired-endpoint" in fields(raw).get("tags", ""):
out.add(str(path.relative_to(root)))
return out
def main(argv):
root = Path(argv[1] if len(argv) > 1 else "serverAppConfigs-docs")
exempt = frozenset(live_endpoints(root)) | retired_era_pages(root)
violations = []
notes = []
citation_gaps = 0
pages = 0
for path in sorted(root.rglob("*.md")):
s = str(path)
if ".AGENTTRASH" in s or ".obsidian" in s:
continue
pages += 1
for code, rel, detail in check_file(path, root, exempt):
(notes if code == "LENGTH-OK" else violations).append((code, rel, detail))
text = path.read_text(errors="replace")
if len(text.splitlines()) > 40 and not re.search(
r"^##\s+(Citations|Sources)\b", text, re.M):
citation_gaps += 1
violations += check_links(root)
for code, rel, detail in violations:
print(f" {code:12s} {rel}: {detail}")
for code, rel, detail in notes:
print(f" {code:12s} {rel}: {detail}")
print(f"{pages} concept pages, {len(violations)} OKF violations"
f" ({citation_gaps} pages without a Citations section — soft;"
f" {len(notes)} length exemptions honoured)")
return 1 if violations else 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+6 -1
View File
@@ -31,7 +31,12 @@ def strip_md(path: str) -> str:
def build_index(vault: pathlib.Path):
pages = sorted(vault.rglob("*.md"))
# `.AGENTTRASH` is deleted-but-held state, not vault content: it must not be
# scanned (its relative links resolve against the trash dir, not the vault)
# and must not answer as a link target — a live page pointing into the trash
# is an unresolved link, not a resolved one.
pages = sorted(p for p in vault.rglob("*.md")
if ".AGENTTRASH" not in str(p))
stems: dict[str, list[pathlib.Path]] = {}
for page in pages:
stems.setdefault(page.stem.casefold(), []).append(page)
+1 -1
View File
@@ -1459,7 +1459,7 @@ LLM_ZEALOT_RUNTIME="${LLM_ZEALOT_RUNTIME:-/home/dank/GitPackages/llama-prism-b10
# image path — a deploy that starts without it silently ships a text-only
# endpoint. Checked unconditionally, like frank-dragoon's. Q8_0 is MANDATORY,
# not a preference: the card says the BF16 tower can crash llama-server when
# several image requests run at once, and this endpoint runs --parallel 2. Its
# several image requests run at once, and this endpoint runs --parallel 3. Its
# 868.32 MiB worst-case VRAM (595.29 model + 248.10 compute in the boot ledger)
# lands AFTER the engine's fit check, so it is not inside the q8_0 budget.
LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf}"
+37 -21
View File
@@ -11,35 +11,51 @@ LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server
# metadata dequantizes wrong, silently.
MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf
# Q8_0 IS MANDATORY: the publisher states the BF16 tower can crash llama-server
# under concurrent image requests, which is exactly --parallel 2.
# under concurrent image requests, which is exactly --parallel 3.
MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
# Served id — the harness/family name, not a fact about the payload.
# Id-keyed consumers: OpenWebUI (model row, access_grant, api_configs), models.yml.
ALIAS=frank-zealot
BIND_ADDR=0.0.0.0
LISTEN_PORT=7771
# CTX_SIZE is the POOL llama.cpp divides across slots: 524,288 / 2 = 262,144 a
# conversation, each still at the trained window. MUST stay divisible by
# PARALLEL (the seam enforces it) and models.yml contextWindow quotes the
# PER-SLOT number — a longer prompt is a 400, never a truncation. The pool
# cannot go back to 786,432 under exact f16 KV: 24,576 MiB of cache against the
# 24,123 MiB the driver leaves addressable, before a weight loads.
CTX_SIZE=524288
PARALLEL=2
BATCH_SIZE=4096
# CTX_SIZE is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
# conversation. MUST stay divisible by PARALLEL (the seam enforces it) and
# models.yml contextWindow quotes the PER-SLOT number — a longer prompt is a
# 400, never a truncation. The pool is bought, not granted: K+V at 25,088
# B/token is 14,406.00 MiB here and the MTP draft context bills its own f16 KV
# over the WHOLE pool (4,096 B/position ⇒ 2,352.00 MiB), so the old 786,432
# pool cannot carry a drafter at any KV dtype — 18,816 + 3,072 of cache alone
# against the 24,123 MiB the driver leaves addressable.
CTX_SIZE=602112
PARALLEL=3
# The logical batch is VRAM- and throughput-neutral at a fixed ubatch (the law
# on [[frank-zealot]]); 1024×6 keeps it a clean multiple of the ubatch.
BATCH_SIZE=6144
# Ubatch is a prefill knob (decode never moves with it) and it sets the
# compute-buffer size — which is what decides whether a drafter fits.
# UBATCH_SIZE=1024 is the documented MTP arm.
UBATCH_SIZE=2048
# Exact K and V: the retrieval-fidelity ask, paid with one conversation of
# concurrency. Rate law and the arithmetic: [[frank-zealot]] → Laws.
# compute-buffer size, which decides whether a drafter fits. 1024 is the rung
# every measured MTP arm here ran on (213.5 t/s against 109.2 at 2048). Its
# VRAM credit is SHAPE-specific: the 786,432-pool arm census showed ~1,306 MiB
# back, this boot measured 2,089.88 MiB of target compute against 2,496.13 at
# ubatch 2048 — 406 MiB back, with the split moving (CUDA0 +257, CUDA_Host
# −663). The drafter is funded by CACHE_TYPE_V below, not by this knob.
UBATCH_SIZE=1024
# K exact, V in the 8-bit class: 16,384 + 8,704 = 25,088 B/token (8,192
# elements per token per cache × 2 B for f16, × 1.0625 for q8_0 — there is no
# true fp8 KV dtype in this build). That 7,680 B/token saved against exact V is
# what funds BOTH the third conversation and the drafter. Quantized V is
# illegal without --flash-attn, which the serve script pins. Rate law and the
# arithmetic: [[frank-zealot]] → Laws.
CACHE_TYPE_K=f16
CACHE_TYPE_V=f16
# Speculation OFF — measured, not assumed: the draft context bills its own f16
# KV over the WHOLE pool, and there is no room beside exact KV (the arm census
# is on the page). SPEC_TYPE=none strips the WHOLE set (--spec-type and every
# --spec-draft-*); half a set has segfaulted at load.
SPEC_TYPE=none
CACHE_TYPE_V=q8_0
# Speculation ON (Frank, 2026-10-19): the head is in the file
# (qwen35.nextn_predict_layers = 1, blk.32.nextn.*) and rides the MAIN model's
# weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of
# draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for
# by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V
# beside MTP needs fork build >= b10776; LLAMA_BIN is 10777.
# SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and
# every --spec-draft-*); half a set has segfaulted at load.
SPEC_TYPE=draft-mtp
SPEC_DRAFT_N_MAX=2
SPEC_DRAFT_N_MIN=0
SPEC_DRAFT_P_MIN=0
+206
View File
@@ -0,0 +1,206 @@
#!/usr/bin/env python3
"""Regenerate serverAppConfigs-docs/index.md from page frontmatter.
The root index is a locator (AGENTS.md -> *Documentation Policy*): one row per
page, one clause, no numbers. Hand-maintaining it is how rows go missing, and
the docs seam fails when a page is not linked from it, so it is generated:
* preamble everything above the first `## ` in the existing file is kept
verbatim (that is the curated statement of the vault's law)
* section from the page's directory, overridden by its frontmatter tags
(`endpoint` / `retired-endpoint` split the LLM pages)
* row text the page's frontmatter `description`, verbatim
A page with no description is a hard error rather than an empty cell, so the
generator cannot silently produce a silent index. Sections with no pages are
skipped. Deterministic: the same vault always yields the same bytes.
Sub-directory indices are hand-grouped (they carry curated prose), so they are
synced rather than regenerated: dead rows dropped, row text refreshed from the
page's description, uncovered siblings appended.
"""
import pathlib
import re
import sys
RESERVED = {"index.md", "log.md"}
SECTIONS = [
("Inference endpoints", "endpoint"),
("Retired endpoints", "retired-endpoint"),
]
DIRECTORY_SECTIONS = [
("Dankclaw", "Dankclaw services"),
("Dankserver", "Dankserver services"),
("FrankBook-Pro", "FrankBook-Pro"),
("Desktop", "Workstations"),
("Infrastructure", "Cross-host infrastructure"),
("Architecture", "Repo architecture and policy"),
("Operations", "Operations and incidents"),
("Agent", "Agent / harness"),
("Research", "Research and methodology"),
]
PIPELINE_SECTION = "Pipeline Parallelism (series)"
LLM_FALLBACK = "LLM technique and hardware references"
CHANGE_SECTION = "Change history"
ORDER = [t for t, _ in SECTIONS] + [t for _, t in DIRECTORY_SECTIONS] + \
[PIPELINE_SECTION, LLM_FALLBACK, CHANGE_SECTION]
def frontmatter_fields(text):
if not text.startswith("---"):
return {}
end = text.find("\n---", 3)
if end == -1:
return {}
raw = text[3:end]
fields = {}
for key in ("description", "type"):
m = re.search(rf"^{key}:\s*(.*)$", raw, re.M)
if m:
fields[key] = m.group(1).strip().strip("\"'")
tags = []
m = re.search(r"^tags:\s*\[(.*?)\]$", raw, re.M)
if m:
tags = [t.strip().strip("\"'") for t in m.group(1).split(",") if t.strip()]
elif re.search(r"^tags:\s*$", raw, re.M):
block = re.search(r"^tags:\s*\n((?:\s+-\s+.*\n?)+)", raw, re.M)
if block:
tags = [l.strip().lstrip("-").strip().strip("\"'")
for l in block.group(1).splitlines() if l.strip()]
fields["tags"] = tags
return fields
def section_for(page, vault, fields):
rel = page.relative_to(vault)
if len(rel.parts) == 1:
return CHANGE_SECTION if page.name == "log.md" else "Uncategorised"
top = rel.parts[0]
if top == "LLM":
if PIPELINE_SECTION_DIR in rel.parts:
return PIPELINE_SECTION
for title, tag in SECTIONS:
if tag in fields.get("tags", []):
return title
return LLM_FALLBACK
for prefix, title in DIRECTORY_SECTIONS:
if top == prefix:
return title
return "Uncategorised"
ROW = re.compile(r"^(\| )\[\[([^\]|]+)\]\]( \|)( .*?)( \|$)")
BULLET = re.compile(r"^(- )\[\[([^\]]+)\]\]( — )(.*?)$")
def sync_subindex(index: pathlib.Path, vault: pathlib.Path):
"""Keep a hand-grouped sub-index honest without erasing its grouping.
Rows are locators, so their text is the linked page's own description; a
row whose page is gone is a dead promise and is dropped; a direct sibling
with no row is appended under `## Pages` (that is what the link-integrity
seam checks). Curated headings and prose are untouched.
"""
here = index.parent
known = {p.stem for p in here.rglob("*.md")
if ".AGENTTRASH" not in str(p) and p.name != "index.md"}
lines = index.read_text().split("\n")
out, dropped, refreshed = [], 0, 0
for line in lines:
for pat, wrap in ((ROW, "| [[{}]] |{} {} |"), (BULLET, "- [[{}]] — {}")):
m = pat.match(line)
if not m:
continue
stem = m.group(2).strip()
if stem not in known:
dropped += 1
line = None
break
cand = next((p for p in here.rglob(f"{stem}.md")
if ".AGENTTRASH" not in str(p)), None)
desc = frontmatter_fields(cand.read_text(errors="replace")).get("description") \
if cand else None
if desc:
current = m.group(4).strip()
if current != desc:
line = wrap.format(stem, " " if pat is ROW else "", desc) \
if pat is ROW else wrap.format(stem, desc)
refreshed += 1
break
if line is not None:
out.append(line)
text = "\n".join(out)
# Presence must be the wikilink itself: a bare stem is a substring of other
# page names (`frank-medium` lives inside `frank-medium-rocmfpx`), and a
# sibling matched that way silently never gets a row.
missing = sorted(
(p.stem, frontmatter_fields(p.read_text(errors="replace")).get("description", ""))
for p in here.glob("*.md")
if p.name != "index.md" and f"[[{p.stem}]]" not in text
)
if missing:
block = ["", "## Pages", "", "| Page | Owns |", "|---|---|"]
block += [f"| [[{stem}]] | {desc} |" for stem, desc in missing]
text = text.rstrip("\n") + "\n" + "\n".join(block) + "\n"
if text != index.read_text():
index.write_text(text)
return dropped, refreshed, len(missing)
PIPELINE_SECTION_DIR = "Pipeline Parallelism"
def main(argv):
vault = pathlib.Path(argv[1] if len(argv) > 1 else "serverAppConfigs-docs")
index = vault / "index.md"
if not index.is_file():
print(f"missing {index}")
return 2
existing = index.read_text()
preamble = existing.split("\n## ", 1)[0].rstrip("\n")
pages = [p for p in vault.rglob("*.md")
if ".AGENTTRASH" not in str(p) and ".obsidian" not in str(p)
and p.name != "index.md" and p != index]
buckets, errors = {}, []
for page in pages:
fields = frontmatter_fields(page.read_text(errors="replace"))
if page.name != "log.md":
desc = fields.get("description")
if not desc:
errors.append(str(page.relative_to(vault)))
continue
else:
desc = "Chronological incident and law log for the whole bundle."
buckets.setdefault(section_for(page, vault, fields), []).append((page.stem, desc))
if errors:
print("pages with no description (fix the frontmatter, not the index):")
for e in sorted(errors):
print(" " + e)
return 1
out = [preamble, ""]
for title in ORDER + sorted(set(buckets) - set(ORDER)):
rows = sorted(buckets.get(title, []), key=lambda r: r[0].lower())
if not rows:
continue
out += [f"## {title}", "", "| Page | Owns |", "|---|---|"]
out += [f"| [[{stem}]] | {desc} |" for stem, desc in rows]
out.append("")
out.append("<!-- generated by scripts/gen-vault-index.py; edit page frontmatter instead -->")
index.write_text("\n".join(out) + "\n")
touched = 0
for sub in sorted(vault.rglob("index.md")):
if sub == index:
continue
dropped, refreshed, added = sync_subindex(sub, vault)
if dropped or refreshed or added:
touched += 1
print(f" {sub}: -{dropped} dead rows, {refreshed} refreshed, +{added} added")
print(f"{index}: {sum(len(v) for v in buckets.values())} rows in "
f"{len([t for t in ORDER if buckets.get(t)])} sections, {touched} sub-indices synced")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env python3
"""Port census: prove a merge/collapse lost no fact.
For every line that existed in the OLD set and not in the NEW set, check that
each of its significant tokens (numbers, long identifiers, paths) still occurs
somewhere in the NEW corpus. A line whose every token survives is a reworded
line; a line with an unheld token is a candidate fact loss and gets printed.
port-census.py <new-glob-dir> <old-file> [<old-file> ...]
Old files may be `git-rev:path` (read from git) or a path (e.g. a trash copy).
"""
import pathlib
import re
import subprocess
import sys
def toks(line):
out = []
for n in re.findall(r"\d[\d,]*(?:\.\d+)?", line):
if len(n.replace(",", "").split(".")[0]) >= 3 or (
"." in n and len(n.split(".")[-1]) >= 2 and len(n.replace(",", "")) >= 4
):
out.append(n)
# Identifiers, flags, paths and backticked spans are the facts worth
# tracking; ordinary prose words are not — they reword on every rewrite.
# A hyphen alone is not evidence of an identifier (`long-standing`), so a
# dashed word counts only with two hyphens (`frank-tts-engine`) or with a
# digit, dot, slash, colon or `--` inside it.
for w in re.findall(r"[A-Za-z][\w./:-]{7,}", line):
core = w.strip(".,;:!?\"')]")
if (core.startswith("--") or core.isupper()
or re.search(r"[\d_/:.]", core) or core.count("-") >= 2):
out.append(core)
out += [s.strip() for s in re.findall(r"`([^`]+)`", line) if len(s) > 3]
return out
def held(tok, corpus):
t = tok.replace(",", "")
alt = rf"\b{re.escape(t)}\b"
for c in corpus:
if tok in c or t in c:
return True
if re.search(alt, c.replace(",", "")):
return True
return False
def main(argv):
new_dir = pathlib.Path(argv[1])
def corpus_excluding(exclude):
"""Every vault page except the one under test.
The corpus is what must still hold a fact AFTER the file is gone, so a
file that lives inside the scanned directory has to be removed from it —
otherwise every token is "held" by the file itself and the census proves
nothing (it reports 0 losses for every page, which is the bug this
guards).
"""
out = []
for p in new_dir.rglob("*.md"):
s = str(p)
if ".AGENTTRASH" in s or ".obsidian" in s:
continue
if exclude is not None and p.resolve() == exclude:
continue
out.append(p.read_text(errors="replace"))
return out
lost = checked = 0
for spec in argv[2:]:
if spec.startswith("git-"):
ref, path = spec[4:].split(":", 1)
old = subprocess.run(["git", "show", f"{ref}:{path}"],
capture_output=True, text=True).stdout
label = f"{ref}:{path}"
exclude = None
else:
old = pathlib.Path(spec).read_text(errors="replace")
label = spec
exclude = pathlib.Path(spec).resolve()
blob = "\n".join(corpus_excluding(exclude))
for line in old.splitlines():
s = line.strip().strip("->*|# ").strip()
if len(s) < 30 or s.startswith("```"):
continue
checked += 1
missing = [t for t in toks(s) if not held(t, [blob])]
if missing:
lost += 1
print(f" {label}\n L: {s[:150]}\n unheld: {', '.join(missing[:6])}")
print(f"{checked} old lines checked, {lost} with unheld tokens")
return 1 if lost else 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+69 -60
View File
@@ -5,33 +5,45 @@
#
# 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export, then the same
# evening TWO 262,144-token conversations (--ctx-size 524288 --parallel 2) over
# an UNQUANTIZED f16 KV cache. The 9B bills 8,192 elements per token per cache
# (8 full-attention layers instead of the 27B's 16), so exact K and V costs
# 32,768 B/token — 16,384.00 MiB at the 524,288 pool, which is what a 24,123 MiB
# addressable card can carry beside the trunk. Verbosity 4 ships so the VRAM-fit
# forensics (KV layout, buffer sizes, graph capture) land in the journal.
# an UNQUANTIZED f16 KV cache.
#
# 2026-10-19 (Frank): THREE 200,704-token conversations over a 602,112 pool
# (--ctx-size 602112 --parallel 3), K left at the engine's own default (f16),
# V dropped to q8_0, batch 1024×6 / ubatch 1024, and the MTP drafter ARMED
# (--spec-type draft-mtp). The 9B bills 8,192 elements per token per cache (8
# full-attention layers instead of the 27B's 16), so f16 K is 16,384 B/token and
# q8_0 V is 8,704: 25,088 B/token = 14,406.00 MiB at the 602,112 pool. That
# 7,680 B/token against exact V is exactly what funds the third conversation
# AND a draft context that bills f16 KV over the whole pool. Verbosity 4 ships
# so the VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in
# the journal.
#
# Architecture (engine ledger, not the marketing page): qwen35 hybrid, 32
# layers with qwen35.full_attention_interval = 4 ⇒ 8 full-attention + 24 Gated
# DeltaNet, 4 KV heads × 256 dim, native 262,144. The KV pool scales with the 8
# full layers (16,384.00 MiB of f16 at the 524,288 pool); the DeltaNet layers
# cost a fixed recurrent state per SLOT — ~50.25 MiB a cell, so 100.50 MiB for
# two. The runtime is a hard lock, not a preference: custom GGML type 142 plus
# prism.hadamard.* metadata, which only the vendor build registers (the b10776
# gate that moved the prebuilt 10754 → 10777 was MTP beside quantized V, which
# exact KV now makes moot — the type-142 lock alone keeps the build pinned).
# full layers (14,406.00 MiB of f16 K + q8_0 V at the 602,112 pool); the
# DeltaNet layers cost a fixed recurrent state per SLOT — ~50.25 MiB a cell, so
# 150.75 MiB for three, and an armed depth-2 drafter triples that again. The
# runtime is a hard lock, not a preference: custom GGML type 142 plus
# prism.hadamard.* metadata, which only the vendor build registers — and the
# b10776 gate that moved the prebuilt 10754 → 10777 (MTP beside a quantized V)
# is LOAD-BEARING again on this shape, not moot as it was under exact KV.
# Vision comes from the publisher's Q8_0 projector (the BF16 one can crash the
# server on concurrent images, and this endpoint runs two slots), so --mmproj is
# server on concurrent images, and this endpoint runs three slots), so --mmproj is
# what gives this endpoint eyes; MMPROJ_FILE="" is the documented text-only
# opt-out.
#
# Speculative decoding ships OFF, measured: the head loads and accepts well
# (0.798/0.617 per position, mean length 2.41) but the draft context bills its
# own f16 KV over the WHOLE pool — 4,096 B/position × 524,288 = 2,048 MiB plus
# buffers — and the trunk already projects 21,234 MiB with exact KV. Exact KV
# and MTP cannot share this card; frank-zealot.env carries the q8_0-era arm table
# and the two doors back to the drafter (one window, or KV back to q8_0 at
# --ubatch-size 1024 for 213.5 t/s).
# Speculative decoding ships ON (Frank, 2026-10-19). The head is IN the file and
# accepts well (0.798/0.617 per position, mean length 2.41, measured 2026-10-18);
# what made it lose that evening was the bill, not the drafts — the draft context
# is built against the target's WHOLE pool, so blk.32 adds 4,096 B/position
# (2,352.00 MiB at 602,112) plus 480.97 MiB of its own compute buffers on the
# card, and it triples the recurrent state per slot (452.25 MiB for three cells
# against 50.25 a cell with the drafter off). CACHE_TYPE_V=q8_0 is what pays
# that: 7,680 B/token cheaper than exact V, 4,410 MiB at this pool. The ubatch
# drop to 1024 contributes ~406 MiB here, NOT the ~1,306 the 786,432-pool census
# showed — that figure does not transfer between shapes. SPEC_TYPE=none is
# the rollback and strips the whole flag set.
#
# Sampling: the checkpoint carries the publisher's thinking recipe as GGUF
# metadata (general.sampling.* = temp 1.0 / top_p 0.95 / top_k 20 / min_p 0 /
@@ -58,49 +70,46 @@ fi
: "${ALIAS:=frank-zealot}"
: "${BIND_ADDR:=0.0.0.0}"
: "${LISTEN_PORT:=7771}"
# ctx-size is the POOL llama.cpp divides across slots: 524,288 / 2 = 262,144
# per conversation — every slot still at the model's trained window. The harness
# contextWindow MUST quote the PER-SLOT number — a longer prompt is answered
# with a 400, never truncated. Under exact KV the pool cannot stay at 786,432:
# 32,768 B/token × 786,432 = 24,576 MiB, more than the card's whole addressable
# 24,123 MiB before a single weight loads.
: "${CTX_SIZE:=524288}"
: "${PARALLEL:=2}"
# ctx-size is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
# conversation. The harness contextWindow MUST quote the PER-SLOT number — a
# longer prompt is answered with a 400, never truncated. The pool is priced by
# the KV bill, not by the trained window: at 25,088 B/token (f16 K + q8_0 V) the
# 602,112 pool costs 14,406.00 MiB and the armed drafter adds its own f16 KV over
# the whole pool (2,352.00 MiB). Exact V at this pool (18,816 MiB) or the old
# 786,432 pool at any dtype cannot boot inside the 24,123 MiB the driver leaves
# addressable.
: "${CTX_SIZE:=602112}"
: "${PARALLEL:=3}"
# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
# 2048 is the measured CUDA optimum (MiMo era, same arch/shape: +4.6 % prefill
# at 8.9k depth, decode flat, 616 MiB). The logical batch is VRAM- and
# throughput-neutral at a fixed ubatch. It ALSO sets the compute-buffer size,
# which is the term that decides whether an MTP draft context fits beside a KV
# pool: 1024 hands back 1,306 MiB and was the arm that ran the drafter at three
# q8_0 windows (frank-zealot.env carries the table).
: "${BATCH_SIZE:=4096}"
: "${UBATCH_SIZE:=2048}"
# Exact KV: f16 K + f16 V — no quantization error anywhere in the attention
# state. 32 KiB/token at this geometry (8 full-attention layers × 4 KV heads ×
# 256 dim × 2 B × 2 caches); 16,384.00 MiB at the 524,288 pool. That is the whole
# reason the slot count is 2: the same dtype at the old 786,432 pool is 24,576
# MiB and dies at cudaMalloc. q8_0 K + q8_0 V (17,408 B/token, 13,056.00 MiB at
# 786,432) is the documented fallback rung and funds three windows; CACHE_TYPE_V
# alone is the halfway door. Explicit either way so the contract is visible in
# the argv, and --flash-attn stays pinned because quantized V — the fallback —
# is illegal without it (no true fp8 KV dtype exists in this build).
# The logical batch is VRAM- and throughput-neutral at a fixed ubatch, so 6144
# (1024×6) is free; the ubatch is what sets the compute-buffer size, and 1024 is
# the rung every measured MTP arm here ran on (213.5 t/s at 1024 against 109.2
# at 2048) — worth ~406 MiB at THIS shape, not the ~1,306 the old pool showed.
: "${BATCH_SIZE:=6144}"
: "${UBATCH_SIZE:=1024}"
# Asymmetric KV: f16 K (the engine's own default, spelled out so the contract is
# visible in argv) + q8_0 V. 8,192 elements per token per cache (8 full-attention
# layers × 4 KV heads × 256 dim) ⇒ 16,384 B/token for K and 8,704 for V (q8_0 is
# 1.0625 B/element — llama.cpp has no true fp8 KV dtype), 25,088 B/token and
# 14,406.00 MiB at the 602,112 pool. That is the halfway door between exact KV
# (32,768 B/token, which costs the third conversation and the drafter both) and
# the fully quantized pair (17,408 B/token, which costs retrieval fidelity in the
# keys); --flash-attn stays pinned because quantized V is illegal without it.
: "${CACHE_TYPE_K:=f16}"
: "${CACHE_TYPE_V:=f16}"
# Speculative decoding defaults OFF, measured on this card and shape: the head
# IS in the checkpoint (qwen35.nextn_predict_layers = 1 — MTP is a property of
# the FILE, so the plain PQ2_0 sibling could not run it at any flag value), it
# loads, and it accepts well (mean accepted length 2.41) — but the draft context
# bills its own f16 KV over the WHOLE pool (4,096 B/position, 2,048 MiB at the
# 524,288 pool) and at ubatch 2048 that squeeze halves decode: 167.2 → 109.2 t/s
# on the q8_0 3-slot shape. The draft cache dtype IS a knob (--spec-draft-type-k
# / --spec-draft-type-v, both default f16 — verified in this build's --help), so
# an armed drafter can halve its own bill; there is no --spec-draft-model because
# the draft rides the main model's weights. Card gate: quantized V alongside MTP
# needs fork build >= prism-b10776, which is why LLAMA_BIN moved to the 10777
# prebuilt; exact KV removes that gate but NOT the type-142 loader lock.
# SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024 is the documented MTP arm (213.5 t/s);
# SPEC_TYPE=none strips the whole set (flags move together).
: "${SPEC_TYPE:=none}"
: "${CACHE_TYPE_V:=q8_0}"
# Speculative decoding defaults ON: the head IS in the checkpoint
# (qwen35.nextn_predict_layers = 1 — MTP is a property of the FILE, so the plain
# PQ2_0 sibling could not run it at any flag value) and the draft context rides
# the MAIN model's weights, so there is no --spec-draft-model. Its bill is the
# target's whole pool at 4,096 B/position plus ~0.45 GiB of buffers at this
# ubatch and a tripled recurrent state; CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024
# above are what pay for it. The draft cache dtype IS a further knob
# (--spec-draft-type-k / --spec-draft-type-v, both default f16 — verified in this
# build's --help) and stays at its default here. Card gate: quantized V alongside
# MTP needs fork build >= prism-b10776, which is why LLAMA_BIN is the 10777
# prebuilt; the type-142 loader lock keeps it pinned regardless.
# SPEC_TYPE=none is the rollback and strips the whole set (flags move together).
: "${SPEC_TYPE:=draft-mtp}"
: "${SPEC_DRAFT_N_MAX:=2}"
: "${SPEC_DRAFT_N_MIN:=0}"
: "${SPEC_DRAFT_P_MIN:=0}"
@@ -1,7 +1,7 @@
---
type: reference
title: Agent Docs and Edits
description: Two self-inflicted damage classes — structurally broken markdown after parallel agent edits, and botched hashline PUT/CUT repairs — with the audit checks and recovery order for each.
description: "Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each."
tags: [agents, docs, markdown, hashline, edit-recovery]
timestamp: 2026-10-18T00:00:00Z
---
@@ -1,7 +1,7 @@
---
type: reference
title: Command Center Dotfiles
description: The ~/.omp repo is itself the deployment — stow-equivalent packages, full-replica rsync transport, the pre-overwrite audit for non-blank hosts, deploy.sh mechanics, and the fish/sshpass transport gotchas.
description: "How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas."
tags: [dotfiles, omp, deploy, rsync, blankpad]
timestamp: 2026-10-05T00:00:00-07:00
---
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: OMP Harness
description: The omp agent harness as configured on this fleet — compaction chain arithmetic, per-model sampler routing via compat.extraBody, and the static-preamble/skill-library audits that keep session context lean.
description: "The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits."
tags: [omp, harness, compaction, models-yml, context, samplers]
timestamp: 2026-10-18T00:00:00Z
---
+3 -14
View File
@@ -1,23 +1,12 @@
---
type: index
description: Pages about the agent harness itself — omp config, vault editing discipline, and the dotfiles repo that deploys the agent's own home.
tags: [agent, index]
timestamp: 2026-10-05T00:00:00-07:00
---
# Agent
Pages about the agent's own tooling and context — how omp is configured, how
docs get edited safely, and how the agent's home is deployed.
- [[OMP Harness]] — compaction chain arithmetic, per-model samplers via
`compat.extraBody`, preamble and skill-library audits
- [[Agent Docs and Edits]] — doc-structure audits after parallel edits;
hashline edit recovery
- [[Command Center Dotfiles]] — the `~/.omp` repo-as-deployment, rsync
transport, override packages
- [[OMP Harness]] — The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits.
- [[Agent Docs and Edits]] — Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each.
- [[Command Center Dotfiles]] — How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas.
## Related
- [[Testing Policy]] — the seams these pages' checks ride
- [LLM](../LLM/index.md) — the fleet these agents operate
@@ -1,8 +1,7 @@
---
type: reference
title: Container Network Namespaces
description: Which local services are addressable from which Podman network namespace on
Dankclaw, and why bridge-network containers cannot dial host-netns loopback listeners.
description: "How Podman builds container networks here: the network inventory, the netns sharing mechanism, and the isolation limits."
tags:
- networking
- podman
@@ -1,7 +1,7 @@
---
type: reference
title: Deploy State Triage
description: Reading and reconciling the deploy-state markers — stale marker vs live container, pending-service lockout, interrupted/raced deploys — without restarting healthy heavy endpoints.
description: "Reading deploy-state markers against live containers and deciding, safely, whether to redeploy."
tags: [deployment, deploy-state, systemd, podman, triage]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,5 +1,5 @@
---
type: guide
type: howto
title: Deployment Flow
description: Service-scoped Git deployment for the two homelab hosts.
tags:
@@ -1,6 +1,6 @@
---
type: reference
description: "`tests/e2e/` drives the **live fleet** through a real browser: fresh contexts"
description: "The Playwright suite that drives the live fleet through a real browser, its project set, and the FrankBook conditional."
tags:
- architecture
- e2e
@@ -1,7 +1,7 @@
---
type: reference
title: Quadlet Lifecycle
description: The full Quadlet discipline on this fleet — ownership auditing, compose-to-Quadlet cutover, drift/crash-loop diagnosis, the podman 5.x v1 generator quirks, log retrieval, and compose bring-up with ground truth.
description: "The Quadlet discipline: ownership audits, compose cutover, drift and crash-loop triage, generator quirks, log retrieval."
tags: [podman, quadlet, systemd, boot-persistence, compose, drift]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: Stale Signal Audit
description: The audit battery for surfaces that rot without failing — unreachable probes, vacuous assertions, invalid checkers, gates that never run, and prose invalidated by a change. Four passes found real defects every time.
description: "The audit battery for surfaces that rot without failing: dead probes, vacuous assertions, gates that never run."
tags: [testing, observability, audit, signals, gates]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,6 +1,6 @@
---
type: howto
description: "Qwen-Image 2.1 workflows on Dankclaw: two proven txt2img workflows (art + legible text), the API recipe, and which models/LoRAs to add next."
description: "Proven Qwen-Image text-to-image workflows on Dankclaw: the recipes, the API call shape, and which models to add next."
tags:
- dankclaw
- comfyui
@@ -1,6 +1,6 @@
---
type: reference
description: "frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker, on a private podman bridge behind frank-tts — deployed 2026-10-18, and it publishes no port on purpose."
description: "The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes."
tags:
- dankclaw
- tts
+4 -1
View File
@@ -1,11 +1,13 @@
---
type: reference
description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; deployed 2026-10-18 and wired as the harness speech role."
description: "The fleet's cloned-voice speech endpoint and the one voice surface consumers address; wired as the harness speech role."
tags:
- endpoint
- dankclaw
- tts
- voice
- quadlet
timestamp: 2026-10-19
---
# Frank TTS
@@ -202,3 +204,4 @@ harness bundle and measured the day of the flip:
| First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
| The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
---
+1 -1
View File
@@ -1,5 +1,5 @@
---
type: configuration
type: reference
title: Gitea
description: Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service.
tags:
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: Hermes-WebUI (Hermex server)
description: The hermes-webui container on Dankclaw — the API server that the Hermex iPhone app talks to, reachable over an SSH tunnel, from the whole LAN (it binds all interfaces), and from https://hermes.franksdank.cloud (internal-only vhost).
description: "The iPhone companion app for Hermes on Dankclaw and the ways it is reached from the LAN."
tags:
- hermes
- hermes-webui
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Marreta on Dankclaw
description: The content-cleaning proxy (manualdousuario/marreta, Laravel/serversideup fpm-nginx) deployed as a rootless Quadlet pair that shares SearXNG's Gluetun netns for egress and sits behind the Anubis + Authentik guard chain at marreta.franksdank.cloud.
description: "The content-cleaning proxy SearXNG borrows egress from: a quadlet pair in the VPN netns behind the guard chain."
tags:
- dankclaw
- marreta
@@ -1,6 +1,6 @@
---
type: reference
description: "open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out — script written by frank-zealot, voices rendered by frank-tts, no cloud provider anywhere in the path."
description: "The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house."
tags:
- dankclaw
- open-notebook
@@ -13,7 +13,7 @@ timestamp: 2026-10-18
open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook)) is the notebook layer that turns existing notes and sources into a **spoken episode**: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by **frank-zealot**, the voices are rendered by **frank-tts**, and nothing leaves the LAN.
**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Deploying it is a deliberate next step, not a pending one.
**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Re-censused 2026-10-19 and unchanged: no `*notebook*` unit in `systemctl --user list-unit-files`, nothing installed under `~/.config/containers/systemd/` by that name, no container in `podman ps` — the repo's three quadlet files have never been installed on the host. Deploying it is a deliberate next step, not a pending one.
**Host:** Dankclaw (rootless Podman, user `dank`) — **Ports:** 8502 UI and 5056 API, published on the LAN, **no vhost and no Caddy route** — **Deployer:** `open-notebook` (one service, two units) — **Docs:** https://github.com/lfnovo/open-notebook
+3 -3
View File
@@ -1,6 +1,6 @@
---
type: reference
description: "Seerr is the unified request manager for the *arr stack (the maintained `seerr-team/seerr` successor to the archived Overseerr/Jellyseerr line)."
description: "The request manager for the *arr stack — the maintained successor to the archived Overseerr line."
tags:
- dankclaw
- seerr
@@ -40,8 +40,8 @@ the next outpost reload), and the missing app bounced post-login to
authentik's own user UI instead of Seerr. Both backfilled live: the three
standard flows copied from a known-good provider, plus an application
(`seerr-guard`) linked to the provider. Verified by the hardened SSO sweep
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik]] for the two
provisioning rules this incident established.
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik Outpost Reference]] for the
two provisioning rules this incident established.
## *arr wiring
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: Dankclaw Swap Configuration
description: The host swapfile on Dankclaw — 64 GiB at /swap.img on ext4, fstab-driven via the generated swap.img.swap unit. Covers why it was raised from 8 GiB, the safe resize ordering that avoids an OOM on a host whose MemAvailable is inflated by pinned model weights, and the swappiness caveat.
description: "The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat."
tags:
- dankclaw
- host
@@ -1,6 +1,6 @@
---
type: reference
description: "How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material gate scripts/build-voice-pack.sh enforces, how to add a fourth voice, and what to do when a pinned engine bump changes the extractor's output sizes."
description: "How a voice pack is built: the files every pack must carry, the source-material gate, adding a voice, engine-bump drift."
tags:
- dankclaw
- frank-tts
+13 -6
View File
@@ -143,13 +143,13 @@ Copyparty v1.20.9+ supports WOPI (Web Application Open Platform Interface) for e
## Security Advisories
| Version | CVE | Issue |
|---------|-----|-------|
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | FTP upload outside volume |
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | dirkey/filekey confusion |
| v1.20.9 | CVE-2026-27948 / GHSA-62cr-6wp5-q43h | XSS via file upload |
| Version | Advisory | CVE | Issue |
|---------|----------|-----|-------|
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | CVE-2026-104102 | FTP upload outside the defined volumes (limited path traversal) |
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | CVE-2026-70657 | dirkey/filekey confusion |
| v1.20.9 | GHSA-62cr-6wp5-q43h | CVE-2026-27948 | reflected XSS via the `?setck=` URL parameter (not the upload path — the separate upload/DOM-XSS advisory is GHSA-m2jw-cj8v-937r) |
Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`.
Upgrade to v1.20.19+ to cover all known advisories. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`. Each row's patched version is the advisory's own `first_patched_version`.
## Gotchas
@@ -174,3 +174,10 @@ Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters
- [[Dankserver Stack]]
- [[Caddy]] — TLS termination and authentication in front
- [[ARR-Stack]] — Copyparty as a viewer alongside automated download management
## Citations
- GitHub Security Advisories, `9001/copyparty` —
https://github.com/9001/copyparty/security/advisories (accessed 2026-10-19);
each row's patched version read from `first_patched_version` in the
`https://api.github.com/advisories/<GHSA-id>` response.
@@ -1,6 +1,6 @@
---
type: reference
description: "Dankserver (192.168.0.235) runs the media and library stack."
description: "Service inventory and routing overview for the media and library host."
tags:
- dankserver
timestamp: 2026-09-03
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Jellyfin
description: Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and hardware-transcode status.
description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status."
tags: [jellyfin, dankserver, media-server]
timestamp: 2026-09-03T00:00:00Z
---
@@ -1,6 +1,6 @@
---
type: reference
description: "This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping workflow for RomM on Dankserver."
description: "Metadata providers, cover art, ROM naming and the scraping workflow behind the RomM library."
tags:
- dankserver
- romm
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: MariaDB (RomM Backend)
description: MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and backup path.
description: "The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path."
tags: [mariadb, romm, dankserver]
timestamp: 2026-09-03T00:00:00Z
---
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Redis (RomM Cache / Queue)
description: Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup path.
description: "The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path."
tags: [redis, romm, dankserver]
timestamp: 2026-09-03T00:00:00Z
---
@@ -1,7 +1,7 @@
---
type: reference
title: Noctalia and Niri
description: The noctalia v5 desktop shell on niri (workstation + blankpad) — config layers, remote IPC over ssh, custom-palette recolor, offline template rendering, and output rotation/pinning.
description: "The noctalia shell on niri across workstations: config layers, remote IPC, custom palettes, output rotation and pinning."
tags: [desktop, noctalia, niri, blankpad, workstation, ghostty]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: "*arr Stack"
description: "The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) — FrankBook-Pro only, every container on the gluetun WireGuard netns (all egress VPN): topology, wiring, auth posture, and gotchas."
description: "The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas."
tags: [arr-stack, prowlarr, sonarr, radarr, lidarr, readarr, qbittorrent, frankbook-pro, dankserver]
timestamp: 2026-09-04T00:00:00Z
---
+7 -7
View File
@@ -7,10 +7,10 @@ Gluetun WireGuard netns. The host overview lives at [[FrankBook-Pro]]
## *arr Stack
- [[ARR-Stack]] — the full *arr stack: indexers, managers, and the download client
- [[sonarr]] — TV show management
- [[radarr]] — movie management
- [[lidarr]] — music management
- [[readarr]] — ebook/audiobook management
- [[prowlarr]] — indexer management
- [[qbittorrent]] — torrent download client
- [[ARR-Stack]] — The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas.
- [[sonarr]] — Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
- [[radarr]] — Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
- [[lidarr]] — Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
- [[readarr]] — The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client.
- [[prowlarr]] — Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas.
- [[qbittorrent]] — The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe.
@@ -1,7 +1,7 @@
---
type: reference
title: Prowlarr
description: Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and verified live gotchas.
description: "Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas."
tags: [prowlarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
@@ -1,7 +1,7 @@
---
type: reference
title: qBittorrent
description: qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns, PBKDF2 WebUI credential recipe.
description: "The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe."
tags: [qbittorrent, arr-stack, frankbook-pro, gluetun]
timestamp: 2026-09-03T00:00:00Z
---
+17 -9
View File
@@ -1,18 +1,19 @@
---
type: reference
title: Readarr
description: Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB download client.
description: "The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client."
tags: [readarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
# Readarr
> **RETIRED UPSTREAM (May 2026).** The Servarr Team retired Readarr: its
> metadata became unusable and the community migration to Open Library stalled.
> There is no upstream development. Alternatives: the Bookshelf community fork,
> or the Calibre content server. This instance still runs on a pinned image
> tag; see below.
> **RETIRED UPSTREAM (announced 2025-06-27).** The Servarr Team retired
> Readarr: its metadata became unusable and the community migration to Open
> Library stalled. There is no upstream development — the GitHub repository is
> archived and its last push is 2025-06-27. Alternatives: the Bookshelf
> community fork, or the Calibre content server. This instance still runs on a
> pinned image tag; see below.
Readarr manages ebooks and audiobooks in the *arr stack. The stack runs on
**FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
@@ -69,9 +70,9 @@ Restart=always
## Gotchas
- Readarr is abandoned upstream (retired May 2026; last line was 0.4.x before
the final tags). The pinned `amd64-1.0.980` tag exists precisely because
`:latest` does not exist for this image on LSIO.
- Readarr is abandoned upstream (retired 2025-06-27; last line was 0.4.x
before the final tags). The pinned `amd64-1.0.980` tag exists precisely
because `:latest` does not exist for this image on LSIO.
- Calibre integration is folder-based, not API-based. Readarr drops files into
the books root which [[Calibre-Web]] serves.
- Not all indexers carry all formats (epub, mobi, pdf). Goodreads/StoryGraph
@@ -81,3 +82,10 @@ Restart=always
- [[ARR-Stack]] — full *arr stack documentation
- [[Calibre-Web]] — ebook web frontend that serves Readarr downloads
## Citations
- Servarr wiki, *Readarr* (Retired) — "Announcement: Retirement of Readarr
2025-06-27" — https://wiki.servarr.com/readarr (accessed 2026-10-19)
- `Readarr/Readarr` on GitHub: `archived: true`, last push 2025-06-27 —
https://api.github.com/repos/Readarr/Readarr (accessed 2026-10-19)
@@ -1,6 +1,6 @@
---
type: reference
description: "Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to the host-side Signal relay — which sends it through..."
description: "Receives firing rules from Prometheus, groups and dedupes them, and forwards each group to the host-side Signal relay."
tags:
- infrastructure
timestamp: 2026-09-17
@@ -0,0 +1,230 @@
---
type: reference
description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for the homelab's Authentik SSO."
tags:
- infrastructure
- authentik
timestamp: 2026-10-19
---
# Authentik Outpost Reference
Configuration lookup for the [[Authentik]] deployment: the live proxy-provider
inventory behind the guarded vhosts, the provisioning rules the embedded outpost
enforces, this fork's OAuth2 relying-party quirks, the LDAP provider objects, the
self-serve user-settings flow, and the live database shape. What the service is,
how it is deployed, wired and operated is [[Authentik]]; the three core
containers are [[Authentik Processes]].
## Live proxy-provider inventory
Live provider inventory (read from the `authentik-postgres` container with
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
members in `authentik_outposts_outpost_providers`), all with
`internal_host = http://caddy:8017`, all carrying the **same three flow
UUIDs** (`default-authentication-flow`,
`default-provider-authorization-implicit-consent`,
`default-provider-invalidation-flow`), and each backed by an Application
(the `slug` column below):
| Provider pk | name | external_host | app slug |
|---|---|---|---|
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
the lockstep rule in [[Authentik]] holds today. Two corrections against the 2026-09-02 table:
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
not proxy providers, so the proxy outpost does not deserialize them: pk 9
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
below rejects), and pk 14 `Open WebUI`
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
`external_host` (the runbook-recipe style) and bare origins both intercept
correctly — the outpost matches on hostname.
## Provisioning rules for proxy providers
Two provisioning rules, each verified live by an outage:
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
null wedges the embedded outpost's config fetch: the Rust side rejects the
whole config JSON (serde: `invalid type: null, expected a formatted UUID
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
forever, and **everything behind the outpost dies at the next outpost
reload** — including hosts whose providers were fine (verified 2026-09-02
when three providers provisioned without any flows took down every guarded
vhost after a server restart). Backfill all three flows (copy from pk 1:
`default-authentication-flow`,
`default-provider-authorization-implicit-consent` (the implicit-consent
authorization flow, `default-invalidation-flow`) and the retry loop
recovers on its next 3-second cycle. The silent killer: while the outpost
process keeps its last good in-memory config, everything works — the
outage only surfaces on restart/reload.
2. **Every proxy provider needs an attached Application.** The outpost
resolves the post-login redirect through the provider's application; a
provider with no app completes the login (authentik session established)
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
the app. No error is logged anywhere — the SSO "works" and the app never
appears. Create the Application with `provider_id` set (the UI does this
automatically; raw API/ORM drives must do it explicitly).
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
implementation diverges from upstream in ways that bit us; all verified live:
- **Application row required.** `OAuth2Provider.objects.create()` does not
create the single-table-inheritance `Application` row; without it the flow
executor can never complete an authorize request. Create both rows.
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
contains `"authorization_code"` (enum value with underscore), the token
endpoint rejects every exchange with `invalid_request`.
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
`parse_token_request` see no `grant_type` → `unsupported_grant_type`;
POST-body client secrets are not accepted — authenticate the confidential
client with `Authorization: Basic base64(client_id:client_secret)`.
- **Provider resolution is by `client_id` query param** on the global
`/application/o/authorize/` route; per-slug authorize paths do not exist in
this build. Token/userinfo live at `/application/o/token/` and
`/application/o/userinfo/?api=1`.
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
becomes the *expression source* instead of the context; every call raises a
swallowed `PropertyMappingExpressionException` and the claim is skipped.
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
provider FK (linkage lives in the `ProviderPropertyMapping` through model),
so `filter(provider=...)` always yields an empty queryset and tokens asked
to carry scopes get zero. Net effect: userinfo/introspection answers with
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
build until the evaluator is patched or upgraded. Clients consuming this
provider must key identity off `sub`.
- Scope mappings for this provider exist — `grafana-email-scope` and
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
the evaluator bug is fixed; no config change is needed then.
## LDAP provider objects (provisioned via `ak shell` ORM)
The LDAP outpost's own deployment — unit, image digest, published ports, token
env var — is in [[Authentik]]. The objects it serves:
Read from the live database (read-only `SELECT`s against the
`authentik-postgres` container, 2026-10-18):
| Object | Value |
|---|---|
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — an earlier revision's "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — an earlier revision recorded `intent=verification`; the live value is `app_password` |
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (an earlier revision said slug `jellyfin`, which does not exist) |
Hard-won rules (the first is now contested by the live database):
- **Token intent: contested, do not trust either value blindly.** An earlier
revision asserted that a bind token with `intent=app_password` is rejected by
the flow's password stage with `Invalid password` and only
`intent=verification` passes. The live token for `svc-jellyfin-ldap` is
`app_password` (`authentik_core_token`, 2026-10-18), so either the rule is
wrong or the current Jellyfin bind path is not that token. Nothing observable
settles it today: the LDAP outpost's retained log holds only its 5-minute
`initialized direct binder` / `Update providers` config cycles and no bind
activity *(unverified 2026-10-18: no bind was performed — reproducing it
requires the bind secret)*. Re-verify with a real bind before changing the
token's intent either way.
- **Search needs `search_full_directory`.** Without the RBAC grant the
outpost returns only the bind user itself — a `(memberOf=…)` filter then
matches zero entries and the plugin logs `Found no users matching X`.
The permission is object-level on the provider; grant it via an RBAC role
and retest.
- `ak shell` import path for the provider model in 2026.8:
`from authentik.providers.ldap.models import LDAPProvider`.
*(unverified 2026-10-18: importing it outside a configured Django shell fails
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
auth service for this check)*
## Self-serve user-settings flow (password change)
Users change their own account details through the `default-user-settings-flow`
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
target of the homepage footer's "account" link. It is a `stage_configuration`
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
user.
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
insertion order):
| order | stage | type | what it does |
|---|---|---|---|
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
**Password change is a prompt field, not a stage.** The `UserWriteStage`
writes whatever the preceding `PromptStage` collected: for the key `password`
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
is a `password` field on the `default-user-settings` prompt
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
`validate_password` runs `authenticate()` against the backends to check the
*current* password; it never sets a new one. (A `PasswordStage` was added on
2026-09-25 and removed the same day for exactly this reason.)
Two ORM gotchas: `Flow.stages.add()` fails on this through model
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
and a prompt field is a `Prompt` row linked through `PromptStageField`
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
**A blank password must not set an empty password.** `set_password("")` would
make the account log in with no password, so a validation policy
(`default-user-settings-no-empty-password`, bound to the prompt via
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
is `""` and allows it when set. Verified with the `PolicyEvaluator`
(empty → deny, set → allow, absent → allow) and end-to-end by driving
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
invalid).
## Live database shape
For live DB truth prefer the postgres container, and it works without any
credential — the local socket is trusted, so a read-only audit is just
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
before guessing: 2026.x renamed both tables and columns, and the models are
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
three `*_flow_id` columns; the proxy table is
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
table in 2026.8), the LDAP table is
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
`authentik_core_application` attaches to a provider through `provider_id`
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
is the same story from the ORM side: `type` lives in the subclass tables.
Never select secret columns (`client_secret`, `key`, `password`) into a log.
## Cross-references
- [[Authentik]] — what the service is, its deployment, wiring, operations and
failure modes
- [[Authentik Processes]] — the postgres, server and worker containers
- [[Caddy]] — the reverse proxy carrying the `:8016`/`:8017` auth-chain listeners
- [[Jellyfin]] — consumes the LDAP outpost via its LDAP-Auth plugin
- [[Grafana]] — guarded vhost *and* OAuth2 relying party of this provider
@@ -0,0 +1,76 @@
---
type: reference
description: "The three core Authentik containers — database, server and worker — with their units, images, roles and probes."
tags:
- infrastructure
- authentik
- postgres
timestamp: 2026-10-19
---
# Authentik Processes
The three core containers of the [[Authentik]] stack on Dankclaw. The two
outpost units are not covered here — they live on [[Authentik]] and
[[Authentik Outpost Reference]]. Server and worker run the same image; the
`command` selects the role (`server` / `worker`). Data persists under
`/home/dank/Containers/authentik/` as bind mounts (repo policy — no named
volumes), moved from `/home/dank/authentik/` in the 2026-09-01 quadlet
standardization; the retired stubs this page replaces recorded the pre-move
paths `/home/dank/authentik/data` (server and worker) and
`/home/dank/authentik/postgres` (database).
## authentik-postgres
The PostgreSQL 16 database for Authentik — the only stateful store in the stack
(users, flows, sessions, events). Sessions, caching, and WebSockets moved to
PostgreSQL in 2025.10 (Redis removed).
- **Image:** `docker.io/library/postgres:16-alpine`
- **Data:** `/home/dank/Containers/authentik/postgres` (bind mount; no named
volumes)
- **Health:** `pg_isready -d authentik -U authentik`
- **Retired wording:** the stub called this a "Compose service … in
`dankclaw/compose.yaml`"; the compose stack is gone and the unit is a quadlet.
- The side-by-side with `gitea-postgres` (network, published ports, secret
source, boot persistence) is owned by [[Postgres]].
## authentik-server
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
It also serves the login page at `https://auth.franksdank.cloud`.
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container`
(`authentik-server.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: server`)
- **Port:** `9000` (published) — the outpost, the admin API and the login UI all
live here
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
worker)
- **Health:** recorded by the retired stub as
`http://localhost:9000/-/health/`; superseded — [[Authentik]] → Operations
owns the live probe.
- **Retired wording:** the stub described "the Caddy `forward_auth` vhosts"
proxying here; the chain Caddy actually runs is [[Authentik]] → Caddy
integration.
## authentik-worker
The Authentik worker — background tasks: event processing, provider
synchronization, scheduled flows, and certificate renewal.
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container`
(`authentik-worker.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: worker`)
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns
the data dir)
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
server)
- **No port** — the worker has no HTTP endpoint.
## Cross-references
- [[Authentik]] — what the service is, its deployment, wiring, operations and
failure modes
- [[Authentik Outpost Reference]] — provider, flow and token configuration
- [[Postgres]] — both PostgreSQL clusters on Dankclaw, side by side
+23 -212
View File
@@ -16,7 +16,7 @@ OpenWebUI, Jellyfin (LDAP outpost), the *arr stack, Pi-hole (basicauth edge) —
they handle their own session. Grafana is the exception that is both: it is a
guarded vhost (`grafana-proxy`, pk 8, locked to Authentik SSO) *and* an OAuth2
relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
`dankclaw/Caddyfile:374-377`; live provider table below).
`dankclaw/Caddyfile:374-377`; live provider table in [[Authentik Outpost Reference]]).
- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`;
the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`,
@@ -35,6 +35,13 @@ relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
pre-quadlet era, retired.
- **LDAP outpost:** serves LDAP for Jellyfin — see the section below.
**Split off from this page:** [[Authentik Processes]] owns the per-container
facts for `authentik-postgres`, `authentik-server` and `authentik-worker`;
[[Authentik Outpost Reference]] owns the provider/flow/token lookup material —
the live proxy-provider inventory, the proxy-provider provisioning rules, the
OAuth2 relying-party quirks, the LDAP provider objects, the user-settings flow
design, and the live database shape.
## Why Authentik
The comparison that chose it (against Authelia, Keycloak, Zitadel, Ory) was
@@ -100,23 +107,10 @@ contract:
The chain terminates there — nothing dials back into Authentik from
`:8017`.
## The three compose services
## The stack's processes
| Service | Role | Image |
|---|---|---|
| `authentik-postgres` | PostgreSQL 16 (users, flows, sessions, events) | `docker.io/library/postgres:16-alpine` |
| `authentik-server` | Core + admin API + the embedded Caddy outpost (port 9000) | `ghcr.io/goauthentik/server:2026.8.3` |
| `authentik-worker` | Background tasks (event processing, provider sync, flows) | `ghcr.io/goauthentik/server:2026.8.3` |
- **Postgres** is the only stateful store. Sessions, caching, and WebSockets
moved to PostgreSQL in 2025.10 (Redis removed).
- **Server** and **worker** run the same image; the `command` selects the role
(`server` / `worker`).
- The worker runs as `0:0` (rootless: container root maps to the host user that
owns the data dir).
- Data persists under `/home/dank/Containers/authentik/{data,postgres}`
(bind mounts, repo policy — no named volumes; moved from `/home/dank/authentik/`
in the 2026-09-01 quadlet standardization).
Per-container facts — unit path, image and role, port, run-as, data directory,
health probe — are owned by [[Authentik Processes]].
## Caddy integration
@@ -142,104 +136,9 @@ routing table, and the Authentik proxy-provider list must be the same set of
hosts. A guarded host with no provider 404s at login time; a provider with no
DNS record breaks off-LAN clients.
Live provider inventory (read from the `authentik-postgres` container with
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
members in `authentik_outposts_outpost_providers`), all with
`internal_host = http://caddy:8017`, all carrying the **same three flow
UUIDs** (`default-authentication-flow`,
`default-provider-authorization-implicit-consent`,
`default-provider-invalidation-flow`), and each backed by an Application
(the `slug` column below):
| Provider pk | name | external_host | app slug |
|---|---|---|---|
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
the lockstep rule holds today. Two corrections against the 2026-09-02 table:
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
not proxy providers, so the proxy outpost does not deserialize them: pk 9
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
below rejects), and pk 14 `Open WebUI`
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
`external_host` (the runbook-recipe style) and bare origins both intercept
correctly — the outpost matches on hostname.
Two provisioning rules, each verified live by an outage:
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
null wedges the embedded outpost's config fetch: the Rust side rejects the
whole config JSON (serde: `invalid type: null, expected a formatted UUID
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
forever, and **everything behind the outpost dies at the next outpost
reload** — including hosts whose providers were fine (verified 2026-09-02
when three providers provisioned without any flows took down every guarded
vhost after a server restart). Backfill all three flows (copy from pk 1:
`default-authentication-flow`,
`default-provider-authorization-implicit-consent` (the implicit-consent
authorization flow, `default-invalidation-flow`) and the retry loop
recovers on its next 3-second cycle. The silent killer: while the outpost
process keeps its last good in-memory config, everything works — the
outage only surfaces on restart/reload.
2. **Every proxy provider needs an attached Application.** The outpost
resolves the post-login redirect through the provider's application; a
provider with no app completes the login (authentik session established)
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
the app. No error is logged anywhere — the SSO "works" and the app never
appears. Create the Application with `provider_id` set (the UI does this
automatically; raw API/ORM drives must do it explicitly).
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
implementation diverges from upstream in ways that bit us; all verified live:
- **Application row required.** `OAuth2Provider.objects.create()` does not
create the single-table-inheritance `Application` row; without it the flow
executor can never complete an authorize request. Create both rows.
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
contains `"authorization_code"` (enum value with underscore), the token
endpoint rejects every exchange with `invalid_request`.
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
`parse_token_request` see no `grant_type` → `unsupported_grant_type`;
POST-body client secrets are not accepted — authenticate the confidential
client with `Authorization: Basic base64(client_id:client_secret)`.
- **Provider resolution is by `client_id` query param** on the global
`/application/o/authorize/` route; per-slug authorize paths do not exist in
this build. Token/userinfo live at `/application/o/token/` and
`/application/o/userinfo/?api=1`.
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
becomes the *expression source* instead of the context; every call raises a
swallowed `PropertyMappingExpressionException` and the claim is skipped.
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
provider FK (linkage lives in the `ProviderPropertyMapping` through model),
so `filter(provider=...)` always yields an empty queryset and tokens asked
to carry scopes get zero. Net effect: userinfo/introspection answers with
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
build until the evaluator is patched or upgraded. Clients consuming this
provider must key identity off `sub`.
- Scope mappings for this provider exist — `grafana-email-scope` and
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
the evaluator bug is fixed; no config change is needed then.
The live proxy-provider inventory (the per-vhost `external_host` → app mapping),
the two provisioning rules the outpost enforces, and this fork's OAuth2
relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
## LDAP outpost (Jellyfin auth, live 2026-09-01)
@@ -267,44 +166,9 @@ rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect
(2026-10-18). The outpost token travels in `AUTHENTIK_LDAP_OUTPOST_TOKEN`
from the host `dankclaw/.env` (git-ignored).
### Authentik-side objects (provisioned via `ak shell` ORM)
Read from the live database (read-only `SELECT`s against the
`authentik-postgres` container, 2026-10-18):
| Object | Value |
|---|---|
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — the page's earlier "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — the page previously recorded `intent=verification`; the live value is `app_password` |
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (the page previously said slug `jellyfin`, which does not exist) |
Hard-won rules (the first is now contested by the live database):
- **Token intent: contested, do not trust either value blindly.** This page
long asserted that a bind token with `intent=app_password` is rejected by the
flow's password stage with `Invalid password` and only `intent=verification`
passes. The live token for `svc-jellyfin-ldap` is `app_password`
(`authentik_core_token`, 2026-10-18), so either the rule is wrong or the
current Jellyfin bind path is not that token. Nothing observable settles it
today: the LDAP outpost's retained log holds only its 5-minute
`initialized direct binder` / `Update providers` config cycles and no bind
activity *(unverified 2026-10-18: no bind was performed — reproducing it
requires the bind secret)*. Re-verify with a real bind before changing the
token's intent either way.
- **Search needs `search_full_directory`.** Without the RBAC grant the
outpost returns only the bind user itself — a `(memberOf=…)` filter then
matches zero entries and the plugin logs `Found no users matching X`.
The permission is object-level on the provider; grant it via an RBAC role
and retest.
- `ak shell` import path for the provider model in 2026.8:
`from authentik.providers.ldap.models import LDAPProvider`.
*(unverified 2026-10-18: importing it outside a configured Django shell fails
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
auth service for this check)*
The objects behind it — LDAP provider, bind DN, service account and bind token,
RBAC grant, groups, application slug — and the hard-won bind rules are in
[[Authentik Outpost Reference]].
## Configuration (secrets)
@@ -340,7 +204,8 @@ every outpost session (see the ORM playbook).
Applications** → add an application + a **Proxy** provider whose
`external_host` is exactly the browser-facing origin
(`https://<host>`), `internal_host = http://caddy:8017`, and **all three**
flows set (see the provisioning pitfall above). Attach the provider to the
flows set (see the provisioning rules in
[[Authentik Outpost Reference]]). Attach the provider to the
embedded outpost.
4. For apps with native OIDC: add an **OAuth2/OpenID Connect** provider, create
the app with the correct redirect URI, and paste the `client_id` /
@@ -373,47 +238,6 @@ every outpost session (see the ORM playbook).
the LDAP section) — bumping only the tag would leave the LDAP outpost behind.
Authentik migrates the schema on startup.
## Self-serve user-settings flow (password change)
Users change their own account details through the `default-user-settings-flow`
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
target of the homepage footer's "account" link. It is a `stage_configuration`
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
user.
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
insertion order):
| order | stage | type | what it does |
|---|---|---|---|
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
**Password change is a prompt field, not a stage.** The `UserWriteStage`
writes whatever the preceding `PromptStage` collected: for the key `password`
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
is a `password` field on the `default-user-settings` prompt
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
`validate_password` runs `authenticate()` against the backends to check the
*current* password; it never sets a new one. (A `PasswordStage` was added on
2026-09-25 and removed the same day for exactly this reason.)
Two ORM gotchas: `Flow.stages.add()` fails on this through model
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
and a prompt field is a `Prompt` row linked through `PromptStageField`
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
**A blank password must not set an empty password.** `set_password("")` would
make the account log in with no password, so a validation policy
(`default-user-settings-no-empty-password`, bound to the prompt via
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
is `""` and allows it when set. Verified with the `PolicyEvaluator`
(empty → deny, set → allow, absent → allow) and end-to-end by driving
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
invalid).
## Outpost claim cache (`proxysession`) — 403 after a group add
The embedded outpost does not re-evaluate group membership per request. Each
@@ -457,7 +281,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
(the inventory table carries both bare origins and trailing slashes;
routing matches on hostname, verified 2026-09-02).
`internal_host = http://caddy:8017`, all three flows set, and an
Application attached — the two provisioning rules above.
Application attached — the two provisioning rules in [[Authentik Outpost Reference]].
2. **Route flip in ONE commit/deploy**: the vhost becomes
`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
inside the `http://:8017` listener add `@<svc> host <vhost>` +
@@ -521,22 +345,9 @@ EOF
is no `authentik.policies.group` module in this build.
*(unverified 2026-10-18: these module paths were not re-executed — importing
outside a configured Django shell fails with `ImproperlyConfigured`)*
- For live DB truth prefer the postgres container, and it works without any
credential — the local socket is trusted, so a read-only audit is just
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
before guessing: 2026.x renamed both tables and columns, and the models are
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
three `*_flow_id` columns; the proxy table is
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
table in 2026.8), the LDAP table is
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
`authentik_core_application` attaches to a provider through `provider_id`
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
is the same story from the ORM side: `type` lives in the subclass tables.
Never select secret columns (`client_secret`, `key`, `password`) into a log.
- The live database's table and column names (the 2026.x renames, the MTI parent
keys, the read-only `psql` audit recipe) are reference lookup:
[[Authentik Outpost Reference]].
## User provisioning playbook (user-level account)
@@ -69,7 +69,7 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
client_id `grafana-21ca190cde75d60f`, global authorize/token/userinfo paths,
Basic client auth, form-encoded token body, and the fork's userinfo-claims
bug that leaves identity at the `sub` hash only) is in
[[Authentik]] → "OAuth2 relying-party quirks". Consequences for Grafana:
[[Authentik Outpost Reference]] → "OAuth2 relying-party quirks". Consequences for Grafana:
The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`,
`ORG_ID=1` and **no** role-attribute mapping (`grafana.container:52-55`), so
users land as Viewer with `login = <sub hex>`; no email/groups/role claims
@@ -1,7 +1,7 @@
---
type: reference
title: Homepage Status Probes
description: The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage procedure, local render check, deploy, and the e2e gate interactions (asleep Mac mini, raw-IP matrix).
description: "How probe targets are chosen, the three false-down shapes, the triage procedure, and the e2e gate interactions."
tags: [homepage, status, probes, e2e, homepage-status]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,6 +1,6 @@
---
type: reference
description: "A password-protected web site on Dankclaw lists the services that are up, with a short description of each and a link to both the service and its usage documentation."
description: "The password-protected service list on Dankclaw: what each card shows and where its data comes from."
tags:
- infrastructure
- homepage
@@ -1,7 +1,7 @@
---
type: reference
title: Observability Provisioning
description: How Prometheus config/rules and Grafana datasources/dashboards are provisioned as code in serverAppConfigs — the config triad, deploy copiers, planner mappings, seam tests, entry points for each change type, and API verification.
description: "Monitoring config as code: the config triad, deploy copiers, planner mappings, and where each change type lands."
tags: [observability, prometheus, grafana, provisioning, deployment]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: "Observability on Dankclaw"
description: "The Dankclaw observability stack — Prometheus, Grafana, Loki, Alloy, node-exporter and Alertmanager (Signal delivery): what runs, how it is deployed, how it is exposed, and what is still unconfigured."
description: "The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured."
tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw]
timestamp: 2026-09-11
---
@@ -1,6 +1,6 @@
---
type: reference
description: "Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints, and feeds the provisioned Grafana datasource. Down targets today are structural, not outages."
description: "Scrapes each service's metrics endpoint and feeds the provisioned Grafana datasource; down targets are structural."
tags:
- infrastructure
- prometheus
@@ -1,24 +0,0 @@
---
type: reference
description: "The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /"
tags:
- infrastructure
- authentik
- postgres
timestamp: 2026-08-28
---
# authentik-postgres
The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /
OIDC provider. It is the only stateful store in the stack (users, flows,
sessions, events). Redis was removed in Authentik 2025.10; sessions and caching
now live here.
- **Compose service:** `authentik-postgres` (in `dankclaw/compose.yaml`)
- **Image:** `docker.io/library/postgres:16-alpine`
- **Data:** `/home/dank/authentik/postgres` (bind mount; no named volumes)
- **Health:** `pg_isready -d authentik -U authentik`
See [Authentik.md](Authentik.md) for the full architecture, setup, backups
(`pg_dump`), and failure modes.
@@ -1,24 +0,0 @@
---
type: reference
description: "The Authentik server — the core, the admin API, and the embedded Caddy outpost."
tags:
- infrastructure
- authentik
timestamp: 2026-09-16
---
# authentik-server
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
This is the service the Caddy `forward_auth` vhosts proxy to
(`authentik-server:9000`). It also serves the login page at
`https://auth.franksdank.cloud`.
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container` (`authentik-server.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `server`)
- **Port:** `9000` (published; the outpost + admin + login all live here)
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the worker)
- **Health:** `http://localhost:9000/-/health/`
See [Authentik.md](Authentik.md) for the full architecture, the Caddy
forward_auth pattern, first-time setup, and failure modes.
@@ -1,23 +0,0 @@
---
type: reference
description: "The Authentik worker — background tasks: event processing, provider"
tags:
- infrastructure
- authentik
timestamp: 2026-09-16
---
# authentik-worker
The Authentik worker — background tasks: event processing, provider
synchronization, scheduled flows, and certificate renewal. It runs the same
image as the server; the `command: worker` selects the role.
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container` (`authentik-worker.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `worker`)
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns the data dir)
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the server)
- **No port** — the worker has no HTTP endpoint.
See [Authentik.md](Authentik.md) for the full architecture, setup, and failure
modes.
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: Dankserver System Specs
description: Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI PRO R9700 gfx1201 on CPU lanes x8/x8) serving frank-medium.
description: "Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture."
tags:
- dankserver
- hardware
@@ -1,7 +1,7 @@
---
type: research
type: reference
title: Dual R9700 Cooperation
description: Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic routing), what the 2026-09-20 ProArt B850-Creator swap unblocked, and what still has to be probed.
description: "Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked."
tags:
- frank-medium
- rocm
@@ -1,7 +1,7 @@
---
type: research
type: reference
title: FastFlowLM NPU Inference
description: FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on Dankclaw, published benchmarks, verified install state, and a test runbook.
description: "The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook."
tags:
- fastflowlm
- npu
@@ -1,7 +1,7 @@
---
type: reference
title: FreeToken Builds
description: Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the CUDA vLLM venv build and the ROCm build, with the dependency traps each one hit.
description: "Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit."
tags: [freetoken, build, dankclaw, vllm, mtp]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: GGUF Compatibility and Runtime Lock-in
description: How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the runtime, and the discipline for pinning a fork build that can actually serve the file.
description: "Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can."
tags: [gguf, llama.cpp, forks, quantization, speculative]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: Heterogeneous Split Triage
description: Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X + LPDDR APU, PCIe x4 link) — link characterisation, the real bottleneck, what placement never buys, UMA pools, drafter splits, and the verdict ranking.
description: "Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys."
tags: [heterogeneous, multi-device, llama-cpp, triage, pcie]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: LLM Bench Windows and Measurement
description: Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm vLLM offload sweeps), honest benchmark methodology, and the usage-counter measurement law under speculative decoding.
description: "Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting."
tags: [benchmarking, llama-bench, ab-testing, speculative, vllm, fleet]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: LLM Performance Diagnosis
description: Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time attribution via the profiler window, live telemetry without benchmarking, RAM-bandwidth ceilings, IBS tensor attribution and prefix-cache-aware routing.
description: "Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution."
tags: [performance, profiling, moe, pcie, prefix-cache, routing, telemetry]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: Llama Cpp Multi-Device Splits
description: What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by measurement, drafter KV sizing from GGUF headers, and heterogeneous placement triage.
description: "What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing."
tags: [llama.cpp, multi-gpu, layer-split, pcie, sizing, speculative]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: Llama Cpp Serving Sizing
description: Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA flash-attn fix — the measurement-first procedures for llama.cpp serving shapes.
description: "Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache."
tags: [llama.cpp, vram, kv-cache, slots, flash-attn, sizing]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: Model Swap and Rollback
description: The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag checks, git-only config, live validation, systemd handoff, close-out, the three-pin revert law, and research-verification discipline.
description: "The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert."
tags: [models, swap, gguf, huggingface, rollback, deployment, verification]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: PCIe Link and Atomics
description: Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic, and the PCIe-atomics wall that decides whether AMD multi-GPU collectives can exist at all.
description: "Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives."
tags: [pcie, rccl, atomics, multi-gpu, bandwidth, amd, nvidia]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,11 +1,11 @@
---
type: reference
title: Ethernet Bandwidth Math
description: Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node benchmark numbers.
description: "Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it."
tags:
- pipeline-parallelism
- research
timestamp: 2026-08-17
timestamp: 2026-10-19
---
# Bandwidth Analysis: Multi-Node LLM Inference over Ethernet
@@ -147,19 +147,11 @@ bytes_per_boundary = batch_size × hidden_size × 2 [fp16 = 2 bytes]
- Per-token, per-boundary: `6144 × 2 = 12,288 bytes = 12 KB`
- With Nₚₚ stage boundaries: total = `Nₚₚ × 12 KB/tok`
**Full example — 27B model, 2 PP stages (1 boundary), batch=1:**
**Worked examples (1 boundary = 2 PP stages):**
```
1 boundary × 4096 × 2 bytes = 8,192 bytes/tok
```
**Example — same model, batch=64:**
```
1 boundary × 4096 × 64 × 2 bytes = 524,288 bytes = 512 KB/tok
```
**Example — 36B model, 2 PP stages, batch=1:**
```
1 boundary × 6144 × 2 bytes = 12,288 bytes = 12 KB/tok
27B, batch=1: 1 boundary × 4096 × 2 bytes = 8,192 bytes/tok = 8 KB
27B, batch=64: 1 boundary × 4096 × 64 × 2 bytes = 524,288 = 512 KB/tok
36B, batch=1: 1 boundary × 6144 × 2 bytes = 12,288 bytes/tok = 12 KB
```
### 1.2 Tensor Parallelism (TP) — for comparison
@@ -244,57 +236,13 @@ total_bytes_per_tok = L × batch_size × hidden_size × 2 [L all-reduces total
| 50 | 10.24 Mbit/s | 163.84 Mbit/s | 655.36 Mbit/s |
| 100 | 20.48 Mbit/s | 327.68 Mbit/s | 1,310.72 Mbit/s |
### 2.3 Feasibility matrix
### 2.3 Feasibility
Green = feasible (< 50% of link capacity). Yellow = marginal (50–90%). Red = saturated (> 90%).
**27B model — PP (1 boundary), batch=1:**
| | 1 GbE (118 MB/s) | 2.5 GbE (294 MB/s) | 10 GbE (1175 MB/s) |
|---|---|---|---|
| 10 tok/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s |
| 50 tok/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s |
| 100 tok/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s |
**27B model — PP (1 boundary), batch=16:**
| | 1 GbE | 2.5 GbE | 10 GbE |
|---|---|---|---|
| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
**27B model — PP (1 boundary), batch=64:**
| | 1 GbE | 2.5 GbE | 10 GbE |
|---|---|---|---|
| 10 tok/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s |
| 50 tok/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s |
| 100 tok/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s |
**27B model — TP (32 layers), batch=1:**
| | 1 GbE | 2.5 GbE | 10 GbE |
|---|---|---|---|
| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
**27B model — TP (32 layers), batch=16:**
| | 1 GbE | 2.5 GbE | 10 GbE |
|---|---|---|---|
| 10 tok/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s |
| 50 tok/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s |
| 100 tok/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s |
**27B model — TP (32 layers), batch=64:**
| | 1 GbE | 2.5 GbE | 10 GbE |
|---|---|---|---|
| 10 tok/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s |
| 50 tok/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s |
| 100 tok/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s |
Thresholds: **feasible** = under 50% of link capacity, **marginal** = 50–90%,
**saturated** = over 90%. Cross-checking §2.2 against §2.1: every PP case is
feasible on every link class — even the heaviest, 27B at batch=64 and 100
tok/s (81.92 Mbit/s), is feasible on 1 GbE — and every TP case is feasible
except batch=64:
**27B TP batch=64 at 100 tok/s: 1310.72 Mbit/s > 1 GbE usable (1000 Mbit/s) — link-saturated on 1 GbE**
@@ -368,10 +316,7 @@ With TP, each layer transfers `H × 2` bytes but this happens L times. For 256 K
**Petals [1]:**
- Uses pipeline parallelism with dynamic 8-bit quantization (halves bandwidth)
- For BLOOM-176B (H=14336), per-boundary communication is ~14 KB/tok with quantization
- Recommended bandwidth: 25 Mbit/s bidirectional for interactive inference [1]
- Key finding: "For inference, performance does not depend much on bandwidth or sequence length but degrades with higher latency" — RTT matters more than throughput
- At 1 Gbit/s with <5 ms RTT: 1.71 steps/s; at 100 Mbit/s with <5 ms: 1.66 steps/s (only 3% difference)
- At 100 Mbit/s with 100 ms RTT: 1.23 steps/s (28% degradation)
**llama.cpp RPC [2]:**
- Model weights and KV cache distributed across nodes proportionally to memory
@@ -493,38 +438,17 @@ With Nₚₚ = 4 and 30 µs per-hop RTT: ~120 µs network, still <6% of compute
## Appendix: Full bandwidth formulae
### Pipeline Parallelism (generation, batch=1)
### Pipeline and Tensor Parallelism (generation)
`N` is the batch size; `N = 1` is the batch=1 form.
```
bytes_per_boundary_per_tok = hidden_size × 2 [fp16]
total_bytes_per_tok = num_boundaries × hidden_size × 2
bytes_per_boundary_per_tok = hidden_size × N × 2 [fp16, PP boundary]
bytes_per_layer_allreduce = hidden_size × N × 2 [TP, one all-reduce per layer]
total_bytes_per_tok = num_boundaries × hidden_size × N × 2 [PP]
total_bytes_per_tok = num_layers × hidden_size × N × 2 [TP]
bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
```
### Pipeline Parallelism (generation, batch=N)
```
bytes_per_boundary_per_tok = hidden_size × N × 2
total_bytes_per_tok = num_boundaries × hidden_size × N × 2
```
### Tensor Parallelism (generation, batch=1)
```
bytes_per_layer_allreduce = hidden_size × 2
total_bytes_per_tok = num_layers × hidden_size × 2
bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
```
### Tensor Parallelism (generation, batch=N)
```
total_bytes_per_tok = num_layers × hidden_size × N × 2
```
### With 8-bit quantization (like Petals)
```
# Simply divide all fp16 byte counts by 2
bytes_per_boundary_per_tok = hidden_size × 1 [int8]
# With 8-bit quantization (like Petals): simply divide all fp16 byte counts by 2
bytes_per_boundary_per_tok = hidden_size × 1 [int8]
```
@@ -1,7 +1,7 @@
---
type: reference
title: Expected Performance
description: Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors.
description: "Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on."
tags:
- pipeline-parallelism
- performance
@@ -13,7 +13,7 @@ timestamp: 2026-08-14
This page estimates the speed of a 25-40B model split across two machines
over ethernet. It uses the bandwidth math in [[Ethernet Bandwidth Math]]
and the measured numbers in [[Mixed-Vendor Experiments]].
and the mixed-vendor measurements in [[x4-pair-experiment]].
## The Bandwidth Ceiling Is Not the Limit
@@ -93,5 +93,5 @@ roughly 200k-260k tokens across both nodes.
## See Also
- [[Ethernet Bandwidth Math]]
- [[Mixed-Vendor Experiments]]
- [[Qwen3.8 PP Deployment]]
- [[Sample Script Set]]
@@ -1,9 +0,0 @@
---
type: reference
title: "Mixed-Vendor Experiments"
description: "Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks (mixed CUDA+ROCm vLLM), measured tok/s."
tags:
- pipeline-parallelism
- research
timestamp: 2026-08-14
---
@@ -1,9 +0,0 @@
---
type: reference
title: "Qwen3.6 27B Spec Methods"
description: "Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC compatibility, memory/context math."
tags:
- pipeline-parallelism
- research
timestamp: 2026-08-14
---
@@ -1,6 +1,6 @@
---
type: reference
description: "Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX"
description: "The real split deployment: which node is head and which is worker, the flags, and the verified gotchas."
tags:
- llm
- pipeline
@@ -1,7 +1,7 @@
---
type: reference
title: Sample Script Set
description: An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or deploy.
description: "An educational two-node RPC split script set, for reading only — never run it or deploy it."
tags:
- pipeline-parallelism
- sample
@@ -255,7 +255,7 @@ is 30 GiB.
## See Also
- [[llama.cpp RPC]]
- [[Qwen3.8 PP Deployment]]
- [[Expected Performance]]
- [[Ethernet Bandwidth Math]]
- [[Qwen3.6 27B Spec Methods]]
- [[DFlash DSpark Speculative Decoding]]
@@ -28,28 +28,24 @@ graph LR
end
```
- [[llama.cpp RPC]] — the working, production-capable path for two
heterogeneous nodes over ethernet.
- [[vLLM Pipeline Parallelism]] — supported but heavier; needs Ray or
multiprocessing and does not compose with speculative decoding yet.
## What the Research Found
- [[Ethernet Bandwidth Math]] — a PP boundary moves only the hidden state
- [[Ethernet Bandwidth Math]] — Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it.
(about 10 KiB per token for a 27B model). 1 GbE supports hundreds of
tokens per second on paper; the real limit is latency and RPC overhead.
- [[Mixed-Vendor Experiments]] — NVIDIA + Strix Halo multi-node inference
works today via llama.cpp RPC with separate CUDA and ROCm builds. The
exact NVIDIA-desktop + Strix-Halo-NUC pair has not been benchmarked with
published numbers yet.
- [[Qwen3.6 27B Spec Methods]] — Q8 GGUF variants, and which speculative
methods (MTP, DFlash, ngram-mod) work with RPC. DSpark is vLLM-only.
- [[Expected Performance]] — estimated tokens per second for a 27B Q8 split
- [[Expected Performance]] — Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on.
across an RTX 3090 and a Strix Halo NUC over 1/2.5/10 GbE.
## Deployment
- [[Qwen3.8 PP Deployment]] — the real Qwen3.8-27B split: head on Dankclaw
- [[Qwen3.8 PP Deployment]] — The real split deployment: which node is head and which is worker, the flags, and the verified gotchas.
(Strix), worker on Dankserver (RTX 3090), b10433 on both sides, scripts in
`scripts/`, morning-of test runbook. Verified gotchas: no `--device` with
`--rpc`, default split is proportional to reported free memory (pin 1,1),
@@ -57,7 +53,7 @@ graph LR
## Sample Scripts
- [[Sample Script Set]] — a two-node llama.cpp RPC split of
- [[Sample Script Set]] — An educational two-node RPC split script set, for reading only — never run it or deploy it.
Qwen3.6-27B-Q8_K_P with MTP, DFlash, and ngram-mod variants, sized for
the largest context that fits. For educational reading only. Do not run
it in production.
@@ -68,3 +64,8 @@ graph LR
- [[frank-zealot]]
- [[Dankserver System Specs]]
- [[Dankclaw System Specs]]
## Pages
| Page | Owns |
|---|---|
@@ -1,9 +0,0 @@
---
type: reference
title: "llama.cpp RPC"
description: "llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and RPC-compatible speculative decoding."
tags:
- pipeline-parallelism
- research
timestamp: 2026-08-14
---
@@ -1,9 +0,0 @@
---
type: reference
title: "vLLM Pipeline Parallelism"
description: "vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP + speculative decoding support."
tags:
- pipeline-parallelism
- research
timestamp: 2026-08-14
---
@@ -1,3 +1,13 @@
---
type: "howto"
title: "Quantization Production"
description: "Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file."
tags:
- llm
- quantization
timestamp: 2026-10-19
---
# Quantization Production — making our own quants, per card and runtime
**Research page (2026-10-08).** Question: how do WE produce quantizations
@@ -1,7 +1,7 @@
---
type: reference
title: RDNA4 Container Serving
description: Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class gating, fetch verification, container GPU-init triage, memory budgeting and the expert-offload plan.
description: "Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage."
tags: [rdna4, gfx1201, r9700, vllm, expert-offload, podman, rocm]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,7 +1,7 @@
---
type: reference
title: RDNA4 MoE Expert Offload VRAM Budget
description: The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the CUDA-graph charge, what free VRAM really is, knife-edge survivability, parallelism ceilings and the context ladder.
description: "Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder."
tags: [vllm, expert-offload, vram, arena, kv-cache, ladder, tcclaviger]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -1,6 +1,6 @@
---
type: reference
description: "Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series."
description: "Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate."
tags:
- llm
timestamp: 2026-09-03
@@ -1,6 +1,6 @@
---
type: reference
description: "Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series."
description: "Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape."
tags:
- llm
timestamp: 2026-09-03
@@ -1,7 +1,7 @@
---
type: reference
title: Tcclaviger Fork Ops
description: Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken cutover) — the ad-hoc sweep harness, measured knob verdicts, sweep hygiene, the large-MoE serving recipe, and compile-cache persistence.
description: "Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence."
tags: [tcclaviger, vllm, dankserver, expert-offload, sweeps, cache]
timestamp: 2026-10-05T00:00:00-07:00
---
+1 -1
View File
@@ -1,6 +1,6 @@
---
type: reference
description: "This index contains all documentation for vLLM deployment on Dankserver and Dankclaw."
description: "Locator for every vLLM page in this vault."
tags:
- llm
- vllm
@@ -1,7 +1,7 @@
---
type: research
type: reference
title: Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)
description: Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a measured prefill cost of nothing (+2.0% on byte-identical traffic). Records the 2026-09-11 amd_iommu=off era that bought a vendor-claimed 13-16% figure never measured here, what that cost, the AMD PerfOpt research (Linux 7.4), the tooling that owns the boot line, and the revert path.
description: "Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line."
tags:
- dankclaw
- iommu
@@ -1,7 +1,7 @@
---
type: research
type: reference
title: Frontier MoE on Dankserver — what fits, on which runtime
description: VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded.
description: "The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded."
tags:
- dankserver
- r9700
@@ -1,7 +1,7 @@
---
type: reference
title: dealign Ornith-1.5-35B-A3B CRACK on Dankserver
description: dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with ngram-mod speculative decoding; selected expert weights offloaded to CPU with an explicit --override-tensor. frank-zealot.
description: "Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card."
tags:
- llamacpp
- cuda
@@ -9,6 +9,7 @@ tags:
- moe
- ornith
- mtp
- retired-endpoint
timestamp: 2026-08-24
---
@@ -123,4 +124,4 @@ llama-server \
- [[deployment]] — deploy and update process
- [[Dankserver System Specs]] — the box
- [[frank-redo-2026-08-17]] — fleet topology history
- [[frank-redo-2026-08-17]] — fleet topology histor
+1 -1
View File
@@ -1,5 +1,5 @@
---
type: guide
type: howto
title: LLM Deployment
description: Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot).
tags:
+12 -4
View File
@@ -1,7 +1,7 @@
---
type: reference
title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)"
description: "The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras table carrying every rollback path including the re-download doors for the purged payloads, the laws this endpoint taught (pinned-RAM/TTM preflight, the KV dtype law, the one per-card VRAM budget, descriptor selection), the wire contract, and the files that own it. It replaces the nine per-era runbooks."
description: "The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught."
tags:
- llm
- frank-archon
@@ -11,6 +11,7 @@ tags:
- r9700
- tp2
- expert-offload
- endpoint
timestamp: 2026-10-18
---
@@ -139,15 +140,21 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
| 2026-09-26 → 09-27 | unsloth Q8_0, 175.30 GiB / 6 parts + shared-Q8_0 MTP 2,786,568,256 B + `mmproj-BF16.gguf` 907,542,944 B | unsloth llama.cpp b11030-mix prebuilt (commit `6ba30d05b`), ROCm gfx120X | `-ts 1,1` layer split, routed experts **and** the PLE table computed in host RAM (`-ot exps=CPU,per_layer_token_embd.weight=CPU`), 262,144 × 1 slot, then the same-day reshape to 4 × 262,144 with V `q8_0` and hot expert blocks 5/3 | the R9V radiance-vLLM fork took the port at Frank's direction (true TP=2 plus MTP4) | trio at commit `4992e5a`+; the GGUF/MTP/mmproj set is no longer on disk → re-fetch, then redeploy |
| 2026-09-26 (same day) | `Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF`, 377.2 GiB, sha256 `2daea6ca…` | `localhost/lucebox-rocm:gfx1201`, built from `Dockerfile.rocm` + 3 local patches | ONE R9700, `--max-ctx 131072`, `--chunk 8192`, q4_0 K+V, `--ds4-prefill sparse`, routed experts on CPU from the page cache | 377 GiB fits neither 63.7 GiB of VRAM nor 246 GiB of RAM → NVMe page-cache thrash; Qwen Flash-Next Q8 gave better quality, vision and 262k at comparable speed | image still in dank's rootless storage; payload gone; the quadlet and its entrypoint were deleted by `8500600` — `git show 8500600^:dankserver/frank-primary/quadlet/frank-primary.container` (and `…:dankserver/frank-primary/frank-primary-entrypoint.sh`), stop the root unit, start the user quadlet |
Before 2026-09-26 the port belonged to the **`frank-medium`** service: [[frank-medium-radiance-mxfp4]]
(vllm-radiance TP=2 MXFP4, 2026-09-20 → 09-26) is that era's page and is **not** superseded here.
Before 2026-09-26 the port belonged to the **`frank-medium`** service, and its era
pages are **not** superseded here: [[frank-medium]] is the locator for that
endpoint's whole shape history, including the unresolved end date of the
vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
## Laws learned here
### The radiance engine
- **KV dtype law.** `--kv-cache-dtype` takes only `bf16|fp8` — ONE dtype for the whole cache,
so llama.cpp's exact-K / quantized-V pair (what [[frank-zealot]] runs as `--cache-type-k`/
a **radiance** restriction, not stock vLLM's: upstream advertises `auto|fp8|fp8_e5m2|fp8_e4m3`
(`docs.vllm.ai` engine-args) and even there the advertised set is not the usable set — several
accepted values crash engine init at the attention layer's own dtype check
([vllm-project/vllm#42587](https://github.com/vllm-project/vllm/issues/42587)).
So llama.cpp's exact-K / quantized-V pair (what [[frank-zealot]] runs as `--cache-type-k`/
`--cache-type-v`) is unservable here: the rung below bf16 is fp8 on both halves. Measured
per-rank cost out of the boot ledger's group table (fp8 boot 2026-10-08, bf16 boot
2026-10-11): `kv_attn` 13 layers / 4-token blocks pages **26.00 → 52.00 KiB**, while
@@ -438,3 +445,4 @@ Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]],
[[Dual-R9700-Cooperation]], [[deployment]], [[swift15-campaign-worklog]],
[[dankserver-frontier-moe-feasibility]], [[x4-pair-experiment]], [[RDNA4 Container Serving]],
[[frank-medium-radiance-mxfp4]], [[LLM/index]].
---
+3 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
description: "The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env, every retired era with its rollback path, the durable laws this endpoint paid for, the wire contract, and the files that own it."
description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it."
tags:
- llm
- frank-dragoon
@@ -9,6 +9,7 @@ tags:
- strix-halo
- gfx1151
- halogen
- endpoint
timestamp: 2026-10-18
---
@@ -234,3 +235,4 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
- Related: [[deployment]] · [[Model Swap and Rollback]] · [[frank-zealot]] · [[frank-archon]] ·
[[dankclaw-iommu-translation-off]] · [[frank-redo-2026-08-17]] · [[Dankclaw System Specs]] ·
[[DFlash DSpark Speculative Decoding]] · [[qwen3.8-spec-matrix]] · [[qwen3.8-strix-optimizations]].
---
@@ -2,7 +2,7 @@
timestamp: 2026-09-17
type: reference
title: frank-medium — Batch and Ubatch Sweep
description: llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the deployed context shape, and the rebalance it forced. Prefill peaks at ubatch 512 and falls on BOTH sides.
description: "The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced."
tags:
- benchmark
- llamacpp
@@ -11,6 +11,7 @@ tags:
- dankserver
- layer-split
---
- retired-endpoint
# frank-medium — Batch and Ubatch Sweep
@@ -222,4 +223,5 @@ re-checking that 35:30 and 37:28 still fail so the gate keeps discriminating.
difference: with no KV to attend over, per-pass overhead dominates and a
bigger micro-batch wins; at 64k of resident KV it does not.
- [[Dual-R9700-Cooperation]] — why the cards are layer-split at all
- [[Dankserver System Specs]] — hardware inventory
- [[Dankserver System Specs]] — hardware inventory---
timestamp: 2026-09-17
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: frank-medium llama.cpp Runbook (DFlash2 drafter)
description: The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj.
description: "Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on."
tags:
- frank-medium
- dankserver
@@ -12,6 +12,7 @@ tags:
- gfx1201
- r9700
- layer-split
- retired-endpoint
timestamp: 2026-09-20
---
@@ -730,3 +731,4 @@ fleet.
- [[frank-medium-batch-ubatch]] — the batch/ubatch sweep that set the deployed
geometry and moved this split from 36:29 to 35:30: the ubatch optimum at 512,
why the logical batch is free, and the harness traps
---
@@ -1,7 +1,7 @@
---
type: runbook
type: howto
title: frank-medium Model Swap
description: RETIRED-era procedure (frank-medium is a retired endpoint; `:7770` on Dankserver is [[frank-archon]] today and the live swap law is [[Model Swap and Rollback]]) — how a checkpoint swap behind frank-medium worked: the streamlining toolchain (hf-manifest.py, hf-fetch.sh, swap-frank-medium.sh), the quantization-aware entrypoint contract, the quadlet edit, the deploy, the capability battery, and the rollback path.
description: "Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path."
tags:
- frank-medium
- dankserver
@@ -9,6 +9,7 @@ tags:
- vllm-radiance
- huggingface
- runbook
- retired-endpoint
timestamp: 2026-09-20
---
@@ -155,9 +156,10 @@ back:
2. `deploy.sh dankserver --service frank-medium --skip-e2e`.
3. Re-run the capability battery.
The 2026-09-20 rollback target is the MXFP4 checkpoint at
`/home/dank/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (19.37 GiB single
safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`.
The 2026-09-20 rollback target is the MXFP4 checkpoint with
`SPEC_METHOD=dflash`, `SPEC=7`; [[frank-medium-radiance-mxfp4]] owns that
payload's identity, and as of the 2026-10-19 host census it is a re-fetch, not a
local file.
## 2026-09-20 swap record (MXFP4 → FP8)
@@ -184,3 +186,4 @@ safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`.
DFlash2 drafter's 2.1 GiB costs the pool vs MTP's 461,540 (1.76×); both
smaller than MXFP4's 943,581 (3.59×) because FP8 weights are bigger (30.9
vs 19.4 GiB).
---
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: frank-medium vllm-radiance MXFP6 Runbook
description: RETIRED era runbook (endpoint renamed frank-archon 2026-10-10; this shape is a row in [[frank-archon]]'s Eras table) — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era.
description: "Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload."
tags:
- frank-medium
- dankserver
@@ -14,6 +14,7 @@ tags:
- rccl
- pcie-atomics
- quadlet
- retired-endpoint
timestamp: 2026-09-21
---
@@ -37,11 +38,14 @@ e4m3fn scale) with `--kv-cache-dtype auto` (unquantized bf16 KV) and
`--gpu-memory-utilization 0.98`. The Swift variant is a finetune of
`ukisai/Swift-Qwen3.8-27b` (a different base from `Qwen/Qwen3.8-27B`), so the
DFlash2-FP8 drafter's hidden-state distribution is a mismatch — acceptance may
drop from the MXFP4 era's 4.24/7, but it is still a net win over no spec
decode. **The MXFP4 checkpoint is the on-disk rollback**: flip `SNAP` back to
`/models/Qwen3.8-27B-Uncensored-MXFP4-awq`, and the image
back to `localhost/radiance-vllm:0.9.3` (re-pull `docker.io/stilldeadcode/
vllm-radiance:0.9.3` if it is no longer on disk; the MXFP4 shape auto-sized to
drop from the 4.24/7 that the DFlash2 arm measured on the FP8 target (the A/B
below ran both arms on FP8, not on MXFP4). **The MXFP4 checkpoint is the
rollback**, and as of the 2026-10-19 host census it is **no longer on disk**:
`/home/dank/models/` holds `StillDeadcode`, `rad` and `tpl.jinja` only, so a
rollback re-fetches `/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (Hub
`just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq`) and flips `SNAP` and the
image back to `localhost/radiance-vllm:0.9.3` (re-pull
`docker.io/stilldeadcode/vllm-radiance:0.9.3` if the local tag is gone; the MXFP4 shape auto-sized to
486,464 tokens with bf16 KV and held the full 262,144 window).
**Deployed shape since 2026-09-20: one rootless Quadlet, `--tensor-parallel-size 2`.**
@@ -528,9 +532,8 @@ soak (three concurrent loops, each re-sending a ~30k-token document every turn
with growing history) shows what happens at that limit: turns alternate between
**~1.1 s** when the prefix is resident and **12-22 s** when it has been evicted
and must be prefilled again, because two families per card plus their cached
prefixes exceed the pool. Nothing failed — ~120 requests, no errors, no failover,
both replicas steady — and per-family pinning kept the fast turns on the replica
that holds the prefix.
prefixes exceed the pool. Nothing failed, and per-family pinning kept the fast
turns on the replica that holds the prefix.
Read that as a capacity budget, not a defect: two concurrent large-context
sessions per card are comfortable, three or more of that size trade turns
@@ -632,3 +635,4 @@ if that person is looking. That is the honest state of "alerting" here.
the same image and kernel 7.0.0-30 failing identically.
- `ggz14/radiance-vllm-mxfp4` — the patch set and the reference benchmark table
this page compares against.
---
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: frank-medium ROCmFPX Runbook
description: charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver (2x AMD Radeon AI PRO R9700, gfx1201).
description: "Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required."
tags:
- frank-medium
- dankserver
@@ -10,6 +10,7 @@ tags:
- mtp
- gfx1201
- r9700
- retired-endpoint
timestamp: 2026-09-15
---
+65
View File
@@ -0,0 +1,65 @@
---
type: reference
title: frank-medium (retired) — the Dankserver endpoint before frank-archon
description: "Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree."
timestamp: 2026-10-19
tags:
- llm
- dankserver
- frank-medium
- retired
- retired-endpoint
---
# frank-medium (retired) — the Dankserver endpoint before frank-archon
`frank-medium` was the LLM service on Dankserver port 7770. It is **not** a live
endpoint: the same port is [[frank-archon]] today, and [[frank-archon]] owns every
shape served from 2026-09-26 onward, including the `frank-medium` → `frank-archon`
name cutover dated 2026-10-10 in `AGENTS.md`. This page owns only the **locator** —
which era page holds which payload's evidence — and carries no measurements: every
number stays in the era page that measured it.
## Eras
Windows are quoted from each era page's own banner; no date here is derived.
| Window | Runtime | Payload | Shape | Page | Rollback |
|---|---|---|---|---|---|
| 2026-09-15 → 2026-09-16 | `charlie12345/ROCmFPX` llama.cpp `main` at `c49ebdb`, self-contained ROCm 7.14 | `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` (custom tensor type 103) + an MTP Q4_0 draft | 2× R9700 `-sm layer` over `ROCm0,ROCm1`, 131,072 × 1 slot, f16 KV + flash attention, draft-MTP pinned to `ROCm1`; the fork is the only loader for that tensor type | [[frank-medium-rocmfpx]] | re-fetch the GGUF, rebuild the fork |
| 2026-09-16 → 2026-09-19 | vllm-radiance on the old TUF GAMING B650-E board | ParoQuant MXFP4 | two single-GPU replicas behind an adaptive-routing proxy | [[frank-medium-radiance-mxfp4]] (sections below its deployed-shape header) | the replica pair is reconstructable from that page's flags |
| 2026-09-16 → 2026-09-20 | llama.cpp `fb27a525`, ROCm `7.14.0a` | its own GGUF build, DFlash drafter | `-sm layer` split across the two cards, drafter pinned to one device | [[frank-medium-llamacpp-dflash2]] | the page is the named rollback target for the llama shape |
| 2026-09-20 → disputed, see Open questions | vllm-radiance on the replacement ProArt board | ParoQuant MXFP4 → FP8 E4M3 128×128 (`orcarouter/…-FP8`, same day, [[frank-medium-model-swap]]) → ParoQuant MXFP6 (`paroquant_mxfp6`) from 09-21 | `--tensor-parallel-size 2` over x8/x8 Gen5 root ports | [[frank-medium-radiance-mxfp4]] (deployed-shape header) | re-fetch the MXFP4 checkpoint (host census 2026-10-19: not local); image `localhost/radiance-vllm:0.9.3` |
`[[frank-medium-batch-ubatch]]` is not an era: it is the batch-versus-ubatch sweep run
on the layer-split pair at its deployed shape, and its numbers belong to that
campaign. `[[frank-medium-model-swap]]` is likewise a campaign ledger — the record of
the 2026-09-20 swap procedure — while the procedure itself is owned by
[[Model Swap and Rollback]].
## Laws
None restated. Each era page carries its own laws and links them to the technique
page that owns the mechanism ([[PCIe Link and Atomics]],
[[Llama Cpp Multi-Device Splits]], [[DFlash DSpark Speculative Decoding]],
[[Llama Cpp Serving Sizing]], [[Dual-R9700-Cooperation]]).
## Open questions
Two date claims in the era pages do not reconcile, and both are left standing rather
than silently corrected:
- `frank-medium-rocmfpx.md` banner says the MXFP4 stack replaced that era and, in the
next sentence, names [[frank-medium-llamacpp-dflash2]] as the active successor. The
MXFP4 replica era and the llama layer-split era both open on 2026-09-16, so which
one directly followed the ROCmFPX fork cannot be settled from the repo.
- `frank-medium-radiance-mxfp4.md` dates its own retired era to 2026-10-10, while
[[frank-archon]]'s Eras table opens on 2026-09-26. The gap is either an unrecorded
fortnight on that endpoint or a stale banner; nothing else in the repo covers
2026-09-26 → 10-10 for port 7770.
## Files
No repo files own this endpoint: its units, env and serve scripts were renamed to
`frank-archon` on 2026-10-10 and are owned by [[frank-archon]]. `scripts/old/` holds
retired serve scripts from these eras as reference only.
+2 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: frank-npu (retired) — FLM NPU endpoint on Dankclaw
description: The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the amd_iommu=off decision, and where every dependent route went.
description: "The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went."
tags:
- flm
- fastflowlm
@@ -10,6 +10,7 @@ tags:
- dankclaw
- frank-npu
- retired
- retired-endpoint
timestamp: 2026-09-11
---
@@ -1,9 +1,10 @@
---
type: reference
description: "This page records the full redo of the inference fleet on 2026-08-17."
description: "The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way."
tags:
- llm
- frank
- retired-endpoint
timestamp: 2026-09-03
---
@@ -223,3 +224,4 @@ repeated prompt prefixes. Measured on a 2023-token prefix (2026-08-17
evening): 4.6s cold -> 1.1s cached (~4x TTFT), output unchanged. This is
the main TTFT lever for pi, which re-sends the same system prompt + tool
schema every turn. Toggled via `ENABLE_PREFIX_CACHING` in the env file.
---
+80 -54
View File
@@ -1,7 +1,7 @@
---
type: reference
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 2 × 262144, exact f16 KV"
description: "The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env ships it, every era with its rollback path, the deduplicated laws of this card, the wire contract, and the owning files."
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 3 × 200704, f16 K + q8_0 V, MTP armed"
description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws."
tags:
- llm
- frank-zealot
@@ -9,13 +9,15 @@ tags:
- llama-cpp
- gguf
- rtx3090
- endpoint
timestamp: 2026-10-18
---
# frank-zealot — llama.cpp (PrismML) Bonsai-2-9B PQ2_0-MTP, 2 × 262,144, exact f16 KV
# frank-zealot — llama.cpp (PrismML) Bonsai-2-9B PQ2_0-MTP, 3 × 200,704, f16 K + q8_0 V, MTP armed
Dankclaw's RTX 3090 endpoint — `http://192.168.0.143:7771`, wire id `frank-zealot`, the fleet's **compaction hub**
(widest window it has). One page per endpoint: eras are rows below, not pages. Siblings: [[frank-archon]],
(three conversations at 200,704 each, the window the other two endpoints also cap at). One page per
endpoint: eras are rows below, not pages. Siblings: [[frank-archon]],
[[frank-dragoon]]. Deploy with `./scripts/deploy.sh dankclaw --service frank-zealot`; swap mechanics in
[[Model Swap and Rollback]], sizing math in [[Llama Cpp Serving Sizing]]. `scripts/frank-zealot.env` and
`scripts/serve-frank-zealot.sh` are the authority for the shape; where any prose disagrees with them, the prose
@@ -31,74 +33,89 @@ is the bug.
| Runtime | `/home/dank/GitPackages/llama-prism-b10777/llama-server` — PrismML **fork**, `0.2.0-dev (build 10777, commit e3de71522)`; the only loader for this payload |
| Trio | `scripts/frank-zealot.service` (root, relinked, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`, `TimeoutStopSec=180`) · `scripts/frank-zealot.env` · `scripts/serve-frank-zealot.sh` |
| Address / alias | `BIND_ADDR=0.0.0.0`, `LISTEN_PORT=7771`, `ALIAS=frank-zealot` (a flag, not a fact) |
| Flags | `--ctx-size 524288 --parallel 2` ⇒ 262,144 per slot · `--cache-type-k f16 --cache-type-v f16` (exact, **16,384 MiB**) · `--flash-attn on` · `--batch-size 4096 --ubatch-size 2048` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · `--log-verbosity 4` · `--presence-penalty 1.0` · **`SPEC_TYPE=none`** |
| Flags | `--ctx-size 602112 --parallel 3` ⇒ 200,704 per slot · `--cache-type-k f16 --cache-type-v q8_0` (asymmetric, **14,406.00 MiB**) · `--flash-attn on` · `--batch-size 6144 --ubatch-size 1024` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · `--log-verbosity 4` · `--presence-penalty 1.0` · **`--spec-type draft-mtp`** with `--spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0` |
**Why this shape.** `general.architecture qwen35`: 33 blocks (32 trunk + `blk.32` = the MTP block),
`full_attention_interval 4` ⇒ 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, `nextn_predict_layers 1`
(the head **is in the file**), trained window 262,144, `general.sampling.*` 1.0/0.95/20. Only the 8 full layers bill
per token (32,768 B at exact f16 ⇒ 16,384 MiB over the pool) and the DeltaNet layers add ~50 MiB of recurrent state
**per slot**: exact KV costs the third conversation and nothing else.
per token and the DeltaNet layers add ~50 MiB of recurrent state **per slot**, so the shape is a single budget
equation at 25,088 B/token (f16 K 16,384 + q8_0 V 8,704): **14,406.00 MiB** of cache over a 602,112 pool +
**2,352.00 MiB** of draft cache (blk.32 at its own f16 rate, billed over the WHOLE pool) + ~7.7 GiB of weights,
projector, buffers and states, inside the 24,123 MiB the driver leaves addressable. Exact V (32,768 B/token ⇒
18,816 MiB here) buys neither the third conversation nor the drafter; the fully quantized pair (17,408 B/token)
buys both and costs the keys their precision.
**MTP is OFF by measurement, not by absence of a head.** The drafter is *good* — 2026-10-18 measured
acceptance 0.798 / 0.617 per position, mean accepted length 2.41, and the publisher's card says depth 2 works
best — and it still loses here, because the draft context is built against the *target's* pool: `blk.32` bills
4,096 B/position at its default f16 ⇒ 2,048 MiB at 524,288 positions on a trunk that projects 21,234 MiB of the
24,123 MiB addressable (that projection predicts 2,889 MiB free; replace it with the measured pair). Exact KV and a
drafter cannot share this card. `--spec-draft-type-k`/`--spec-draft-type-v` (default f16, verified in this build's
`--help`) would cut the draft to 1,088 MiB — legal on b10777, **never booted beside exact KV**. There is no
`--spec-draft-model`: the MTP draft rides the main model's weights. The env carries the recipe
(`SPEC_DRAFT_N_MAX=2`, `SPEC_DRAFT_N_MIN=0`, `SPEC_DRAFT_P_MIN=0`); `SPEC_TYPE=none` never sends it.
**MTP is ON (Frank, 2026-10-19).** The drafter was always good — 2026-10-18 measured acceptance 0.798 / 0.617 per
position, mean accepted length 2.41, and the publisher's card says depth 2 works best — and arming it is a
budget decision, not a flag: the draft context is built against the *target's* pool, so `blk.32` bills 4,096
B/position (2,352.00 MiB at 602,112) plus 480.97 MiB of its own CUDA0 buffers, and it triples the recurrent
state (452.25 MiB for three cells against 150.75 with the drafter off). `CACHE_TYPE_V=q8_0` is what pays that:
7,680 B/token cheaper than exact V = **4,410.00 MiB** at this pool. `UBATCH_SIZE=1024` — the rung every measured
MTP arm here ran on — contributes only ~406 MiB at THIS shape, not the ~1,306 the 786,432-pool census showed,
and `UBATCH_SIZE=2048` beside an armed drafter is still the shape that halved decode (167.2 → 109.2 t/s).
`--spec-draft-type-k`/`--spec-draft-type-v` (default f16, verified in this build's `--help`) would cut the draft's bill again and stay at their default; there is no
`--spec-draft-model` — the draft rides the main model's weights, so a draft file would load the trunk twice.
`SPEC_TYPE=none` is the rollback and strips the whole set.
**Harness contract** (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw-3090)`, id `frank-zealot`):
`contextWindow: 262144` — **the per-slot window**, because llama.cpp divides `--ctx-size` across slots and answers a
`contextWindow: 200704` — **the per-slot window**, because llama.cpp divides `--ctx-size` across slots and answers a
longer prompt with a 400, never a truncation — plus `maxTokens: 16384`, `input: [text, image]`,
`compat.reasoningContentField: reasoning_content`, `compat.reasoningEffortMap` (**load-bearing**: the template takes
only `xhigh|medium|low` and raises on anything else), `extraBody` 1.0 / 0.95 / 20 / `min_p 0`, and the
`compactionModel` of [[frank-archon]], [[frank-dragoon]] **and** frank-zealot all pointing here.
**The exact-KV boot, measured** (2026-10-18 evening, under the unit, verbosity 4). KV **16,384.00 MiB** =
32,768 B/token × 524,288 positions, to the MiB — rate derivation and ledger agree with no slop, which is what
buries the doc's old "f16 needs 26,112 MiB and cannot boot" figure for good: that was the 27B's 34,816 B/token
applied to the 9B. RS **100.50 MiB** (2 cells, one 50.25 MiB cell lighter than the q8_0 era), compute
1,408.06 + 1,088.07 CUDA_Host, mmproj worst case **868.32 MiB** charged *after* the engine's own fit check,
settled `nvidia-smi` **21,176 used / 2,949 free** of 24,123, `n_slots = 2`, `n_ctx_slot = 262144`,
`DropInPaths` empty, no clamp line, no `cudaMalloc` error — within 24 MiB of the pre-flight projection. That is
a desktop margin: the ~2 GiB of slack is what an image storm has to live inside.
**No rates, by law.** Frank ordered the reslice, not a number: the evidence a swap needs is
the seam's fit gate, the deployer's health gate, and the boot ledger below. The previous era's
exact-KV boot and its rate floors live in that era's row; those floors were measured while
another consumer drove the other slot (56 prompts in three minutes, 96 % GPU-util), the exact
condition a bench window exists to exclude.
**Rates on this shape are FLOORS, not the shape's rates** — measured the same evening greedy with `ignore_eos`
at 512 tokens, first request discarded, while another consumer drove the other slot (56 new prompts in three
minutes, 96 % GPU-util): the exact condition the bench window exists to exclude, and one contended deep arm
even came back truncated (a 512 KB body accepted as `prompt_n = 4`).
**The 2026-10-19 boot, measured** — verbosity-4 ledger of `ba69ea5`, started 03:21:56 PDT.
Recorded here because projections drift, and nowhere else:
| Arm | q8_0 3-slot record | f16 2-slot, contended |
|---|---|---|
| single-stream decode | 168.7 | **134.1 – 141.5** (145.8 and 163.2 in a lighter minute) |
| concurrent aggregate | ≈300 across 3 (102.8 each) | **256.6 / 262.4 / 265.1** across 2 (≈128–133 each) |
| prefill at ~106k | 2,877 at 106,418 | **3,159.5 at 106,282** |
| decode at 106k depth | never measured | **71.5** |
| Term | Ledger |
|---|---|
| Target KV over the 602,112 pool | **14,406.00 MiB** — K (f16) 9,408.00 + V (q8_0) 4,998.00, `200704 cells, 8 layers, 3/3 seqs` |
| Draft KV (`blk.32`, its own f16 context) | **2,352.00 MiB** — 1,176.00 + 1,176.00, `200704 cells, 1 layers, 3/3 seqs`: the whole pool, again |
| Recurrent state | **452.25 MiB** — `3 cells, 32 layers, 3 seqs, 2 rs_seq`, R (f32) 20.25 + S (f32) 432.00 |
| Compute buffers | target 1,665.00 CUDA0 + 424.88 CUDA_Host; draft context 480.97 + 424.86 |
| Weights | 2,283.29 CUDA0 + 257.66 CPU-mapped |
| Projector | 595.29 model + 248.10 compute + 24.93 CPU (its 868.32 worst case is still charged AFTER the engine's fit check) |
| Engine's own verdict | projected 21,639 vs 23,404 free, then **"cannot meet free memory target of 1,867 MiB, need to reduce device memory by 102 MiB"** |
| Settled | **22,870 used / 1,255 free**, `n_slots = 3`, `n_ctx_slot = 200704`, `kv_unified = false` |
What the floors already support: per-conversation concurrency **improved** (two streams hold ~130 t/s each where
three held 102.8), endpoint aggregate sits ~12 % under the 3-slot record, prefill beat the record while
contended, and single-stream decode is between parity and −20 %. The 1.882× KV byte bill is real; its true size
on this card is still unmeasured, so quoting 134 or 163 t/s as "the" number is a guess in either direction —
re-run the same arms with the box quiet before a figure is carried anywhere else.
This is the first boot on this card that does **not** meet llama.cpp's own free-memory target,
and 1,255 MiB settled sits under the ~2 GiB band every previous era held. It boots, `/health`
answers, and the seam's gate still passes — it priced 22,428 MiB, conservative by 442 against
this ledger. The cheapest rungs back into the band are `CTX_SIZE=401408` (two conversations) or
`SPEC_TYPE=none`; both are one line and neither changes the payload.
**The drafter is engaged, measured on the same boot** (three ordinary requests, not a bench):
acceptance 0.706 and 0.763 accepted-per-generated, mean accepted length 2.41 / 2.53,
per-position rates (0.882, 0.765) and (0.871, 0.699) — depth 2 earning its keep at 0.3–1.5 MiB
of draft state per saved prompt. The three probes that prove a swap: chat → `"zealot alive"`
with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY content with
`finish_reason: length` — the thinking-budget trap, not a fault); tools →
`finish_reason: tool_calls` with `{"a":24,"b":7}`; vision → a 512 px solid-red PNG answers
"Red" at `prompt_tokens` 280.
**Fallback rungs**, cheapest first — one line of `frank-zealot.env` each, and each must keep the seam's fit gate green:
| Knob | Effect |
| Knob | Effect from the shipped shape |
|---|---|
| `UBATCH_SIZE=1024` | hands back 1,306 MiB of compute buffers; prefill −17 %; it is also the arm that ran the drafter at three windows (213.5 t/s) |
| `CACHE_TYPE_V=q8_0` | exact K + quantized V: KV 16,384 → **12,544 MiB**, freeing 3,840 (llama.cpp has no true fp8 KV dtype — q8_0 is the 8-bit class) |
| `CTX_SIZE=401408` | two 200,704 conversations, exact KV at 12,544 MiB (same 3,840 freed) — `contextWindow` MUST follow the per-slot number |
| `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` | the era below this one: 13,056 MiB of cache, the third conversation back |
| `CACHE_TYPE_K/V=q4_0` | 10,240 B/token ⇒ 5,120 MiB (the 27B era measured q4_0 at 16,392 used / 7,733 free at 2 × 200,704) |
| `SPEC_TYPE=none` | the drafter off: hands back its 2,352.00 MiB of cache, 480.97 MiB of CUDA0 buffers and the 301.50 MiB of rollback planes it adds to the recurrent state (~3.1 GiB), at the cost of the ~+28 % decode the 2026-10-18 census measured whenever the card has room |
| `UBATCH_SIZE=2048` | +4.6 % prefill and only ~406 MiB of compute buffers at THIS shape (the ~1,306 MiB the 786,432-pool census showed does not transfer) — and the rung that HALVED decode beside an armed drafter (167.2 → 109.2 t/s) |
| `CTX_SIZE=401408` | two 200,704 conversations: frees 4,802 MiB of cache — `contextWindow` MUST follow the per-slot number |
| `CACHE_TYPE_V=f16` | exact V: +4,410 MiB at this pool with the drafter armed, which is a `cudaMalloc` death the seam refuses — legal only once the drafter or a slot is gone |
| `CACHE_TYPE_K=q8_0` | the fully quantized pair (17,408 B/token): frees 4,410 MiB and moves the quantization error into the keys |
## Eras
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---|
| 2026-10-18 21:06 → 2026-10-19 | the same 9B MTP file and projector | PrismML fork 10777 | 2 × 262,144 (pool 524,288), exact f16 K+V = 16,384.00 MiB, `SPEC_TYPE=none`, batch 4096 / ubatch 2048; boot ledger KV 16,384.00 / RS 100.50 / compute 1,408.06 + 1,088.07 CUDA_Host / mmproj worst case 868.32 charged after the fit check, settled **21,176 used / 2,949 free**, `n_ctx_slot = 262144`, within 24 MiB of the projection; floors 134.1–141.5 t/s single-stream, 256.6–265.1 aggregate over two, 3,159.5 t/s prefill at 106,282, 71.5 t/s decode at 106k depth (contended — floors, not the shape's rates) | Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352 | one line each — `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` — and redeploy; no payload move |
| 2026-10-18 13:19 → same evening (the boot `68afc6d` replaced) | the same 9B MTP file and projector | PrismML fork 10777 | 3 × 262,144 (pool 786,432), q8_0 K+V = 13,056 MiB, settled 17,956 / 6,169 MiB; 168.7 t/s decode (three runs 168.4–168.7), ~300 t/s over three streams (102.8 each), 2,877 t/s prefill at 106,418 | Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable | one env line — `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` — and redeploy; no payload move |
| 2026-10-18, 24 min (arm under the unit) | `Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf` 5,946,648,928 B + hardlinked Q8 mmproj | PrismML fork 10777 | 2 × 200,704, batch/ubatch 3072/1536 | Frank scratched the 27B plan — ~48 t/s decode vs 76.6 for PQ2_0 on the same weights | `/home/dank/models/Bonsai2-PTQ1_0/` + `scripts/download-frank-zealot-bonsai2-ptq1.sh` |
| 2026-10-18 03:14 → 13:19 (arm under the unit, then the leftover drop-in) | `Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf` 5,946,648,928 B + hardlinked Q8 mmproj | PrismML fork 10777 | arm: 2 × 200,704, batch/ubatch 3072/1536. The leftover `frank-zealot.service.d/ptq1.conf` then made the 13:16 cutover deploy boot THIS payload at 3 × 133,888 (`new slot, n_ctx = 133888` x3, journal read 2026-10-19) until the 13:19 boot | Frank scratched the 27B plan — ~48 t/s decode vs 76.6 for PQ2_0 on the same weights | `/home/dank/models/Bonsai2-PTQ1_0/` + `scripts/download-frank-zealot-bonsai2-ptq1.sh` |
| 2026-10-17 → 10-18 | Ternary-Bonsai-2-**27B** PQ2_0 7,206,168,928 B `3907dc16…` + Q8_0 mmproj 629,246,976 B | PrismML fork **10754** (`~/GitPackages/llama-prism/`) | 2 × 200,704 (pool 401,408), q8_0 K+V = 13,328 MiB, settled 22,664 / 1,461; 76.6 t/s decode, 109.4 aggregate over two streams, 691.7 t/s prefill at 166,027; MTP impossible — no head in the file | the 9B bills half the KV per token (8 full layers, not 16), which funded three slots at the native window and beat it on every rate | `/home/dank/models/Bonsai2-PQ2_0/` + the 10754 fork tree, `scripts/download-frank-zealot-bonsai2.sh`, revert the swap commit |
| 2026-10-07 → 10-17 | Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K 7,359,259,008 B `c0ba7beb…` + BF16 mmproj 921,704,448 B | llama.cpp master `5fc4f3c` CUDA sm_86 (`~/GitPackages/llama-master/build-cuda/`) | 2 × 200,704, exact f16 K+V, MTP off (no `nextn` tensors) | the ternary 27B bought a larger model class at the same window | `/home/dank/models/Qwen35-9B-HauhauCS-Q6K/` + the master build, `scripts/download-frank-zealot.sh` |
| 2026-10-07 (same day) | Gemma4-26B-A4B-QAT Q4_K_M 16,796,015,520 B + BF16 mmproj 1,194,827,776 B + unused `gemma4-assistant` MTP 251,937,728 B | llama.cpp master CUDA | 1 × 200,704, f16 K + q8_0 V, ubatch 1024, MTP off; 21,868 / 2,257 settled, 139.9 t/s decode (the ~4 B-active MoE is at home here); MTP ON would cost ~1.9 GiB and boot at 319 MiB free | ONE slot serialized compaction; the 9B's lighter weights bought two slots and exact KV | `/home/dank/models/Gemma4-26B-A4B/` (ungated — plain `curl -fL`) |
@@ -129,13 +146,19 @@ re-run the same arms with the box quiet before a figure is carried anywhere else
3 cells, and an armed depth-2 drafter triples it to 452.25), compute buffers at graph capture, the mmproj after the
fit. On gemma4-class hybrids `attention_k_eq_v` does **not** shrink the allocation — the cache reserves the K/V pair
twice, 6.125 B/elem at f16 K + q8_0 V.
- **`--flash-attn on` is pinned, not decorative**: a quantized V exists only on the FA path and the engine falls back silently without it. llama.cpp has **no true fp8 KV dtype** — q8_0 is the 8-bit class (1.0625 B/elem, ~8.5 effective bits). `GGML_CUDA_FA_ALL_QUANTS` is vestigial and build-specific: the PrismML fork carries it ON (since 2026-09-17) while the master build had it OFF and still dispatched Q4/Q5 KV unconditionally (`ggml-cuda/fattn.cu` has no guard), so quantized KV never required it.
- **`--flash-attn on` is pinned, not decorative**: a quantized V exists only on the FA path and the engine falls back silently without it. llama.cpp has **no true fp8 KV dtype** — q8_0 is the 8-bit class (1.0625 B/elem, ~8.5 effective bits). `GGML_CUDA_FA_ALL_QUANTS` is **deprecated upstream and inert in both trees on this host** (no reference anywhere in `ggml/src/ggml-cuda/`, read 2026-10-19): master's `ggml/cmake/common.cmake` warns `use GGML_CUDA_FA_QUANTS=all instead`, and the successor's default — `GGML_CUDA_FA_QUANTS="q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16"` (`ggml/CMakeLists.txt:208`) — already compiles the quantized-KV kernels with the old flag OFF (master's `build-cuda` cache says `FA_ALL_QUANTS:BOOL=OFF` and it dispatched q8_0/q4_0 KV regardless). Prove a build's quantized-KV ability from its `CMakeCache.txt` `GGML_CUDA_FA_QUANTS` line, never from the retired flag or the build command you remember.
- **Prices come from boots, not formulas, and an arm belongs to the payload it was measured on**: 3 × 180,224 +
q8_0 V fits with Q5_K_M (21,684 used / 2,441 free) and does **not** fit with Q6_K inside the 2–3 GiB band. The
mmproj worst case (~870 MiB) is printed before fitting but never billed by the engine's own fit check. Only
q8_0 V fits with Q5_K_M (21,684 used / 2,441 free) and does **not** fit with Q6_K inside the 2–3 GiB band.
The mmproj worst case is printed before fitting (`estimated worst-case memory usage of mmproj is
873.10 MiB` on the Bonsai-2 27B boot) but never billed by the engine's own fit check: that boot
projected **21,439 MiB** and settled at **22,664**. Only
`tests/frank_zealot_unit.bats`'s fit gate can see a boot that would die at `cudaMalloc`, so every retune keeps it
green — and the first verbosity-4 ledger plus settled `nvidia-smi` of a new dtype belongs on this page, because
projections drift.
The same trap one knob down: the ubatch 2048 → 1024 step handed back **~1,306 MiB** at the 786,432 / q8_0-pair
arm and **406 MiB** at the 602,112 / f16-K + q8_0-V boot (target compute 2,496.13 → 2,089.88, with CUDA0 going
UP 257 while CUDA_Host went down 663). Changing the dtype or the pool moves the compute buffers too, so a
credit measured at one shape is not a budget line at the next.
The two boots that priced non-KV: 2 × 262,144 **without** the projector — 20,190 used, KV 12,800 ⇒ non-KV
**7,390 MiB**; 3 × 196,608 **with** it — 22,972 used, KV 14,113 ⇒ non-KV **8,859 MiB**. The difference prices the
vision tower at **~1.4 GiB for a 918 MB file**, because the encoder's activation and compute buffers ride with it.
@@ -160,7 +183,7 @@ re-run the same arms with the box quiet before a figure is carried anywhere else
## Contract
- `/health` → `{"status":"ok"}`; `/props` reports `n_ctx` and `presence_penalty`; `/slots` shows occupancy; a third
- `/health` → `{"status":"ok"}`; `/props` reports `n_ctx` and `presence_penalty`; `/slots` shows occupancy; a fourth
concurrent conversation **waits** for a slot — the slot count is the concurrency contract.
- `/metrics` answers 200 with 15 `llamacpp:*` families **only with `--metrics`**; without it llama.cpp answers **501**, which Prometheus reads as DOWN while `/health` stays 200. The scrape target is `host.containers.internal:7771/metrics` and the oldest retained sample of `up{job="frank-zealot"}` (2026-10-11) is already 0 — this target was never successfully scraped, across every payload era. `METRICS=0` is the documented opt-out and re-breaks it; the counters cost nothing on the request path.
- Reasoning arrives in **`reasoning_content`** (`--reasoning-format deepseek`) and **counts against `max_tokens`**: a short budget yields empty `content` with `finish_reason: "length"`, which reads as a broken model, not a mid-thought one. The slot prefix cache is live (`cached_tokens` in `usage`).
@@ -168,9 +191,9 @@ re-run the same arms with the box quiet before a figure is carried anywhere else
(1.0 / 0.95 / 20). `presence_penalty` is the one recipe value with no GGUF metadata key, so it must reach argv;
the serve script uses `${PRESENCE_PENALTY=1.0}`, deliberately **not** `:=`, because `""` is the documented opt-out.
- Vision: only the OpenAI **`image_url`** content part reaches the model — the legacy top-level `images` field bumps `usage.prompt_tokens` while the model invents an answer. Probe with a **≥ 256 px** solid color (a postage stamp answers "video") and budget **≥ 512** tokens; a 512 px image bills ~280 prompt tokens.
- **Q8_0 mmproj, not BF16**: the publisher warns the BF16 tower "can crash llama-server when several image requests run at once", which is exactly the `--parallel 2` shape. `MMPROJ_FILE=` (explicitly empty) is the text-only opt-out and returns ~1.4 GiB; a path at a missing file is a boot **refusal**, never silent blindness.
- **Q8_0 mmproj, not BF16**: the publisher warns the BF16 tower "can crash llama-server when several image requests run at once", which is exactly the `--parallel 3` shape. `MMPROJ_FILE=` (explicitly empty) is the text-only opt-out and returns ~1.4 GiB; a path at a missing file is a boot **refusal**, never silent blindness.
- Tools: native via `--jinja`; `finish_reason: tool_calls` with valid JSON arguments.
- Two non-faults: a `400` naming 262,144 is one conversation outgrowing its **slot**, not pool exhaustion; and
- Two non-faults: a `400` naming 200,704 is one conversation outgrowing its **slot**, not pool exhaustion; and
the raw `/completion` path is a **dead instrument on this fork build** — a greedy `ignore_eos` run answers
**500 "The model produced output that does not match the expected Content-only format"**, because build 10777
guards the raw path the moment reasoning markup leaks (older builds only logged `failed to parse reasoning
@@ -179,7 +202,10 @@ re-run the same arms with the box quiet before a figure is carried anywhere else
`predicted_per_second`). Keep `VERBOSITY=4` — 3 is the engine default and cannot prove the KV or draft sizes,
while levels above 4 flood the journal.
- llama.cpp does **not** validate the requested model id: it answers and echoes the served `ALIAS`, so a rename breaks id-keyed consumers (the OpenWebUI `model` row, its `access_grant`, `openai.api_configs`) and nothing server-side. Alias history: `frank-small` → `frank-secondary` (2026-10-06) → `frank-zealot` (`7c77326`, 2026-10-09); the service and unit took the name 2026-10-10 (`55f87fc`).
- The first request after a boot is a warmup artifact (37 t/s class) and is never a measurement; never time a wall-clock aggregate on a generation that stops on its own (use `ignore_eos`).
- The first request after a boot is a warmup artifact, never a measurement: on the Bonsai-2 27B
boot request #1 gave 2.6 t/s prefill / 37.7 t/s decode and request #2 gave 1,258 t/s prefill /
54.7 t/s per stream (two concurrent, the 109.4 aggregate in the Eras row) — a 3x spread on one
shape. Never time a wall-clock aggregate on a generation that stops on its own (use `ignore_eos`).
## Files
@@ -1,7 +1,7 @@
---
type: research
type: reference
title: FreeToken Benchmark Sweep — Dankserver RTX 3090
description: KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2.
description: "KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime."
tags:
- freetoken
- benchmark
@@ -1,7 +1,7 @@
---
type: reference
title: ik_llama.cpp on Dankserver (Ornith Q8_0)
description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V q6_0 with Hadamard on V only (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 163840 ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
description: "Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required."
tags:
- ikllama
- llamacpp
@@ -11,7 +11,8 @@ tags:
- ornith
- kv-cache
- hadamard
timestamp: 2026-08-26
- retired-endpoint
timestamp: 2026-10-19
---
# ik_llama.cpp on Dankserver (Ornith-1.5-35B-A3B CRACK)
@@ -259,4 +260,4 @@ spec / `--spec-type` flags. The model file is not changed either way.
build flags incl. `GGML_CUDA_FA_ALL_QUANTS` and `GGML_IQK_FA_ALL_QUANTS`
- [ik_llama.cpp serverAppConfigs-docs/parameters.md](https://github.com/ikawrakow/ik_llama.cpp/blob/main/docs/parameters.md) —
q8_KV (PR 208), K/V Hadamard options, `-cuda graphs=0` workaround, and
`--split-mode graph` MoE note (PR 1080)
`--split-mode graph` MoE note (PR 1080)
+54 -53
View File
@@ -9,37 +9,33 @@ each of those owns exactly that one thing. The incidents behind the laws are in
| Page | Owns |
|---|---|
| [[frank-archon]] | The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras… |
| [[frank-dragoon]] | The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env,… |
| [[frank-zealot]] | The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env… |
| [[frank-archon]] | The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught. |
| [[frank-dragoon]] | The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it. |
| [[frank-zealot]] | The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws. |
## Retired endpoints
| Page | Owns |
|---|---|
| [[frank-medium-batch-ubatch]] | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… |
| [[frank-medium-llamacpp-dflash2]] | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… |
| [[frank-medium-model-swap]] | RETIRED-era swap procedure for the frank-medium endpoint (`:7770` is now [[frank-archon]]) — the… |
| [[frank-medium-radiance-mxfp4]] | RETIRED frank-medium era runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
| [[frank-medium-rocmfpx]] | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… |
| [[frank-npu]] | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… |
| [[frank-redo-2026-08-17]] | This page records the full redo of the inference fleet on 2026-08-17. |
| [[ikllama-ornith-dankserver]] | ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V… |
| [[kingjones-q8agent-rocmfpx]] | **Status (historical, 2026-09-13):** was to deploy on Dankclaw. |
| [[qwen3.8-spec-matrix]] | This page records the speculative-decoding benchmark history and the |
| [[frank-medium-batch-ubatch]] | The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced. |
| [[frank-medium-llamacpp-dflash2]] | Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on. |
| [[frank-medium-model-swap]] | Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path. |
| [[frank-medium-radiance-mxfp4]] | Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload. |
| [[frank-medium-rocmfpx]] | Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required. |
| [[frank-npu]] | The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went. |
| [[frank-redo-2026-08-17]] | The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way. |
| [[ikllama-ornith-dankserver]] | Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required. |
| [[kingjones-q8agent-rocmfpx]] | Historical: a checkpoint and loader pairing staged for this host and never deployed on it. |
| [[qwen3.8-spec-matrix]] | The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out. |
## Pipeline Parallelism (series)
| Page | Owns |
|---|---|
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node… |
| [[Expected Performance]] | Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors. |
| [[llama.cpp RPC]] | llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and… |
| [[Mixed-Vendor Experiments]] | Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks… |
| [[Qwen3.6 27B Spec Methods]] | Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC… |
| [[Qwen3.8 PP Deployment]] | Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX |
| [[Sample Script Set]] | An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or… |
| [[vLLM Pipeline Parallelism]] | vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP +… |
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it. |
| [[Expected Performance]] | Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on. |
| [[Qwen3.8 PP Deployment]] | The real split deployment: which node is head and which is worker, the flags, and the verified gotchas. |
| [[Sample Script Set]] | An educational two-node RPC split script set, for reading only — never run it or deploy it. |
## Techniques, hardware, trials
@@ -47,43 +43,48 @@ each of those owns exactly that one thing. The incidents behind the laws are in
|---|---|
| [[Dankclaw Improvements]] | This page collects concrete, sourced recommendations to improve Dankclaw |
| [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. |
| [[dankclaw-iommu-translation-off]] | Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a… |
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI… |
| [[dankserver-frontier-moe-feasibility]] | VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and… |
| [[dankclaw-iommu-translation-off]] | Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line. |
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture. |
| [[dankserver-frontier-moe-feasibility]] | The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded. |
| [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. |
| [[dealign-ornith-crack]] | dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with… |
| [[dealign-ornith-crack]] | Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card. |
| [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). |
| [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. |
| [[Dual-R9700-Cooperation]] | Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic… |
| [[FastFlowLM NPU Inference]] | FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on… |
| [[FreeToken Builds]] | Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the… |
| [[freetoken-benchmark-sweep]] | KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2. |
| [[GGUF Compatibility]] | How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the… |
| [[Heterogeneous Split Triage]] | Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X +… |
| [[Dual-R9700-Cooperation]] | Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked. |
| [[FastFlowLM NPU Inference]] | The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook. |
| [[FreeToken Builds]] | Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit. |
| [[freetoken-benchmark-sweep]] | KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime. |
| [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. |
| [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. |
| [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. |
| [[ling-3.0-flash-trial]] | `bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`. |
| [[ling-3.0-flash-vllm-plan]] | This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62… |
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by… |
| [[Llama Cpp Serving Sizing]] | Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA… |
| [[LLM Bench Windows and Measurement]] | Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm… |
| [[LLM Performance Diagnosis]] | Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time… |
| [[Model Swap and Rollback]] | The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag… |
| [[PCIe Link and Atomics]] | Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic,… |
| [[ling-3.0-flash-trial]] | Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict. |
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing. |
| [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. |
| [[LLM Bench Windows and Measurement]] | Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting. |
| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. |
| [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. |
| [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. |
| [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
| [[Quantization Production]] | — |
| [[qwen3.8-strix-optimizations]] | This page answers one question: **how do we make Qwen3.8-27B serve faster on |
| [[Quantization Production]] | Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file. |
| [[qwen3.8-strix-optimizations]] | How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought. |
| [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. |
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class… |
| [[RDNA4 MoE Expert Offload VRAM Budget]] | The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the… |
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. |
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series. |
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage. |
| [[RDNA4 MoE Expert Offload VRAM Budget]] | Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder. |
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate. |
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape. |
| [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
| [[swift15-campaign-worklog]] | — |
| [[Tcclaviger Fork Ops]] | Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken… |
| [[ubatch-benchmark-large]] | Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. |
| [[vLLM Boot Forensics and Config Interrogation]] | Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone),… |
| [[VLLM Index]] | This index contains all documentation for vLLM deployment on Dankserver and Dankclaw. |
| [[swift15-campaign-worklog]] | Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm. |
| [[Tcclaviger Fork Ops]] | Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence. |
| [[ubatch-benchmark-large]] | The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness. |
| [[vLLM Boot Forensics and Config Interrogation]] | Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config. |
| [[VLLM Index]] | Locator for every vLLM page in this vault. |
| [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. |
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that… |
| [[vllm_research_findings]] | vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method… |
| [[x4-pair-experiment]] | What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the… |
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage. |
| [[vllm_research_findings]] | Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method. |
| [[x4-pair-experiment]] | Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement. |
## Pages
| Page | Owns |
|---|---|
| [[frank-medium]] | Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree. |
@@ -1,10 +1,11 @@
---
type: reference
description: "**Status (historical, 2026-09-13):** was to deploy on Dankclaw."
description: "Historical: a checkpoint and loader pairing staged for this host and never deployed on it."
tags:
- llm
- rocm
- frank
- retired-endpoint
timestamp: 2026-09-15
---
@@ -239,3 +240,4 @@ reads `--host/--port/--temp` as CLI flags, never as env vars.
`gl=45`. See the `set_rows` build bug section above.
- Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
retired the same day for the AGENT model). See [[frank-dragoon]].
---

Some files were not shown because too many files have changed in this diff Show More