Compare commits

..
Author SHA1 Message Date
Frank 065d796d73 docs(frank-tts): renderD128 is the NVIDIA node — pass /dev/dri, not a guessed number
The trial's quadlet line said AddDevice=/dev/dri/renderD128. Resolved the nodes by
driver instead of assuming: on this hybrid host renderD128 -> nvidia and
renderD129 -> amdgpu, so the guess pointed the engine at the wrong card. Also
records the gate-1 command that works (vulkaninfo --text; --batch prints help and
exits 1) and the probe script path.
2026-10-19 14:12:05 -07:00
Frank 1e6b39c382 docs(frank-tts): gate 1 passes — RADV gives a container tenant the aperture, not the carve-out
vulkaninfo inside the engine's own base image (ubuntu:24.04 + mesa-vulkan-drivers
vulkan-tools, --device /dev/dri, rootless) reports Radeon 8060S Graphics (RADV
GFX1151) / driver radv / API 1.4.318 with two heaps: heap 0 host-visible 38.67 GiB
(budget 32.58) and heap 1 DEVICE_LOCAL 77.33 GiB (budget 65.16) with frank-dragoon
loaded. ggml's Vulkan backend asks for DEVICE_LOCAL, so the ~4 GiB the TTS engine
needs lands in the 65 GiB budget: memory is not the constraint, and the 2 GiB DRM
carve-out never enters the picture. Both heaps are GTT — the same host RAM, so the
budget is a ceiling on mappings, not free silicon memory.

Traps recorded: vulkaninfo --batch is not a flag in Ubuntu 24.04's vulkan-tools
(prints help, exits 1; use --text), and the container has no NVIDIA ICD, so Vulkan
device 0 happens to be the iGPU — select by name, not index.

Probe: /home/dank/spikes/npu-tts/vk_probe.sh on Dankclaw. Enumeration only; no
allocation, no compute, production untouched.
2026-10-19 14:10:12 -07:00
Frank 1e52e45fae docs(frank-tts): record the NPU spike verdict and the iGPU co-tenancy plan
The overnight spike settled whether frank-tts can run on the Strix Halo NPU.
It cannot, and the wall is the loader, not the silicon: AMD's own Linux
runtime opens /dev/accel/accel0, succeeds at the GEM ioctls, then declines
the NuTTS-Air graph without emitting a single log line — and the one run that
did load has never reproduced. The accelerator that does exist is the gfx1151
iGPU behind -DGGML_VULKAN=ON, a flag our own Containerfile.engine writes OFF,
and Frank set the target shape to co-tenancy: the engine resident beside
frank-dragoon, not a handover.

- report/tts-npu-spike-verdict-2026-10-19.md owns the measured result: the
  exoneration chain, the two arms' real-time bars (12.5 codes/s deployed vs
  50.00 for NuTTS-Air), the 65,536-entry speech-token census (ids
  151,671-217,206), the host gates (rootless cannot read accel0; dpkg-deb the
  vendor XRT, never dpkg -i), the iGPU census (18,523 of 116,733 MiB GTT in
  use, 17 GiB host RAM available, GTT pages are host pages), the two gates
  that decide co-tenancy (RADV heap for an APU tenant; dragoon's TTFT with a
  synthesis in flight), and the five-file change surface for the trial.
- report/npu-tts-runtime-landscape-2026-10-19.md and
  report/qwen3-tts-payload-portability-2026-10-19.md are the research slices;
  both now point at the verdict page instead of ending in loose ends.
- The vault page gains "Where the engine computes", so an operator doing a TTS
  task learns the NPU is a closed door without re-walking it.

No fleet change: frank-tts / frank-tts-engine still serve narrator1-3 on CPU.
tests: ./tests/run.sh 450/450 green.
2026-10-19 14:02:34 -07:00
dank 88342c144b Merge pull request 'fix(frank-archon): declare two slots, revert the inert host-pool raise' (#32) from fix/frank-archon-two-slots into main 2026-10-19 11:09:24 -07:00
Frank 4d41366f4c fix(frank-archon): declare two slots, revert the inert host-pool raise
FRANK_ARCHON_MAX_SEQS 3 -> 2. At 1.8.1 the budget resolver sizes the KV cache
from the cache bound (--max-model-len x --max-num-seqs), --gpu-headroom-mib and
--host-pool-mib, and the shipped CLI has no KV-size flag (the
--vram-weights-mib/--vram-kv-mib pair was removed at 1.3.0). The live boot
ledger reads that cache at 4.20 GiB = 556,348 tokens = 2.8 sessions at 200,000,
so a third full-height conversation is refused while /health answers green and
the harness advertises capacity the engine cannot honour. Two slots is what the
shape actually serves at the full window; a slot costs KV, never decode, and the
fused TP all-reduce drops from 9 rows to 2 x (1 + spec) = 6, still inside
wht6's 25-row bit-exact bound.

FRANK_ARCHON_HOST_POOL_MIB back to 44928 and FRANK_ARCHON_TTM_MIN_MIB back to
150000, reverting PR #30. That raise bought cache the resolver never hands out:
the two boot ledgers differ in exactly one line -- host_pool 44.47 -> 45.09 GiB
-- while vram_weights (25.81 GiB) and vram_kv (4.20 GiB / 556,348 tokens, both
ranks) are byte-identical, so the extra 640 MiB sat unclaimed in the host pool
while the TTM floor moved out from under the preflight's arithmetic. The
compute-bound cap on --host-pool-mib is unchanged either way (45,568 -> 22,222
MiB of experts either side of the 22,222 clamp).

The bats seam pins the new contract: MAX_SEQS=2, pool 44,928, and the ledger
floor that computes to 148,920 MiB at that pool against the 150,000 minimum.
2026-10-19 11:07:11 -07:00
dank d09d2745cf Merge pull request 'docs(vault): documentation exists for point of use — destroy the ledgers (−13.7k lines)' (#31) from docs/vault-destruction-20261019 into main 2026-10-19 10:59:51 -07:00
Frank a3bce7d123 docs(agents): documentation exists for point of use
*Documentation Policy* rewritten: a page earns its bytes only if it changes what
the reader does next. Measurements are disposable — a rate for anything not
running and not being compared is destroyed, not filed in an Eras row or a
technique page. "It is the only copy" is explicitly NOT a reason to keep a fact.
`log.md` retired from the policy; the Runbook Index no longer promises a page per
retired endpoint (rollback is a commit, not a document).

*Key Lessons Learned* purged of the dead: the dual-R9700 board-swap narrative and
the layer-split/x4-pair experiment laws (no live endpoint uses either), the QSA
patch note and the retired Kairic/FastMTP traps, and the throughput figures inside
the laws that survive — the law stays, the number goes. Two lessons added, both
from this campaign's own failures: a verdict is a lead, the scout's and the tool's
alike; and never let a census decide what to keep, because a tool can say where a
fact lives but not whether anything needs it.

997 → 820 lines.
2026-10-19 10:56:37 -07:00
Frank 327e78b950 docs(vault): destroy the dead weight — 53 pages and the change diary
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for
payloads that no longer run. A throughput figure for a runtime nobody operates is
not a record: AI moves faster than the vault, so the number is wrong before
anyone reads it, and a wrong number in a runbook gets believed. Per the new
purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed.

Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest):
- the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM,
  the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch
  campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two
  "Running Qwen3.6 …" pages;
- the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag)
  and the multi-device/x4-pair/hybrid/offload experiment pages;
- every vLLM page — no vLLM endpoint exists on this fleet;
- `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced
  owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff,
  self-hosted app census, the TTS survey) that were obsolete by construction;
- `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the
  page that owns the thing it governs, never to a diary.

Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras
tables keep the window, payload, runtime, reason and rollback pointer; the rate
columns are blanked. The live shapes stay verbatim — those numbers are what the
deployer and `models.yml` consume.
2026-10-19 10:56:37 -07:00
Frank 8775e64ed0 feat(scripts): tools to destroy documentation, not preserve it
Point-of-use enforcement needs hands, not advice:

- `destroy-vault.py` — moves the pages on a kill list to `.AGENTTRASH/` and then
  deletes every reference to them from the survivors, including the whole line
  when the dead link was the line's only content. Stems are seeded from the trash
  so the pass is re-runnable; `pathlib.Path.stem` is NOT used to compare link
  targets, because it strips the real suffix off names like `sd.cpp`.
- `axe-measurements.py` — deletes bench/throughput ledger sections and rate rows,
  blanks any table cell that reports a rate, and folds Eras older than the current
  one into a single row that says the numbers were destroyed and that rollback is
  a commit. Never touches a shape number or a Law/Wire/Rollback section.
- `repair-damage.py` — fixes the two artifacts those passes can leave (a list
  marker that lost its space, a table separator left without its header).
- `restore-punctuation.py` — restores a line whose only difference from HEAD is
  stripped trailing punctuation. Written after `repair-damage.py`'s first version
  matched every wrapped prose line ending in a comma and deleted 660 of them: the
  rule is now `… see .` only, and the incident is recorded in its docstring.
- `gen-vault-index.py` — never walks into `.AGENTTRASH/`, and sweeps rows whose
  page cell is empty (a description left nameless by a destruction).

Full suite: 451 tests pass.
2026-10-19 10:56:37 -07:00
dank 64765c6a47 Merge pull request 'fix(frank-archon): size the host pool so the KV cache clears 3 x 200,000' (#30) from fix/frank-archon-cache-ceiling into main 2026-10-19 10:30:13 -07:00
Frank d5934a6b47 fix(frank-archon): size the host pool so the KV cache clears 3 x 200,000
The 1.8.1 boot resolved the budget resolver to 4.20 GiB of cache a card —
556,348 tokens, 2.8 sessions at full height — so three concurrent
full-height requests are refused with a 400 at admission: the endpoint's
3-slot contract is broken by the cutover, silently, because /health and
every shorter conversation still answer.

The resolver gives the expert plane whatever the pinned host pool cannot
hold and the cache the rest, so the pool is the lever: +640 MiB a rank
moves that much expert demand into host RAM and the cache wins it back.
Tokens scale with the GiB won (8.11 KiB/token at fp8), so ~0.63 GiB buys
~44,000 tokens — 600,000 with margin — at ~2.5% off the expert slab.
TTM floor moves with the pool (2 x pool + 48,824 + prefix + MNBT =
150,200 MiB computed by the ledger test).

Verified by the boot ledger's `kv:` line and the probe's capacity leg,
not by this arithmetic.
2026-10-19 10:29:20 -07:00
dank 40ad1f4a60 Merge pull request 'feat(frank-archon): radiance 1.8.1 — the budget resolver, the sharing knobs, and the CLI breaks between them' (#29) from feat/frank-archon-radiance-181 into main 2026-10-19 10:21:27 -07:00
Frank 9de58ed6fd feat(frank-archon): move the endpoint to radiance 1.8.1 and its budget model
The decode stall is step contention (measured: 18.7 ms decode-only steps
against 317 ms steps that carry a prefill chunk, decode at 13 t/s median
while prefill runs), and the two engine knobs that fix it arrived after this
endpoint's pin. Moving to 1.8.1 (2026-10-18) brings both, and brings two CLI
breaks with them:

  - --vram-weights-mib / --vram-kv-mib were REMOVED at 1.3.0. The per-card
    split between the expert slab and the KV pool is now derived by the
    engine's budget resolver from --gpu-headroom-mib, the cache bound
    (--max-model-len, --max-num-seqs) and --host-pool-mib, and the cache may
    lend idle blocks to the expert slab. The slab size and the pool's token
    count are boot-ledger lines from here on, so the capacity contract
    (3 x 200,000 fp8) is verified live by the probe's capacity leg instead of
    computed from an env number.
  - The prefix cache has no disk tier any more: --prefix-cache-dir and
    --prefix-cache-disk-mib are refusals and --prefix-cache-host-mib is
    --prefix-cache-mib. 64 GiB of parked KV on NVMe becomes re-prefill, so
    prefix_hit_rate is the thing to watch after this boot.

env: image re-pinned to the 1.8.1 digest; the pair and the disk tier dropped;
FRANK_ARCHON_PREFIX_CACHE_MIB replaces the host/disk pair; new
FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto (explicit, because auto vs off is the
difference between decode waiting one chunk and waiting a whole step) and
FRANK_ARCHON_PREFILL_INTERLEAVE=0 (opt-in through a guarded argv array).

serve: argv follows the new CLI, /data mount and the pre-created kvcache path
go with the disk tier, and the retired env keys are no longer required (they
would abort ExecStart). autoresearch-frank-archon.sh takes the same CLI, with
a cold prefill now coming from --prefix-cache-mib 0 instead of a per-run disk
directory. run-swift15-bf16.sh is untouched: it pins its own engines by design.

tests: the two budget-arithmetic seams asserted knobs that no longer exist and
are replaced by one seam that asserts the retired knobs are ABSENT from env and
argv - a stale line is a boot refusal, not a no-op. The container-contract seam
now asserts the tune mount and the absence of /data.
2026-10-19 10:18:24 -07:00
Frank 52dbe18486 docs(vault): the census was lying; corrected, and the honest result is flat docs
`scripts/port-census.py` built its corpus from the new directory, so each page
under test was held by itself: every era page reported 0 orphan lines. With the
corpus excluding the file under test, each candidate holds 14-63 sole-record
lines, so no era page is padding and none was trashed.

Deletions of restatement only:
- Multi-Node Runtimes.md (-257): restated launch material owned by
  Qwen3.8 PP Deployment.md; index rows and two Related bullets repointed.
- log.md: 66 headings folded into 35 dated entries (one entry per date, newest
  first), titled headings kept as `date — Title` per the spec's wording, and 78
  lines of transient deploy chatter dropped where every fact token in the line
  is held by an owner page (preserved in .AGENTTRASH/20261019-log-transient/).
- Authentic: the 5-page split reverted (-93); the parent carries
  `okf-exempt: length` and the three stub pages stay deleted.

check-okf.py accepts `## YYYY-MM-DD — Title`; port-census.py excludes the file
under test and gained a docstring note about the trap. One-off compaction tool
moved to ~/.AGENTTRASH. Full suite: 451 tests pass, 0 OKF violations,
155 pages, 0 unresolved links.
2026-10-19 10:17:52 -07:00
Frank 2a12487653 fix(vault): delete the restated consolidation page, not the vault
`Multi-Node Runtimes.md` (+257) restated launch material that
[[Qwen3.8 PP Deployment]] (build target, two-process RPC table, cache-prune
cron), [[Sample Script Set]] (runnable scripts), [[Ethernet Bandwidth Math]]
and [[Expected Performance]] already own, and mixed-vendor numbers owned by
[[x4-pair-experiment]]. Consolidation means the facts end up in ONE place, not
that a new page is written beside the old owners: the four stubs it replaced
were frontmatter-only (9 lines each) and needed no successor. Page trashed,
three inbound links repointed at the real owners.
2026-10-19 09:58:04 -07:00
Frank a483e51804 Merge origin/main: reconcile the frank-zealot reslice with the dedupe pass
main moved under this campaign — PR #25/#26 resliced the RTX 3090 endpoint.
The frontmatter conflict is resolved in favour of the live shape, confirmed
by re-probe the same day: `--ctx-size 602112 --parallel 3` (3 × 200,704),
`--cache-type-k f16 --cache-type-v q8_0`, `--spec-type draft-mtp`, `--jinja
--reasoning auto`, `--metrics`. The 262,144 figures that remain on the page
are era rows and the checkpoint's trained window, not the live shape.

log.md's 2026-10-19 fleet-state bullet now records both probes and the law
this taught: a docs pass that reports live shapes must re-probe before
writing the number, because main moving mid-campaign makes the first probe
stale.
2026-10-19 05:12:41 -07:00
Frank c860fb3c8c docs(vault): the frank-medium locator's two wrong cells
The ROCmFPX row said "single card, pinned at 36e9acd": `36e9acd` is the
SHA that `scripts/build-rocmfpx.sh` pins for the *build*, and the era page
records the deployed commit as `c49ebdb` running a 2× R9700 `-sm layer`
split over `ROCm0,ROCm1` at 131,072 × 1 slot with draft-MTP pinned to
`ROCm1`. The 2026-09-20 row skipped a payload generation: the same day the
endpoint swapped MXFP4 → FP8 E4M3 128×128 ([[frank-medium-model-swap]])
before MXFP6 arrived on 09-21. Both cells now quote the era pages.
2026-10-19 05:09:48 -07:00
Frank fb8862b24c docs(vault): verified corrections with sources, not assertions
- frank-archon KV dtype law: `bf16|fp8` is a **radiance** restriction. Stock
  vLLM advertises `auto|fp8|fp8_e5m2|fp8_e4m3` (docs.vllm.ai engine args), and
  even there the advertised set is not the usable set — several accepted
  values crash engine init at the attention layer's own dtype check
  (vllm-project/vllm#42587). The page stated the restriction as if upstream
  had it.
- Open Notebook: re-censused 2026-10-19 — no unit file, no installed quadlet,
  no container. The page already said NOT deployed; it now says what was
  checked, so the claim is re-checkable instead of remembered.
2026-10-19 05:08:47 -07:00
Frank 293c25c50b docs(agents): compress the load-bearing lessons, repoint their owner links
Four bullets in Key Lessons Learned carried narrative a vault page owns: the
2026-09-20 board swap ([[Dual-R9700-Cooperation]]), the `--override-tensor`
gate ([[Llama Cpp Multi-Device Splits]]), the x4 link budget
([[x4-pair-experiment]]), and the silent-RAM incident plus the Prometheus
`--metrics` law ([[frank-archon]], [[frank-zealot]]). Each keeps its law, its
numbers and its error strings; the causal narration that duplicated a page is
what went. The RAM bullet linked to [[log]] — the incident is owned by
[[frank-archon]], so the link now names the owner.
2026-10-19 05:08:29 -07:00
Frank 483fd843b7 fix(scripts): port-census must not call prose words facts
A hyphen or a trailing period inside a word was enough to make it a tracked
token, so `long-standing`, `deployed:` and `cleaned.` were reported as lost
facts while real identifiers hid behind them. Require a digit, dot, slash,
colon, `--`, two hyphens (`frank-tts-engine`) or ALL-CAPS in the stripped
word. The census of the log rewrite went from 979 noisy lines to a set whose
unheld tokens are all fact-shaped.
2026-10-19 05:08:22 -07:00
Frank a5fe045b58 docs(vault): one fact, one owner — repair the doubled pages, single-source the facts
Structural repair:
- 13 pages were the same document twice (each exactly 2x its HEAD size),
  a second frontmatter block whose closing fence had been glued onto a body
  line by a frontmatter rewrite that re-read its own output. Un-doubled by
  cutting at the second frontmatter and keeping the copy whose body is a
  superset; the orphan census reports 0 lost tokens against HEAD.
- log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first —
  66 dated headings merged into 34, titles demoted to bold leads, every
  content line verified present. The file had been descending at the head
  and ascending at the tail.

Ownership moves:
- New `LLM/frank-medium.md` is the locator for the retired Dankserver
  endpoint: era windows quoted from each era page's own banner, no numbers
  restated, and the two unresolved date conflicts recorded instead of
  silently resolved. `frank-archon.md` links to it instead of restating the
  pre-rename window.
- The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone;
  `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor
  verdicts and cross-split spec decode, and the four links that pointed at
  the stubs were repointed at it or at the drafter page.
- Authentik's three stub pages fold into `Authentik.md`, with the process
  detail and the outpost reference split out so the parent stays under the
  length hint.

Verified corrections (host census + live probe, 2026-10-19):
- The MXFP4 checkpoint two pages called the "on-disk rollback" is gone:
  /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on
  the era page that owns that payload; the swap page now links to it.
- frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2
  arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run
  census is kept once (155 requests) instead of twice (~120 and 155).
- Fleet shapes re-confirmed against the live endpoints before touching any
  page: 3x200000 fp8 KV, 2x200704, 2x262144.

Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
2026-10-19 05:00:43 -07:00
Frank 3880503057 feat(scripts): OKF conformance linter, trash-aware vault seams
Add scripts/check-okf.py: enforces the OKF v0.1 hard rules the repo had
been carrying by convention — parseable frontmatter with type/title/
description/timestamp, description shape (length, not-equals-title, no
restated quantity), the reserved-name and index.md rules, log.md's date
heading shape (bare ISO date, one entry per date, newest first), the
500-line hint, and the doubled-page signature (a second `type:` key in
the body). Citations are reported as a soft tally, not a failure. The
500-line ceiling is escapable two ways, both explainable: `okf-exempt:
length` in a page's own frontmatter, and pages derived as one-per-
endpoint — live endpoints from scripts/, retired-endpoint era pages from
their tag (AGENTS.md lets a campaign ledger live on the page that
measured it).

scripts/gen-vault-index.py: sibling-presence was tested as a bare stem
substring, so a page whose name is a prefix of another page's name
(`frank-medium` inside `frank-medium-rocmfpx`) silently never got an
index row. Test the wikilink form instead.

scripts/check-vault-links.py: `.AGENTTRASH` is deleted-but-held state,
not vault content. It is now excluded from both the scan set and the
target index, so a live link into the trash reads as unresolved instead
of resolved.

tests/docs_completeness.bats: the same exclusion in both page walks —
trashed pages need no index row, and a trashed page must not satisfy the
docs requirement for a live compose service. Adds the OKF conformance
and index-freshness tests.

scripts/port-census.py: the port-before-cut orphan census used to prove
a merge lost no fact, kept so the next collapse can re-run it.
2026-10-19 04:54:06 -07:00
dank b9d21ae947 Merge pull request 'docs(frank-zealot): boot ledger of ba69ea5, corrected ubatch and q8_0 credits' (#26) from chore/frank-zealot-mtp-reslice into main 2026-10-19 03:38:40 -07:00
Frank 30e8ed55a0 docs(frank-zealot): record the ba69ea5 boot ledger and correct the carried credits
The boot of record for the new shape, from the verbosity-4 journal plus
settled nvidia-smi: target KV 14,406.00 (K f16 9,408.00 + V q8_0 4,998.00),
draft KV 2,352.00 over the same 200,704 cells x 3 seqs, recurrent state
452.25, compute 1,665.00 + 424.88 (draft 480.97 + 424.86), weights 2,283.29
+ 257.66, projector 595.29 + 248.10 + 24.93, settled 22,870 used / 1,255
free with n_slots = 3 / n_ctx_slot = 200704.

Two inherited numbers did not survive it, and both are corrected in the env,
the serve rationale and the page:

  - The ubatch 2048 -> 1024 step was quoted at "-~1,306 MiB". That was the
    786,432 / q8_0-pair arm. At THIS shape the target's compute buffers went
    2,496.13 -> 2,089.88: 406 MiB, with CUDA0 going UP 257 while CUDA_Host
    went down 663. q8_0 V (4,410 MiB, not the 3,840 previously written) is
    what funds the drafter, alone.
  - The fit gate priced 22,428 against a real 22,870: conservative by 442
    MiB, so it stays honest as written.

And one finding stated plainly: this is the first boot on this card that
misses llama.cpp's own free-memory target - the engine printed "cannot meet
free memory target of 1,867 MiB, need to reduce device memory by 102 MiB"
and settled at 1,255 MiB free, under the ~2 GiB band every previous era
held. Cheapest rungs back are CTX_SIZE=401408 or SPEC_TYPE=none, both one
line.

Probes on the same boot go in too: chat turn (448 tokens, 1,755 chars of
thinking - empty content at max_tokens 64 is the documented trap), tools
(finish_reason tool_calls, {"a":24,"b":7}), vision (512 px solid red ->
"Red", 280 prompt tokens), and the drafter's own statistics (acceptance
0.706 / 0.763, mean accepted length 2.41 / 2.53).
2026-10-19 03:38:03 -07:00
dank ba69ea563e Merge pull request 'feat(frank-zealot): 3 x 200,704, q8_0 V, MTP armed' (#25) from chore/frank-zealot-mtp-reslice into main 2026-10-19 03:19:04 -07:00
Frank a7062d1cb3 docs(frank-zealot): carry the 3 x 200,704 shape, retire the exact-KV era row
The endpoint page owns its shape: the flags row, the budget equation behind
it (25,088 B/token, 14,406.00 MiB of cache + 2,352.00 MiB of draft cache
inside 24,123 MiB addressable), the "MTP is ON" section replacing the "OFF by
measurement" one, the harness contract at the per-slot 200,704, and the
fallback rungs rewritten as deltas FROM the shipped shape.

The 2026-10-18 exact-KV boot (KV 16,384.00 / RS 100.50 / compute 1,408.06 +
1,088.07 / mmproj 868.32 charged after the fit check, settled 21,176/2,949)
and its contended rate floors move into that era's Eras row with the one-line
rollback, so the two pages that carried them collapse to one. Rates for the
new shape are deliberately absent - no bench was ordered, and the fit gate,
the health gate and the boot ledger are the evidence a swap needs.

AGENTS.md follows the same numbers: contextWindow 200704 is the per-slot
window and the trio's shape line.
2026-10-19 03:16:48 -07:00
Frank 25a8b00043 feat(frank-zealot): reslice to 3 x 200,704 with q8_0 V and MTP armed
Frank's order, 2026-10-19: --parallel 3 at 200,704 a conversation (pool
602,112), K at the engine default (f16), V to q8_0, batch 1024x6 / ubatch
1024, and the MTP drafter armed explicitly.

The three requests are one budget equation. At 8,192 elements per token per
cache the pair now bills 25,088 B/token = 14,406.00 MiB over the 602,112
pool, and the armed draft context adds its own f16 KV over that whole pool
(4,096 B/position = 2,352.00 MiB) plus buffers and recurrent cells. Dropping
V to the 8-bit class frees 4,410 MiB against exact V and the ubatch drop
hands back ~1,306 - that is what pays for the third slot AND the drafter,
neither of which exact V could fund.

The serve script's written-down fallbacks move with the env so a missing env
file cannot boot the retired shape, and the seam now: asserts the per-slot
contract (divides evenly, <= the trained 262,144), bills an armed drafter in
the VRAM fit gate, proves the gate still REFUSES both boots this shape can
walk into, pins the shipped MTP set (and the absence of --spec-draft-model,
which would load the trunk twice), and pins SPEC_TYPE=none as the rollback
that strips the whole set.
2026-10-19 03:16:35 -07:00
Frankandomp 1ed7f0ddf2 docs(agents): compress Key Lessons to one law per bullet, correct the FA-quants law
Key Lessons Learned 483 -> 340 lines. Every law survives; only the essays
went. Port-before-cut census over the diff: 454 deleted lines, 0 orphaned
facts (every number still has a holder in the vault).

- Each lesson is now one bullet with the page that owns the detail.
- De-duplicated the page's own internal repeats: the layer-split law was
  told twice (PCIe and ROCm sections), the FA-quants law twice more.
- CORRECTION, verified on dankclaw 2026-10-19: `GGML_CUDA_FA_ALL_QUANTS` is
  deprecated upstream (master's ggml/cmake/common.cmake tells you to use
  GGML_CUDA_FA_QUANTS=all), the successor's default pair list
  `q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16` compiles the quantized-KV kernels
  with the old flag OFF (master build-cuda cache: FA_ALL_QUANTS=OFF), and
  neither llama tree references the old macro under ggml/src/ggml-cuda/.
  The "required or prefill collapses ~12x" law is obsolete; the new law is
  to read the CMakeCache pair list.
- frank-zealot: ported the three facts AGENTS.md alone carried, and verified
  them against the host first — the 2026-10-18 13:16 cutover boot really did
  load PTQ1_0 at `new slot, n_ctx = 133888` x3 (journal), the mmproj fit
  check really printed 873.10 MiB against a 21,439 MiB projection, and the
  warmup spread is 2.6 -> 1,258 t/s prefill.
- New tooling: scripts/check-okf.py (OKF v0.1 conformance linter) and
  scripts/gen-vault-index.py (root index regenerated from frontmatter).

Co-Authored-By: omp <omq@openclaw.sh>
2026-10-19 02:51:32 -07:00
dank a5fe8db877 Merge pull request 'docs(vault): one fact, one owner - de-stale the technique and era pages (C1-C6)' (#24) from wt/20261017-181502 into main 2026-10-19 02:06:12 -07:00
Frank c195c839b4 docs(agents): say a scout verdict is a lead, not a finding
The 2026-10-19 vault de-dup campaign sent four read-only scouts over the same
pages and had to parent-verify every claim: one inverted era-page law (flagged
pages stale for documenting the retired payloads they exist to record), one
listed files that do not exist, one reported duplications grep disproved, one
cited a line past the end of the file it judged. Record the rule beside the
other evidence laws so the next agent does not integrate a yield unchecked.

Refs: vault de-dup campaign, PR #24.
2026-10-19 02:04:40 -07:00
Frank 500ac2b871 docs(vault): consolidate the vLLM offload-knob claim to one owner (C2, C6)
The "`--swap-space` is removed; use `--kv-offloading-size`" claim was carried by
six pages, two of them with their own copy of the upstream source citation. It
now lives only in [[RAM CPU Offloading in VLLM]], which carries the 2026-10-19
verification against `vllm/config/cache.py` (no `swap_space`; `kv_offloading_size`
activates only with `kv_offloading_backend`):

- `VLLM Overview` (two argument tables), `VLLM Index` (parameter table),
  `DFlash DSpark Speculative Decoding`, `Shared Memory Architecture Dankclaw`,
  `Hybrid CPU GPU Processing Dankserver` -> cite the owner.
- `VLLM Index`'s example command used the dead flag; it now shows the live pair.
- `dankserver-vram-growth`: banner disambiguates the 2026-08 `frank-zealot`
  (llama.cpp on the 3090 when that card was in Dankserver) from today's endpoint
  of the same name on Dankclaw, so a retired crash signature does not read as a
  live fault on a same-card-class service.

Refs: vault de-dup campaign, clusters C2 and C6.
2026-10-19 02:01:14 -07:00
Frank 6a8cbde260 docs(vault): retire the ACTIVE labels on frank-medium era pages (cluster C5)
The root and LLM index pages still advertised two retired endpoints' pages as
live procedure:

- `frank-medium-radiance-mxfp4`: frontmatter said "ACTIVE frank-medium
  runbook" and the body opened "The ACTIVE frank-medium deployment". Both
  re-keyed to the era it ran (2026-09-21 -> 10-10), with [[frank-archon]]'s
  Eras table named as the owner of that shape.
- `frank-medium-model-swap`: read as live how-to for a retired endpoint.
  Re-keyed to historical, pointing at [[Model Swap and Rollback]] for the
  current swap law.
- Both index pages' rows re-keyed to match (rows are hand-copied from
  frontmatter - no generator - so both sites move together).

Refs: vault de-dup campaign, cluster C5.
2026-10-19 02:00:21 -07:00
Frank 73fb27254e docs(vault): era-correct the vLLM/Ling era pages (cluster C3)
- `Running Qwen3.6 27B FP8 on Dankclaw`: had no banner at all and opened as a
  live how-to. Added the RETIRED banner (iGPU endpoint today is
  [[frank-dragoon]]), era-tagged the hardware table, and rewrote the
  "primary service now runs vLLM-ROCm with Ling CIRU INT4" production note,
  which was false by three fleet generations.
- `ling-3.0-flash-vllm-plan`: "frank-dragoon is the Qwen3.8-27B PP head" ->
  past tense with the 2026-08-17 redo as the anchor and the live owner named.
- `frank-redo-2026-08-17` banner: stripped restated live shapes that had
  already drifted (radiance 1.0.13, PrismML b10754 - the fleet runs 10777)
  and the 3090-provenance clause owned by the PP deployment page; banner is
  now locator-only (names + ports + links).

Verified before cutting: 457.9 t/s and the ping-RTT figures are solo-held by
Ethernet Bandwidth Math.md (no duplicate), so both scout-reported "duplicates"
were kept.

Refs: vault de-dup campaign, cluster C3.
2026-10-19 01:59:13 -07:00
Frank e32df1b04f docs(NPU page): era-tag host-RAM and PP-rate restatements
One fact, one owner (de-dup campaign C4):

- `FastFlowLM NPU Inference.md` carried the host RAM figures three times
  (29 GiB budget input kept as era data; the 2026-10-18 `free -g` re-reads
  of 121/246 GiB cut -> [[Dankclaw System Specs]]/[[Dankserver System Specs]]).
- The 14-18 t/s PP-split decode is this page's own solo-held era measurement:
  kept, tagged to the 2026-08 era with [[frank-redo-2026-08-17]].
- Promotion section: `frank-zealot` "now runs the NPU endpoint" era-tagged
  (the name has been reused twice since; the NPU unit is `frank-npu`, masked).
- Repaired the Test Runbook splice introduced by the range edit above.

Refs: vault de-dup campaign, cluster C4.
2026-10-19 01:58:03 -07:00
Frank a518fc7c22 docs(vault): de-stale the quant and swap method pages
Parent-written (scout summaries were corroborated against the files, not
trusted): every restatement of an endpoint shape now cites the endpoint page,
and four claims contradicted by today's fleet state are corrected.

- Quantization Production: the challenger-contract line quoted the retired
  compaction-hub window; now cites [[frank-zealot]]'s Current shape. The 3090
  worked example is era-tagged. Sources listed [[frank-dragoon]] three times.
- Quantization by Hardware Architecture: said the 3090 lives on Dankserver
  (it moved to Dankclaw 2026-09-16; the Eras table owns the move) and ranked
  the retired Ling-3.0 Quark INT4 service as the ACTIVE Dankclaw workload. The
  active service is [[frank-dragoon]] on a closed `.hgn` format.
- GGUF Compatibility: Related claimed the ftype-115 ROCmFPX file was "in
  production"; that is an era now, so it points at [[kingjones-q8agent-rocmfpx]].
- Model Swap and Rollback: dropped the fragment byte-count ledger (AGENTS.md
  owns it), the Ampere FP8 hardware claim ([[Quantization by Hardware
  Architecture]] owns it), halogen's HTTP surface ([[frank-dragoon]]'s wire
  contract) and three brittle unit-file line refs (accepted loss: line numbers
  are derivable, the root-unit law stays). Fixed the per-slot example, which
  described the retired 3-slot arithmetic, and a Related bullet that called
  [[frank-archon]] retired while linking the same page twice.

Census over the diff: 1 orphan line, the deleted unit line-number refs above.
2026-10-19 01:52:30 -07:00
Frank 9f1a4e4e51 docs(dankclaw): drop the stale frank-zealot drift note
The page claimed the unit had been failed since 2026-10-18 03:40 with a
hand-run process on the port. Verified live: both units are active under
their repo units (zealot since 21:06, the deployer's 2-slot reslice).
2026-10-19 01:39:54 -07:00
Frank 4a13dcc5b7 docs(agents): say who owns a campaign ledger
The dedupe passes kept trying to port single-owner experiment figures into
endpoint Eras tables. The law now states the test explicitly: two pages
carrying one figure is the defect; a figure with exactly one holder is fine
wherever it lives.
2026-10-19 01:39:54 -07:00
Frank e7bfa82853 docs(log): compress the 2026-10-18 entries to incident + law
The 3-slot -> 2-slot reslice entry and the contended-rate entry carried the
full measurement ledger; the numbers belong to [[frank-zealot]] (shape block +
Eras rows), so the entry keeps the causal story (pool halving funds exact KV;
rates taken under a live consumer are floors) and links for the figures.
2026-10-19 01:34:06 -07:00
Frank 04f4e2c6d4 docs(vault): strip restated fleet facts from 17 technique pages
Enforces "one fact, one owner": technique pages keep mechanism, laws, traps
and verdicts; per-endpoint shapes, digests and era measurements are cited to
the endpoint page that owns them.

Also repairs the four pointers the pass left empty (each verified against the
cited owner before landing):
- EMULATION dequant doubles host RAM (118 GiB -> ~238 GiB) -> [[frank-archon]]
- 12.11 tok/s flat 304 -> 16,095 tokens -> [[Tcclaviger Fork Ops]]
- bundled-head acceptance ladder 0.739/0.554/0.427/0.340 -> [[Qwen3.8 PP Deployment]]
- llama.cpp PR #23897 dedup provenance -> [[qwen3.8-spec-matrix]]

Replaces the retired-shape illustration in Llama Cpp Serving Sizing with the
owned MiMo Q5_K_M row (6.40 GiB payload -> 20,530 MiB used), and unwraps a
hard-wrapped [[Dankserver System Specs]] link that failed link integrity.

Orphan census over the diff: 0 numbers lost (comma-insensitive match against
the whole vault plus AGENTS.md).
2026-10-19 01:33:47 -07:00
dank 129b953843 Merge pull request 'docs: one fact, one owner — endpoint pages own every fleet fact' (#23) from wt/20261017-181502 into main 2026-10-19 01:15:56 -07:00
Frank eb21dbc705 Merge origin/main (PRs #21, #22) into the documentation-law restructure
Conflicts and how each was resolved:

- AGENTS.md: kept the 2026-10-18 Model Swaps law (a payload change is the trio
  edit, the deploy, /health and one probe; no bench unless Frank orders one).
  The incoming Benchmarking-Policy wording it replaced is not lost: its one
  durable fact -- on the PrismML fork build 10777 the per-request `timings`
  object only reaches you on /v1/chat/completions, because a greedy ignore_eos
  run on the raw /completion path answers 500 "Content-only format" -- is now
  carried inside "When a bench IS ordered", step 2, and in [[frank-zealot]]'s
  Contract section.
- frank-zealot-llama-prism-bonsai9-mtp.md: kept deleted. PR #22's additions
  (the exact-KV boot ledger, the contended floor rates, the /completion
  instrument trap) were ported into LLM/frank-zealot.md before the merge, which
  is where the one-page-per-endpoint law says they live.
- log.md: auto-merged, then compressed to incident plus law -- the reslice
  decision, the "a KV byte rate belongs to the payload it was measured on"
  lesson that buried the invented 26,112 MiB figure, and the instrument law.
  The boot ledger and the floor table now live only on [[frank-zealot]].
2026-10-19 01:07:54 -07:00
Frank 846f81ddb8 docs(vault): the work log keeps incidents and laws; the numbers move to the page that owns them
log.md drops 466 lines of per-swap ledger prose (1,169 lines out, 703 back) and
keeps the causal stories: why the host went down, why a knob resets at boot, why
a stale drop-in served a different model for 24 minutes, why the RAM was the
fault. Date sections are unchanged (65 before, 65 after) -- this is compression
inside sections, not deletion of days.

Every number the compression would have orphaned was ported to its owner first,
and a census of the deleted lines against the whole vault is what found them:

- frank-zealot: the two boots that priced non-KV (20,190/KV 12,800 => 7,390
  without the projector; 22,972/KV 14,113 => 8,859 with it, so the vision tower
  costs ~1.4 GiB for a 918 MB file), the 1,153 MiB-free 3-slot boot that made
  the seam price non-KV at 9 GiB, and the f16 wall's own figures (fit 33,198 vs
  23,558 free on a 25,088.00 MiB KV request).
- frank-archon: the pool-cut-is-headroom-not-arena law (2.75 GiB/rank bought
  zero slots; d(reserve) = d(arena) = +248 slots; 14.87 GiB / 6,496 slots
  against a 398,658-token pool), the MNBT compile-RANGE vs SEQS+1 law, and the
  2 x 196,919 prefix-cache proof (393,837 tokens resident above the pool's
  386,904; 94.1/102.3/117.6 t/s, acceptance 0.7284/0.7525).
- frank-dragoon: the KL arm of the 2026-10-17 A/B (mean 0.155 / median 0.057,
  top-1 agreement 85.1 %) joins the PPL and NIAH figures already on the era row.
- Payload and Disk Reclaim: the 121 GB layer census lesson -- a huge image layer
  can be the engine image, not a payload.

Also fixes a wikilink the rewrite had hard-wrapped across two lines, which the
link checker reads as a target that does not exist.

New seam: 'one page per live LLM endpoint'. The live set is derived from
scripts/ (an endpoint has both an env and a unit), and any <endpoint>-*.md page
in the vault fails it -- the enforcement of the one-page-per-endpoint law, so a
re-created era page cannot pass tests.
2026-10-19 01:05:32 -07:00
Frank 6bdfd2853e docs(vault): service and infrastructure pages stop restating endpoint facts
18 pages cut their per-endpoint shape, payload, port and window claims and
link [[frank-archon]] / [[frank-dragoon]] / [[frank-zealot]] instead. Several
of the cut claims had already drifted -- the stack page still described the
kingjones ROCmFP4 era on ROCmFPX, OpenWebUI carried the 27B/3-slot bench
narrative, Hermes named a retired max_model_len and the old service name, and
the OMP harness page's live-window block had drifted off frank-zealot's real
262,144 (it is now labelled a dated worked example whose authority is the
endpoint page).

Two facts gain a single owner instead of two: the up{job=~"frank-.*"} scope
lives in Infrastructure/Prometheus.md, and the blackbox/Anubis probe form in
Architecture/Stale Signal Audit.md, each cited from the other page.

Voice Packs and Frank TTS carried live-state claims that contradicted the
engine page's 2026-10-18 deploy record (nothing listening on 7772/8880, model
dir absent); they now point at the pages that own host state.
2026-10-19 00:56:02 -07:00
Frank 219a097b39 docs(vault): one page per endpoint, forever -- 45 era pages fold into three
Every payload swap a live endpoint has ever run is now one row in that
endpoint's Eras table, with its rollback path in the same row. No per-attempt
page, no per-swap ledger, nowhere else.

- Added: LLM/frank-archon.md (radiance, 2x R9700 TP=2), LLM/frank-dragoon.md
  (halogen flash-server, 19 eras), LLM/frank-zealot.md (llama.cpp PrismML fork
  on the 3090). Each carries Current shape, Eras, Laws, wire contract, files.
- Deleted: 45 per-payload runbooks -- the frank-dragoon line (11 including the
  pre-name halogen/kingjones/kairic/CIRU/eemin/agentionai/dflash2 pages and the
  benchmark page), the frank-zealot line (llama.cpp/vLLM/ik_llama/SGLang eras),
  the frank-archon line (vLLM/FreeToken/R9V/llama.cpp/radiance eras), plus
  frank-medium, frank-npu and the ubatch/PCIe one-offs that owned nothing.
- Rewired: every wikilink and markdown-path reference to a folded page points
  at the endpoint page instead; the three Research see-also lists and the
  ubatch sweep's runbook link included. No anchored links existed in the set.
- Regenerated both index pages as pure locators -- one row, one clause, no
  numbers -- so the docs_completeness locator law holds for all 153 pages.

tests/docs_completeness.bats green: index links every page, no unresolved
links. The pages moved to .AGENTTRASH/20261018-doc-dedupe/LLM/ (gitignored,
and git history holds every byte).
2026-10-19 00:49:03 -07:00
Frank af5ddfc54c chore(scripts): env files carry knobs plus the reason, not the swap ledger
Every KEY=value is byte-identical to the previous revision (diffed against
git show HEAD: -- 28/25/20 keys). What left is the per-swap narration: the
archon header drops 207 lines of era prose, dragoon 127, zealot 104. Each
header now keeps the format lock, the digest/identity law, and one line per
knob explaining what a reader editing that line cannot recover anywhere.

The facts moved to their one owner: LLM/frank-archon.md, LLM/frank-dragoon.md,
LLM/frank-zealot.md. One stale claim dies with the trim -- the dragoon header
still said the KV pool is ONE 200,704-position budget behind 2 slots, which
contradicted its own 401,408 paragraph; the page carries the correct ledger.

Also kills two wrong arithmetic claims in the zealot header: the q8_0-V and
CTX_SIZE=401408 rungs each free 3,840 MiB at the 524,288 pool, not 8,192 and
not ~6.7 GiB.
2026-10-19 00:48:36 -07:00
Frank 61011b0a81 docs(agents): one fact, one owner — AGENTS.md carries operator-critical numbers and cross-cutting laws only
The runbook index collapses to three live endpoints plus a retired-row
locator: a payload swap is an Eras row on the endpoint's vault page, never
a page of its own. Restated shapes, digests and measured numbers are
replaced with links to [[frank-archon]], [[frank-dragoon]], [[frank-zealot]]
and scripts/<svc>.env, which is the authority for every value.

Carries the frank-zealot instrument correction from d457985: on the PrismML
fork build 10777 the per-request timings object only reaches you on
/v1/chat/completions — the raw /completion path answers 500 on a greedy
ignore_eos run.
2026-10-19 00:48:09 -07:00
dank e06092b24d Merge pull request 'docs(frank-zealot): measured exact-KV boot, floor rates, chat-path instrument' (#22) from docs/zealot-f16-measured-record into main 2026-10-18 22:07:07 -07:00
Frank d5a43b7192 Merge remote-tracking branch 'origin/main' into wt/20261018-204501 2026-10-18 21:28:04 -07:00
Frank d457985b8d docs(frank-zealot): measured exact-KV boot, floor rates, chat-path instrument
Deployed shape verified live: KV 16,384.00 MiB (ledger-exact 32,768 B/token x
524,288), RS 100.50 MiB (2 cells), settled nvidia-smi 21,176 used / 2,949 free,
n_slots 2, n_ctx_slot 262144, DropInPaths empty.

Rates recorded as FLOORS: another consumer drove the endpoint through the whole
window (56 prompts on the other slot in three minutes, 96% GPU-util). Single
stream 134.1-141.5 (145.8/163.2 in a lighter minute) vs the 168.7 q8_0 record;
two-stream aggregate 256.6-265.1 vs ~300 across three; prefill 3,159.5 at
106,282 beats the 2,877 record while contended; decode at 106k depth 71.5 (new
baseline, no prior). Per-stream concurrency improved: ~130 each vs 102.8.

Instrument correction: the PrismML fork build 10777 answers 500 'The model
produced output that does not match the expected Content-only format' for a
greedy ignore_eos run on the raw /completion path, so the per-request timings
must be read off /v1/chat/completions. AGENTS.md Benchmarking Policy corrected.
2026-10-18 21:28:04 -07:00
dank 5550d32a2e Merge pull request 'fix(frank-zealot): 2 × 262,144 over unquantized f16 KV' (#21) from fix/zealot-2x262144-f16-kv into main 2026-10-18 21:06:08 -07:00
Frank 71bc1735d3 docs(frank-zealot): re-key the exact-KV shape across the vault
AGENTS.md: topology row, the frank-zealot runbook section (shape, KV
accounting, MTP verdict, compaction-hub concurrency), and two Key
Lessons bullets -- the draft-context bill restated with the real knob
names, plus a new lesson that a quant's scale overhead belongs in its
byte rate (the 6.25 % invented by doubling 13,056.00 MiB).

LLM/frank-zealot-llama-prism-bonsai9-mtp.md: retitled to the live shape;
the 3-slot q8_0 boot ledger and its measurements are relabelled as the
arm this replaced (byte-identical numbers, honest attribution), the
corrected f16 rate derivation is added, the MTP census is labelled
q8_0-era, the fallback rungs are re-derived for the 524,288 pool, and the
boot-ledger section carries an explicit PROJECTION that must be replaced
with the measured ledger from the first f16 boot.

Pointers re-pointed (live-shape sentences only; every historical ledger
left byte-identical): LLM/index.md, vault index.md, Dankclaw/OpenWebUI.md,
and the retired-runbook "today" pointers in mimo-q5km, mimo-q6k,
qwen35-9b-q6k, swift15, sglang-mimo-v26. log.md gains the dated entry.
2026-10-18 21:02:42 -07:00
Frank 68afc6dbaf fix(frank-zealot): 2 x 262,144 over unquantized f16 KV
Frank: "shift frank-zealot to 2 parallels of unquantized kv".

Shape: CTX_SIZE 786432 -> 524288, PARALLEL 3 -> 2, CACHE_TYPE_K/V
q8_0 -> f16. The per-slot window is UNCHANGED at 262,144 (the native
trained window, and models.yml contextWindow), so what the exact cache
buys is fidelity and what it pays is one concurrent conversation.

The POOL is the term that had to move: exact KV bills 32,768 B/token at
this geometry (8 full-attention layers x 4 KV heads x 256 dim, 2 B,
K and V), so the old 786,432 pool asks 24,576 MiB against the 24,123 MiB
the driver leaves addressable. 524,288 positions = 16,384 MiB, projected
~21,200 used / ~2,900 free from the 13:19 boot ledger (only the KV pool
and one recurrent-state cell move).

Also corrects a repo-wide arithmetic error: "f16 would bill 26,112 MiB"
doubled the q8_0 ledger figure INCLUDING its 1/32 scale overhead. q8_0 is
1.0625 B/element, f16 exactly 2.0, so the true ratio is 1.882x. The wall
stands; the number was wrong.

And it corrects a false claim about the drafter: this build DOES expose
--spec-draft-type-k / --spec-draft-type-v (verified against
llama-server --help), so the draft cache is quantizable -- the
exact-KV-plus-MTP exclusion is a measurement gap, not a law, and the
1,088 MiB quantized-draft arm is recorded as unmeasured.

Seam: tests/frank_zealot_unit.bats re-pins the shape contract (2 slots,
524,288 pool, 262,144 per slot) and adds a VRAM fit gate that computes
dtype x pool + the measured non-KV ledger terms against 24,123 MiB with
>=1 GiB of slack -- and proves the gate REFUSES exact KV at the old
pool, so passing means something. 18/18 green.
2026-10-18 21:02:37 -07:00
dank 100b62ba22 Merge pull request 'docs(openwebui): the TTS voice rejector is in-browser Kokoro, not frank-tts' (#20) from docs/openwebui-browser-kokoro-tts into main 2026-10-18 20:38:50 -07:00
Frank 9a17b93e8a docs(frank-tts): crosslink the browser-Kokoro rejector both ways
A consumer that lists the 29-name af_*/am_*/bf_*/bm_* pack and refuses
narrator1 without a request reaching the façade is synthesizing in the
browser. Frank TTS's troubleshooting table gains that symptom; OpenWebUI's
See Also gains the service page.
2026-10-18 20:17:19 -07:00
Frank 3e779823e0 Merge remote-tracking branch 'origin/main' into wt/20261017-181502
# Conflicts:
#	serverAppConfigs-docs/log.md
2026-10-18 20:17:19 -07:00
Frank 30e3d36b1e docs(openwebui): the TTS voice rejector is in-browser Kokoro, not frank-tts
The `Voice "narrator1" not found. Should be one of: af_heart, …, bm_fable`
toast is OpenWebUI 0.11.4's bundled Kokoro.js validator: the enum is
Kokoro's fixed 29-voice pack, not frank-tts's registry. Frank-tts answers
200 audio/mpeg for the exact OpenWebUI body from inside the container, and
the failing click leaves no `audio/speech` line in the container log — that
contrast is the instrument. Admin config DB is correct
(engine=openai, base URL :7772/v1, model frank-tts, voice narrator1); no
container env and no user row carries an audio override, so the
`browser-kokoro` choice is per-browser storage. Fix: Settings → Audio →
TTS Engine → Default.
2026-10-18 20:15:28 -07:00
dank 5fbbea2c2a Merge pull request 'docs(frank-tts): the contract table gains its Default voice row' (#19) from docs/frank-tts-voice-lore into main 2026-10-18 19:33:06 -07:00
Frank 96b7687b34 docs(frank-tts): the contract table gains its Default voice row
#18 shipped DEFAULT_VOICE and #17 explained why the harness cannot name a voice from a setting, but the endpoint page still described voice resolution as if every request carried a name. One contract row, one paragraph in Naming a voice (a call that names nothing is the harness default shape, not a client bug), the env-file row now lists the default voice among its knobs, and the worklog records the rollout: both merges, the one-dispatch deploy with the 33/33 gate, the card flipping to up on the next checker tick, and the previously failing call returning 430124 bytes with the journal reading "speech voice=narrator1 (default)".
2026-10-18 19:32:03 -07:00
dank 7889a68497 Merge pull request 'fix(frank-tts): a voice-less request now speaks the declared default instead of 400ing' (#18) from fix/frank-tts-default-voice into main 2026-10-18 19:26:08 -07:00
dank b590695e2a Merge pull request 'docs(frank-tts): the voice stack is deployed — fix the six pages and the card that still deny it' (#17) from docs/frank-tts-live into main 2026-10-18 19:25:38 -07:00
Frank b91873578b fix(frank-tts): answer a voice-less request with the declared default
The omp speech device forwards a voice only when the call passes voice_id - there is no setting that feeds one (speech.voice is the TUI speak-aloud Kokoro enum, tts.localVoice belongs to the local backend) - so every harness voiceover arrives with no voice key and the shim answered 400 "unknown voice None". Measured on the deployed endpoint 2026-10-18, from a restarted session with the speech role pointed here.

DEFAULT_VOICE makes the absent case a spoken operator choice instead of a code surprise: the pack named there answers a voice-less request and the journal line is marked (default), so an unrequested voice never reads later like a requested one. An UNKNOWN name still 400s with the list that exists - the no-silent-fallback law that this endpoint was built on is untouched, and the strict shape stays the code default (empty DEFAULT_VOICE), which the second new seam test pins. Ships narrator1 in scripts/frank-tts.env.
2026-10-18 19:21:28 -07:00
Frank 0d70f115d1 docs(frank-tts): the harness sends a voice only in voice_id
The section this PR adds claimed speech.voice was the forwarded voice key. It is not: the openai-speech transport builds {model, input, response_format, voice?} and sets voice only when the CALL passed voice_id, tts.localVoice is the local Kokoro backend key, and speech.voice is the TUI speak-aloud voice whose enum is Kokoro-only. A restarted session proved it the honest way - the tts device arrived at 7772 with no voice key and got "unknown voice None; available voices: narrator1, narrator2, narrator3" - so the fresh-process success earlier that day was the run that was told to pass voice_id, not evidence of a setting.

The law now reads: voice comes from voice_id alone, so a voice-less call is the harness default shape (the endpoint grew DEFAULT_VOICE in its own PR to answer it), and speech.voice must stay a Kokoro name - a pack name there breaks the vocalizer and does nothing for synthesis. config.yml reverted to am_fenrir on the operator box, and the worklog entry that recorded narrator1 as wiring carries the correction.
2026-10-18 19:21:22 -07:00
Frank 7e391d4325 docs(frank-tts): the voice stack is deployed, and the harness speaks through it
Six statements still claimed the endpoint did not exist: both vault descriptions, the two index.md bullets, the Open Notebook blocker, and the engine page deploy order that opened with "Nothing below has been run". All re-keyed to the deployed state, with the ledger the deploys printed: engine healthy 31 s after the build at 4.0 GiB resident, three voices warm, the ladder measured at 1883 ms then 1226-1283 ms (not the retired 6.2-8.0 s), and two voices rendering the same sentence into different bytes.

Adds a The harness speech role section to the endpoint page with the three wiring laws measured the day of the flip: speech.voice is the forwarded key (tts.localVoice belongs to Kokoro), an explicit speech role has no fallback candidate, and models.yml loads at process start while config.yml is read live - so a session older than the frank-tts row fails exactly like a dead endpoint. Fresh-process proof: omp -p --tools tts saved 180524 bytes with voice=narrator1 while the facade logged the matching speech line.
2026-10-18 19:11:41 -07:00
Frank 5768680542 fix(homepage): the frank-tts card is a real probe now
The endpoint deployed on 2026-10-18, so the admin card must carry its state rather than a placeholder: wip becomes false (the registry schema wants the key on every service; false is what the live cards use) and the description drops the "Hidden until deployed" line. showLink stays false and the card stays admin-only: an HTTP API has no page to open, and homepage-status.py counts ANY HTTP response as up, so the card is honest with no link and no /health path in the probe. The voice seam that pinned wip == down is re-keyed to the deployed contract and now also refuses a link or a user-category placement.
2026-10-18 19:11:41 -07:00
dank d2ce6693da Merge pull request 'docs(frank-tts): the retired cold-voice claim still lived in six more places' (#16) from docs/retire-cold-voice-ladder into main 2026-10-18 18:59:13 -07:00
Frank 2dc31ff09c docs(frank-tts): retire the 5.5 s cold-voice claim everywhere it still lived
PR #15 corrected the engine's page, the research brief's law bullet and the
warmer's header, but the retired ladder had six more homes: both quadlet files
("6.2-8.0 s… 0.5 ms", "~6-8 s each", "paying the 5.5 s cold decode"),
`scripts/frank-tts-engine.env` (the whole law paragraph and the WARM_TEXT note),
`scripts/frank-tts.env`, the [[Frank TTS]] troubleshooting row that told an
operator to expect ~5.5 s, the research brief's consequences paragraph, and one
test comment.

All re-keyed to what was measured on the deployed container 2026-10-18: the
first synthesis of a freshly registered voice costs 1,883 ms against 1,226-1,283
ms afterwards, so the warm pass buys about 0.6 s — and the reason it exists at
all, which the pages never led with, is that the registry is memory-only: without
the hook every voice name is a 400 until somebody synthesizes. The trial's dated
worklog entry stays as written; the correction points at it.

No behavior change: comments, unit lore and docs only, so no cfhash move and no
image rebuild.
2026-10-18 18:52:50 -07:00
dank 0315a62d84 Merge pull request 'fix(frank-tts): the warm hook died on the kernel argument cap; the cold-voice law is retired by measurement' (#15) from fix/frank-tts-warm-argv into main
none
2026-10-18 18:50:00 -07:00
Frank 0773b26559 docs(frank-tts): the cold-voice ladder did not survive the deployment
The first real boot gave the chance to measure what warming actually buys, and
the trial's numbers were wrong:

- Empty registry, register only, then time: the FIRST synthesis of a freshly
  registered voice cost 1 883 ms; the next two 1 283 and 1 226 ms. A ~110-word
  request cost 9.4 s and then 9.8 s.
- So the published 6.2–8.0 s cold / first-audio-byte 5 530 ms / later-requests
  0.5 ms ladder does not reproduce, and request time tracks the audio generated
  rather than snapshot state. The 0.5 ms figure is an internal state-restore
  cost, never an end-to-end request time.
- Warming still earns its place for a different reason than the page claimed:
  the registry is memory-only and re-registering does not warm, so a boot without
  the hook leaves the façade answering 400 for every name. The saving is ~0.6 s.

Law 1 of [[Frank TTS Engine]] is re-titled the registry law and carries the
measured numbers; its example journal line is a real one from today; the
troubleshooting row for "first request of the day is slow" — the retired law's
symptom — is replaced by the argument-cap fault that actually happened. The same
ladder in the research brief is annotated rather than erased, and [[Voice Packs]]
step 6 stops claiming the hook registers only packs it has not seen (the engine
replaces a name it already holds).
2026-10-18 18:45:31 -07:00
Frank 4235c1d4cb fix(frank-tts): send the registration body through a file, not an argument
The deploy of the newly built engine image came back `0 voice(s) warm` with
`/app/warm-voices.sh: line 152: /usr/bin/curl: Argument list too long` above
every line, and the unit was `active` with a green `/health` beside it. The hook
built `{"name":…,"wav_b64":…}` in a shell variable and passed it as ONE curl
argument; the kernel caps a single argument at MAX_ARG_STRLEN = 131,072 bytes,
and the shipped reference clips base64 to about 1.27 MB, so curl was never
executed and the hook reported the absence as "HTTP no response".

The body now goes to a mktemp file and curl reads `@file`; the audio enters
through a redirection instead of a command substitution, so the script never
holds a 1.3 MB string at all. Every pack in the fleet is ~1 MB, i.e. ~13x the
cliff, so this was always going to happen — it took the first real packs to show
it because no fixture was ever that big.

The regression test therefore ships a 300 KB pack against the seam's REAL stub
listener with the REAL curl: it fails with the same kernel error the host did
(`Argument list too long`, then the 400-for-every-name summary), and passes once
the body is a file. Proved red before the fix, green after.
2026-10-18 18:45:31 -07:00
dank 747c6656d5 Merge pull request 'feat(frank-tts-engine): Qwen-TTS as a boot-persistent endpoint, plus the three silent faults auditing it found' (#14) from feat/frank-tts-engine into main
none
2026-10-18 18:30:22 -07:00
Frank 6956189bb6 docs(frank-tts): the worklog wikilink must not straddle a line break
The vault link check reads [[Frank TTS Engine]] literally, so wrapping it made
the target "Frank\nTTS Engine" and the unresolved-link seam failed. Rewrapped,
not reworded.
2026-10-18 18:10:25 -07:00
Frank 3380b7da10 docs(frank-tts): worklog entry for the voice-stack audit
Three faults, all of the reporting-success-while-doing-nothing species: the
build that could not start (no adduser in the base), the warm pass that could
not warm (podman's --env-file keeps the quotes), and the image that would never
rebuild (cfhash over the Containerfile alone). Records the proofs, the two
behaviour seams, and what stays undone: nothing is deployed, #2 phase 1 stays
open until 7772 answers.
2026-10-18 18:10:02 -07:00
Frank 7178e1c789 docs(frank-tts): what makes a locally built voice image stale, and what the warm text must survive
Three pages, all from faults found this session:

- `LLM/deployment.md` gains "Locally built images": which images have no
  registry digest, what the `cfhash` skip is for, why it must cover every COPY'd
  file (the silent shape: build skipped, /health green, old code serving), and
  the base-image half of the law — the engine's `useradd`, because the pinned
  ubuntu:24.04 has no `adduser`.
- `Dankclaw/Frank TTS Engine.md` troubleshooting gains the two lines an operator
  actually needs: every voice cold on a fresh boot with a hand-run hook
  succeeding (= `WARM_TEXT` arrived quoted, `podman exec … env | grep WARM_TEXT`
  proves it), and a 502 from the façade beside a 200 `/health` from the engine
  (model loaded ≠ voices registered). The stale-warmer row is re-keyed to the
  new rebuild rule.
- `Dankclaw/Voice Packs.md` step 4 was wrong: it told the next agent to restart
  the engine *instead of* the warm hook because the hook "only registers packs it
  has not seen". It registers every pack on every run and the engine replaces a
  name it already holds — `tools/tts-server.cpp:148` at the pinned commit:
  "Re-registering a name replaces the previous entry." A restart is what an
  engine bump needs (the registry is memory-only), never worse, only slower.
- `Dankclaw/Frank TTS.md` deploy command re-keyed to `./scripts/deploy.sh
  dankclaw --service frank-tts`, the way it is actually deployed; the raw
  `deploy-remote.sh` form is plumbing.
2026-10-18 18:07:19 -07:00
Frank 59e021da2c fix(deploy): hash every file a locally built image COPYs, not just its Containerfile
build_compose_image skips `podman build` when the installed image's `cfhash`
label matches a hash taken from the checkout now — the skip is what keeps
frank-tts-engine's ten-minute recursive source clone off every re-deploy. The
hash covered the Containerfile ALONE, so any change to a file the image COPYs
was invisible: edit dankclaw/voiceservice/app.py or tts-engine-warm.sh, re-run
the deploy, and podman reports the existing image is fine. The redeployed
container runs the previous code, /health answers 200, and the e2e gate has no
way to see it. It bit the boot warmer twice this week.

Each case arm now names its own extra=(…) — the paths its Containerfile COPYs —
and the label hashes the Containerfile plus those. The two arms are kept apart
deliberately: the façade bakes app.py and tts-start.sh, the engine bakes
tts-engine-start.sh and tts-engine-warm.sh, and comfyui/loki/alloy have no extra
files, so their hash is unchanged and they do not rebuild on this commit.
Hashing rather than word-splitting because these are repo-relative paths under
a fixed checkout root.

The seam drives the real function against a stub podman and asserts observable
output — did the deploy ask podman to build, or did it skip? — in five steps:
first deploy builds, unchanged tree does not, editing the engine's COPY'd warm
hook rebuilds the ENGINE image and not the façade, editing app.py rebuilds the
façade. Reverted scripts/deploy-remote.sh to prove it red on the old hash: the
warm-hook step is exactly where it fails.
2026-10-18 18:06:43 -07:00
Frank d336bdaa66 fix(frank-tts): make the boot warm pass survive both readers of its env
`WARM_TEXT` reaches the container **with its quotes**. The unit reads the env
file through podman's `--env-file`, which takes the literal remainder of the
line — no shell parsing — while bash `source` strips one layer. The hook
interpolated the raw value into the synthesis body, so with the quotes intact
the JSON was invalid and every voice logged "registered, but the warm synthesis
failed" while a hand-run hook (bash shape) succeeded. Reproduced both ways
against a real `podman run --env-file`, not by reading the man page.

The hook now strips exactly one matched pair — only when both ends are the same
quote AND the inner content carries another quote or a backslash, so a value
that legitimately contains quotes is never mangled — and then JSON-escapes what
is left, which is what makes a sentence like `He said "hi\" once"` arrive as
itself instead of corrupting the body.

The seam gains a test that reads `scripts/frank-tts-engine.env` BOTH ways —
`source` and the way `podman --env-file` does — and asserts the engine receives
the same sentence either way, in both shapes. A test that only sources the file
cannot see this bug, which is how it survived the first deploy.
2026-10-18 18:06:06 -07:00
Frank d135ae7f02 fix(frank-tts): create the engine user with useradd — the base has no adduser
The runtime stage of Containerfile.engine ran `adduser --system …`. The pinned
ubuntu:24.04 base ships no adduser, so the build died at that RUN line with
crun's "executable file not found in $PATH".

Verified in the base itself on Dankclaw, 2026-10-18: `command -v adduser`
prints nothing, and the exact replacement line — `useradd --system --uid 10000
--home-dir /app --shell /usr/sbin/nologin tts` — answers `uid=10000(tts)
gid=999(tts)`. Same account as before: numeric uid/gid 10000, non-login shell,
no password entry. Its one warning ("uid 10000 is greater than SYS_UID_MAX
999") is cosmetic and is what keeps the uid equal to the façade's, so the
comment says so before someone "fixes" it by dropping --system.
2026-10-18 18:05:07 -07:00
Frank c1087f214e docs(frank-tts): plain wikilink to the merged voice-pack page 2026-10-18 17:44:43 -07:00
Frank 1312c29fe2 Merge remote-tracking branch 'origin/main' into feat/frank-tts-engine
# Conflicts:
#	serverAppConfigs-docs/index.md
#	serverAppConfigs-docs/log.md
2026-10-18 17:44:31 -07:00
Frank 8fefe6e1a9 feat(frank-tts): deploy wiring for the 7772 voice endpoint, and voice naming written down
Stage 2 of the voice stack. Repo state only: nothing deployed, no host touched,
no container started.

- `frank-tts` is now registered everywhere a dankclaw service has to be:
  `SERVICE_ORDER` (after `frank-tts-engine`, before the consumers), the planner's
  dankclaw set, `deploy-remote.sh`'s allowlist + dispatch + a
  `deploy_frank_tts` handler, and `deploy.sh`'s dispatch regex — plus the
  `build_compose_image` arm naming `dankclaw/voiceservice/Containerfile`, without
  which the build silently returns 0 and the unit's `Pull=never` tag never
  exists.
- The unit joins TWO networks, in order: `dankclaw_default` first (default route,
  where consumers and open-notebook live), then the voice bridge. `TTS_UPSTREAM`
  moves from `host.containers.internal:8880` to `http://frank-tts-engine:8880` —
  the actual fix for a shape that could never work, since a bridge container
  cannot reach a host-loopback bind and the engine publishes nothing at all.
- `Wants=`/`After=` on the engine, never `Requires=`: the `llama-tts-cli` rollback
  backend needs no upstream, so a broken engine may delay this endpoint but must
  not take its unit down.
- The handler installs the engine's quadlet pair too, because a referenced
  `Network=` file the generator cannot find means no façade unit is produced at
  all, and `--only frank-tts` has no other way to get it. It creates
  `/home/dank/voices` as the deploy user (rootless podman would auto-create a
  missing bind source root-owned, which reads as "no voices"), and gates on
  podman's recorded health.
- Voice naming is now written down where it can be found: a voice is a lowercase
  directory name matching `^[a-z0-9][a-z0-9._-]*$`, at most 64 chars. That shape
  is the intersection of two rules that disagree — the shim would accept mixed
  case, the engine lowercases names — so a mixed-case pack resolves on one side
  only, and the boot warmer refuses it loudly. `narrator1/2/3` are load-bearing
  for open-notebook.
- Homepage card for the pair, `wip: down` and hidden, probing `127.0.0.1:7772`;
  the engine gets no card because it publishes nothing and could never be probed.
- **No Prometheus job, deliberately.** No `/metrics` exists on the shim, this
  Prometheus 3 refuses HTML/JSON bodies (the existing `comfyui` and `services`
  probes are documented down for exactly that), and `LlmEndpointDown` is scoped
  `up{job=~"frank-.*"}` — so a `/health` job would page on Signal for a healthy
  endpoint. Recorded in the page instead of quietly skipped.
- Vault: new [[Frank TTS]] page linked from the index; [[Frank TTS Engine]] now
  points at it for the pair's public signal; log entry for the day.
- Seam grows to 25 tests: the generator's rendering of the pair (both
  attachments, first-network order, one publish, Wants-not-Requires, bounded
  per-unit blocks), the env file's upstream checked as a network JOIN rather than
  a string, planner separation and order, the handler's steps, the card's gating,
  routing that selects both seams for either deployable.
2026-10-18 17:43:08 -07:00
dank d421cb69cf Merge pull request 'feat(voice-packs): the builder that turns a crop into a deployable voice pack' (#13) from feat/voice-pack-builder into main 2026-10-18 17:41:11 -07:00
Frank 8d76854a1e feat(frank-tts-engine): the arm A engine as a boot-persistent quadlet unit
Repo state only: nothing deployed, no host touched, no container started.

- `frank-tts-engine` on Dankclaw: `qwentts.cpp` commit
  5916bf7c79dbb4e66fc059cdc507a79a23a28bb4, resident `tts-server`, 1.7 B base
  talker Q4_K_M + Q8_0 tokenizer, container port 8880. It is deliberately NOT a
  fleet endpoint — `frank-tts` on 7772 is the endpoint consumers address.
- Publishes NOTHING. A host-loopback publish would have been useless to the
  façade (netavark DNATs published ports only, so a bridge container cannot
  reach a host loopback bind — the shape that 502'd hermes) and would have been
  the one LAN-reachable surface of an unauthenticated synthesiser. Both voice
  containers sit on one committed quadlet network, `frank-tts-engine.network`
  pinned to 10.89.5.0/24 (next free /24 after open-notebook's 10.89.4.0/24), and
  peers address the engine by container name. Diagnostics and the deployer's
  health gate run inside the netns (`podman exec`, `wait_container_healthy`).
- Cold-voice law honoured in the unit: the first request for a registered voice
  decodes its whole reference to snapshot codec state (6.2-8.0 s, first audio
  byte 5 530 ms), later ones restore in 0.5 ms, and the registry is memory-only.
  `ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh`
  with `TimeoutStartSec=900`; the `-` prefix keeps a failed warm from
  crash-looping the endpoint under `Restart=always`. It logs cold=/warm= per
  voice, so the under-a-second second request is proven on every boot.
- Thread law recorded: `tts-server` has no `--threads`; src/backend.h uses
  hardware_concurrency()/2, so a cpuset pin silently changes the thread count.
- Image builds from the pinned commit with the proven line (GGML_BLAS=OFF
  forced by this host, every GPU backend off, -j 8, no -DGGML_NATIVE because
  ggml defaults to -march=native for the building host). Two stages; the runtime
  stage COPYs both the entrypoint and the warmer, because a container that
  publishes nothing is reached with `podman exec`, not a port.
- `scripts/download-frank-tts-engine-qwentts.sh` fetches the two GGUFs with byte
  and LFS-oid checks (talker 1 219 245 248 B, tokenizer 291 150 624 B); the
  deployer gates on --verify-only instead of pulling 1.4 GiB mid-deploy.
- Registered in SERVICE_ORDER, DANKCLAW_SERVICES, the deploy-remote.sh dispatch,
  its pre-deploy requirement, deploy.sh's regex, the planner's path map (the
  whole `dankclaw/voiceservice/` tree mapped to nothing before this), and
  test-changed.sh ABOVE the `dankclaw/*.container` case whose `*` crosses `/`.
- New seam tests/frank_tts_engine.bats (19): the Quadlet generator's own output
  for network/publish/boot/warm shape, entrypoint argv driven from the tracked
  env file, the warmer against a stub tts-server over a real socket
  (registration keys, ref_text absent, late engine, hung synthesis, name
  refusal), the fetcher's verifier, the pinned cmake line, and both routing
  tables. Suite: 422 passed.
- Vault: [[Frank TTS Engine]] linked from the index; log.md entry for the day.
2026-10-18 17:26:50 -07:00
Frank 2427fc448f docs(voice-packs): runbook, index link, and the log entry
The page a new operator needs: what the three required files are and who
reads each one, how a pack is built (with the extractor's usage text
verbatim, because the flags are not guessable and there is no -o), the
10-second gate and where it fires twice, how to add a fourth voice
including the warm hook that registers a pack without restarting the
engine, and the engine-bump procedure for when the artifact sizes move.
Plus the host census as of 2026-10-18 — the trial packs carry no
artifacts and their manifest is the trial's own schema, so the builder's
first real run is a --force migration — and the three gaps left open on
purpose: no curated ICL transcript, no 16 kHz variant, no loudness pass.

Linked from the Dankclaw services list; log entry appended oldest-first.
2026-10-18 17:25:34 -07:00
Frank 3824d03ac5 Merge remote-tracking branch 'origin/main' into feat/voice-pack-builder 2026-10-18 17:18:31 -07:00
Frank 042003a969 feat(voice-packs): tracked tunables, the bats seam, and its wiring
scripts/voice-packs.env carries the decisions worth a diff and nothing
else: the pack list, each voice's source file, crop offset, crop length
and pinned source digest, the extractor path with its engine repo and
commit, and the artifact arithmetic. Paths and numbers only — the pack
directories themselves stay host state under /home/dank/voices.

tests/voice_pack_unit.bats pins behaviour, from inputs to observable
output: a crop inside the first ten seconds exits nonzero naming the
invariant and creates nothing, a legal one builds the whole pack, the
reference comes out 24 kHz mono inside 18-25 s (ffprobe, on a locally
generated tone), the manifest round-trips through jq with offsets and a
sha256 that matches the source bytes, a re-run without --force leaves
every byte untouched, --force retires rather than deletes, --dry-run
needs no extractor at all, and a stub extractor that changed its artifact
size cannot publish a voice. No engine binary, no GPU, no network, no
fleet host: qwen-codec is stood in by a stub with the real tool's I/O
shape, the way frank_tts_unit.bats drives its backends.

test-changed.sh routes both new paths to that seam instead of letting
them land on the generic scripts/* syntax fallback.
2026-10-18 17:18:30 -07:00
Frank 2242fcfec0 feat(voice-packs): builder — crop, pinned artifacts, manifest, 10 s gate
Turns a source recording plus a crop offset into the pack frank-tts
resolves by name: reference.wav (24 kHz mono pcm_s16le, 18-25 s), the
engine's own reference.spk / reference.rvq made from it, and manifest.json
provenance (source path + sha256, crop start/end, sample rate, duration,
tool, engine commit, voice name).

The extractor is qwen-codec from the pinned qwentts.cpp tree, and the
flags are its own usage text, not a guess: --model <codec gguf>, --talker
<base talker gguf>, -i <wav>, optional --device; there is no -o, so it
names both artifacts next to the input with the extension swapped, which
is why the pack's WAV is staged under its final stem.

The 10-second source-material invariant is enforced, not annotated: an
offset inside the forbidden region exits 1 before a directory is created,
and the gate is asserted a second time from the manifest that was just
written, together with the recorded sha256. Artifact sizes are checked
against the codec's own arithmetic (8,192 B x-vector; 22 B per 12.5 Hz
frame), so an engine bump fails the build instead of shipping a pack the
engine rejects at synthesis time — and the crop is refused up front when
it is too short to fill the .rvq window.

reference.wav is the file the façade gates on, so it lands last, after
the artifacts and the manifest: a voice name never exists on disk without
its artifacts behind it. Nothing is ever deleted — an existing pack is
refused unless --force, which retires it whole to .AGENTTRASH/ — and
--dry-run runs every step except the extractor, so the seam can test the
gate without the engine.
2026-10-18 17:18:21 -07:00
dank e13f875c94 Merge pull request 'docs(tts): record the measured CPU engine A/B behind frank-tts' (#10) from docs/tts-engine-trials into main 2026-10-18 17:15:00 -07:00
Frank 1d082a7f1d Merge remote-tracking branch 'origin/main' into docs/tts-engine-trials
# Conflicts:
#	serverAppConfigs-docs/log.md
2026-10-18 16:54:13 -07:00
Frank 5c8ac208a0 Merge remote-tracking branch 'origin/main' into docs/tts-engine-trials 2026-10-18 16:51:01 -07:00
Frank dbc2c67bcb docs(tts): record the arm A engine decision and the loopback-publish law
Frank heard every audition pair and passed it, so likeness stopped being the
tiebreak and the engine was picked on the live-output criterion: arm A
qwentts.cpp 5916bf7, resident tts-server, 1.7 B talker, llama-tts-cli kept as
the rollback. Records the discriminators (first audio byte, warm RTF, spread),
the three consequences that are configuration (boot-warm every voice, no cpuset
pin, WAV-only registration), the two follow-ups left open on purpose, and the
corrected addressing law: a host-loopback publish is unreachable from a bridge
container, so the pair shares one quadlet network.
2026-10-18 16:51:01 -07:00
dank 1eb8b2199a Merge pull request 'feat(open-notebook): podcast pipeline on Dankclaw (blocked on frank-tts)' (#12) from feat/open-notebook-podcast into main 2026-10-18 16:50:47 -07:00
dank 0e4bb327da Merge pull request 'docs: attribution footer for machine-opened issues and PRs' (#11) from chore/pr-attribution-trailer into main 2026-10-18 16:46:00 -07:00
Frank e030486ce6 docs: pin cwd meaning in attribution footer
Mirror of command-center clarification: first dogfooding round stamped the
build worktree instead of the session's own working directory. Rule text
now says which one, and to note the build worktree in the body.
2026-10-18 16:41:21 -07:00
Frank d89ae23195 docs: attribution footer for machine-opened issues and PRs
Mirror of dank/command-center's rule (there: PR off docs/session-trailer-rule).
Every agent-opened issue/PR body ends with cwd + omp session id so machine-filed
artifacts trace back to the box and session that produced them.

Refs #11 (command-center tracker)
2026-10-18 16:32:59 -07:00
Frank 0e7936f94e docs(tts): record the measured CPU engine A/B behind frank-tts
Both candidate :8880 engines were measured on Dankclaw's CPU (no GPU,
no container, nothing deployed). arm A qwentts.cpp 5916bf7 resident at
RTF 0.55-0.57 warm / 0.67 ICL / 1.65 cold; arm B mainline llama-tts at
0.93-0.99 with a +-0.25 spread. Records the four configuration laws the
trial produced (boot-warm every voice, .spk+.rvq replay is byte-identical,
streaming != buffered, /v1/audio/voices rejects a lone spk_b64), the
likeness measurement's resolution limit, and re-keys the stale
8890/voiceshim shape on that page to frank-tts on 7772. The engine pick
stays open: the likeness metric cannot rank clones, so the audition clips
decide.
2026-10-18 16:19:51 -07:00
235 changed files with 8854 additions and 28136 deletions
+471 -1016
View File
File diff suppressed because it is too large Load Diff
+140
View File
@@ -0,0 +1,140 @@
# frank-tts-engine — the arm A TTS engine: qwentts.cpp's resident `tts-server`
# (the CPU binary that frank-tts on :7772 forwards to). The runbook is
# [[Frank TTS Engine]]; the fleet-facing endpoint is [[Frank TTS]].
#
# WHAT THIS IS. The upstream source is NOT vendored here: the image clones
# ServeurpersoCom/qwentts.cpp at one pinned commit and builds it. Rebuilding is
# therefore a network operation (github.com plus objects.githubusercontent.com
# for the `ggml` submodule), and build_compose_image in scripts/deploy-remote.sh
# skips it whenever this Containerfile's sha256 label already matches the built
# image — the same offline-safe idiom as loki/alloy/comfyui, so a re-deploy of
# an unchanged engine needs no network.
#
# THE PINNED COMMIT IS THE CONTRACT. 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4
# is the commit the 2026-10-18 CPU trial on Dankclaw measured (server-side TTFA
# 94-112 ms, warm RTF 0.55-0.67, run spread +-0.01 — the numbers that won the
# engine decision). Moving it means re-measuring, not just editing this line.
# `ggml` is a submodule of the repo (the only one; vendor/cpp-httplib and
# vendor/yyjson are ordinary directories in the tree), and the submodule commit
# is recorded in the parent tree, so cloning the parent commit recursively is
# reproducible without a second pin.
#
# THE BUILD LINE IS THE ONE PROVEN ON DANKCLAW, verbatim:
#
# cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF \
# -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON \
# -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF \
# -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF \
# && cmake --build build -j 8
#
# Every flag is load-bearing, so do not "tidy" it:
# GGML_BLAS=OFF is FORCED, not a preference — Dankclaw has libopenblas.so.0
# but no BLAS dev headers, so the upstream default (BLAS on) fails to
# configure. Upstream's own Dockerfile and buildcpu.sh both pass BLAS on,
# which is why this file does not copy them.
# GGML_CCACHE=OFF because a container build layer has no persistent ccache
# directory to hit; on, it only writes a useless one.
# GGML_OPENMP=ON is what makes the runtime need libgomp1 (installed below).
# GGML_BACKEND_DL=OFF / GGML_CPU_ALL_VARIANTS=OFF: upstream's images build
# every x86 ISA as a loadable backend so one image runs anywhere. This one
# is built on the machine that runs it, so the single native CPU backend is
# what was measured. That is also why no -DGGML_NATIVE is passed: ggml
# defaults it ON outside wasm/cross-compiles, i.e. -march=native for THIS
# host — correct while build host == run host, wrong if anyone ever builds
# this image elsewhere, which is why the fleet builds it on Dankclaw.
# CUDA/HIP/VULKAN/METAL/SYCL/RPC all OFF: this is a CPU endpoint (the trial
# ran on CPU; the box's GPUs belong to the text endpoints).
#
# BUILD SHARED LIBS, RUNTIME SMALL. BUILD_SHARED_LIBS defaults ON on Linux, so
# tts-server links libggml*.so rather than swallowing them; the runtime stage
# copies the binary and its shared objects out of the build tree and points
# LD_LIBRARY_PATH at them, exactly as upstream's own Dockerfile does (the RPATH
# baked at link time names the build directory, which does not exist here).
#
# BASE PINNED BY DIGEST, like every other image this fleet builds: the digest
# is the multi-arch INDEX of ubuntu:24.04 as resolved from Docker Hub on
# 2026-10-18; Dankclaw takes the amd64 entry. One base for both stages so there
# is exactly one thing to re-resolve.
#
# NO HEALTHCHECK IN THIS FILE. The probe lives in the Quadlet unit
# (HealthCmd=), which is the fleet pattern for anything with a shell — see the
# alertmanager/prometheus units, and the opposite case in dankclaw/loki, whose
# distroless image has no /bin/sh for the generator's CMD-SHELL wrapping.
# `curl` is therefore installed in the runtime stage, for that probe only.
ARG BASE_IMAGE=docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac
# ------------------------------------------------------------------ build
FROM ${BASE_IMAGE} AS build
RUN apt-get update -qq \
&& apt-get install -y -qq --no-install-recommends \
ca-certificates cmake g++ git make \
&& rm -rf /var/lib/apt/lists/*
# Checked out detached at a full SHA, then the recorded submodule. No branch
# name anywhere: a branch moves, this commit does not.
ARG QWENTTS_COMMIT=5916bf7c79dbb4e66fc059cdc507a79a23a28bb4
RUN git clone --no-checkout https://github.com/ServeurpersoCom/qwentts.cpp /src \
&& git -C /src checkout --detach "${QWENTTS_COMMIT}" \
&& git -C /src submodule update --init --depth 1 \
&& git -C /src rev-parse HEAD \
&& git -C /src submodule status
WORKDIR /src
RUN cmake -B build -DCMAKE_BUILD_TYPE=Release \
-DGGML_CCACHE=OFF \
-DGGML_BACKEND_DL=OFF \
-DGGML_CPU_ALL_VARIANTS=OFF \
-DGGML_OPENMP=ON \
-DGGML_BLAS=OFF \
-DGGML_CUDA=OFF \
-DGGML_HIP=OFF \
-DGGML_VULKAN=OFF \
-DGGML_METAL=OFF \
-DGGML_SYCL=OFF \
-DGGML_RPC=OFF \
&& cmake --build build -j 8
# ---------------------------------------------------------------- runtime
FROM ${BASE_IMAGE}
# libgomp1: the OpenMP runtime the build above links against. curl: the unit's
# HealthCmd. ca-certificates: so that curl can speak https if anyone ever
# points a probe at a URL with a scheme this host terminates elsewhere.
RUN apt-get update -qq \
&& apt-get install -y -qq --no-install-recommends \
ca-certificates curl libgomp1 \
&& rm -rf /var/lib/apt/lists/*
# Same unprivileged uid as the frank-tts façade (10000): the voice packs and the
# model directory are host state mounted read-only, and one uid across the pair
# means one thing to remember about what is world-readable.
#
# `useradd`, NOT `adduser`: this trimmed ubuntu:24.04 base ships no `adduser` at
# all (verified 2026-10-18 against this digest — `adduser` is absent from the
# base and is not pulled in by ca-certificates/curl/libgomp1 either), so an
# `adduser` line dies at image-build time with crun's "executable file `adduser`
# not found in $PATH". `useradd --system --uid 10000` succeeds and prints one
# cosmetic warning (`uid 10000 is greater than SYS_UID_MAX 999`); the uid is the
# point, and it is what matches the façade. The alpine façade Containerfile keeps
# its own busybox `adduser -D -u 10000` — different base, different tool, same uid.
RUN useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin tts
WORKDIR /app
COPY --from=build /src/build/tts-server /src/build/*.so* /app/
COPY dankclaw/voiceservice/tts-engine-start.sh /app/start.sh
# The boot warmer lives INSIDE the image because the engine publishes no port:
# the unit reaches it with `podman exec`, and the script needs the same curl,
# base64 and /voices the engine's own namespace has. It is a sibling of
# start.sh, not a second entrypoint — CMD stays the server.
COPY dankclaw/voiceservice/tts-engine-warm.sh /app/warm-voices.sh
RUN chmod 755 /app/start.sh /app/warm-voices.sh
# The binary's RPATH points inside the build tree; this is what makes it find
# the libggml*.so copied beside it.
ENV LD_LIBRARY_PATH=/app
USER tts
EXPOSE 8880
CMD ["/app/start.sh"]
+17 -6
View File
@@ -7,7 +7,10 @@ Consumers (the omp harness, OpenWebUI, open-notebook) speak the OpenAI
directory under ``$VOICES_DIR`` and ``reference.wav`` inside it is what makes
the name real. A name that has no reference is not a voice — the shim answers
400 with the list that is, instead of silently narrating in a fallback voice
(silent fallback is how a wrong-voice endpoint stays wrong for weeks).
(silent fallback is how a wrong-voice endpoint stays wrong for weeks). A request
that names NO voice at all takes ``$DEFAULT_VOICE``: that is a spoken choice in
the env file rather than a surprise in the code, and the journal marks it
``(default)`` so a name nobody asked for is never mistaken for one that was.
Two backends sit behind the one surface, chosen by ``TTS_BACKEND``:
@@ -70,7 +73,7 @@ class Config:
def __init__(self, voices_dir, backend, upstream, upstream_model, llama_bin,
llama_model, llama_mmproj, llama_lang, llama_threads, port,
max_chars, timeout):
max_chars, timeout, default_voice=""):
self.voices_dir = Path(voices_dir)
self.backend = backend
self.upstream = upstream
@@ -83,6 +86,7 @@ class Config:
self.port = port
self.max_chars = max_chars
self.timeout = timeout
self.default_voice = default_voice
@classmethod
def from_env(cls, environ=None):
@@ -100,6 +104,7 @@ class Config:
port=_int(env, "SHIM_PORT", 7772),
max_chars=_int(env, "SHIM_MAX_CHARS", 15000),
timeout=float(_int(env, "SHIM_TIMEOUT", 600)),
default_voice=(env.get("DEFAULT_VOICE") or "").strip(),
)
@@ -271,8 +276,13 @@ class SpeechHandler(BaseHTTPRequestHandler):
"endpoint serves wav and mp3" % fmt))
return
# `voice_id` is the alias the omp harness forwards for `voice`.
voice = body.get("voice") or body.get("voice_id")
# `voice_id` is the alias the omp harness forwards for `voice`. A request
# that names nothing takes the declared default — the harness has no
# setting that feeds a voice, so voice-less calls are the normal shape
# there, not a client bug.
requested = body.get("voice") or body.get("voice_id")
defaulted = not requested and bool(cfg.default_voice)
voice = requested or cfg.default_voice
reference = resolve_voice(cfg.voices_dir, voice)
if reference is None:
available = list_voices(cfg.voices_dir)
@@ -297,8 +307,9 @@ class SpeechHandler(BaseHTTPRequestHandler):
content_type = MP3_CONTENT_TYPE if fmt == "mp3" else WAV_CONTENT_TYPE
self._bytes(200, audio, content_type)
self.log_message("speech voice=%s format=%s chars=%d -> %d bytes %s",
voice, fmt, len(text), len(audio), content_type)
self.log_message("speech voice=%s%s format=%s chars=%d -> %d bytes %s",
voice, " (default)" if defaulted else "", fmt,
len(text), len(audio), content_type)
# -- backends ----------------------------------------------------------
def _speech_http(self, cfg, model, text, voice, reference):
@@ -0,0 +1,115 @@
# Quadlet unit: frank-tts-engine (rootless, boot-persistent).
#
# Install path on dankclaw: ~/.config/containers/systemd/frank-tts-engine.container
# Unit name: frank-tts-engine.service (container name: frank-tts-engine)
#
# WHAT IT IS. The arm A TTS engine: qwentts.cpp (pinned commit
# 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4) built into an image by
# dankclaw/voiceservice/Containerfile.engine, running its resident `tts-server`
# with the 1.7 B base talker Q4_K_M and the Q8_0 12 Hz tokenizer loaded ONCE at
# startup. It is deliberately NOT a fleet endpoint: the endpoint consumers
# address is **frank-tts on 7772** ([[Frank TTS]]), which resolves voice names,
# re-encodes to mp3 and owns the timeout budget. Runbook: [[Frank TTS Engine]].
#
# WHY THIS ENGINE (Frank's call, 2026-10-18, criterion "whatever is better for
# live output"): server-side time-to-first-audio 94-112 ms (86-451 ms end to
# end through the façade), streaming that stays ahead of real time at warm RTF
# 0.55-0.67, and a run-to-run spread of +-0.01 — against the mainline CLI path's
# 0.93-0.99 at +-0.25, which crosses 1.0 often enough to stall a live stream.
# The A/B that decided it is "Decision: arm A for live output" in
# [[Local TTS and Voice Cloning]].
#
# IMAGE. Built locally by deploy-remote.sh (build_compose_image), like
# loki/alloy/frank-tts. `Pull=never` plus the `localhost/` prefix are required:
# this host's quadlet path cannot resolve a locally built tag from local storage
# otherwise. The build is a network operation (it clones the pinned commit) and
# is skipped when the Containerfile hash label already matches.
#
# NETWORK, AND WHY IT PUBLISHES NOTHING. The container sits on exactly one
# network: the private `frank-tts-engine.network` (10.89.5.0/24). The façade
# joins the same network and dials `http://frank-tts-engine:8880` — the
# container DNS name, on the network that owns it. There is no `PublishPort` of
# any kind, not even `127.0.0.1:8880`, and the reason is the lesson that 502'd
# hermes: netavark DNATs published ports only, so a host-loopback publish is not
# a path to a peer container — the façade could never have used it, and a bridge
# container cannot reach the host's own loopback bind either (see
# [[Container Network Namespaces]]). A publish would therefore buy exactly one
# thing: an extra reachable surface on an engine that has no authentication, on
# a box whose CPU is also carrying the text endpoints. Diagnostics do not need
# it either — `podman exec frank-tts-engine curl http://127.0.0.1:8880/health`
# speaks to the server from inside its own namespace, which is where the address
# is valid.
#
# MEMORY / NO GPU. Peak RSS measured 5.7 GiB with both GGUFs resident
# (2026-10-18 trial); no memory limit is set because the box has the room and a
# limit would convert a busy moment into an OOM kill mid-utterance. No
# /dev/kfd, no /dev/dri, no keep-groups: every GPU backend is compiled out of
# the image, the endpoint is CPU-only by decision, and the GPUs on this host
# belong to the text endpoints.
#
# THE VOICE MOUNT IS READ-ONLY AND IS FOR THE WARMER, NOT THE ENGINE. A voice
# becomes known to the engine only through POST /v1/audio/voices, and the
# registration is MEMORY ONLY — it does not survive a restart, which is exactly
# why the warm hook below runs on every start. That hook runs INSIDE this
# container (see ExecStartPost), so it reads the packs at /voices; `tts-server`
# itself never opens the directory, it receives the reference audio as base64 in
# the registration body. The façade mounts the same host directory read-only for
# its own name list, so one directory stays the single source of a voice name.
#
# REGISTRY LAW, THE REASON EXECSTARTPOST EXISTS. The voice registry is memory
# only: a restart forgets every voice, and re-registering does not warm one —
# only synthesizing does — so a boot without this hook leaves the façade
# answering 400 for every name. The per-voice cold penalty is real but small, and
# not the trial's 6.2-8.0 s: measured on the built container 2026-10-18 against an
# empty registry, the first synthesis of a freshly registered voice cost 1,883 ms
# against 1,226-1,283 ms for the requests after it. The warm pass runs serially
# (two short syntheses per voice, ~5 s each) and `TimeoutStartSec=900` keeps a growing library from
# tripping systemd's default 90 s start deadline. The leading `-` on
# ExecStartPost is load-bearing: with `Restart=always`, a warmer that exits
# non-zero (a pack missing its reference.wav, the engine still loading past the
# warmer's own patience) would restart the whole container and crash-loop the
# endpoint forever. A failed warm must be a journal line, not a restart.
#
# The hook is `podman exec`, not a host script, because there is no address the
# host could dial: this container publishes nothing (above). Exec-ing into it
# also means the warmer inherits this unit's EnvironmentFile, so every knob it
# has lives in scripts/frank-tts-engine.env beside the engine's own, and it sees
# the same /voices and 127.0.0.1:8880 the engine does.
#
# HEALTH. `GET /health` answers `{"status":"ok"}` from the HTTP thread, not the
# synthesis queue, so a busy engine still reads healthy and a dead one does not.
# HealthStartPeriod covers the first model load. The command is argument-only
# and needs /bin/sh — the generator wraps HealthCmd as CMD-SHELL — so the image
# installs curl (see the opposite case in dankclaw/loki, whose distroless base
# has no shell).
[Unit]
Description=frank-tts-engine (arm A qwentts.cpp TTS engine, internal :8880)
After=network-online.target
Wants=network-online.target
[Container]
ContainerName=frank-tts-engine
Image=localhost/franksdank/frank-tts-engine:latest
Pull=never
Network=frank-tts-engine.network
# No PublishPort, at all: the only reachable address this server has is the
# bridge address the façade resolves by name. Diagnostics use `podman exec`.
EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts-engine.env
Volume=/home/dank/models/Qwen3-TTS-GGUF:/models:ro
Volume=/home/dank/voices:/voices:ro
HealthCmd=curl -fsS http://127.0.0.1:8880/health
HealthInterval=30s
HealthTimeout=10s
HealthRetries=3
HealthStartPeriod=120s
[Service]
Restart=always
RestartSec=10
# Model load (tens of seconds) plus two short syntheses per registered voice.
TimeoutStartSec=900
ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh
[Install]
WantedBy=default.target
@@ -0,0 +1,52 @@
# Quadlet network: the private bridge of the voice pair (rootless).
#
# Install path on dankclaw: ~/.config/containers/systemd/frank-tts-engine.network
# Unit name: frank-tts-engine-network.service (podman network name:
# systemd-frank-tts-engine — quadlet prefixes "systemd-", so probe the runtime
# name, never the filename; see [[Container Network Namespaces]]).
#
# WHY A PRIVATE BRIDGE FOR A PAUSE THAT LOOKS PUBLIC-FACING. frank-tts (:7772,
# published, the one endpoint consumers address) and frank-tts-engine (:8880,
# never published, the arm A qwentts.cpp server behind it) could both have
# stayed on the fleet-wide dankclaw_default and resolved each other by name.
# They are split off anyway, for two reasons:
#
# 1. THE HOST-LOOPBACK TRAP. The phase-1 shape in `scripts/frank-tts.env` still
# names `host.containers.internal:8880` as the upstream, on the assumption the
# engine runs on the host and publishes `127.0.0.1:8880`. That cannot work:
# netavark DNATs published ports only, a published port is not on any podman
# bridge, and a container cannot reach the host's own loopback bind — the same
# shape that 502'd hermes, and the reason the signal-relay hop needs a host LAN
# bind instead ([[Container Network Namespaces]], [[Homepage Status Probes]]).
# Putting both containers on one committed network removes the guess: the
# façade dials the engine's container DNS name on the network that owns it, and
# the engine publishes nothing at all.
#
# 2. THE ENGINE HAS NO AUTH AND NO QUEUE LIMIT WORTH THE NAME. `tts-server`
# answers anyone who can open a socket, and at `--max-batch 1` a second
# caller simply waits behind the first. Left on dankclaw_default every
# container on that net — open-webui, hermes, the *arr proxies — could drive
# a CPU synthesiser that occupies every other core on a box that also serves
# two text endpoints. On this bridge the only dialer is frank-tts: the engine
# publishes no port, so not even the host has an address to reach it at, and
# diagnostics go through `podman exec` instead.
#
# Kernel bridge, not slirp4netns: the 2026-08-22 open-webui incident (a dead
# per-container userspace helper left a container that accepted TCP and reset
# every request for two days, logging nothing). A bridge has no such helper.
#
# Subnet registry on dankclaw (rootless):
# 10.88.0.0/16 default "podman" bridge
# 10.89.0.0/24 gitea (systemd-gitea)
# 10.89.1.0/24 compose stack (dankclaw_default)
# 10.89.2.0/24 searxng (systemd-searxng)
# 10.89.3.0/24 open-webui (systemd-open-webui)
# 10.89.4.0/24 open-notebook (systemd-open-notebook)
# 10.89.5.0/24 voice pair (systemd-frank-tts-engine) <- this file
# Pinned so the two container IPs stay deterministic across reboots, like the
# other pinned nets; 10.89.5.0/24 was the next unclaimed /24 in this registry.
[Network]
Driver=bridge
Subnet=10.89.5.0/24
Gateway=10.89.5.1
@@ -9,9 +9,9 @@
# /home/dank/voices holding reference.wav; the shim refuses any name without
# one (400 + the list that does exist) rather than narrating in a fallback
# voice nobody chose. The engine behind it is a separate decision —
# TTS_BACKEND in scripts/frank-tts.env picks between forwarding to an
# OpenAI-shaped upstream on host :8880 and running a local CLI binary per
# request.
# TTS_BACKEND in scripts/frank-tts.env picks between forwarding to the
# OpenAI-shaped engine (`frank-tts-engine`, [[Frank TTS Engine]]) and running a
# local CLI binary per request.
#
# IMAGE. Built locally from dankclaw/voiceservice/Containerfile by
# deploy-remote.sh (build_compose_image), like loki/alloy: python 3.12-alpine
@@ -19,16 +19,27 @@
# Pull=never and the localhost/ prefix are required — this host's quadlet
# path cannot resolve a locally built tag from local storage otherwise.
#
# NETWORK. Bridge (dankclaw_default), NOT host: nothing here needs a shared
# loopback namespace, and the comfyui/loki/authentik units are bridge too.
# Consequences both ways: consumers reach it on the published 7772, and the
# shim reaches the host-side TTS engine through host.containers.internal
# (never the LAN IP — a bridge netns cannot route to the host's own address
# for every bind shape, and the fleet's law is the gateway name). NOTE for
# whoever starts the :8880 engine: host.containers.internal reaches the host
# network namespace, so the engine must bind an address that namespace
# accepts; a 127.0.0.1-only bind is not it (the 2026-09-17 signal-relay hop
# needed the host LAN bind for the same reason).
# NETWORK. TWO attached networks, and the order is the one the open-notebook
# pair uses: `dankclaw_default` first (the default route, and where consumers
# live — they reach the published 7772, and open-notebook dials
# `http://frank-tts:7772/v1` by container name), then the voice pair's private
# bridge, `frank-tts-engine.network`, where the arm A engine lives. The shim's
# upstream is therefore the engine's container DNS name
# (`http://frank-tts-engine:8880`, scripts/frank-tts.env) and NOT
# host.containers.internal: netavark DNATs published ports only, so a bridge
# container cannot reach a host-loopback bind — the shape that 502'd hermes —
# and the engine publishes nothing at all. That file MUST be installed beside
# this unit (the engine's handler does it; the façade handler re-installs it for
# `--only frank-tts`), because with a referenced Network= file missing the v1
# generator refuses to produce this container unit at all.
#
# ORDERING. Wants+After on frank-tts-engine.service, not Requires: the façade
# has a second backend (`TTS_BACKEND=llama-tts-cli`) that needs no upstream, and
# an engine that will not start must not take this endpoint's unit down with it.
# When both boot, this comes up after the engine, whose ExecStartPost warm pass
# is part of its start — so the first request of the day meets a REGISTERED voice
# (the registry is memory-only) rather than a 400, and skips the ~0.6 s penalty
# the first synthesis of an unwarmed voice costs.
#
# VOICES. /home/dank/voices is host state, never repo state, and is mounted
# READ-ONLY: this endpoint has no business writing voice packs. Because the
@@ -41,14 +52,16 @@
# carries them deliberately.
[Unit]
Description=frank-tts (cloned-voice OpenAI-compatible TTS facade)
After=network-online.target
After=network-online.target frank-tts-engine.service
Wants=network-online.target
Wants=frank-tts-engine.service
[Container]
ContainerName=frank-tts
Image=localhost/franksdank/frank-tts:latest
Pull=never
Network=dankclaw_default
Network=frank-tts-engine.network
PublishPort=7772:7772
EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts.env
Volume=/home/dank/voices:/voices:ro
+89
View File
@@ -0,0 +1,89 @@
#!/usr/bin/env bash
# tts-engine-start.sh — the entrypoint of the frank-tts-engine image
# (dankclaw/voiceservice/Containerfile.engine). It exists for one reason: the
# knobs of the arm A engine are command-line flags, while the fleet's knobs
# live in scripts/frank-tts-engine.env and reach the container through
# EnvironmentFile=. This is the translation layer, and nothing else — no
# supervision, no retries, no logging of its own. PID 1 stays the server, so
# `podman stop` and the unit's Restart=always drive it directly.
#
# THE FLAGS THAT ARE NOT HERE, and must stay absent:
#
# --threads / -t does not exist in this build. src/backend.h computes the
# CPU thread count as std::thread::hardware_concurrency()/2
# (the comment there: GEMM shares SIMD units across
# hyperthreads), and there is no flag to override it. So a
# CPU affinity pin (CPUQuota=, taskset, a cpuset) silently
# changes the engine's thread count instead of just its
# placement — pinning is deliberately not done anywhere in
# this stack. See the env file header for the same warning
# aimed at the operator.
# --device left at its default. Every GPU backend is compiled out of
# the image, so the default device pick can only land on
# CPU; naming a device would add a way to be wrong.
#
# Everything defaults to the shape the 2026-10-18 trial measured, so an empty
# environment still boots the decided configuration: 1.7 B talker Q4_K_M with
# the Q8_0 tokenizer, one request at a time, on the private bridge.
set -euo pipefail
# The binary path. Overridable for one reason: the bats seam runs a stub
# instead of a 1.2 GiB model. Operators never set it — the image ships exactly
# one binary at /app/tts-server.
: "${QWENTTS_BIN:=/app/tts-server}"
# Paths are CONTAINER paths (the host directories they come from are named in
# scripts/frank-tts-engine.env and mounted by the Quadlet unit).
: "${QWENTTS_MODEL:=/models/qwen-talker-1.7b-base-Q4_K_M.gguf}"
: "${QWENTTS_CODEC:=/models/qwen-tokenizer-12hz-Q8_0.gguf}"
# Reported by GET /v1/models, and what scripts/frank-tts.env's
# TTS_UPSTREAM_MODEL names. A model id, not a filename: the file name would
# leak a quantization choice into every consumer's config.
: "${QWENTTS_ALIAS:=frank-tts-engine}"
# 0.0.0.0 inside the container is not "public". tts-server defaults to
# 127.0.0.1 (its own usage prints `--host <ip> Listen address (default:
# 127.0.0.1)`), and inside a container netns a loopback bind is reachable by
# nothing outside that namespace — the façade dials this container by its bridge
# address on the private voice network, which is why the address has to be
# broad. The unit PUBLISHES NOTHING (see
# dankclaw/voiceservice/quadlet/frank-tts-engine.container: no PublishPort at
# all, not even 127.0.0.1:8880), so the only namespaces that can reach
# 0.0.0.0:8880 are that bridge and this container's own loopback — and its own
# loopback is exactly where the boot warmer and the health probe speak.
: "${QWENTTS_HOST:=0.0.0.0}"
: "${QWENTTS_PORT:=8880}"
# The language label used when a request omits one. `auto` is the upstream
# default and what the trial ran.
: "${QWENTTS_LANG:=auto}"
# One request at a time. The server-side queue is a serialised FIFO at this
# setting, which is what keeps the measured per-voice timings true; batching
# on a CPU build splits the same cores between two syntheses and the warm RTF
# margin (0.55-0.67 against real time 1.0) disappears. The engine's own
# default is also 1; it is written here because it is a decision, not an
# accident.
: "${QWENTTS_MAX_BATCH:=1}"
# Fail loudly BEFORE binding, so the journal says which artifact is missing
# instead of the unit reporting a container that starts and dies. This is the
# self-healing path: Restart=always re-runs it every RestartSec, and the same
# sentence repeats until the operator fetches the weights
# (scripts/download-frank-tts-engine-qwentts.sh).
for artifact in "$QWENTTS_MODEL" "$QWENTTS_CODEC"; do
if [[ ! -r "$artifact" ]]; then
echo "frank-tts-engine: cannot read ${artifact}; run scripts/download-frank-tts-engine-qwentts.sh on the host (the /models mount is read-only and holds the two GGUF files)" >&2
exit 1
fi
done
exec "$QWENTTS_BIN" \
--model "$QWENTTS_MODEL" \
--codec "$QWENTTS_CODEC" \
--alias "$QWENTTS_ALIAS" \
--host "$QWENTTS_HOST" \
--port "$QWENTTS_PORT" \
--lang "$QWENTTS_LANG" \
--max-batch "$QWENTTS_MAX_BATCH"
+222
View File
@@ -0,0 +1,222 @@
#!/usr/bin/env bash
# tts-engine-warm.sh — the boot warmer of the arm A TTS engine, shipped inside
# the frank-tts-engine image at /app/warm-voices.sh
# (dankclaw/voiceservice/Containerfile.engine).
#
# WHO RUNS IT. Two callers, same binary:
#
# 1. The Quadlet unit, as
# `ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh`
# — after every start of the container, including every boot.
# `podman exec` rather than a host script because the engine publishes no
# port at all, so there is no address the host could dial; and because an
# exec inherits the unit's EnvironmentFile, which is what puts every knob of
# this script in scripts/frank-tts-engine.env next to the engine's own.
# 2. scripts/deploy-remote.sh's deploy_frank_tts_engine, once per deploy, so a
# voice that did not warm becomes a deploy result instead of only a journal
# line (the unit's `-` prefix deliberately cannot fail the unit).
#
# WHY IT EXISTS AT ALL (the registry law). The voice registry is MEMORY ONLY: a
# restart forgets every voice, and re-registering a voice does not warm it — only
# synthesizing does. A boot that registers nothing therefore leaves frank-tts
# answering 400 for every name, so this script registers every pack it finds. The
# per-voice cold penalty is real but small, and smaller than the trial reported:
# measured on the deployed build 2026-10-18 against an empty registry, the FIRST
# synthesis of a freshly registered voice cost 1,883 ms against 1,226-1,283 ms for
# the requests after it — about 0.6 s, not the trial's 6.2-8.0 s. Each voice gets
# two identical requests so the journal carries both numbers, `voice <name>
# cold=…ms warm=…ms`; cold ≈ warm is the expected shape, and a large gap is what
# would mean the snapshot path is broken.
#
# THE VOICES. /voices — the host directory /home/dank/voices mounted read-only by
# the unit, the same packs frank-tts mounts for its own name list. A pack is a
# directory that contains reference.wav; everything else in it (manifest.json, the
# engine's reference.spk / reference.rvq extraction, a 16 kHz variant) is not read
# here. The registration payload is `{"name": <dir>, "wav_b64": <base64 of
# reference.wav>}` with NO ref_text, which selects the embedding-only (x-vector)
# voice — the shipped default, warm RTF ~0.57-0.63 against ICL's 0.67, and the one
# this fleet can produce honestly because it has no trustworthy transcript for the
# audition crops. Adding ref_text (with the codes) would flip that voice to
# in-context-learning mode: better fidelity on a good transcript, slower, and only
# worth it for batch work like a podcast render. That is a per-voice opt-in this
# script deliberately does NOT exercise, and the door stays open: a pack that
# wants ICL needs its text in the pack plus one key in the payload below.
#
# A lone `spk_b64` is a 400 (src/tts-server.h:480 — it is `wav_b64` or BOTH
# `spk_b64` and `rvq_b64`), which is why the WAV travels and the .spk stays home.
#
# NOTHING HERE IS A SECRET, and nothing is logged that could be: the audio bytes
# are discarded, the synthesized text comes from the env file, and the reference
# clip is base64'd into a request body that is written to a temporary file and
# handed to curl as `@file` — never echoed, and never placed on a command line,
# because of the size law explained at the mktemp below.
set -euo pipefail
# The engine's own address, from inside its namespace. Loopback is valid HERE:
# this script runs in the container, next to the server. There is deliberately no
# host-visible address to use instead — the unit publishes nothing, because a
# host-loopback publish would be unreachable from the façade anyway (netavark
# DNATs published ports only) while adding a surface on an unauthenticated
# server.
: "${ENGINE_URL:=http://127.0.0.1:8880}"
# Container path; the host directory behind it is named in the Quadlet unit.
: "${VOICES_DIR:=/voices}"
# How long to wait for the engine to answer /health. Model load dominates: the
# two GGUFs (1.2 GiB + 291 MB) are read and the context built before the HTTP
# thread binds, and a boot races every other unit on this box.
: "${HEALTH_WAIT:=300}"
# One short sentence, long enough to exercise the codec snapshot and short enough
# that warming a library of voices is seconds, not minutes. Never varied per run:
# the cold/warm numbers in the journal should mean the same thing every boot. The
# synthesis asks for `wav`, the buffered path — the streamed path (`pcm`) is
# different audio at the same seed, and `wav` is also what frank-tts asks for.
: "${WARM_TEXT:=Warm up, one two three.}"
# Per-request ceiling. A cold voice is 6-8 s by measurement; anything approaching
# this is a stalled engine and should be reported rather than waited on forever —
# systemd counts the whole warm pass against the unit's start deadline.
: "${REQUEST_TIMEOUT:=120}"
# The warm text goes into a JSON string literal by interpolation. Two steps make
# whatever the two readers of scripts/frank-tts-engine.env deliver arrive as the
# sentence itself:
#
# 1. Unwrap ONE shell-style quoting layer. bash (`set -a; source`) strips the
# quotes around a value with spaces; podman's `--env-file` does NOT — measured
# 2026-10-18 against the image's pinned base, `WARM_TEXT="…"` reaches the
# container WITH its quotes. Unwrapped only when the whole value is exactly one
# quoted string (starts AND ends with `"` and holds no quote inside), so a
# sentence that legitimately contains quotes is never mangled.
# 2. Escape the two characters that would corrupt or end a JSON string: a
# backslash, and a double quote.
#
# Without step 1 the body is `{"input":""Warm up…"",…}`, the engine's JSON parser
# rejects it, every voice stays cold, and the boot still reads healthy. Voice NAMES
# need neither step: the character class checked below admits no quote, no
# backslash and no control byte.
warm_text="$WARM_TEXT"
if (( ${#warm_text} >= 2 )) && [[ "$warm_text" == \"*\" ]]; then
inner="${warm_text:1:${#warm_text}-2}"
[[ "$inner" != *'"'* ]] && warm_text="$inner"
fi
warm_json="${warm_text//\\/\\\\}"
warm_json="${warm_json//\"/\\\"}"
# The registration body is a FILE, never an argument. The kernel caps a single
# command-line string at 32 pages — MAX_ARG_STRLEN = 131,072 bytes — and a shipped
# reference clip base64's to about 1.27 MB, so passing the body as one argument
# fails with E2BIG before curl is even executed. The first real boot of the unit
# (2026-10-18, the three shipped packs) proved it end to end: `/usr/bin/curl:
# Argument list too long` for every voice, reported as the misleading
# "registration failed (HTTP no response)". `mktemp` lands in the container's
# /tmp as the unprivileged `tts` user; one file for the whole pass, overwritten
# per voice, removed on any exit — it is container scratch, not host state.
body_file="$(mktemp -t frank-tts-warm.XXXXXX)"
trap 'rm -f "$body_file"' EXIT
fail=0
log() { printf 'frank-tts-engine-warm: %s\n' "$*"; }
# Wait for /health. It answers from the HTTP thread, so a 200 means the server is
# listening; it says nothing about voices, which is what the rest of this script
# is for.
ready=0
for ((i = 1; i <= HEALTH_WAIT; i++)); do
if curl -fsS --max-time 5 "${ENGINE_URL}/health" >/dev/null 2>&1; then
ready=1
break
fi
sleep 1
done
if (( ready == 0 )); then
log "engine did not answer ${ENGINE_URL}/health within ${HEALTH_WAIT}s; nothing warmed"
exit 1
fi
if [[ ! -d "$VOICES_DIR" ]]; then
log "no voice directory at ${VOICES_DIR}; nothing to warm (frank-tts will answer 400 with an empty voice list until packs are built)"
exit 0
fi
warmed=0
for dir in "$VOICES_DIR"/*/; do
[[ -d "$dir" ]] || continue
name="${dir%/}"
name="${name##*/}"
if [[ ! -f "${dir}reference.wav" ]]; then
continue # not a pack; frank-tts refuses this name too
fi
# The engine lowercases every voice name (tts_voice_name) while frank-tts
# matches the directory name exactly, so a mixed-case pack would work by
# accident on one side and not the other. Refuse it loudly instead. Pack
# names are also JSON string content here, so the character class is what
# keeps a directory name from writing structure into the request body.
if [[ ! "$name" =~ ^[a-z0-9][a-z0-9._-]*$ ]]; then
log "voice '${name}': name must be lowercase [a-z0-9._-] (the engine lowercases names; the shim does not) — skipped"
fail=1
continue
fi
# The audio enters through a redirection rather than a command substitution:
# the file is several hundred KB, and both a shell variable and an argument of
# that size are what this script used to die on.
{
printf '{"name":"%s","wav_b64":"' "$name"
base64 -w0 "${dir}reference.wav"
printf '"}'
} > "$body_file"
if ! code="$(curl -sS --max-time "$REQUEST_TIMEOUT" -o /dev/null -w '%{http_code}' \
-H 'Content-Type: application/json' --data-binary "@${body_file}" \
"${ENGINE_URL}/v1/audio/voices")" || [[ "$code" != "200" ]]; then
log "voice '${name}': registration failed (HTTP ${code:-no response}) — the shim will answer 400 for this name"
fail=1
continue
fi
# The cold pass. Its output is discarded; the point is the codec snapshot it
# leaves inside the engine.
start=$(date +%s%3N)
if ! curl -fsS --max-time "$REQUEST_TIMEOUT" -o /dev/null \
-H 'Content-Type: application/json' \
--data-binary "{\"input\":\"${warm_json}\",\"voice\":\"${name}\",\"response_format\":\"wav\"}" \
"${ENGINE_URL}/v1/audio/speech"; then
log "voice '${name}': registered, but the warm synthesis failed — first live request stays cold"
fail=1
continue
fi
cold=$(( $(date +%s%3N) - start ))
# The warm pass, same request. This is the number the fleet actually
# experiences, and it is what "the second request answers under a second" is
# measured against.
start=$(date +%s%3N)
if ! curl -fsS --max-time "$REQUEST_TIMEOUT" -o /dev/null \
-H 'Content-Type: application/json' \
--data-binary "{\"input\":\"${warm_json}\",\"voice\":\"${name}\",\"response_format\":\"wav\"}" \
"${ENGINE_URL}/v1/audio/speech"; then
log "voice '${name}': warm synthesis failed after a cold one that worked"
fail=1
continue
fi
warm=$(( $(date +%s%3N) - start ))
log "voice ${name} cold=${cold}ms warm=${warm}ms"
warmed=$(( warmed + 1 ))
done
# Failure first, emptiness second, and not the other way round: a pass where
# every voice failed also has `warmed == 0`, and reading that as "nothing to do,
# success" is how an endpoint that answers 400 for every voice name gets deployed
# on a Friday.
if (( fail )); then
log "${warmed} voice(s) warm, at least one failed (see the lines above)"
exit 1
fi
if (( warmed == 0 )); then
log "no voice with a reference.wav under ${VOICES_DIR}; engine is up but speaks nothing"
exit 0
fi
log "${warmed} voice(s) registered and warm"
@@ -0,0 +1,319 @@
# XDNA2 NPU inference runtimes on Linux (Strix Halo) — can any of them serve TTS?
Research slice, 2026-10-19. Worker: `NpuRuntimeLandscape`. Research only, no code.
Question: every way to execute model inference on an AMD XDNA2 NPU on Linux as of
2026-10, and whether any of them can serve a speech-synthesis model.
> **Status, same night — the spike that followed this slice settled it.** NO-GO on the
> NPU (the loader declines the graph, not the silicon); the actionable door is the gfx1151
> iGPU behind `-DGGML_VULKAN=ON`. Measured contract, exoneration chain and host gates:
> `tts-npu-spike-verdict-2026-10-19.md`. That page supersedes §8–§9 below.
Convention: vendor claims carry URL + the version/date they hold for.
**[host]** = Dankclaw state verified on the live machine 2026-10-19 (Main), not a
vendor claim. `[INFERENCE]` = my deduction from the cited artifacts.
## 0. Answers to the three questions Main ranked
**(1) A runtime that already runs TTS on XDNA2 today: it exists at AMD, not on our
host.** Ryzen AI Software **1.8.0** (docs `latest` == 1.8.0; released 2026-07-23)
lists `neutts-air` and `neutts-nano` as model type **TTS**, variant
*full-optimized* (NPU-only) — <https://ryzenai.docs.amd.com/en/latest/llm_list.html>;
release note "New TTS model support: neutts-air, neutts-nano" —
<https://ryzenai.docs.amd.com/en/latest/relnotes.html>. Pre-compiled artifacts are
published: `amd/neutts-air_rai_1.8.0_npu_4K` (arch `qwen2`, lastModified
2026-07-23, Apache-2.0, "AWQ / Group 128 / Asymmetric / BFP16 activations / UINT4
Weights") and `amd/neutts-nano_rai_1.8.0_npu_4K` (arch `llama`).
**Neither artifact is a TTS engine.** Both file lists are LM-only (`model.onnx`,
`genai_config.json`, BPE tokenizer, AIE-ML compiler metastate: `dd_metastate_*`
`.ctrlpkt/.super/.state`, `cache/txn_bins.zip`) with **no codec, no vocoder, no
reference-audio encoder**. [INFERENCE from HF file lists]
**(2) FastFlowLM: no TTS, and we cannot add one** — see §2. Halogen: no TTS entry
**[host]**. So "something installed already serves TTS on the NPU" is FALSE.
**(3) Our own ONNX graph compiled for the NPU on Linux:** possible for the plain
ONNX flows (they compile on Linux), **impossible for a new LLM/OGA model** — model
generation is Windows-only in 1.8.0 — and gated by an operator table where BF16
lacks Softmax/LayerNorm/ReLU/Gelu (see §4).
## 1. Landscape: every way to reach the XDNA2 array on Linux
| # | Path | License | Strix Halo + Linux, 2026-10 | Speech synthesis |
|---|------|---------|------------------------------|------------------|
| 1 | Ryzen AI SW → ONNX Runtime + Vitis AI / Ryzen AI EP; compile INT8 (`X2`) or BF16 (`VAIML`) ONNX at session creation | AMD proprietary (tarball) | supported: our device ID/rev is in AMD's STX table; compile+run works on Linux (§3, §4) | user-authored ONNX only; no TTS recipe/demo |
| 2 | Same EP driven by **OGA** (`onnxruntime-genai`), "full fusion" NPU-only LLM flow; runner = AMD `model_benchmark` / OGA C++ API | AMD + MS OGA | Linux: **vendor-precompiled models only**, no hybrid flow | **neutts-air / neutts-nano = TTS** (LM half) |
| 3 | **FastFlowLM** `flm` | MIT + free proprietary kernels | Strix Halo "Supported"; Ubuntu 24.04/25.10/**26.04**, Arch | none (ASR input only) |
| 4 | **IRON / mlir-aie** + Peano → hand-written AIE-ML design → XRT/`amdxdna` | Apache-2.0 + LLVM exception | explicitly documented "AMD Ryzen™ AI on Linux"; same driver/XRT stack | anything, hand-written; nothing ships TTS |
| 5 | Vendor-curated AIE-ML graph sets (halogen `HALOGEN_NPU_MODELS`, FLM packs) | vendor | present **[host]** | none **[host]** |
| 6 | Windows-only: DirectML, Ryzen AI Windows EP (`ryzenai-llm`), Studio Effects, `whispercpp` NPU | — | not usable | (per Lemonade matrix: Windows only) |
| 7 | ggml/llama.cpp/**qwentts.cpp**, whisper.cpp, sherpa-onnx, Piper | — | no XDNA backend in ggml/ORT-mainline | CPU / iGPU only |
| 8 | IREE, TVM, ONNX Runtime mainline EPs (OpenVINO = Intel NPU, MIGraphX/ROCm = AMD GPU) | — | no XDNA2 target; AMD's NPU EP is only "Xilinx Vitis-AI (preview)" in the ORT EP list and ships inside AMD's own build | n/a |
Sources: §2–§6 below.
## 2. FastFlowLM (`/usr/bin/flm` 1.0.1 **[host]**) — catalog + HTTP surface
* Repo moved into AMD's ROCm org **2026-08-11** (`github.com/ROCm/FastFlowLM`,
v1.0.0). "Supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo,
Kraken, and Gorgon Point)."
<https://github.com/ROCm/FastFlowLM> (README fetched 2026-10-19).
* Linux support landed 2026-03; distros: Ubuntu 24.04, 25.10, **26.04**, Arch,
generic. Needs NPU firmware ≥ 1.1.0.0, kernel **7.0+** with `amdxdna` (or
`amdxdna-dkms`), XRT stack (`ppa:lemonade-team/stable` → `libxrt-npu2`,
`amdxdna-dkms`); `flm validate` reads the DRM path (`/dev/accel/accel0`), `flm
run` goes through **XRT** and fails with `No such device with index '0'` unless
`xrt-smi examine` lists the NPU; `memlock` must be raised.
<https://fastflowlm.com/docs/install_lin/> (page dated 2026-03-05, fetched
2026-10-19). Static build bundles **XRT v2.21.75** — the same XRT series as the
distro package on the host **[host]**.
* HTTP surface: `flm serve <model>` → OpenAI-compatible REST, **port 52625**, no
auth. Maintained endpoints: `v1/models`, `v1/chat/completions`,
`v1/audio/transcriptions`, `v1/embeddings`. **There is no
`/v1/audio/speech`.**
<https://fastflowlm.com/docs/instructions/server/openapi/>
* Catalog 2026-10-19 (<https://fastflowlm.com/docs/models/>): LLaMA, DeepSeek,
Qwen, Gemma, MedGemma, TranslateGemma, gpt-oss, LiquidAI/LFM, Phi, Nanbeige,
**Whisper — card states "Type: Speech-to-Text (ASR)"**
(<https://fastflowlm.com/docs/models/whisper/>), EmbeddingGemma, SmolVLA,
Hy-MT2, MiniCPM. No speech-synthesis entry.
* Engines are **prebuilt, closed, downloaded from Hugging Face per architecture**
("Internet access to HuggingFace is required to download the optimized model
kernels"). The only in-tree engine build is Gemma-4E
(`cmake --preset linux-default -DFLM_BUILD_GEMMA4E=ON`).
→ **A new TTS architecture cannot be added to `flm` by us.**
## 3. AMD Ryzen AI Software 1.8.0 on Linux — install source, device support
* **Install channel (important correction to the pypi.amd.com evidence).** The pip
index is Windows-only for the EP runtime (`ryzenai_dynamic_dispatch-1.7.0-…
win_amd64.whl`, `onnxruntime_vitisai-1.23.2-…win_amd64.whl`; `flexmlrt` has a
single **linux_aarch64** wheel) — but AMD's Linux stack is not on pip. It ships
as a tarball that is, as of 2026-10-19, **publicly reachable without an AMD
account**: `https://download.amd.com/opendownload/RyzenAI/1.8.0/ryzen_ai-1.8.0.tgz`
→ HTTP 206 on a range probe, `Content-Range: bytes 0-8/**7819506855**`
(**≈7.8 GiB**, magic `1f 8b 08` = gzip). (The docs link it behind the
EULA form `account.amd.com/...ryzenai-eula-public-xef.html`; the CDN path
serves it directly. Verified by me with curl 2026-10-19. I did not finish
downloading 7.8 GiB, so the tarball's *contents* list is unverified —
`[INFERENCE]` only where noted.)
* Driver/XRT package (public, 90,016,137 bytes, `application/zip`):
`https://download.amd.com/opendownload/RyzenAI/Driver/RAI_1.8_Linux_NPU_XRT.zip`
contains `xrt_202620.2.25.37_24.04-amd64-{base,base-dev,npu}.deb` +
`xrt_plugin.2.25.260102.56.release_24.04-amd64-amdxdna.deb` (verified by
download + unpack). **The plugin deb ships firmware for
`/usr/lib/firmware/amdnpu/17f0_11/npu.dev.sbin`** — exactly the directory our
NPU loads **[host]** — plus `1502_00`, `17f0_10`, `17f1_10`, `17f1_13`,
`libvxdna.so`, `libxrt_driver_xdna.so.2.25.260102.56`, and the DKMS driver
source tarball. **No xclbin/PDI and no compiler binary in this package**: it is
driver + XRT shim + firmware only.
* **Our device is officially STX.** AMD's APU table
(<https://ryzenai.docs.amd.com/en/latest/app_development.md#apu-types>):
`0x1022:0x17F0 rev 0x11 → STX` (0x00/0x10 also STX, 0x20 = KRK, `0x1502` rev 0
= PHX/HPT). Release notes: "Strix and **Strix Halo** as STX", and the STX/KRK
column enables CNN INT8, CNN BF16, NLP BF16, LLM (OGA).
* Vendor driver source (same deb, `share/amdxdna/amdxdna.tar.gz`, source dated
2026-06-29) maps `0x17f0 rev 0x11 → dev_npu5_info`, `.fw_path =
"amdnpu/17f0_11/"`, `default_vbnv = "RyzenAI-npu5"`, and the revision→name table
has `{ AIE2_DEV_REVISION_HALO, "NPU Strix Halo" }`; `xrt-smi examine` therefore
reports the device as **NPU Strix Halo**.
* Linux install recipe (1.8.0 docs
<https://ryzenai.docs.amd.com/en/latest/linux.html>): Ubuntu 24.04 packages,
**Python 3.12** (`apt install python3.12 python3.12-venv`),
`libboost-filesystem1.74.0`, dkms; then `install_ryzen_ai.sh -a yes -p <venv>`;
`LD_LIBRARY_PATH=…:${RYZEN_AI_INSTALLATION_PATH}/onnxruntime/lib/`;
`quicktest.py` "picks up a simple CNN model, **compiles it** and runs on AMD's
Neural Processing Unit" — i.e. **compilation runs on Linux** for the non-LLM
flows. "Linux currently supports NPU only flow" (no iGPU EP).
Host friction: our box is Ubuntu 26.04 + **Python 3.14** **[host]**, so the venv
must be built with a 3.12 interpreter and the packages are 24.04-built
`[INFERENCE]`.
* LLM flow on Linux
(<https://ryzenai.docs.amd.com/en/latest/llm_linux.html>): prequantized
ready-to-run models from HF, `deployment/lib`,
`RYZENAI_EP_PATH=$PWD/deployment/lib/libonnxruntime_providers_ryzenai.so`,
run via the `model_benchmark` binary; **"Linux does not support Hybrid flow"**;
**"Model generation is not supported in current release"** (repeated in
relnotes 1.8 Known Issues: "Models generated on Windows are compatible with
Linux"); needs `memlock unlimited`.
Sample throughput in that doc: **49.1 tokens/s decode**, 753 tokens/s prefill
(Phi-3.5-mini, 4K model).
* Third-party confirmation of the OS split: Lemonade's backend matrix
(<https://raw.githubusercontent.com/lemonade-sdk/lemonade/main/README.md>,
fetched 2026-10-19) — `flm`/npu/XDNA2 = **Windows, Linux**;
`ryzenai-llm`/npu = **Windows only**; `whispercpp`/npu (STT) = **Windows only**;
text-to-speech engines `kokoro` and `acestep` = metal / cuda / vulkan / rocm,
**no npu device at all**. Lemonade is the "Server Interface" layer of AMD's own
Ryzen AI stack (docs index), so this matrix is the vendor-sanctioned view.
## 4. Can we compile our own graph on Linux? (the no-recipe path)
Mechanics from <https://ryzenai.docs.amd.com/en/latest/modelrun.md> (1.8.0):
ONNX only, opset 17 recommended; the EP compiles on first `InferenceSession` and
caches (VitisAI EP cache, or ORT **EP context cache** — `ep.context_enable`, and
"precompile the EP context model using Python and deploy it using a C++ program"
is explicitly supported); EP auto-partitions unsupported nodes to CPU;
BF16 = `target VAIML` (`preferred_data_storage: unvectorized` for transformers),
BF16 **STX/KRK-or-newer only**; INT8 = `target=X2` with XINT8/A8W8/A16W8 and
"general asymmetric quantization … to enable third-party quantized models";
**deployment EP cannot compile BF16 on the fly** — ship a pre-compiled model
(<https://ryzenai.docs.amd.com/en/latest/app_development.md>);
cache dirs are not portable across EP or NPU-driver versions.
Operator coverage is the hard gate
(<https://ryzenai.docs.amd.com/en/latest/ops_support.md>, "operators supported in
Ryzen AI 1.8.0"; "Y" = broad coverage, some configurations may still fail):
* Good for TTS front-ends/vocoders: `ConvTranspose` **Y in BF16/A16W8/A8W8**
(absent only from XINT8), `Conv`, `Resize`, `Upsample`, `Pad`, `LSTM` (BF16),
`CumSum` (BF16/XINT8), `STFT` (XINT8 only).
* **Blockers for a decoder-style transformer in the plain flows:** in **BF16** the
table has **no Softmax, no LayerNormalization, no Relu/Gelu**; LayerNorm is
XINT8-only and Softmax is A16W8/A8W8/XINT8-only. So attention either lives in
the **LLM/OGA flow** (whose op set is exactly
`SimplifiedLayerNormalization`, `SkipSimplifiedLayerNormalization`, `MatMulNBits`
(W4ABF16/W4ABFP16), `RotaryEmbedding`, **`GroupQueryAttention`**, `Sigmoid`,
`Mul`, `Add`) or must be decomposed into a quantized INT8 flow.
* Missing entirely: `GRU`, `Trilu`, `OneHot`, `EyeLike`, `Multinomial`,
`NonMaxSuppression`, `CastLike`, `MelWeightMatrix`/`MelFilterGroup`,
`RMSNormalization` → graphs that emit them must be rewritten before they
partition onto the NPU. [INFERENCE from absence in the table.]
* Quantization is a separate tool, not in the driver package: **AMD Quark**
(`quark.torch` / `quarkonnx`), which documents `ConvTranspose2d`, `conv1d`,
softmax/LayerNorm/BFP quantization (relnotes 1.3/1.2/1.1/0.9) — Quark is *not*
on pypi.org under the obvious names **[host probe]** and ships from AMD's
gated/pip-private channel.
* **New LLM-shaped model (our Qwen-Talker analogue): Windows-only today.**
OGA model generation is not available on Linux in 1.8.0 (§3). So a *new*
speech-LM cannot be self-served onto the NPU on Linux without AMD (or a
Windows box) in the loop.
* Practical shape cost: the neutts artifacts carry **one compiled variant per
padded sequence length** (128/256/512/768/1024/1536/2048/2304/3072/4096 for
prefill; 256…4096 for token-phase attention mask) — i.e. per-shape compile
artifacts are the norm for the LLM flow. [INFERENCE from the HF file list.]
## 5. Does the AMD TTS artifact actually produce speech? (cloned-voice survival)
NeuTTS is **LM + separate neural codec** ("Simple LM + codec architecture"),
clone from ~3 s of reference audio, and the codec is **NeuCodec (50 Hz, single
codebook)**: `neuphonic/neucodec` (PyTorch, encode+decode),
`neucodec-onnx-decoder{,-int8}` (**decode only**), "Decoder-only (ONNX) codecs
require **pre-encoded references** — these ship with the repo for all bundled
voices." Phoneme input; 2048-token context ≈ 30 s of audio; NeuTTS-Air ~360 M
active / 552 M with embeddings; upstream benchmarks are **CPU-only llama.cpp**
(119 tok/s prefill/decode mix on a Ryzen 9 HX 370) and explicitly exclude the
codec. <https://raw.githubusercontent.com/neuphonic/neutts/main/README.md>
(fetched 2026-10-19); `neuphonic/neutts-air` HF metadata (lastModified
2026-08-26, `qwen2`, GGUF + safetensors).
Consequences for the fleet spike:
1. AMD's NPU artifact = LM only; **waveform decode and reference encoding are CPU
work either way**. Our `reference.wav` voices survive only if we run the
NeuCodec **encoder** (PyTorch, per the codec table there is no ONNX encoder)
once per voice and then feed pre-encoded codes. That is a one-time offline
step per voice; after that the compiled graph takes codes/prompt prefix as
ordinary inputs, so nothing about the compiled graph forbids a caller-supplied
speaker prefix. `[INFERENCE]`
2. Real-time math: 50 Hz codec means **≥50 tokens/s decode** for 1× RT. AMD's own
Linux sample output for a 4K Phi-3.5-mini is **49.1 tok/s** — so budget for
roughly RT-1× on one 4K "full fusion" graph, before the codec's CPU cost.
`[INFERENCE — different model; must be measured.]`
3. Host RAM: the same doc reports **peak working set 3.54 GB** for that 4K model.
NeuTTS-Air is ~552 M params at W4A-BFP16, so expect a smaller footprint than
3.5 GB `[INFERENCE]`; still non-trivial against 14 GiB available **[host]**.
4. AMD ships **no TTS demo, runner, or server** on any OS: the examples page has
only CNN/BF16/DistilBERT/super-resolution/OGA-LLM/VLM/RAG
(<https://ryzenai.docs.amd.com/en/latest/examples.md>); the RyzenAI-SW repo
tree (669 paths, `main`, fetched 2026-10-19) contains speech demos **only under
`Demos/ASR/{Whisper,Parakeet-TDT}` and `Transformer-examples/ASR/Whisper-AI`**
— no TTS directory. Serving TTS therefore means writing our own OGA host
(or driving `model_benchmark`-style C++ code) plus phonemizer + NeuCodec glue.
## 6. Two graphs on one NPU, and the iGPU hazard
* **Concurrency is a first-class driver feature.** The vendor `amdxdna` source
(in `xrt_plugin.2.25.260102.56`, 2026-06-29) sets `.hwctx_limit = 16` for
npu4/npu5 — i.e. **16 hardware contexts** on our device — and allocates each
context a set of AIE columns with QoS `opc = col_opc (4096) × num_col`
(`aie2_ctx.c`, `aie2_solver.h`), with `frame_boundary_preempt` /
`fill_preempt_slot` in `aie2_pci.h` (firmware preemption at frame boundaries).
So halogen's armed models and a TTS graph can hold separate contexts
simultaneously; they contend for columns and OPC share, they are not mutually
exclusive.
* AMD's user-visible history agrees: 1.4 "Support for up to **16 hardware
contexts** on Ryzen AI 300 series"; 1.5 "Application concurrency: improves the
resource distribution across applications"; 1.3 "Idle detection and suspension
of contexts", "Rebalance feature for AIE hardware resource optimization";
1.0 known issue "Support for multiple applications is limited to up to eight"
(all in <https://ryzenai.docs.amd.com/en/latest/relnotes.html>).
* **Package conflict, the real risk** [INFERENCE]: the vendor stack wants AMD's
XRT 2.25.37 + `xrt_plugin…amdxdna` deb **[host: distro `libxrt-npu2
1:2.21.75+dfsg-4` installed]**, and firmware must match the loaded `amdxdna.ko`
— the vendor README warns that a stale/mismatched `npu.sbin` causes
`ERT_CMD_ABORT`/mailbox timeouts, and that a distro **in-tree** `amdxdna` older
than the XRT SHIM makes `GET_ARRAY`/`QUERY_TELEMETRY` return
`EOPNOTSUPP`/`EINVAL` (<https://github.com/amd/xdna-driver> README,
fetched 2026-10-19). Installing AMD's debs over the distro XRT changes the ABI
under FastFlowLM/halogen, so the spike must decide the XRT version **once**, for
the whole host.
* **iGPU / fabric clock:** no vendor document supports a fabric-clock pin, and no
vendor document addresses NPU+iGPU concurrency hangs. AMD's Strix Halo
optimization doc (ROCm docs, dated 2026-02-20, and its successor "AMD RDNA3.5
system optimization", ROCm 10.1.0) covers GPUVM/GART/GTT, the TTM `pages_limit`
knob, `amd-ttm`, BIOS VRAM sizing, and minimum kernel versions
(<https://rocm.docs.amd.com/en/docs-7.2.0/how-to/system-optimization/strixhalo.html>,
<https://rocm.docs.amd.com/en/latest/reference/system-optimization/rdna3-5.html>).
The `pp_dpm_fclk` level-7 pin and the GPU+NPU hang observation stay
fleet-observed **[host]**, i.e. `[INFERENCE]` territory — no citation exists.
## 7. Paths I checked and ruled out (so nobody re-walks them)
* **Windows-only doors**: DirectML; Ryzen AI Windows EP; Lemonade's
`ryzenai-llm` and `whispercpp`-NPU engines (matrix above); AMD's OGA model
generation. Not usable on Dankclaw.
* **Mainline ONNX Runtime** has no AMD-NPU EP: its EP summary lists AMD ROCm and
AMD MIGraphX (GPU) and "Xilinx Vitis-AI (*preview*)" only
(<https://onnxruntime.ai/docs/execution-providers/>, 2026-10-19); the working
EP is AMD's own build (`libonnxruntime_providers_ryzenai.so` in the tarball's
deployment dir, per llm_linux).
* **ggml / llama.cpp / qwentts.cpp**: no XDNA backend → the current engine
(`tts-server` on GGUF, CPU-only build **[host]**) cannot be "turned on" for the
NPU; migration means a different runtime, not a flag.
* **`flm`**: catalog/engines are closed per-architecture (§2) — the cheapest door
by install cost, and it is not a door for TTS.
* **IRON / mlir-aie** is the only open toolchain that documents running on the
Ryzen AI NPU from Linux ("A close-to-metal Python API for programming AMD Ryzen™
AI NPUs, built on an open-source MLIR-based compiler toolchain"; "Getting
Started for AMD Ryzen™ AI on Linux", Ubuntu 24.04/24.10, XRT from
`ppa:amd-team/xrt`: `libxrt2 libxrt-npu2 … amdxdna-dkms`, `render` group
required) — <https://raw.githubusercontent.com/Xilinx/mlir-aie/main/README.md>,
fetched 2026-10-19. This is how FLM's kernels were made ("Low-level kernels
optimized using IRON + AIE-MLIR", FLM README) — i.e. real, but it is
hand-written per-op tile programming, not a model compiler: writing a TTS model
here is a multi-month engineering project, not a migration.
* **Embedded** Ryzen AI P100 support in the same Linux doc is irrelevant to STX.
## 8. Recommendation implied by the above (for Main's spike)
Spike the AMD path as decided, with these constraints planned in: Python 3.12 venv
+ 24.04-built wheels on Ubuntu 26.04; one host-wide XRT decision (AMD 2.25.37 vs
distro 2.21.75) with firmware/driver pairing checked via `xrt-smi examine` →
"NPU Strix Halo"; 7.8 GiB tarball staged off the public CDN URL (§3); pre-encode
each `reference.wav` with the **PyTorch** NeuCodec encoder because the ONNX codec
is decoder-only; keep phonemizer + codec on CPU; measure decode tokens/s against
the 50 Hz budget and peak RSS against the ~14 GiB headroom; expect to write the
serving shell yourself (AMD ships `model_benchmark`, no TTS demo).
## 9. Loose ends (not verified; I stopped on Main's stand-down)
* Contents of `ryzen_ai-1.8.0.tgz` (7.8 GiB) — whether the Linux tarball contains
the AIE-ML compiler (`vaiml`, `aiecompiler_client`) for the *BF16/INT8* flows
and which Python wheels it carries. My statement that non-LLM compilation works
on Linux rests on AMD's own Linux doc ("quicktest.py … compiles it and runs on
… NPU"), not on the package contents. Download was killed mid-flight.
* Whether AMD's `neutts` artifacts run correctly under the Linux `model_benchmark`
runner (the TTS rows are listed without OS qualifiers; no Linux TTS run is
documented anywhere in 1.8.0 docs).
* Exact AIE column count/total `ncols` exposed for Strix Halo (driver reads it
from AIE metadata: `ndev->total_col = min(XRS_MAX_COL, ndev->aie.metadata.cols)`).
@@ -0,0 +1,438 @@
# Qwen3-TTS payload + qwentts.cpp runtime: architecture and portability
Research deliverable (no code). Slice: *what the fleet's TTS payload actually is, and how
portable it is off the CPU ggml path.* Companion slice: `NpuRuntimeLandscape` (what NPU
runtimes exist on Dankclaw).
Evidence date: 2026-10-19. Everything below is grounded in:
- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
(2026-10-16, upstream HEAD on 2026-10-19) and its ggml submodule
`ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16).
- The two GGUF files as they sit on Dankclaw (`/home/dank/models/Qwen3-TTS-GGUF/`), read
byte-for-byte from the live host.
- Upstream Qwen3-TTS model cards / release material.
Status: COMPLETE for the assigned slice. ANSWER C was confirmed on the live host the same
night — the running `frank-tts-engine` container carries `libggml-base.so.0.27.0` +
`libggml-cpu.so.0.27.0` and nothing else, and our own
`dankclaw/voiceservice/Containerfile.engine` writes `-DGGML_VULKAN=OFF` out explicitly, so
the iGPU change is a flag in a file the fleet owns. Verdict, measured contract and the NPU
exoneration chain: `tts-npu-spike-verdict-2026-10-19.md`.
---
## 1. What the payload is
Two files, both read byte-for-byte off the live host (`/home/dank/models/Qwen3-TTS-GGUF/`,
owned `dank:dank`, mtimes 2026-10-18 18:31), both from
`https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF` (community converter, not Qwen):
| file | bytes | GGUF | tensors | arch string | file_type |
|---|---|---|---|---|---|
| `qwen-talker-1.7b-base-Q4_K_M.gguf` | 1,219,245,248 | v3, kv=64 | 480 | `qwen3-tts` | `Q4_K_M` |
| `qwen-tokenizer-12hz-Q8_0.gguf` | 291,150,624 | v3, kv=51 | 398 | `qwen3-tts-tokenizer` | `Q8_0` |
`general.name` of the talker file is `Qwen3-TTS-12Hz-1.7B-base`; the header sizes add up
to the file sizes (sum of tensor bytes + header + alignment padding), so nothing in these
files is unaccounted for.
### 1.1 Talker file (480 tensors)
Three models in one container, per `docs/ARCHITECTURE.md` §GGUF layout
(qwentts.cpp @ pinned commit) and confirmed by the tensor census:
| prefix | tensors | bytes | what it is |
|---|---|---|---|
| `talker.*` | 316 | 1,061.9 MB | 28-layer Qwen3 decoder (hidden 2048, FFN 6144, 16 Q heads / 8 KV heads, head_dim 128, RoPE θ=1e6, RMS ε=1e-6, ctx 32768), text stream + codec stream summed, `codec_head` → 3072 codebook-0 logits |
| `code_pred.*` | 88 | 114.8 MB | 5-layer Qwen3 MTP head (hidden 1024, FFN 3072, 15 acoustic codebooks × 2048), own KV cache |
| `spk_enc.*` | 76 | 36.6 MB | ECAPA-TDNN speaker encoder (x-vector, base checkpoints only; mel → conv0 → 3 SE-Res2Net → MFA → ASP → 2048-d vector) |
Dtype split (exact tensor byte sums): `q4_K` 250 tensors / 864.1 MB, `q6_K` 17 / 312.0 MB,
`f32` 176 / 25.8 MB, `f16` 37 / 11.4 MB.
Which tensors carry which quant is the load-bearing portability fact:
- `q6_K` (17): `talker.text_embd.weight [2048, 151936]` (255.3 MB), `talker.codec_embd.weight
[2048, 3072]`, and the 15 `code_pred.codec_embd.<g>.weight [2048, 2048]` embedding tables.
These are the llama.cpp `Q4_K_M` "embedding/output rows go to Q6_K" rule, applied to
embedding tables rather than to an `lm_head`.
- `q4_K` (250): every attention/FFN projection of the talker and of the code predictor.
Distinct matmul shapes: `[2048,2048]`×73, `[2048,1024]`×62, `[2048,6144]`×56,
`[6144,2048]`×28, `[1024,2048]`×20, `[1024,3072]`×10, `[1024,1024]`×10, `[3072,1024]`×5,
`[2048,3072]`×2, `[2048,151936]`×1. Every one is 2-D and ≥ 1024 on both axes — good
GEMM shapes, nothing exotic.
- `f32` (176) / `f16` (37): all of `spk_enc` (conv weights F16 because K=1/3/5 rows are not
block-alignable, per the converter's `should_quantize` policy in `tools/quantize.cpp` +
`quantize.sh`; every bias/norm/LayerScale vector F32), plus the talker/predictor norms and
the `text_proj` biases.
Metadata the runtime actually reads (talker file): `qwen3-tts.num_code_groups=16`,
`talker.vocab_size=3072`, `text_vocab_size=151936`, `text_hidden_size=2048`,
`position_id_per_seconds=13`, `rope.mrope_section=[24,20,20]`, `mrope_interleaved=1`,
the 7 codec special ids (pad 2148 / bos 2149 / eos 2150 / think 2154 / nothink 2155 /
think_bos 2156 / think_eos 2157), `language_names` (10: chinese, english, german, italian,
portuguese, spanish, japanese, korean, french, russian) with their codec ids, the text-side
special ids (im_start 151644, im_end 151645, tts_pad 151671, tts_bos 151672, tts_eos 151673),
the whole `generation_config.json` (do_sample, top_k 50, top_p 1.0, temperature 0.9,
repetition_penalty 1.05, `subtalker_*` mirrors, max_new_tokens 8192), and an embedded
**Qwen2 BPE tokenizer** (`tokenizer.ggml.model=gpt2`, 151,676 tokens, 151,291 merges,
eos 151643) — that BPE block is 5.95 MB of the header, i.e. 49 % of the header is vocabulary.
Two live-file details that contradict the upstream doc text (worth knowing when reading
`docs/ARCHITECTURE.md`: the doc example block is stale against the shipped file):
`mrope_interleaved` is **1** (doc shows `false`), and `spk_enc.sample_rate` is **24000**
(doc says 16000). Doc also says "404 tensors on the 1.7B voice_design build" — the base
build shipped here has 480 because it carries `spk_enc.*`.
### 1.2 Tokenizer/codec file (398 tensors)
One neural audio codec, 12.5 frames/s over 24 kHz mono, hop 1920, 16 codebooks × 2048
entries (1 semantic + 15 acoustic). Dtype split: `f16` 50 / 143.6 MB, `q8_0` 110 / 76.9 MB,
`f32` 238 / 70.7 MB. `tok_dec.*` 253 tensors / 201.3 MB, `tok_enc.*` 145 / 89.9 MB.
Composition (metadata + tensor census, both files agree):
- **Encoder** (used only to make voice references, not on the synthesis hot path): SEANet
conv stack (init k=7, 4 stages with ratios 8/6/5/4 → cumulative 960×, 64→512 ch, final
k=3), an 8-layer transformer (hidden 512, 8 heads, head_dim 64, FFN 2048, RoPE 1e4,
LayerNorm+bias, GELU, LayerScale), a downsample conv k=4 s=2 (25 → 12.5 Hz), then the RVQ
argmin loop. Metadata records `encoder.num_quantizers=32` with
`encoder_valid_num_quantizers=16` — 16 codebooks are used, 16 are dead weight carried from
the upstream config.
- **Decoder** (the hot path: every generated frame): 16 codebook lookups + per-split
`output_proj` (256→512), `conv_pre`, an 8-layer transformer (hidden 512, 16/16 heads,
head_dim 64, FFN 1024, **sliding window 72**, RMSNorm, SwiGLU, LayerScale, input_proj
1024→512 / output_proj 512→1024), a ConvNeXt upampler (`upsampling_ratios=[2,2]`, causal
transposed convs + depthwise k=7), then a DAC-style decoder `dec.0..3`
(`upsample_rates=[8,5,4,3]`, channels 1536→768→384→192→96, `decoder_dim=1536`) whose blocks
are SnakeBeta → causal transposed conv (k = 2·stride) → 3 residual units (dilations 1/3/9)
→ snake_post → conv_post k=7 → 96→1.
- All 32 codebook tensors are **F32, 67.1 MB total** (16 encode + 16 decode, `[256, 2048]`),
deliberately unquantized: the converter's `should_quantize` keeps RVQ codebooks and their
wrapping projections at F32 because nearest-neighbour lookup is sensitive to per-row
quantisation noise (`docs/ARCHITECTURE.md` "Quantisation policy"; mirrored in
`quantize.sh`). 49 of the codec's 55 3-D conv tensors are **F16** with non-alignable K
(7, 3, 2, 1, 16, 10, …) — they cannot be block-quantized at all.
- **No flow matching, no diffusion, no vocoder-with-ODE.** It is a pure RVQ codec
(Mimi/SoundStream/Mimi-style: SEANet + transformer + DAC decoder). That matters for the
NPU question: every component is convolution, attention, or embedding lookup.
### 1.3 Where the payload came from, and what the official weights look like
Official upstream (`huggingface.co/api/models`, retrieved 2026-10-19):
| repo | last modified | license | weights |
|---|---|---|---|
| `Qwen/Qwen3-TTS-12Hz-1.7B-Base` | 2026-01-23 | apache-2.0 | `model.safetensors` BF16, 1,928,677,440 params, plus an embedded `speech_tokenizer/model.safetensors` |
| `Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice` | 2026-01-29 | apache-2.0 | same layout, named-speaker tables |
| `Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign` | 2026-01-29 | apache-2.0 | 1.7B only |
| `Qwen/Qwen3-TTS-12Hz-0.6B-Base` / `-0.6B-CustomVoice` | 2026-01-29 | apache-2.0 | narrow variant (hidden 1024) |
| `Qwen/Qwen3-TTS-Tokenizer-12Hz` | 2026-01-29 | apache-2.0 | codec alone, F32, 170,557,441 params |
So the GGUF is **not** the only public form: the safetensors are the canonical form
(Apache-2.0, no separate community license risk), and the GGUF is a lossy re-quantization of
them produced by the same author who wrote the runtime (`Serveurperso`), via
`checkpoints.sh` → `convert.py` (F32 GGUF) → `quantize.sh` (BF16/Q8_0/Q4_K_M).
Sizes to remember for any re-export: the talker container is 1.93 B params in BF16
(that count includes the embedded codec), the codec alone is 170.6 M params in F32.
**Portability consequence #1 — the GGUF is a dead end off-ggml.** Q4_K/Q6_K are ggml
block formats (256-element super-blocks, F16 sub-block scales, 144 B / 210 B per block). No
ONNX graph, no Vitis-AI/AIE-ML compiler, and no vendor NPU runtime consumes them; AMD's own
quantizer works from ONNX (`ryzenai-quantize`, QDQ/QOP power-of-two — Ryzen AI Software
1.8.0 release notes, https://ryzenai.docs.amd.com/en/latest/relnotes.html). Moving the
payload to a non-ggml stack therefore means starting from the safetensors again, not
converting the GGUF.
**Portability consequence #2 — the quantization split is semantic, not cosmetic.** The
converter keeps RVQ codebooks, `input_proj`/`output_proj` around them, and every 1-D and
non-alignable-conv tensor out of the k-quants precisely because quantizing them breaks voice
fidelity. Any re-quantization path (GPTQ/AWQ/QDQ-INT8/INT4 for an NPU compiler) inherits that
constraint: the talker backbone is requantizable, the codec's codebook lookups and SnakeBeta
convs are not, at least not without re-measuring voice quality.
## 2. What the runtime is
`qwentts.cpp` @ `5916bf7c` (2026-10-16, MIT; upstream model Apache-2.0) is a C++17 ggml
application — 43 files under `src/` + `tools/`, three binaries (`qwen-tts`, `qwen-codec`,
`tts-server`) over one public plain-C99 ABI (`src/qwen.h`, `qt_*`, `QT_ABI_VERSION`
range-checked; `tests/abi-c.c` is compiled `-std=c99 -Wall -Werror -pedantic` on every
build). It is not a llama.cpp derivative: it links ggml directly and builds its own graphs.
Five graph families, and their shapes are the whole portability story:
| graph | batch/time axis | notes |
|---|---|---|
| talker prefill | T = prompt length (hundreds–thousands) | summed text+codec embedding streams, 28-layer GQA decoder, KV cache |
| talker decode | **T = 1 per audio frame** | 12.5 frames/s → 80 ms of audio per step; `codec_head` → 3072 logits |
| code predictor (MTP) | **T = 1, 16 steps per frame** (2 prefill + 14 decode, cache reset every frame) | 5-layer decoder, 15 heads × 2048 |
| speaker encoder | one pass per voice registration | ECAPA-TDNN with mel fused into the graph |
| codec decode | offline: T = 300-frame chunks; streaming: **T = 1 frame** with persistent conv/KV-ring state | the streaming graph is built once with fixed topology so the CUDA graph cache stays in pure replay |
Per generated frame the host therefore runs ~1 + 16 + 1 tiny graph computes for 80 ms of
audio, and `tts-server` serializes synthesis FIFO (`--max-batch` defaults to 1, and the
fleet runs it at 1). This is the archetypal latency-bound, bandwidth-bound,
batch-1 workload; the only big-M matmuls in the whole system are the prefill and the
chunked (buffered) codec decode.
Op contract actually issued to ggml (`src/`, counted): `mul_mat` 77, `cont` 70, `add` 57,
`mul` 40, `rms_norm` 22, `clamp` 15, `rope_ext` 12, `get_rows` 12, `set_rows` 8,
`scale_bias` 6, `swiglu_split` 5, `sqr` 5, `mean` 5, `elu` 4, `col2im_1d` 4, `soft_max` 3,
`conv_1d` 3, `im2col` 2, `cumsum` 2, `sin`, `sigmoid`, `tanh`, `div`, `argsort`, `transpose`,
`permute`, `reshape/view`, plus the CPU-side Philox multinomial sampler.
**No custom ggml op is required today.** The repo doc still says the fork "provides the two
custom ops the codec needs: `GGML_OP_SNAKE` and `GGML_OP_COL2IM_1D`"; that is stale at the
pinned commit. Verified 2026-10-19:
- the submodule `ServeurpersoCom/ggml` @ `cc5df49` is upstream `ggml-org/ggml` master
(`a05c992`, 2026-10-15) **+2 commits, 0 behind** (GitHub compare API), and
`include/ggml.h` is byte-identical to upstream master (`diff` empty);
- the 2 commits are `193e339` (2026-06-19) "cpu: add fused snake activation
(mul -> sin -> sqr -> mul -> add)" — a **CPU subgraph fusion**, not an op (the runtime
builds snake with `ggml_sin`/`ggml_sqr`/`ggml_mul`/`ggml_add` in `src/dac-decoder-v2.h:222`),
and `cc5df49` "sycl: fix get_rows on reordered quantized weights";
- `ggml_col2im_1d` is an **upstream ggml API** (same declaration, line 2097, in both trees);
the Vulkan backend fuses the snake chain through ggml's generic subgraph fuser
(`ggml-vulkan.cpp:14548 ggml_vk_can_fuse_snake`, `vulkan-shaders/snake.comp`).
Consequence: any stock ggml build (upstream, or llama.cpp's vendored ggml) can execute this
payload; only the CPU snake fusion is fork-specific, and it is a performance nicety.
Backend/device support: every tool takes `--device <name>` (`CUDA0`, `Vulkan0`, `MTL0`,
`CPU…`, default best available), resolved through the ggml backend registry; the deployed
container turns CUDA/HIP/Vulkan/Metal/SYCL/RPC/BLAS all OFF and compiles the native-CPU
backend with OpenMP, with CPU threads = `hardware_concurrency()/2` (`src/backend.h`).
The registry in this ggml knows CPU, CUDA, Metal, SYCL, Vulkan, WebGPU, ZDNN, OpenCL,
**Hexagon**, BLAS, RPC, VirtGPU, **CANN**, ZenDNN, **OpenVINO**, ET — and nothing AMD-NPU
(no `xdna`/`aie`/`vitis`/`ryzen` string anywhere in the ggml tree).
Voice handling (what the fleet depends on): the registry is **process RAM only**; a restart
empties it, which is why the unit has a boot warmer. `POST /v1/audio/voices` accepts either
`{name, ref_text, wav_b64}` (server extracts) or `{name, ref_text, spk_b64, rvq_b64}`
(pre-extracted latents verbatim); `ref_text` present ⇒ ICL clone mode, absent ⇒ x-vector-only
mode. `.spk` = the ECAPA x-vector; `.rvq` = 16 codes/frame packed 11 bits LSB-first
(`src/rvq-file.h`). `wav` and `pcm` `response_format` are different audio at the same seed
(buffered chunked decode vs stateful streaming decode) — the fleet already encodes that rule.
Parity expectations for any port: the upstream harness matches the PyTorch reference per
stage (embed cos 0.999988, all 28 hidden taps ≥ 0.999893, logits 0.999824) but explicitly
reports free-run divergence (`CodesFull exact 4.96 %`, `Audio cos 0.095`) and explains why —
one argmax tie at FP epsilon diverts the AR trajectory. So a migrated endpoint must be
accepted by listening + per-stage cosine, never by waveform diff against today's audio.
---
## 3. The three answers Main asked for
### ANSWER A — Qwen3-TTS GGUF → ONNX: no, and it does not matter
There is **no GGUF→ONNX path** and there will not be a useful one: GGUF is a weight
container for ggml, its Q4_K/Q6_K block layouts have no ONNX consumer, and nothing in the
GGUF carries the *graph* (the talker/predictor/codec structure lives in C++ code, only the
op-level structure exists in ggml at runtime). But the GGUF is not the payload's only form —
official Apache-2.0 safetensors are upstream, and **four independent ports of the full
pipeline already exist**:
| port | form | what it proves |
|---|---|---|
| `xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (HF, updated 2026-03-03, apache-2.0) | FP16+FP32 ONNX, fp16/voice_clone + fp16/voice_design splits: `talker_decode.onnx`, `code_predictor.onnx`, `code_predictor_kv.onnx`, `code_predictor_embed_g0..15.onnx`, `text_embedding.onnx`, `codec_embedding.onnx`, shared `speaker_encoder.onnx`, `speech_tokenizer_encoder.onnx`, `speech_tokenizer_decoder.onnx`, `model_cache.npz` | every component of this architecture exports to ONNX, codec decoder included |
| `zhangluoyang/qwen3-tts-onnxruntime` | export tooling + Python inference for Base/CustomVoice/VoiceDesign | the export is reproducible from the official checkpoints |
| `Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO`, `aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03) | OpenVINO IR, **INT8**: `talker`, `code_predictor`, `codec_embedding`, `text_model`, `openvino_speaker_encoder_model`, `speech_tokenizer/speech_decoder` | the talker and predictor INT8-quantize for an NPU-targeting stack |
| `litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17, apache-2.0) | `qwen3_tts_int8.litertlm` / `_int4.litertlm` (LiteRT-LM `OmniEngine`), legacy split graphs `talker_int4.tflite` (prefill_32/prefill_128/decode signatures, KV 1024, blockwise-32 OCTAV INT4), `mtp_folded_int8.tflite` (16 inner steps × 5 layers folded into one graph, in-graph argmax+gather, GPTQ int8, ~5× faster), `codec_partA` (transformer, fp32) + `codec_partB` (conv upsampler, XNNPACK FORCE_FP16, ~2.5×, "ASR-identical") | an NPU/mobile compiler accepts the whole thing — at the price of graph splitting, static prefill buckets, and folding the per-frame inner loop |
What the ports had to concede, and what any port inherits:
- the generation structure does not fit existing engine loops: LiteRT-LM's Engine "does not
support this generation structure yet", so their first shape was **three graphs + a
host-side loop**, and the MTP step was "invoked 17× per frame" until they folded it;
- per-step dispatch cost dominates: even on a Snapdragon 8 Elite Gen 5 they keep the MTP
loop, text projection, embedding gathers and the codec on CPU "avoiding 16 synchronous
CPU↔GPU round-trips per 80 ms frame", putting only the 28-layer talker on the GPU;
- RTF on that hardware is **2.57–2.97 (Galaxy S26) and 5.65–8.48 (Pixel 10 Pro)** for the
0.6 B model, i.e. 2.6–8× slower than real time. Dankclaw's current CPU engine measures
**warm RTF 0.55–0.67 with TTFA 94–112 ms** on the 1.7 B model (fleet trial, 2026-10-18).
An NPU port is not "faster by default" for this workload; it has to beat that.
- quantize the LM (int8/int4: proven), keep the codec's codebook lookups and SnakeBeta convs
in fp16/fp32 (the LiteRT split does exactly this).
So: the answer to "can the fleet's payload be exported" is *yes — from the safetensors, not
from the GGUF*, and the work is a pipeline rewrite (multi-graph host loop, per-frame fold,
static prefill buckets), not a conversion. That is the same conclusion NeuTTS-Air's
precompiled-graph route encodes: on AMD's Linux stack the graph is vendor-curated, so the
fleet chooses the model that AMD already compiled rather than compiling Qwen3-TTS.
### ANSWER B — is there any ggml backend for an AMD NPU? Not upstream; three out-of-tree, none for TTS
Checked against the trees today (2026-10-19), because this was the crux:
- **Nothing in-tree.** `ggml-org/llama.cpp` master (4,237 paths, GitHub tree API) contains no
`xdna`/`aie`/`vitis`/`ryzen` file or directory; the ggml backend set is CPU, BLAS, CUDA,
HIP, Metal, Vulkan, SYCL, OpenCL, WebGPU, MUSA, RPC, CANN (Ascend), **Hexagon** (Qualcomm),
OpenVINO (Intel CPU/GPU/NPU), ZenDNN, ZDNN, VirtGPU, ET. The deployed fork's registry
matches. The feature request "Feature Request: XDNA backend" (llama.cpp issue **#21725**,
opened 2026-04-10) is still **open**, 34 comments; the older #1499 was closed stale
2025-04-07. So Main's premise is confirmed: no ggml XDNA backend exists to flip a flag on.
- **`Cyronius/ggml-xdna`** — a real ggml backend (`XDNA0`) loaded into *stock* llama.cpp via
`GGML_BACKEND_PATH`, no fork. It does bfp16 (8-value blocks sharing an exponent) GEMMs from
GGUF weights, and is **prefill-only**: it takes only matmuls in prompt chunks ≥ 1024 tokens
(`GGML_XDNA_MIN_BATCH` default 1024, `GGML_XDNA_MIN_MFLOP` 256), attention and every decode
token stay on the Vulkan device; pinned to llama.cpp release b10944; tested on Strix Point
(HX 370) only — Strix Halo "should pass, but nobody has run it on one yet"; Linux support
new and untested on hardware by the author; needs `libxrt-npu2`, `/dev/accel/accel0`,
unlimited `memlock`. Best measured prefill gain 1.88× on a 27 B model; models under
2,048-wide **lose** (0.77× at width 1,536).
- **`BrandedTamarasu-glitch/OllamaAMDNPU`** — a llama.cpp fork with `-DGGML_XDNA=ON`,
explicitly targeting **Ryzen AI MAX / XDNA2 on Linux** (`/dev/accel/accel0`, XRT 2.21.75+,
amdxdna, unlimited memlock). It implements **only `GGML_OP_MUL_MAT`**, dispatched to
mlir-aie-compiled `.xclbin`s built per fixed tile shape (e.g. `M=2048 K=2048 N=64`,
int8 in / int32 out, per-row weight scales cached at first use), with a second slot for a
different K. Its own numbers on a Ryzen AI MAX 385 with Llama-3.1-8B Q4_K_M: NPU prefill
12–19 t/s vs **Vulkan iGPU 600–930 t/s**, NPU decode **0.65–1.40 t/s** ("Decode does not
use the NPU … M=1 per token, no xclbin covers it"), 0.6 % AIE utilization, and the stated
conclusion "**For LLaMA 8B on Strix Halo: use Vulkan exclusively**". Single-maintainer,
AI-assisted, self-measured — treat as a proof of concept, but it is the closest published
picture of what XDNA2 graph execution costs.
- Also referenced in the same thread: `Atomic-Germ/OpenFlowLM-Next` (whole-model NPU engine,
LLM-only — and slower than the iGPU in the same comparison), `GaTmaNnes/xdna2-`,
`hardWorker254/Npu-Embeddings-XDNA1` (adds Whisper ASR ops, NPU1/Linux). None is a TTS path.
Why none of them would carry this workload even if installed: the TTS hot loop is **batch-1
by construction** (one talker step + 16 predictor steps + one codec frame per 80 ms), which
sits under every dispatch gate those backends define (≥1024 tokens / ≥32768 M·N·K / tile M
covering only prefill), and the codec half of the pipeline is conv, col2im, gather and
elementwise ops — not `MUL_MAT` — so it would fall back to CPU op-by-op. The NPU case for
this host is therefore *not* "run Qwen3-TTS on the NPU"; it is "run a vendor-compiled AR TTS
graph on the NPU", which is exactly the NeuTTS-Air spike.
### ANSWER C — would `qwentts.cpp` build for the gfx1151 iGPU? Yes, via **Vulkan**, not HIP
- Upstream ships the build scripts and claims the backends: `buildvulkan.sh` ("AMD/Intel GPU
(Vulkan)"), `buildcuda.sh`, `buildsycl.sh`, `buildcpu.sh`, `buildall.sh` (all backends,
runtime dynamic loading), and README line 6 "runs on CPU, CUDA, Metal, Vulkan". CMake knobs
`GGML_VULKAN` / `GGML_HIP` / `GGML_SYCL` are the same ones the fleet's Containerfile turns
OFF, so the engine rebuild is a flag change: `-DGGML_VULKAN=ON` plus
`--device Vulkan0` on the same command line the unit runs today.
- The codec's exotic ops are covered by the Vulkan backend, which is the thing that decides
whether a GPU build is coherent: `vulkan-shaders/col2im_1d.comp`, `snake.comp`,
`swiglu*.comp`, `cumsum*.comp` exist and `supports_op` handles `GGML_OP_COL2IM_1D` /
`CONV_TRANSPOSE_1D` / `CUMSUM`. By contrast `COL2IM_1D` appears in **no** HIP, OpenCL,
WebGPU, Hexagon or CANN source — a **HIP** (ROCm) build would shard the ConvNeXt/DAC
transposed convs back to the CPU per op. On an AMD iGPU the portable path is RADV/Vulkan.
- `--device` also accepts a multi-device split (`-dev`-style naming is ggml's), and the
runtime's documented FP16 guard (`--clamp-fp16`) exists for sub-Ampere CUDA accumulation;
nothing in the tree is NVIDIA-specific.
- Unverified by me: any vendor statement pairing this fork with gfx1151. Supporting evidence
that the hardware/driver combination is viable: this very host already runs a Vulkan
endpoint on the gfx1151 iGPU (fleet ground truth, 2026-10-19), and Mesa's Vulkan
implementation is officially supported on Radeon Software for Linux
(AMD release notes RN-AMDGPU-UNIFIED-LINUX-25.20.3). A gfx1151 Vulkan build of this fork is
a build-and-measure spike, not an unsupported port.
- The real constraint is the one already in the fleet docs: the iGPU belongs to another
endpoint, and GPU+NPU concurrency has hung Strix Halo machines. An iGPU TTS build competes
with the text endpoints for the same silicon; the NPU route's only genuine advantage is
that it does not (the XDNA backend authors make the same argument: "Use NPU when the iGPU
is busy … no GPU contention").
---
## 4. What any replacement must reproduce, and what the voices cost
Endpoint contract in production today (from the live unit/env, not the doc):
`tts-server --model qwen-talker-1.7b-base-Q4_K_M.gguf --codec qwen-tokenizer-12hz-Q8_0.gguf
--alias frank-tts-engine --host 0.0.0.0 --port 8880 --lang auto --max-batch 1`, no published
port (the façade on :7772 dials the private bridge name), `GET /health`, `GET /v1/models`,
`GET|POST /v1/audio/voices`, `DELETE /v1/audio/voices/{name}`, `POST /v1/audio/speech` with
`response_format` wav|pcm (different audio at equal seed) and optional `seed`,
`max_new_tokens`, `temperature`, `top_k`, `top_p`, `repetition_penalty`, `subtalker_*`.
Voice registry is RAM-only, so boot warming is mandatory, and `--alias` keeps the quantization
choice out of every consumer's config.
Live voice census on Dankclaw (2026-10-19, `/home/dank/voices/`): three packs
(`narrator1..3`), each carrying `reference.wav` (24 kHz mono crop), `reference16k.wav`,
`manifest.json` (keys: name, kind, generated_utc, pipeline, source, forbidden_region_guard,
silence_scan, crop, loudnorm, audio, acceptance, acceptance_passed) and ffmpeg analysis
artifacts. **No `.spk` and no `.rvq` exist on the live host** — the repo's
`build-voice-pack.sh` treats them as required-but-they-are-not-there, so today the engine
re-extracts the x-vector (and the ICL codes when a transcript is supplied) from the WAV at
registration time.
That is the portable part, and it is the same shape AMD's NeuTTS flow wants
(`encode_reference(reference.wav)` + a reference transcript): the fleet's voice identity is
*the WAV plus its transcript*, processed at request/registration time by the model's own
speaker encoder + codec — **never a trained per-voice artifact**. Qwen3-TTS cloning is
inference-time conditioning (`generate_voice_clone(ref_audio=…, ref_text=…, x_vector_only_mode=…)`;
`create_voice_clone_prompt` exists purely to avoid recomputing prompt features). So any
replacement that consumes a WAV (+transcript) keeps all three narrators alive with no
re-derivation, and if it wants precomputed latents instead, the LiteRT precedent shows what
that looks like: a 4 KB x-vector `.npy` per voice, extracted offline
(`extract_speaker_embedding.py`) — for the 1.7 B base checkpoint that vector is 2048-dim
(`qwen3-tts.spk_enc.embedding_length = 2048` in the shipped GGUF; the LiteRT 0.6 B bundle
ships a 1024-dim one).
One trap for a mixed-runtime future: `.spk`/`.rvq` files, if they are ever written, are
engine-specific encodings (ECAPA x-vector layout; 11-bit-packed RVQ stream) and are **not**
interchangeable between runtimes. Keep `reference.wav` as the pack's source of truth and
regenerate anything derived per engine — which is what the NeuTTS re-clone plan does anyway.
## 5. Sources
Runtime/payload (all read locally at pinned versions, 2026-10-19):
- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
(commit date 2026-10-16; also upstream HEAD today): `README.md`, `docs/ARCHITECTURE.md`
(860 lines), `src/*`, `tools/*`, `docs/DOCKER.md`, `buildvulkan.sh`/`buildcpu.sh`/`buildall.sh`.
- submodule `ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16);
delta vs `ggml-org/ggml` master `a05c992` (2026-10-15) = ahead 2 / behind 0, per
https://github.com/ServeurpersoCom/ggml/compare/ggml-org:master...HEAD ;
`include/ggml.h` identical to upstream
https://raw.githubusercontent.com/ggml-org/ggml/master/include/ggml.h .
- Live payload bytes: `/home/dank/models/Qwen3-TTS-GGUF/qwen-talker-1.7b-base-Q4_K_M.gguf`
(1,219,245,248 B) and `qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B), headers parsed from
the host; sizes/sha256 match `scripts/download-frank-tts-engine-qwentts.sh`.
- Live units/env/packs: `dankclaw/voiceservice/{Containerfile.engine,tts-engine-start.sh,tts-engine-warm.sh,quadlet/*}`,
`scripts/frank-tts-engine.env`, `scripts/frank-tts.env`, `/home/dank/voices/*`.
Model provenance:
- `https://huggingface.co/api/models/Qwen/Qwen3-TTS-12Hz-1.7B-Base` (+ CustomVoice,
VoiceDesign, 0.6B variants, `Qwen3-TTS-Tokenizer-12Hz`), apache-2.0, last-modified
2026-01-23 / 2026-01-29; safetensors param counts 1,928,677,440 (BF16, incl. embedded
`speech_tokenizer/`) and 170,557,441 (F32 codec).
- `https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base/raw/main/README.md` (clone API:
`generate_voice_clone(ref_audio, ref_text, x_vector_only_mode)`,
`create_voice_clone_prompt`; reference stack = PyTorch + `device_map="cuda:0"` + bf16 +
flash-attn-2, Python 3.12 recommended).
- `https://github.com/QwenLM/Qwen3-TTS`.
Portability precedents:
- `https://huggingface.co/xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (2026-03-03) ·
`https://github.com/zhangluoyang/qwen3-tts-onnxruntime` ·
`https://huggingface.co/Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO` (2026-03-22) ·
`https://huggingface.co/aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03-18) ·
`https://huggingface.co/litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17) incl.
`legacy/README.md` (graph splits, MTP fold, codec partA/partB, RTF table).
AMD NPU / ggml backend status:
- `https://api.github.com/repos/ggml-org/llama.cpp/git/trees/master?recursive=1` (no
XDNA/AIE/Vitis backend, 2026-10-19) ·
`https://github.com/ggml-org/llama.cpp/issues/21725` (open since 2026-04-10) ·
`https://github.com/ggml-org/llama.cpp/issues/1499` (closed stale 2025-04-07).
- `https://github.com/Cyronius/ggml-xdna` (prefill-only, gates, Linux requirements, results
measured 2026-10-05/06 on HX 370) ·
`https://github.com/BrandedTamarasu-glitch/OllamaAMDNPU` (XDNA2 ggml backend on Strix Halo,
phases 1–12, "use Vulkan exclusively", measured on Ryzen AI MAX 385).
- `https://github.com/amd/RyzenAI-SW` (main, 669 paths, 2026-08-18): audio content is ASR
only (`Demos/ASR/Whisper`, `Demos/ASR/Parakeet-TDT`, `Transformer-examples/ASR/Whisper-AI`);
**no TTS example anywhere in the official Ryzen AI Software tree**.
- `https://ryzenai.docs.amd.com/en/latest/relnotes.html` (Ryzen AI SW 1.8.0: ONNX-format
models, ONNX Runtime as the loader, power-of-two QDQ/QOP quantization).
- Mesa/RADV on AMD iGPUs: `https://www.amd.com/en/resources/support-articles/release-notes/RN-AMDGPU-UNIFIED-LINUX-25-20.3.html`
(Mesa Vulkan driver officially supported).
Status: COMPLETE for the assigned slice (payload architecture, runtime architecture,
portability). `[INFERENCE]` marks: none above except the Mesa-version caveat in ANSWER C,
which is flagged there.
+180
View File
@@ -0,0 +1,180 @@
# TTS on the Strix Halo NPU — spike verdict (2026-10-19)
Question: **how can the fleet's TTS engine run on hardware rather than CPU?** Answer in
one line: not on the NPU — the loader, not the silicon, refuses it; the door that exists is
the gfx1151 iGPU behind a build flag we already own.
Spike evidence lives in `report/npu-tts-runtime-landscape-2026-10-19.md` (which runtimes can
reach XDNA2 at all) and `report/qwen3-tts-payload-portability-2026-10-19.md` (what the
payload is and where it can be moved). This page owns only the **measured spike result** and
the **model-side contract** neither slice has. Throwaway work ran in containers under
`/home/dank/spikes/npu-tts` on Dankclaw; nothing in the fleet was changed — `frank-tts` and
`frank-tts-engine` still serve `narrator1|2|3` on CPU.
Legend: **[host]** live census 2026-10-19 · **[spike]** measured in the spike containers ·
**[vendor]** primary vendor source, with the version it holds for.
## 1. Verdict
**NO-GO on the NPU, and the wall is a loader.** The XDNA2 device is healthy by AMD's own
instruments; the runtime that owns this model family opens the device and then declines the
graph, silently. One run loaded successfully; byte-identical runs never reproduced it, and a
path that cannot be reproduced is not a foundation. **[spike]**
**Go on the iGPU, and the lever is in this repo.**
`dankclaw/voiceservice/Containerfile.engine` builds `qwentts.cpp` (commit `5916bf7`) with
`-DGGML_VULKAN=OFF` written out deliberately beside CUDA/HIP/METAL/SYCL/RPC — the running
container carries `libggml-base.so.0.27.0` + `libggml-cpu.so.0.27.0` + `tts-server` and
nothing else. So: `-DGGML_VULKAN=ON` + `--device Vulkan0`, plus `/dev/dri` passthrough in
the engine quadlet (the ComfyUI pattern: `AddDevice` + `GroupAdd=keep-groups` + a
world-writable udev rule). The target shape is **co-tenancy** — the engine resident on the
gfx1151 *beside* `frank-dragoon`, not a handover. §5 carries the census and both gates:
the **heap gate is PASSED** — RADV's `DEVICE_LOCAL` heap is 77.33 GiB with 65.16 GiB of
budget while dragoon is loaded, so memory is not the constraint — which leaves the
contention gate (dragoon's TTFT with a synthesis in flight) and one unproven piece:
whether qwentts.cpp's codec ops have Vulkan kernels for every op. **[host]** **[spike]**
Why the iGPU is worth doing at all, given CPU already works: the deployed payload needs
only **12.5 codes/s** (§2), so this is latency headroom and freed cores, not reviving a
dead service. The independent Strix Halo measurement in the portability slice ("use Vulkan
exclusively"; NPU decode under 1.5 t/s for an 8B) points the same way.
## 2. The two arms do not share a real-time bar
|Arm|Talker|Codec|Codes per second of audio|Bar|
|---|---|---|---|---|
|**Deployed** (`frank-tts-engine`)|`qwen-talker-1.7b-base-Q4_K_M.gguf`|`qwen-tokenizer-12hz-Q8_0.gguf`|**12.5**|~13 tok/s **[host]**|
|**Spike** (`amd/neutts-air_rai_1.8.0_npu_4K`)|NuTTS-Air, 24-layer Qwen2|NeuCodec `24000 / hop 480`|**50.00**|~50 tok/s **[spike]**|
Measured from the staged NPU artifact — the contract any future arm must satisfy:
|Fact|Value|
|---|---|
Vocab|`genai_config.json` `vocab_size` **217,652**; base BPE 151,643; specials 151,644–151,670 **[spike]**|
Speech tokens|**65,536** `<|speech_N|>` entries, ids **151,671–217,206** contiguous; the vendor template maps a codec index straight to its id, `"".join(f"<|speech_{i}|>")`. A 10,001-code assumption silently truncates the space — encoded references really do reach the 60k range. **[spike]**|
Stop token|`<|SPEECH_GENERATION_END|>` = 151,670, but the export's `model.eos_token_id` is `[151645, 151643]` (the *chat turn* stop) and `search.eos_token_ids` is unset — pass 151,670 explicitly or speech ends after one token **[spike]**|
Input ceiling|4096 tokens **total**, input + output **[vendor]**|
Frame rate|50.00 codes/s confirmed across three references (49.96 / 49.99 / 50.00) **[spike]**|
Reference encode|`librosa.load(sr=16000, mono=True)` → `[1,1,T]` float → `codec.encode_code(audio_or_path=…)` (vendor `NeuTTS.encode_reference`); CPU RTF 0.28–0.38 on 8 threads, so codes are a cacheable one-time cost, never a per-request charge. `neucodec` also needs `torchtune`; install both `--no-deps --target <dir>` so a second torch never shadows the venv's. **[spike]**|
Pre-encoded voices|`/spike/logs/codes-narrator{1,2,3}.npy` — 994 / 1,249 / 1,250 codes for 19.90 / 24.98 / 25.00 s **[spike]**|
**Gap that any arm inherits:** the voice packs carry no transcript. `manifest.json` holds
provenance and crop geometry (`crop.source_speech_run` 98.739–118.636 s of a 22,569 s
audiobook) and no text, and Dankclaw has no ASR at all — `whisper`, `whisper.cpp`,
`faster-whisper` all absent. NuTTS conditioning needs `ref_text`; do not invent it. **[host]**
## 3. Exoneration chain — why the blame is not the hardware
Every row is an instrument, not an inference.
|Probe|Result|
|---|---|
`xrt-smi examine`|`[0000:c6:00.1] RyzenAI-npu5 aie2p 6x8`, XRT 2.25.37 (vendor extract), NPU fw 1.1.2.65, `amdxdna 7.0.0-38-generic` **[spike]**|
`xrt-smi validate`|**PASSED**, including `throughput` **[spike]**|
`pyxrt.device(0)` in-container|constructs — XRT enumerates the device **[spike]**|
`strace -f -e trace=openat` on the failing load|`genai_config.json` read → `model.onnx` opened → `libxrt_driver_xdna.so.2` loaded → `/dev/accel/accel0` opened `O_RDWR` → GEM ioctls succeed → **EP declines, before touching `dd_metastate_*` or `cache/txn_bins.zip`** **[spike]**|
`dmesg`, `fuser /dev/accel/accel0`|silent since boot, no holder — the decline never reaches the kernel **[spike]**|
Hardlinked mirror dirs of the model tree (`rundir*`)|retired, no change — the "metastate shadowing" theory is dead **[spike]**|
Every logging lever in that build: `GENAI_ORT_VERBOSE_LOGGING`, `GENAI_LOG_ORT_LIB`, `set_log_options`, `set_log_callback(callable)`, `RYZENAI_EP_TRACING`, `RYZENAI_LLM_PROFILE_INIT`|**zero lines** — the decline path is silent **[spike]**|
Vendor `quicktest`|`Session successfully initialized` — proves the XRT/driver/device path, **not** NPU residency: that build falls back to CPU silently. Prove residency with `/sys/class/accel/accel0/device/mem_usage` during generation. **[spike] [vendor]**|
No NPU token rate was ever measured, so none is quoted here. The single successful load
(13:17, `[load] 0.84s`, finite logits, XRT banner at *generate* time) has not been
reproduced by the byte-identical command since, with the model files pristine by inode and
mtime. **[spike]**
## 4. Host gates the spike proved (so they are not re-litigated)
- **Rootless containers cannot read `/dev/accel/accel0`** (`root:render` 0660): the host
gid never enters the userns map. `--group-add keep-groups`, `--userns=keep-id`,
`--userns=keep-id:gid=990`, `--group-add render`, `--group-add gid:990` — all fail.
Doors: rootful podman (separate storage), or a world-writable udev rule, which the fleet
already does for `kfd`/`renderD*`. **[spike]**
- **Never `dpkg -i` the vendor XRT deb** — it installs a DKMS `amdxdna` over the distro
driver. `dpkg-deb -x` the debs and bind-mount the tree at `/opt/xilinx/xrt`; the plugin
deb is what carries `libxrt_driver_xdna.so`. **[spike]**
- In-tree `amdxdna 0.7.0-38` + `iommu=pt` is enough for context creation; a missing device
node is *not* the modern failure mode. **[host]**
- `xrt-smi` inside a container needs `libboost-filesystem1.83.0` +
`libboost-program-options1.83.0` (**hyphenated** Ubuntu names, after `apt update`);
boost is needed by `xrt-smi` only, never by the OGA/EP path. **[spike]**
- Headroom is tight and it is host RAM, not a separate pool — see §5 for tonight's census.
A multi-GB NPU partition plus an AIE compile workspace does not fit beside
`frank-dragoon`. Benign-looking `Failed to alloc dev bo memory, ret -28` is that
pressure. **[host]**
## 5. Target shape: the TTS engine shares the gfx1151 with `frank-dragoon`
The deployment Frank wants is **both resident at once**, not a handover. Measured 2026-10-19
(read-only, nothing loaded or unloaded):
|Measure|Value|Reading|
|---|---|---|
iGPU GTT in use|**18,523 MiB of 116,733 MiB**|≈96 GiB of aperture free; this is the real budget (fleet law) **[host]**|
DRM "VRAM" carve-out|2,048 MiB total, 174 MiB used|what KFD/DRM calls dedicated — **not** the budget **[host]**|
Host RAM|121 GiB total, 17 GiB available, 79 GiB buff/cache|**GTT pages are system pages**: an iGPU allocation is host RAM consumed, so the two tenants share one pool **[host]**|
TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that same pool **[host]**|
**Two gates, in this order.**
1. **Heap gate — PASSED.** RADV does **not** hand a container tenant the 2 GiB DRM
carve-out. `vulkaninfo` inside the engine's own base image (`ubuntu:24.04` plus
`mesa-vulkan-drivers vulkan-tools`, `--device /dev/dri`, rootless) sees
`Radeon 8060S Graphics (RADV GFX1151)`, driver `radv`, API 1.4.318, and reports:
|Heap|size|budget with dragoon loaded|flags|
|---|---|---|---|
|0|38.67 GiB|32.58 GiB|host-visible, not device-local|
|1|**77.33 GiB**|**65.16 GiB**|`DEVICE_LOCAL`|
The heaps sum to the ~116 GiB aperture in Mesa's ⅓/⅔ APU split — the same RAM twice:
a Vulkan allocation is a GTT mapping, so a budget is a ceiling on mappings, not a pile
of free silicon memory. ggml's Vulkan backend asks for `DEVICE_LOCAL`, which is heap 1,
and the TTS engine needs roughly 4 GiB of it. **Memory stops being the question.** **[spike]**
Probe traps: `vulkaninfo --batch` is not a flag in this `vulkan-tools` (it prints the
help page and exits 1 — use `vulkaninfo --text`), and the container has **no NVIDIA
ICD**, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by
device *name*, or an image that grows an `nvidia_icd.json` silently points TTS at the
3090 slot.
2. **Contention gate.** `frank-dragoon` is the endpoint the operator session runs on, so the
honest measurement with both loaded is dragoon's own TTFT and decode with a synthesis in
flight — one request each, no ladder. Expect compute-unit sharing rather than memory
pressure: the aperture has room, the shaders do not.
**Change surface** — the whole edit, so nobody re-derives it:
1. `Containerfile.engine` build stage: add `libvulkan-dev glslang-dev`. It installs only
`ca-certificates cmake g++ git make` today, and ggml's Vulkan backend compiles its
shaders at build time — the flag alone fails at configure.
2. `-DGGML_VULKAN=OFF` → `ON` in the same cmake block; the flag list is written out
explicitly, so this is a one-word diff.
3. Runtime stage: add `mesa-vulkan-drivers`. Without a Vulkan **ICD inside the image** the
device node buys nothing — there is no `radeon_icd.json` in the container today.
4. `frank-tts-engine.container`: `AddDevice=/dev/dri` + `GroupAdd=keep-groups` — **not**
`renderD128`. On this hybrid box that node is the NVIDIA card; the iGPU is
`renderD129` (`readlink -f /sys/class/drm/renderD129/device/driver` → `amdgpu`).
5. Engine argv: `--device Vulkan0` — qwentts.cpp's tools take `--device <name>` at the
pinned commit.
Gate 1 recipe, already run and **PASSED**: `mesa-vulkan-drivers` + `vulkan-tools` in a
throwaway container off the same base, `--device /dev/dri`, then `vulkaninfo --text` and read
the `VkPhysicalDeviceMemoryProperties` heap sizes — no model, no load, no compute. The probe
trap: `vulkaninfo --batch` is not a flag in Ubuntu 24.04's `vulkan-tools` — it prints the help
page and exits 1, which reads like a driver failure. Every package involved has a candidate on
the host's apt (`vulkan-tools` 1.4.341, `libvulkan-dev` 1.4.341, `glslang-dev`/`glslang-tools`
16.2.0); the build stage runs on the digest-pinned Ubuntu 24.04 base, whose versions are older
but present. Script: `/home/dank/spikes/npu-tts/vk_probe.sh` on Dankclaw.
Constraints that do not relax for a co-tenant: `/dev/dri` passthrough needs the ComfyUI
pattern (`AddDevice` + `GroupAdd=keep-groups` + the world-writable udev rule) because
systemd user-unit spawns carry no supplementary groups; the 64 GiB swap at `/swap.img`
exists to protect co-tenants, and paging a KV pool is worse than the OOM it prevents;
`pp_dpm_fclk` reads `auto` (unheld), which only becomes a hazard if the NPU ever joins.
## 6. Re-visit condition
Retry the NPU when **a Linux loader claims the graph**: an `onnxruntime-genai-ryzenai`
build whose RyzenAI adapter takes the NuTTS-Air Full-Fusion graph on `RyzenAI-npu5`, or that
model appearing in a runtime's own supported catalog with a Linux runner. Then the
prerequisites are already mapped and priced: the gated NeuCodec encoder for cloned voices,
the `ref_text` transcripts, the 4096-token prompt budget, and 50 codes/s as the bar.
+8 -13
View File
@@ -29,10 +29,10 @@
# The scratch mirrors the production container contract exactly (serve-frank-
# primary.sh): --network host, --init, seccomp=unconfined (io_uring for the
# n-gram table), memlock unbounded (the ~132 GiB pinned ledger), /dev/kfd +
# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
# differences: the container name (arbench, so prod's identity is untouched)
# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
# prefill is cold — the prod tier at kvcache/frank-archon is left intact.
# /dev/dri, the .rad ro, the tune cache mount. The ONLY differences: the
# container name (arbench, so prod's identity is untouched) and the prefix cache
# switched OFF (--prefix-cache-mib 0), so every prefill is cold — the tier is
# host RAM now, there is no per-run disk directory to wipe.
set -uo pipefail
ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-archon.env}"
@@ -60,9 +60,8 @@ set -a; . "$ENV_FILE"; set +a
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
FRESH_KV="$FRANK_ARCHON_STATE_DIR/kvcache/$SCRATCH"
rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
# --- the prefix cache is switched off below, so there is no scratch KV tree ---
# to wipe; the cold prefill comes from --prefix-cache-mib 0.
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
# --- boot the scratch: the production run line, scratch name + fresh KV ------
@@ -73,7 +72,6 @@ podman run -d --name "$SCRATCH" \
--ulimit memlock=-1:-1 \
--device /dev/kfd --device /dev/dri \
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-v "$FRANK_ARCHON_STATE_DIR":/data \
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
"$FRANK_ARCHON_IMAGE" \
@@ -87,12 +85,9 @@ podman run -d --name "$SCRATCH" \
--placement "$FRANK_ARCHON_PLACEMENT" \
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
--prefix-cache-dir "/data/kvcache/$SCRATCH" \
--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
--prefix-cache-mib 0 \
--busy-prefill-chunk "${FRANK_ARCHON_BUSY_PREFILL_CHUNK:-auto}" \
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
+164
View File
@@ -0,0 +1,164 @@
#!/usr/bin/env python3
"""Destroy measurement ledgers in the surviving pages.
Point of use (AGENTS.md → Documentation Policy): a rate/throughput figure for
anything not running and not being compared is destroyed, not filed. This pass
deletes
1. any section whose heading is a bench/throughput/rate/ladder/verdict ledger,
2. any table row or standalone bullet that REPORTS a rate (`123.4 t/s`,
`tok/s`, `ms/token`, `TTFT`, acceptance %, `llama-bench`, a bench arm),
and it never touches a shape number (ctx, slots, MiB budgets, ports) or a bullet
under a *Laws* / *Wire contract* / *Rollback* heading, where the sentence is the
payload and the number is decoration. Dry run by default; `--apply` writes.
Usage: axe-measurements.py [--apply] [--verbose]
"""
import pathlib
import re
import sys
VAULT = pathlib.Path("serverAppConfigs-docs")
RATE = re.compile(
r"\d[\d,.]*\s*(t/s|tok/s|tokens?/s|tok s|tps|ms/token|µs/token|GB/s|GiB/s)\b|"
r"\b(llama-bench|bench (arm|ladder|window)|decode ladder|prefill ladder)\b|"
r"\bacceptance (rate|of)\b|\bdraft_n\b|\bttft\b|\bServer-Timing\b",
re.I,
)
# A heading is a ledger only when the noun LEDS it and nothing instructional
# follows: `## Throughput` yes, `## 6. The context ladder (find the max bootable
# window by measurement)` no — that one is the method, which is point-of-use.
LEDGER_HEAD = re.compile(
r"^#{2,4}\s+(?:\d+[.) ]\s*)?(bench(es|mark|marks)?|throughput|results?|timings?|"
r"measured (figures|numbers|results|rates|rates)|rates|verdict|acceptance|ladder)"
r"\b[^|]{0,40}$",
re.I,
)
INSTRUCTIONAL = re.compile(r"how|why|law|policy|method|procedure|find|by measurement|execution|flip|checklist", re.I)
PROTECTED = re.compile(
r"^#{2,4}\s+.*(law|wire|rollback|contract|shape|eras|files|health|see also|related|"
r"boot|preflight|environment)",
re.I,
)
def blank_rate_cells(text):
"""Kill throughput COLUMNS wherever they appear: a cell that reports a rate
becomes an em-dash, so the row keeps whatever else it records (the payload
name, the reason, the rollback) and loses only the dead number."""
out = []
for line in text.split("\n"):
st = line.strip()
if st.startswith("|") and st.endswith("|") and RATE.search(line):
cells = line.split("|")
for i, c in enumerate(cells):
if RATE.search(c):
cells[i] = " — "
line = "|".join(cells)
out.append(line)
return "\n".join(out)
def collapse_eras(text):
"""One Eras row per era is fleet law; the NUMBERS in the old rows are not.
Keep the current era verbatim, fold everything older into one row that says
the numbers were destroyed and where the rollback actually lives (git)."""
lines = text.split("\n")
out = []
in_eras = False
ncol = 0
folded = 0
for line in lines:
if line.startswith("#"):
in_eras = bool(re.match(r"^#{2,4}\s+.*eras\b", line, re.I))
if folded:
out.append("| " + " | ".join(
[f"≤ 2026-09 — {folded} earlier eras: payloads, shapes and every measured "
"number destroyed 2026-10-19 as obsolete"] + ["—"] * max(0, ncol - 1)))
folded = 0
out.append(line)
continue
st = line.strip()
if in_eras and st.startswith("|"):
cells = [c.strip() for c in st.strip("|").split("|")]
if set("".join(cells)) <= set(":- "):
ncol = len(cells)
out.append(line)
continue
if ncol == 0:
ncol = len(cells)
if re.search(r"2026-1[01]|current|active|running", st, re.I):
out.append(line)
else:
folded += 1
continue
if folded and not st.startswith("|"):
out.append("| " + " | ".join(
[f"≤ 2026-09 — {folded} earlier eras: payloads, shapes and every measured number "
"destroyed 2026-10-19 as obsolete"] + ["—"] * max(0, ncol - 1)))
folded = 0
ncol = 0
out.append(line)
if folded:
out.append("| " + " | ".join(
[f"≤ 2026-09 — {folded} earlier eras destroyed 2026-10-19"] + ["—"] * max(0, ncol - 1)))
return "\n".join(out)
def main(argv):
apply = "--apply" in argv
verbose = "--verbose" in argv
total_lines = 0
pages = 0
for page in sorted(VAULT.rglob("*.md")):
s = str(page)
if ".AGENTTRASH" in s or ".obsidian" in s:
continue
text = page.read_text(errors="replace")
before = len(text.split("\n"))
if s.startswith("serverAppConfigs-docs/LLM/"):
text = collapse_eras(blank_rate_cells(text))
lines = text.split("\n")
out = []
skip = False
under_protected = False
cut = 0
for line in lines:
if line.startswith("#"):
under_protected = bool(PROTECTED.search(line))
skip = (bool(LEDGER_HEAD.match(line)) and not under_protected
and not INSTRUCTIONAL.search(line))
if skip:
cut += 1
continue
if skip:
cut += 1
continue
stripped = line.strip()
out.append(line)
if under_protected:
# Eras rows are the rollback record, Law bullets are the payload:
# the number is decoration inside them, and the row/bullet is not
# a ledger entry. Leave the whole section alone.
continue
if stripped.startswith(("|", "-", "*")) and RATE.search(line):
# Destroy only when the line EXISTS to report a number: a table
# row, or a bullet whose verb is measurement.
if stripped.startswith("|") or re.search(
r"\b(measured|measures|bench|benched|ran at|runs at|clocked|hit|"
r"landed at|came in at|peak|peaks at|sustained|wall.clock)\b", line, re.I):
out.pop()
cut += 1
final = re.sub(r"\n{3,}", "\n\n", "\n".join(out))
saved = before - len(final.split("\n"))
if saved or cut:
pages += 1
total_lines += max(saved, cut)
print(f" {s.replace('serverAppConfigs-docs/', '')}: -{max(saved, cut)}")
if apply:
page.write_text(final.rstrip("\n") + "\n")
print(f"{'APPLIED' if apply else 'dry run'}: {pages} pages, {total_lines} lines of ledger")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+483
View File
@@ -0,0 +1,483 @@
#!/usr/bin/env bash
# build-voice-pack.sh — turn a SOURCE recording plus a CROP OFFSET into a
# deployable frank-tts voice pack: the reference WAV, the engine's own speaker
# artifacts made from it, and manifest.json provenance.
#
# THE PACK. $PACKS_DIR/<voice>/ carries the three members the pack law in
# scripts/frank-tts.env names:
#
# reference.wav the crop: 24 kHz mono PCM (pcm_s16le), 18-25 s of clean
# single-speaker audio. This is what makes a voice NAME real
# (dankclaw/voiceservice/app.py lists and resolves exactly
# this file), so it is written LAST, once everything else in
# the pack has been verified.
# reference.spk the engine's x-vector: enc_dim 2048 f32 = 8,192 B.
# reference.rvq the engine's RVQ code stream: 16 codebooks x 11 bits = 22 B
# per 12.5 Hz frame = 22 * floor(samples / 1920).
# manifest.json provenance: source path and sha256, crop start/end, sample
# rate, duration, the tool + engine commit that made the
# artifacts, and the voice name.
#
# Both artifacts come from the pinned engine build's speaker extractor — the
# `qwen-codec` tool of qwentts.cpp at VOICE_PACK_ENGINE_COMMIT — run on the pack's
# own WAV, and the extractor names them by swapping the input's extension, so
# they land beside reference.wav with nothing to rename. Required, never a
# replacement for the WAV: a voice another runtime cannot reuse is not saved.
#
# THE 10-SECOND GATE — THE REASON THIS SCRIPT EXISTS. Project-wide source-material
# invariant: the first 10 s of any sample file used to generate voices is
# forbidden material. An offset that starts inside that region is refused with a
# nonzero exit BEFORE anything is created — no pack dir, no WAV, no artifact — and
# the same gate is asserted a second time from the manifest that was just written
# (plus the recorded sha256 against the source it claims). A pack carrying a
# forbidden crop therefore cannot survive a build.
#
# THE ARTIFACT SIZES ARE CHECKED, NOT ASSUMED. They are a property of the codec
# config inside the GGUF, so an engine bump that changes enc_dim or the codebook
# count fails the build instead of shipping a pack the engine will reject at
# synthesis time. Note the interaction with the WAV window: an 18.0 s crop is a
# legal reference.wav but only 22 * 225 = 4,950 B of .rvq, under the recorded
# 5,456 B floor, so the effective crop window is 19.84-25 s. That is a real
# constraint and the builder says so up front rather than failing after a build;
# serverAppConfigs-docs/Dankclaw/Voice Packs.md covers what to do on an engine bump.
#
# WHERE IT RUNS: the host that holds the engine build (Dankclaw). The extractor is
# a CPU ggml binary — no GPU, no container, no fleet endpoint, no network. ffmpeg
# does the crop; every spawned process runs `nice -n 19` so a pack build never
# competes with the text endpoints on that box.
#
# IT NEVER DELETES. An existing pack is refused unless --force, and --force moves
# the whole old directory aside into $PACKS_DIR/.AGENTTRASH/<voice>-<UTC>/ rather
# than overwriting or unlinking a single byte.
#
# Usage:
# scripts/build-voice-pack.sh [OPTIONS] VOICE PACKS_DIR
# scripts/build-voice-pack.sh [OPTIONS] --all PACKS_DIR
#
# Options (anything not given comes from scripts/voice-packs.env):
# --env FILE tunables file (default scripts/voice-packs.env)
# --source FILE source recording for this voice
# --offset SEC crop start, in seconds into the source (must be >= 10)
# --duration SEC crop length; reference.wav comes out this long
# --sha256 HEX expected sha256 of the source file
# --tool PATH the pinned speaker extractor (qwen-codec)
# --all build every voice in VOICE_PACKS, skipping none
# --dry-run everything except the extractor: validate, cut the crop,
# probe it, print the plan and the manifest it would write
# --force retire an existing pack to .AGENTTRASH/ and rebuild
# -h, --help this text
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
ENV_FILE="$REPO_ROOT/scripts/voice-packs.env"
SOURCE_FORBIDDEN_SECONDS=10 # the source-material invariant; not a knob
WORK_DIR="" # scratch dir a dry run owns, removed on exit
# Only ever removes the scratch directory this script created for itself; pack
# bytes on disk are retired to .AGENTTRASH/, never deleted (see --force).
cleanup_work_dir() {
if [ -n "$WORK_DIR" ]; then
rm -rf -- "$WORK_DIR"
fi
}
trap cleanup_work_dir EXIT
die() {
printf 'ERROR: %s\n' "$*" >&2
exit 1
}
usage() {
sed -n '2,/^set -euo/p' "${BASH_SOURCE[0]}" | sed '$d' | sed 's/^# \{0,1\}//'
}
# fcmp A OP B — float comparison (awk); OP is one of < <= > >= == and A/B are
# already validated as plain decimals, so nothing user-shaped reaches awk as
# program text.
fcmp() {
awk -v a="$1" -v b="$3" "BEGIN { exit !(a $2 b) }"
}
# floorn X — greatest integer <= X, printed. Only ever called on positives.
floorn() {
awk -v x="$1" 'BEGIN { printf "%d\n", int(x) }'
}
is_seconds() {
[[ "$1" =~ ^[0-9]+([.][0-9]+)?$ ]]
}
is_hex64() {
[[ "$1" =~ ^[0-9a-f]{64}$ ]]
}
# ---------------------------------------------------------------- arguments --
VOICE=""
PACKS_DIR=""
ARG_SOURCE=""
ARG_OFFSET=""
ARG_DURATION=""
ARG_SHA=""
ARG_TOOL=""
DRY_RUN=0
FORCE=0
BUILD_ALL=0
while [ "$#" -gt 0 ]; do
case "$1" in
--env) ENV_FILE="${2:?--env needs a path}"; shift 2 ;;
--source) ARG_SOURCE="${2:?--source needs a file}"; shift 2 ;;
--offset) ARG_OFFSET="${2:?--offset needs seconds}"; shift 2 ;;
--duration) ARG_DURATION="${2:?--duration needs seconds}"; shift 2 ;;
--sha256) ARG_SHA="${2:?--sha256 needs a hex digest}"; shift 2 ;;
--tool) ARG_TOOL="${2:?--tool needs a path}"; shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
--force) FORCE=1; shift ;;
--all) BUILD_ALL=1; shift ;;
-h|--help) usage; exit 0 ;;
--*) die "unknown option: $1 (see --help)" ;;
*)
if [ -z "$VOICE" ]; then
VOICE="$1"
elif [ -z "$PACKS_DIR" ]; then
PACKS_DIR="$1"
else
die "unexpected argument: $1"
fi
shift
;;
esac
done
[ -f "$ENV_FILE" ] || die "tunables file not found: $ENV_FILE"
for tool in ffmpeg ffprobe jq sha256sum; do
command -v "$tool" >/dev/null 2>&1 || die "$tool is required and not on PATH"
done
# shellcheck disable=SC1090
source "$ENV_FILE"
# Tunables the env file owes us. Defaults are the engine facts read out of the
# pinned tree; an env file that omits one still builds against the truth.
VOICE_PACKS="${VOICE_PACKS:-}"
SAMPLE_RATE="${VOICE_PACK_SAMPLE_RATE:-24000}"
MIN_SECONDS="${VOICE_PACK_MIN_SECONDS:-18}"
MAX_SECONDS="${VOICE_PACK_MAX_SECONDS:-25}"
FRAME_RATE="${VOICE_PACK_FRAME_RATE:-12.5}"
RVQ_BYTES_PER_FRAME="${VOICE_PACK_RVQ_BYTES_PER_FRAME:-22}"
SPK_BYTES="${VOICE_PACK_SPK_BYTES:-8192}"
RVQ_MIN_BYTES="${VOICE_PACK_RVQ_MIN_BYTES:-5456}"
RVQ_MAX_BYTES="${VOICE_PACK_RVQ_MAX_BYTES:-6864}"
TOOL="${ARG_TOOL:-${VOICE_PACK_TOOL:-}}"
CODEC_MODEL="${VOICE_PACK_CODEC_MODEL:-}"
TALKER_MODEL="${VOICE_PACK_TALKER_MODEL:-}"
ENGINE_REPO="${VOICE_PACK_ENGINE_REPO:-https://github.com/ServeurpersoCom/qwentts.cpp}"
ENGINE_COMMIT="${VOICE_PACK_ENGINE_COMMIT:-5916bf7c79dbb4e66fc059cdc507a79a23a28bb4}"
DEVICE="${VOICE_PACK_DEVICE:-CPU}"
THREADS="${VOICE_PACK_THREADS:-8}"
NICE_LEVEL="${VOICE_PACK_NICE:-19}"
TRASH_DIRNAME="${VOICE_PACK_TRASH_DIRNAME:-.AGENTTRASH}"
DEFAULT_PACKS_DIR="${VOICE_PACK_DIR:-/home/dank/voices}"
[ -n "$PACKS_DIR" ] || PACKS_DIR="$DEFAULT_PACKS_DIR"
[ -n "$PACKS_DIR" ] || die "no packs directory (pass it as an argument)"
is_seconds "$SAMPLE_RATE" || die "VOICE_PACK_SAMPLE_RATE is not a number: $SAMPLE_RATE"
is_seconds "$MIN_SECONDS" || die "VOICE_PACK_MIN_SECONDS is not a number: $MIN_SECONDS"
is_seconds "$MAX_SECONDS" || die "VOICE_PACK_MAX_SECONDS is not a number: $MAX_SECONDS"
# hop / frame arithmetic, derived once: the codec runs at FRAME_RATE Hz over
# SAMPLE_RATE audio, so one frame is HOP_SAMPLES samples and one second is
# FRAME_RATE frames.
HOP_SAMPLES="$(floorn "$(awk -v r="$SAMPLE_RATE" -v f="$FRAME_RATE" 'BEGIN { print r / f }')")"
[ "$HOP_SAMPLES" -gt 0 ] || die "VOICE_PACK_FRAME_RATE ($FRAME_RATE) exceeds the sample rate"
BYTES_PER_SECOND="$(awk -v b="$RVQ_BYTES_PER_FRAME" -v f="$FRAME_RATE" 'BEGIN { print b * f }')"
# rvq_bytes_for DURATION — packed size the extractor will emit for a crop this
# long: whole frames only, because it truncates to the hop boundary.
rvq_bytes_for() {
local samples frames
samples="$(awk -v d="$1" -v r="$SAMPLE_RATE" 'BEGIN { printf "%d\n", int(d * r + 0.5) }')"
frames=$((samples / HOP_SAMPLES))
printf '%d\n' "$((frames * RVQ_BYTES_PER_FRAME))"
}
# ------------------------------------------------------------------- build --
build_voice() {
local voice="$1"
# The pack dir name is the voice name on the wire: app.py's VOICE_NAME_RE
# matches it exactly while the engine lowercases names internally, so an
# uppercase or punctuated name would be reported one way by the façade and
# another by the engine. Lowercase only, and nothing that could escape
# $PACKS_DIR.
[[ "$voice" =~ ^[a-z0-9][a-z0-9._-]{0,63}$ ]] ||
die "voice name '$voice' is not a lowercase [a-z0-9._-] name starting with alphanumerics (app.py resolves it verbatim; the engine lowercases it)"
local prefix
prefix="VOICE_PACK_$(printf '%s' "$voice" | tr '[:lower:]' '[:upper:]' | tr -c '[:alnum:]' '_')"
local source_var="${prefix}_SOURCE" offset_var="${prefix}_OFFSET"
local duration_var="${prefix}_DURATION" sha_var="${prefix}_SHA256"
local source="${ARG_SOURCE:-${!source_var:-}}"
local offset="${ARG_OFFSET:-${!offset_var:-}}"
local duration="${ARG_DURATION:-${!duration_var:-}}"
local want_sha="${ARG_SHA:-${!sha_var:-}}"
if [ -z "$voice" ] || [ -z "$source" ]; then
die "no source for '$voice': pass --source or set ${source_var} in $ENV_FILE"
fi
[ -n "$offset" ] || die "no crop offset for '$voice': pass --offset or set ${offset_var}"
[ -n "$duration" ] || die "no crop duration for '$voice': pass --duration or set ${duration_var}"
is_seconds "$offset" || die "crop offset '$offset' is not a number of seconds"
is_seconds "$duration" || die "crop duration '$duration' is not a number of seconds"
[ -f "$source" ] && [ -r "$source" ] || die "source not readable: $source"
# === THE GATE. Source seconds 0..10 are forbidden voice material, full stop.
if ! fcmp "$offset" '>=' "$SOURCE_FORBIDDEN_SECONDS"; then
die "source-material invariant violated: the crop must start past the first ${SOURCE_FORBIDDEN_SECONDS} s of the source file, and '$voice' asks for ${offset} s into $source. Pick an offset at or past ${SOURCE_FORBIDDEN_SECONDS} s — nothing was written."
fi
local pack_dir="$PACKS_DIR/$voice"
if [ "$DRY_RUN" -ne 1 ] && [ -e "$pack_dir" ] && [ "$FORCE" -ne 1 ]; then
die "$pack_dir already exists; nothing was touched. Re-run with --force to retire it to $PACKS_DIR/$TRASH_DIRNAME/ and rebuild."
fi
local source_duration source_bytes source_sha
source_duration="$(ffprobe -v error -show_entries format=duration -of default=nw=1:nk=1 "$source")"
is_seconds "$source_duration" || die "ffprobe reports no numeric duration for $source: '$source_duration'"
source_bytes="$(stat -c%s "$source")"
if ! fcmp "$(awk -v a="$offset" -v d="$duration" 'BEGIN { print a + d }')" '<=' "$source_duration"; then
die "crop ends at $(awk -v a="$offset" -v d="$duration" 'BEGIN { print a + d }') s but $source is only $source_duration s; the crop would come out short"
fi
if ! fcmp "$duration" '>=' "$MIN_SECONDS" || ! fcmp "$duration" '<=' "$MAX_SECONDS"; then
die "crop duration $duration s is outside the ${MIN_SECONDS}-${MAX_SECONDS} s reference window"
fi
local crop_end planned_rvq
crop_end="$(awk -v a="$offset" -v d="$duration" 'BEGIN { print a + d }')"
planned_rvq="$(rvq_bytes_for "$duration")"
if ! fcmp "$planned_rvq" '>=' "$RVQ_MIN_BYTES" || ! fcmp "$planned_rvq" '<=' "$RVQ_MAX_BYTES"; then
die "a $duration s crop packs to ${planned_rvq} B of .rvq, outside the recorded engine window ${RVQ_MIN_BYTES}-${RVQ_MAX_BYTES} B; lengthen the crop (that window starts at $(awk -v m="$RVQ_MIN_BYTES" -v b="$BYTES_PER_SECOND" 'BEGIN { print m / b }') s) — the source, not the codec, sets this floor"
fi
echo "VOICE $voice"
echo "SOURCE $source ($source_bytes bytes, ${source_duration} s)"
echo "CROP ${offset} s .. ${crop_end} s (${duration} s) — past the ${SOURCE_FORBIDDEN_SECONDS} s forbidden region"
source_sha="$(sha256sum "$source" | awk '{ print $1 }')"
if [ -n "$want_sha" ]; then
is_hex64 "$want_sha" || die "expected sha256 '$want_sha' is not 64 hex digits"
[ "$source_sha" = "$want_sha" ] ||
die "source sha256 is $source_sha, expected $want_sha: $source is not the file these offsets were cut from"
fi
echo "SHA256 $source_sha"
# Where the crop is cut. A dry run keeps its work in a scratch dir it owns;
# a real build stages INSIDE the pack dir so the final rename is on one
# filesystem and the pack becomes visible only when it is whole.
local stage_dir
if [ "$DRY_RUN" -eq 1 ]; then
WORK_DIR="$(mktemp -d "${TMPDIR:-/tmp}/voice-pack-$voice.XXXXXX")"
stage_dir="$WORK_DIR"
else
mkdir -p "$PACKS_DIR"
if [ -e "$pack_dir" ]; then
local trash
trash="$PACKS_DIR/$TRASH_DIRNAME/$voice-$(date -u +%Y%m%dT%H%M%SZ)"
mkdir -p "$(dirname "$trash")"
mv "$pack_dir" "$trash"
echo "RETIRED $pack_dir -> $trash (nothing deleted)"
fi
mkdir "$pack_dir" || die "cannot create $pack_dir (it may exist now); nothing was overwritten"
stage_dir="$pack_dir"
fi
local stage_wav="$stage_dir/staging-reference.wav"
nice -n "$NICE_LEVEL" env OMP_NUM_THREADS="$THREADS" \
ffmpeg -hide_banner -loglevel error -y \
-ss "$offset" -t "$duration" -i "$source" \
-map 0:a:0 -vn -ac 1 -ar "$SAMPLE_RATE" -c:a pcm_s16le "$stage_wav"
local wav_rate wav_channels wav_codec wav_duration
wav_rate="$(ffprobe -v error -select_streams a:0 -show_entries stream=sample_rate -of default=nw=1:nk=1 "$stage_wav")"
wav_channels="$(ffprobe -v error -select_streams a:0 -show_entries stream=channels -of default=nw=1:nk=1 "$stage_wav")"
wav_codec="$(ffprobe -v error -select_streams a:0 -show_entries stream=codec_name -of default=nw=1:nk=1 "$stage_wav")"
wav_duration="$(ffprobe -v error -select_streams a:0 -show_entries stream=duration -of default=nw=1:nk=1 "$stage_wav")"
is_seconds "$wav_rate" || die "ffprobe reports no numeric sample rate for the crop: '$wav_rate'"
is_seconds "$wav_duration" || die "ffprobe reports no numeric duration for the crop: '$wav_duration'"
[ "$wav_rate" = "$SAMPLE_RATE" ] || die "reference.wav came out at ${wav_rate} Hz, not ${SAMPLE_RATE}"
[ "$wav_channels" = "1" ] || die "reference.wav came out with $wav_channels channels, not mono"
[ "$wav_codec" = "pcm_s16le" ] || die "reference.wav came out as $wav_codec, not the pcm_s16le the law names"
if ! fcmp "$wav_duration" '>=' "$MIN_SECONDS" || ! fcmp "$wav_duration" '<=' "$MAX_SECONDS"; then
die "reference.wav is ${wav_duration} s, outside the ${MIN_SECONDS}-${MAX_SECONDS} s window"
fi
echo "REFERENCE ${wav_rate} Hz ${wav_codec} mono, ${wav_duration} s, $(stat -c%s "$stage_wav") bytes"
local spk_file="$stage_dir/staging-reference.spk" rvq_file="$stage_dir/staging-reference.rvq"
local stage_log="$stage_dir/staging-qwen-codec.log"
local spk_bytes=0 rvq_bytes=0 produced=false
if [ "$DRY_RUN" -eq 1 ]; then
if [ -n "$TOOL" ] && [ -x "$TOOL" ]; then
echo "EXTRACTOR $TOOL (present; --dry-run will not run it)"
else
echo "EXTRACTOR not present (${TOOL:-unset}) — --dry-run does not need it"
fi
echo "PLANNED reference.spk ${SPK_BYTES} B, reference.rvq $(rvq_bytes_for "$wav_duration") B"
else
[ -n "$TOOL" ] && [ -x "$TOOL" ] ||
die "speaker extractor missing or not executable: ${TOOL:-VOICE_PACK_TOOL unset} (use --dry-run to build everything up to this point)"
[ -f "$CODEC_MODEL" ] || die "codec model missing: $CODEC_MODEL"
[ -f "$TALKER_MODEL" ] || die "talker model missing: $TALKER_MODEL"
# The extractor takes the WAV and writes its outputs next to it with the
# extension swapped — there is no -o, so the staged name decides where
# the artifacts land.
local extractor=("$TOOL" --model "$CODEC_MODEL" --talker "$TALKER_MODEL" -i "$stage_wav")
if [ -n "$DEVICE" ]; then
extractor+=(--device "$DEVICE")
fi
echo "EXTRACT $(basename "$TOOL") at $ENGINE_COMMIT (device ${DEVICE:-auto}, $THREADS threads)"
if ! nice -n "$NICE_LEVEL" env OMP_NUM_THREADS="$THREADS" \
"${extractor[@]}" 2>&1 | tee "$stage_log" >&2; then
die "$TOOL failed on $stage_wav (its output is in $stage_log); the pack was not published"
fi
[ -f "$spk_file" ] || die "$TOOL wrote no $spk_file (see $stage_log)"
[ -f "$rvq_file" ] || die "$TOOL wrote no $rvq_file (see $stage_log)"
spk_bytes="$(stat -c%s "$spk_file")"
rvq_bytes="$(stat -c%s "$rvq_file")"
local expect_rvq
expect_rvq="$(rvq_bytes_for "$wav_duration")"
[ "$spk_bytes" = "$SPK_BYTES" ] ||
die "reference.spk is $spk_bytes B, expected ${SPK_BYTES} B (enc_dim x f32). The engine changed its speaker encoder — see [[Voice Packs]] before shipping this pack"
[ "$rvq_bytes" = "$expect_rvq" ] ||
die "reference.rvq is $rvq_bytes B, expected ${expect_rvq} B for a ${wav_duration} s crop (22 B per 12.5 Hz frame)"
if ! fcmp "$rvq_bytes" '>=' "$RVQ_MIN_BYTES" || ! fcmp "$rvq_bytes" '<=' "$RVQ_MAX_BYTES"; then
die "reference.rvq is $rvq_bytes B, outside the recorded engine window ${RVQ_MIN_BYTES}-${RVQ_MAX_BYTES} B — an engine bump? see [[Voice Packs]]"
fi
produced=true
echo "ARTIFACTS reference.spk ${spk_bytes} B, reference.rvq ${rvq_bytes} B"
fi
local built stage_manifest
built="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
stage_manifest="$stage_dir/staging-manifest.json"
jq -n \
--arg voice "$voice" \
--arg built "$built" \
--arg builder "scripts/build-voice-pack.sh" \
--arg env_file "$ENV_FILE" \
--arg src "$source" \
--arg src_sha "$source_sha" \
--argjson src_bytes "$source_bytes" \
--argjson src_duration "$source_duration" \
--argjson offset "$offset" \
--argjson end "$crop_end" \
--argjson duration "$duration" \
--argjson forbidden "$SOURCE_FORBIDDEN_SECONDS" \
--argjson rate "$SAMPLE_RATE" \
--argjson wav_duration "$wav_duration" \
--arg wav_codec "$wav_codec" \
--argjson wav_channels "$wav_channels" \
--argjson wav_bytes "$(stat -c%s "$stage_wav")" \
--arg tool "${TOOL:-unset}" \
--arg repo "$ENGINE_REPO" \
--arg commit "$ENGINE_COMMIT" \
--arg codec "$CODEC_MODEL" \
--arg talker "$TALKER_MODEL" \
--arg device "$DEVICE" \
--argjson produced "$produced" \
--argjson spk_bytes "$spk_bytes" \
--argjson rvq_bytes "$rvq_bytes" \
--argjson planned_rvq "$(rvq_bytes_for "$wav_duration")" \
--argjson dry_run "$DRY_RUN" \
'{
voice: $voice,
kind: "voice-pack",
built_utc: $built,
builder: $builder,
tunables: $env_file,
dry_run: ($dry_run == 1),
source: {
path: $src,
sha256: $src_sha,
bytes: $src_bytes,
duration_seconds: $src_duration
},
crop: {
offset_seconds: $offset,
end_seconds: $end,
duration_seconds: $duration
},
reference: {
file: "reference.wav",
codec: $wav_codec,
sample_rate: $rate,
channels: $wav_channels,
duration_seconds: $wav_duration,
bytes: $wav_bytes
},
artifacts: {
produced: $produced,
tool: $tool,
engine_repo: $repo,
engine_commit: $commit,
device: $device,
codec_model: $codec,
talker_model: $talker,
spk: {file: "reference.spk", bytes: $spk_bytes},
rvq: {file: "reference.rvq", bytes: $rvq_bytes, planned_bytes: $planned_rvq}
},
invariant: {
name: "source-material invariant",
rule: "a voice crop MUST start past the first 10 seconds of the source file",
forbidden_source_seconds: $forbidden,
crop_offset_seconds: $offset
}
}' >"$stage_manifest"
# Assert the gate a second time, FROM the manifest, next to the bytes it
# describes. Anything short of a pass leaves the staged pack unpublished.
local recorded_offset recorded_sha
recorded_offset="$(jq -r '.crop.offset_seconds' "$stage_manifest")"
recorded_sha="$(jq -r '.source.sha256' "$stage_manifest")"
jq -e --argjson min "$SOURCE_FORBIDDEN_SECONDS" \
'(.crop.offset_seconds | type == "number") and .invariant.forbidden_source_seconds == $min and (.crop.offset_seconds >= $min) and (.invariant.name == "source-material invariant")' \
"$stage_manifest" >/dev/null ||
die "manifest assertion failed: .crop.offset_seconds ($recorded_offset) does not clear the ${SOURCE_FORBIDDEN_SECONDS} s source-material invariant"
[ "$recorded_sha" = "$source_sha" ] ||
die "manifest assertion failed: .source.sha256 ($recorded_sha) is not the hash of $source ($source_sha)"
jq -e '(.voice | type == "string") and (.reference.sample_rate | type == "number") and (.reference.duration_seconds | type == "number") and (.artifacts.engine_commit | length == 40)' \
"$stage_manifest" >/dev/null ||
die "manifest assertion failed: voice / reference / engine provenance fields are not all present"
echo "MANIFEST crop offset $recorded_offset s clears the ${SOURCE_FORBIDDEN_SECONDS} s invariant"
if [ "$DRY_RUN" -eq 1 ]; then
echo "MANIFEST_BEGIN"
cat "$stage_manifest"
echo "MANIFEST_END"
echo "PLAN would write $pack_dir/{reference.wav,reference.spk,reference.rvq,manifest.json}"
echo "DRYRUN $voice: inputs valid, crop cut and probed, extractor not run"
else
# reference.wav is the gate the shim reads, so it lands last: the voice
# name never exists on disk in a state the artifacts are not behind it.
mv "$stage_log" "$pack_dir/qwen-codec.log"
mv "$spk_file" "$pack_dir/reference.spk"
mv "$rvq_file" "$pack_dir/reference.rvq"
mv "$stage_manifest" "$pack_dir/manifest.json"
mv "$stage_wav" "$pack_dir/reference.wav"
echo "BUILT $pack_dir (voice '$voice': ${wav_duration} s, spk ${spk_bytes} B, rvq ${rvq_bytes} B)"
echo "VOICE_PACK_BUILT $voice $pack_dir"
fi
}
if [ "$BUILD_ALL" -eq 1 ]; then
[ -z "$VOICE" ] || die "--all takes no voice name (got '$VOICE'); the list comes from VOICE_PACKS"
for flag in "$ARG_SOURCE" "$ARG_OFFSET" "$ARG_DURATION" "$ARG_SHA"; do
[ -z "$flag" ] || die "--all reads every crop from $ENV_FILE; --source/--offset/--duration/--sha256 would apply one value to all of them"
done
read -r -a voices <<<"$VOICE_PACKS"
[ "${#voices[@]}" -gt 0 ] || die "VOICE_PACKS is empty in $ENV_FILE"
for voice in "${voices[@]}"; do
build_voice "$voice"
done
else
[ -n "$VOICE" ] || { usage >&2; die "no voice name given"; }
build_voice "$VOICE"
fi
+293
View File
@@ -0,0 +1,293 @@
#!/usr/bin/env python3
"""OKF v0.1 conformance linter for a docs bundle (default serverAppConfigs-docs).
Checks the hard rules of the Open Knowledge Format spec plus the maintenance
guidance, so "conforms to OKF" is a test and not a claim:
FRONTMATTER every non-reserved .md has parseable YAML frontmatter with a
non-empty `type` from the spec's value set
TYPE the value is one of reference / howto / tutorial / explanation
DESC `description` is present, non-blank, not the title, <= 120 chars,
and does not restate a fleet quantity
TIMESTAMP `timestamp` present, ISO 8601, not in the future
LENGTH lines between 1 and 500 inclusive
INDEX-FM no index.md may carry frontmatter (index.md is reserved)
LOG-DATE log.md date headings are `## YYYY-MM-DD` or `## YYYY-MM-DD — Title`,
one entry per date, newest first
INDEX-COVER every index.md links each of its sibling pages; the root index
links every page in the bundle
INDEX-NUM an index table row restates a fleet quantity — a digit run of
three or more, a digits-plus-unit figure, a port, or a
multiplier. Names (Qwen3.6, RDNA4, RTX 3090, PICO-8, P4) are
not quantities and are allowed
CITATIONS informational: pages with factual claims and no Citations
section are counted, never failed
LENGTH lines between 1 and 500 inclusive. A page may opt out of the
ceiling with `okf-exempt: length` in its frontmatter: AGENTS.md
makes one page per endpoint permanent, and an endpoint page with
a full Eras table cannot split without creating a second owner
of the same shape (tests/docs_completeness.bats enforces that)
Exit 0 when there are no violations.
"""
import datetime
import re
import sys
from pathlib import Path
TYPES = {"reference", "howto", "tutorial", "explanation"}
RESERVED = {"index.md", "log.md", "AGENTS.md"}
# A quantity, as opposed to a name: a comma-grouped figure, a run of 3+ digits,
# a unit-bearing figure, a port (":7770"), or a multiplier ("2x", "3 *").
QUANTITY = re.compile(
r"\d{1,3}(?:,\d{3})+" # 262,144
r"|\d{3,}" # 200000, 7770, 3090
r"|\d+(?:\.\d+)?\s*(?:GiB|MiB|GB|MB|TB|tok/s|t/s|x\b|%)"
r"|[:×]\s*\d"
r"|\d+\s*[x×]\s*\d"
)
# Product / model / standard names that legitimately contain digits.
NAMES = re.compile(
r"Qwen[\d.]*[-\w]*|Gemma\s?\d[\w.]*|MiMo[-\w.\d.]*|Ornith[-\w.\d]*"
r"|Bonsai[-\w.\d]*|Ling[-\w.\d.]*|DeepSeek[-\w.\d]*|GLM[-\w.\d]*"
r"|PICO-\d+|RDNA\s?\d+|R\d{4}|RTX\s?\d+|XDNA\d|gfx\d+|CUDA\s?[\d.]+"
r"|\d+\s?GbE|Wi-?Fi\s?\d|v?\d+\.\d+(?:\.\d+)?|b\d+|P\d\b|A\d+\b"
r"|frank-\w*2026-\d\d-\d\d|2026-\d\d-\d\d",
re.I,
)
DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}")
FM_KEY = re.compile(r"^([A-Za-z][\w-]*):\s*(.*)$")
def split_frontmatter(text):
if not text.startswith("---"):
return None, text
end = text.find("\n---", 3)
if end < 0:
return None, text
body_start = text.find("\n", end + 1)
return text[3:end], text[body_start + 1 :] if body_start >= 0 else ""
def fields(raw):
out = {}
if not raw:
return out
key = None
for line in raw.splitlines():
if not line.strip():
continue
m = FM_KEY.match(line)
if m:
key, val = m.group(1), m.group(2).strip().strip("\"'")
out[key] = val
elif line.startswith(" ") and key:
out[key] = (out[key] + " " + line.strip()).strip()
return out
def strip_names(value):
return NAMES.sub(" ", value)
def check_file(path: Path, root: Path, length_exempt=frozenset()):
v = []
rel = str(path.relative_to(root))
text = path.read_text(errors="replace")
lines = text.splitlines()
name = path.name
raw, body = split_frontmatter(text)
fm = fields(raw)
if name not in RESERVED:
if raw is None:
v.append(("FRONTMATTER", rel, "no parseable frontmatter"))
else:
t = fm.get("type", "")
if not t:
v.append(("TYPE", rel, "missing type"))
elif t not in TYPES:
v.append(("TYPE", rel, f"type '{t}' not in {sorted(TYPES)}"))
d = fm.get("description", "")
if not d:
v.append(("DESC", rel, "missing description"))
else:
if len(d) > 120:
v.append(("DESC", rel, f"description is {len(d)} chars (max 120)"))
if d.strip("# ").lower() == fm.get("title", "\0").strip("# ").lower():
v.append(("DESC", rel, "description equals title"))
if QUANTITY.search(strip_names(d)):
v.append(("DESC", rel, "description restates a quantity"))
ts = fm.get("timestamp", "")
if not ts:
v.append(("TIMESTAMP", rel, "missing timestamp"))
else:
try:
stamp = datetime.date.fromisoformat(ts[:10])
if stamp > datetime.date.today():
v.append(("TIMESTAMP", rel, f"timestamp {ts} is in the future"))
except ValueError:
v.append(("TIMESTAMP", rel, f"unparseable timestamp {ts!r}"))
if name == "index.md" and raw is not None:
v.append(("INDEX-FM", rel, "index.md must not carry frontmatter"))
if name == "log.md":
dates = []
for line in lines:
m = re.match(r"^##\s+(\S.*?)\s*$", line)
if not m:
continue
head = m.group(1)
if not DATE_RE.match(head):
continue
dates.append(head[:10])
# OKF requires the heading to USE the date format; a title after an
# em-dash is allowed and costs zero lines, so `## 2026-08-17 — Why the
# pair was rebuilt` is conformant and `## 2026-08-17Rebuilt` is not.
if head != head[:10] and not re.match(r"^\d{4}-\d{2}-\d{2}\s+—\s+\S", head):
v.append(("LOG-DATE", rel,
f"date heading must be the date alone or `date — Title`: '## {head}'"))
if len(dates) != len(set(dates)):
v.append(("LOG-DATE", rel, "duplicate date heading"))
if dates != sorted(dates, reverse=True):
v.append(("LOG-DATE", rel, "date headings not newest first"))
if raw is not None and name not in RESERVED:
n = len(lines)
body_lines = len([ln for ln in body.splitlines() if ln.strip()])
opted_out = "length" in fm.get("okf-exempt", "").lower()
if n > 500:
if rel in length_exempt:
v.append(("LENGTH-OK", rel,
f"{n} lines — over the hint, pinned by the "
"one-page-per-endpoint law"))
elif opted_out:
v.append(("LENGTH-OK", rel,
f"{n} lines — over the hint, opted out by the page"))
else:
v.append(("LENGTH", rel, f"{n} lines (max 500)"))
elif body_lines < 15:
v.append(("LENGTH", rel, "stub: fewer than 15 body lines"))
# A second `type:` key inside the body means the page carries two copies of
# itself with a frontmatter block in the middle — the shape a naive
# frontmatter rewrite produces. It doubles the length, splits the wikilinks
# and makes the page's facts ambiguous, so it is a hard failure.
if raw is not None and name not in RESERVED:
dupes = [i + len((raw or "").splitlines()) + 2
for i, ln in enumerate(body.splitlines())
if re.match(r"^(type|title|description):", ln)]
if dupes:
v.append(("DOUBLED", rel,
f"frontmatter keys inside the body at line(s) "
f"{', '.join(str(d) for d in dupes[:5])} — the page "
"contains a duplicated copy of itself"))
return v
def index_rows(text):
rows = []
for line in text.splitlines():
if line.startswith("|") and "[[" in line:
rows.append(line)
return rows
def check_links(root: Path):
v = []
pages = [
p for p in sorted(root.rglob("*.md"))
if ".AGENTTRASH" not in str(p) and ".obsidian" not in str(p)
]
for index in [p for p in pages if p.name == "index.md"]:
text = index.read_text(errors="replace")
links = set(re.findall(r"\[\[([^\]|#]+)", text))
if index.parent == root:
targets = [p for p in pages if p.name not in RESERVED]
else:
targets = [p for p in index.parent.glob("*.md") if p.name != "index.md"]
for target in targets:
if target.stem not in links and target.name not in links:
v.append(("INDEX-COVER", str(index.relative_to(root)),
f"does not link sibling {target.name}"))
for index in [p for p in pages if p.name == "index.md"]:
for row in index_rows(index.read_text(errors="replace")):
if QUANTITY.search(strip_names(row)):
v.append(("INDEX-NUM", str(index.relative_to(root)),
"row restates a quantity: " + row[:80]))
return v
def live_endpoints(root: Path):
"""Pages pinned by AGENTS.md's one-page-per-endpoint law.
Derived, never hardcoded — the same derivation as
tests/docs_completeness.bats: an endpoint is live when scripts/ holds both
its env and its unit. Those pages carry the Eras table for their whole
history, so the OKF 500-line split hint cannot apply to them without
creating a second owner of the same shape.
"""
scripts = root.parent / "scripts"
out = set()
if scripts.is_dir():
for envf in scripts.glob("frank-*.env"):
if (scripts / f"{envf.stem}.service").exists():
out.add(f"LLM/{envf.stem}.md")
return out
def retired_era_pages(root: Path):
"""Era pages of a retired endpoint: campaign ledgers, one owner each.
AGENTS.md lets a campaign ledger that exists nowhere else live on the page
that measured it, and a retired endpoint keeps exactly one page per era.
Splitting such a ledger to satisfy the 500-line hint would fragment one
campaign across two owners, so the tag opts the page out.
"""
out = set()
for path in root.rglob("*.md"):
s = str(path)
if ".AGENTTRASH" in s or ".obsidian" in s:
continue
raw, _ = split_frontmatter(path.read_text(errors="replace"))
if raw is not None and "retired-endpoint" in fields(raw).get("tags", ""):
out.add(str(path.relative_to(root)))
return out
def main(argv):
root = Path(argv[1] if len(argv) > 1 else "serverAppConfigs-docs")
exempt = frozenset(live_endpoints(root)) | retired_era_pages(root)
violations = []
notes = []
citation_gaps = 0
pages = 0
for path in sorted(root.rglob("*.md")):
s = str(path)
if ".AGENTTRASH" in s or ".obsidian" in s:
continue
pages += 1
for code, rel, detail in check_file(path, root, exempt):
(notes if code == "LENGTH-OK" else violations).append((code, rel, detail))
text = path.read_text(errors="replace")
if len(text.splitlines()) > 40 and not re.search(
r"^##\s+(Citations|Sources)\b", text, re.M):
citation_gaps += 1
violations += check_links(root)
for code, rel, detail in violations:
print(f" {code:12s} {rel}: {detail}")
for code, rel, detail in notes:
print(f" {code:12s} {rel}: {detail}")
print(f"{pages} concept pages, {len(violations)} OKF violations"
f" ({citation_gaps} pages without a Citations section — soft;"
f" {len(notes)} length exemptions honoured)")
return 1 if violations else 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+6 -1
View File
@@ -31,7 +31,12 @@ def strip_md(path: str) -> str:
def build_index(vault: pathlib.Path):
pages = sorted(vault.rglob("*.md"))
# `.AGENTTRASH` is deleted-but-held state, not vault content: it must not be
# scanned (its relative links resolve against the trash dir, not the vault)
# and must not answer as a link target — a live page pointing into the trash
# is an unresolved link, not a resolved one.
pages = sorted(p for p in vault.rglob("*.md")
if ".AGENTTRASH" not in str(p))
stems: dict[str, list[pathlib.Path]] = {}
for page in pages:
stems.setdefault(page.stem.casefold(), []).append(page)
+43
View File
@@ -62,6 +62,11 @@ SERVICE_ORDER = (
"sonarr",
"lidarr",
"readarr",
# The voice pair keeps the inference family's ordering rule: the engine is
# a dependency of the 7772 facade, so it is planned (and therefore
# restarted) first.
"frank-tts-engine",
"frank-tts",
"frank-dragoon",
"frank-zealot",
"frank-archon",
@@ -97,6 +102,8 @@ DANKCLAW_SERVICES = {
"homepage",
"frank-dragoon",
"frank-zealot",
"frank-tts-engine",
"frank-tts",
}
DANKSERVER_SERVICES = {
@@ -326,6 +333,27 @@ def services_for_path(path: str, target: str) -> set[str]:
# plans the pair, which deploy_open_notebook starts database-first.
_add(services, "open-notebook")
return services
if len(parts) >= 2 and parts[1] == "voiceservice":
# The voice stack's containers share one directory but are TWO
# deployables — the arm A engine and the 7772 façade in front of it
# — so this maps per file, the way the authentik trio does, and not
# per directory. Anything that is not named after one of the two
# services plans nothing, exactly as it did before the pair was
# registered.
if parts[-1] in {"frank-tts-engine.container", "frank-tts-engine.network",
"Containerfile.engine", "tts-engine-start.sh",
"tts-engine-warm.sh"}:
_add(services, "frank-tts-engine")
# The 7772 façade is the other deployable in this directory. The
# bridge file is named after the ENGINE unit (install_quadlet copies
# `$unit.network` beside `$unit.container`), yet both units attach
# to it, so a subnet change is a change to both. Anything else here
# belongs to neither deployable and plans nothing.
if parts[-1] in {"frank-tts.container", "app.py", "Containerfile"}:
_add(services, "frank-tts")
if parts[-1] == "frank-tts-engine.network":
_add(services, "frank-tts")
return services
# frank-dragoon left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
# build, root unit): its files live in scripts/ and map there.
if len(parts) >= 2 and parts[1] == "observability":
@@ -429,6 +457,21 @@ def services_for_path(path: str, target: str) -> set[str]:
# what the live frank-dragoon endpoint renders, so it plans
# frank-dragoon.
_add(services, "frank-dragoon")
if name == "frank-tts-engine.env":
# The engine's knobs are command-line flags, and this file is the
# repo's copy of them (the image entrypoint translates one line to
# one flag), so editing it is a restart of the engine. It also
# carries the boot warmer's three knobs, because the warmer is
# exec-ed into this container and inherits the same EnvironmentFile.
# A restart is what re-registers and re-warms every voice: the
# engine's voice registry is memory-only. (The warmer itself lives
# with the image, mapped above with the files it is built from, like
# the serve scripts of frank-zealot and frank-archon are mapped to
# the units that run them.)
# scripts/download-frank-tts-engine-qwentts.sh deliberately plans
# nothing: it fetches host-side state, like every other download
# script in this directory.
_add(services, "frank-tts-engine")
if name in {"frank-archon.env", "frank-archon.service", "serve-frank-archon.sh"}:
# llama.cpp frank-archon (2026-09-26, replacing the Lucebox
# DS4.1 quadlet) keeps its unit, env and serve files here the
+118 -1
View File
@@ -92,6 +92,7 @@ service_is_allowed() {
dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
dankclaw:open-notebook|\
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
dankclaw:frank-tts-engine|dankclaw:frank-tts|\
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
@@ -540,6 +541,14 @@ deploy_observability() {
build_compose_image() {
local service="$1" image cfhash oldhash containerfile
# Every file the image COPIES IN, beside the Containerfile. The skip below
# compares a hash against the image's `cfhash` label, so anything the build
# bakes must be in that hash or the deploy silently keeps serving the old
# bytes: an edited app.py or warm hook leaves the Containerfile untouched,
# the label matches, `podman build` never runs, and the health gate passes on
# a stale container. Services whose Containerfile pulls everything at
# build time (comfyui, loki, alloy) keep an empty list.
local -a extra=()
case "$service" in
comfyui)
image=comfyui-rocm:gfx1151
@@ -549,11 +558,32 @@ build_compose_image() {
image="franksdank/$service:latest"
containerfile="dankclaw/$service/Containerfile"
;;
frank-tts)
# The 7772 façade: stdlib python + ffmpeg, app.py copied in. Listed
# explicitly because build_compose_image's `*)` arm returns 0 — an
# unlisted service builds nothing and the unit's Pull=never then
# fails on a tag that was never made.
image=franksdank/frank-tts:latest
containerfile=dankclaw/voiceservice/Containerfile
extra=(dankclaw/voiceservice/app.py)
;;
frank-tts-engine)
# The arm A TTS engine (qwentts.cpp `tts-server`). Built from
# source at a pinned upstream commit, so the build needs network
# and the cfhash label below is what keeps an unchanged re-deploy
# offline: the clone only happens when this Containerfile changes.
image=franksdank/frank-tts-engine:latest
containerfile=dankclaw/voiceservice/Containerfile.engine
# start.sh is the image's CMD and the warmer is the unit's
# ExecStartPost; both are COPY'd in, so both decide what this image is.
extra=(dankclaw/voiceservice/tts-engine-start.sh
dankclaw/voiceservice/tts-engine-warm.sh)
;;
*)
return 0
;;
esac
cfhash="$(sha256sum "$containerfile" | cut -d' ' -f1)"
cfhash="$(sha256sum "$containerfile" "${extra[@]}" | cut -d' ' -f1 | tr -d '\n')"
oldhash="$(podman image inspect "$image" --format '{{index .Config.Labels "cfhash"}}' 2>/dev/null || echo none)"
if [[ "$oldhash" != "$cfhash" ]]; then
# --format docker is required, not cosmetic: podman defaults to the OCI
@@ -744,6 +774,91 @@ deploy_open_notebook() {
wait_for_health open-notebook http://127.0.0.1:5056/health 180
}
# frank-tts-engine is the arm A TTS engine behind frank-tts: one Quadlet unit
# plus the private bridge its unit defines (install_quadlet copies the sibling
# frank-tts-engine.network — without that file the v1 generator refuses to
# produce the container unit at all, the same trap documented above). The
# façade that consumes it is a separate service on a separate port and a
# separate handler.
deploy_frank_tts_engine() {
# Weights BEFORE the build, and refuse rather than fetch. The payload is
# 1.4 GiB of GGUF with byte and sha256 checks; silently pulling it inside a
# deploy would turn a stalled network into a deploy that looks hung, and
# would hide which of the two failure modes the operator is in. The fetcher
# is idempotent, so re-running this handler after the fetch is cheap.
if ! scripts/download-frank-tts-engine-qwentts.sh --verify-only >/dev/null; then
echo "frank-tts-engine: the talker/tokenizer GGUFs are missing or wrong-sized under /home/dank/models/Qwen3-TTS-GGUF; run scripts/download-frank-tts-engine-qwentts.sh and re-deploy" >&2
return 1
fi
# Build first: Image= is a locally built tag with Pull=never, so the
# generator cannot start a container from it before it exists.
build_compose_image frank-tts-engine
install_quadlet frank-tts-engine dankclaw/voiceservice/quadlet frank-tts-engine
systemctl --user start frank-tts-engine-network.service
# No `systemctl --user enable` (generated Quadlet units cannot be enabled
# on this host); boot persistence is the [Install] section in the unit.
#
# `restart` blocks until the unit reaches started, which includes the
# ExecStartPost warm pass — a boot of this service is therefore already
# warm by the time the next line runs.
systemctl --user restart frank-tts-engine.service
# NOT wait_for_health: this container publishes nothing, so there is no URL
# the deployer can dial from the host. The unit's HealthCmd runs inside the
# container's own namespace, and podman's recorded health status is what the
# host can actually observe.
wait_container_healthy frank-tts-engine 150 || return 1
# The warmer runs a second time here on purpose. ExecStartPost carries a
# `-` prefix so a failed warm can never crash-loop the endpoint, which also
# means the unit cannot fail the deploy on it. This call is where a voice
# that did not warm becomes a deploy result, and where the per-voice
# cold=/warm= numbers land on the operator's console instead of only in the
# journal. The pass itself is idempotent and the second cold cost is gone:
# the voices are already warm, so this costs about a second per voice. It is
# exec-ed into the container for the same reason the health gate is: no host
# address exists.
podman exec frank-tts-engine /app/warm-voices.sh
}
deploy_frank_tts() {
# The 7772 façade in front of frank-tts-engine. Weights and the engine build
# are not this handler's business, but the BRIDGE FILE is: this unit
# references frank-tts-engine.network, and with a referenced Network= file
# absent the v1 generator refuses to produce the container unit at all. So
# `--only frank-tts` installs the engine's quadlet pair too — idempotent, and
# it is what makes the façade deployable on a host where the engine was
# planned separately or not at all.
#
# The voice packs are host state (the pack builder writes them), but the
# directory is created here as the deploy user on purpose: rootless podman
# would otherwise auto-create a missing bind source as root-owned, and the
# container runs as uid 10000.
mkdir -p /home/dank/voices
build_compose_image frank-tts
install_quadlet frank-tts-engine dankclaw/voiceservice/quadlet frank-tts-engine
install_quadlet frank-tts dankclaw/voiceservice/quadlet frank-tts
systemctl --user start frank-tts-engine-network.service
# No `systemctl --user enable` (generated Quadlet units cannot be enabled on
# this host); boot persistence is the [Install] section in each unit. The
# façade unit carries Wants=/After= on frank-tts-engine.service, so this
# restart pulls the engine — and its ExecStartPost warm pass — ahead of it:
# an engine that will not start delays this gate instead of taking the
# endpoint down with it.
systemctl --user restart frank-tts.service
# The gate is podman's recorded health, driven by the unit's own HealthCmd
# (`wget --spider -q http://127.0.0.1:7772/health`, inside the container's
# namespace). /health answers 200 with the backend name and the resolved
# voice list, so an empty /voices still reads healthy while a dead app does
# not — and note what this gate cannot see: the engine publishes no port on
# purpose, so an engine that is down shows up in the /health payload and in
# the first synthesis, never here.
wait_container_healthy frank-tts 120
}
deploy_comfyui() {
# Build first: the unit's Image= resolves against local storage, so the
# image has to exist before the generator starts a container from it.
@@ -1690,6 +1805,8 @@ deploy_service() {
frank-dragoon) deploy_llm_dragoon ;;
frank-zealot) deploy_llm_zealot ;;
frank-archon) deploy_llm_archon ;;
frank-tts-engine) deploy_frank_tts_engine ;;
frank-tts) deploy_frank_tts ;;
hermes|hermes-dashboard|hermes-webui) deploy_hermes ;;
romm|romm-mariadb|romm-redis|mariadb|redis) deploy_romm_stack "$service" ;;
jellyfin) deploy_jellyfin ;;
+1
View File
@@ -142,6 +142,7 @@ valid_service() {
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
dankclaw:open-notebook|\
dankclaw:frank-tts-engine|dankclaw:frank-tts|\
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
+109
View File
@@ -0,0 +1,109 @@
#!/usr/bin/env python3
"""Destroy the vault's dead weight: era ledgers, retired runtimes, dead experiments.
`AGENTS.md` was treating a measurement as a record worth keeping forever. It is not: a
throughput figure for a payload that no longer runs, on a runtime that no longer exists,
is a liability — the next agent spends context on a number that AI progress has already
obsolete. This pass moves those pages to `.AGENTTRASH/` (repo law: never `rm`) and then
deletes every reference to them from the surviving pages, including the sentence that
carried the link when the link was the sentence's only content.
Usage: destroy-vault.py <list-file>
List file: one vault-relative path per line; a directory path destroys the whole tree.
"""
import pathlib
import re
import shutil
import sys
VAULT = pathlib.Path("serverAppConfigs-docs")
TRASH = VAULT / ".AGENTTRASH" / "20261019-destruction"
def targets(list_file):
out = []
for line in pathlib.Path(list_file).read_text().splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
p = VAULT / line
if p.is_dir():
out += [f for f in sorted(p.rglob("*.md"))]
elif p.is_file():
out.append(p)
else:
print(f" SKIP (missing): {line}")
return out
def main(argv):
if len(argv) < 2:
print(__doc__)
return 2
doomed = targets(argv[1])
# Stems come from the trash too, so re-running the pass (or running it after a
# partial destruction) still knows which names are dead and keeps scrubbing.
stems = {p.stem for p in doomed}
if TRASH.is_dir():
stems |= {p.stem for p in TRASH.rglob("*.md")}
moved = 0
for p in doomed:
dest = TRASH / p.relative_to(VAULT)
dest.parent.mkdir(parents=True, exist_ok=True)
shutil.move(str(p), str(dest))
moved += 1
print(f"destroyed {moved} pages ({sum(len(p.read_text(errors='replace').splitlines()) for p in TRASH.rglob('*.md'))} lines staged in trash)")
# Repoint-or-delete: a link to a destroyed page is itself dead information.
link_wl = re.compile(r"\[\[([^\]|#]+)(?:#[^\]|]*)?(?:\|([^\]]*))?\]\]")
link_md = re.compile(r"\[[^\]]*\]\(([^)#]*\.md)(?:#[^)]*)?\)")
pages_cleaned = links_dropped = lines_dropped = 0
for page in sorted(VAULT.rglob("*.md")):
s = str(page)
if ".AGENTTRASH" in s or ".obsidian" in s:
continue
text = page.read_text(errors="replace")
lines = text.split("\n")
out = []
hits = 0
def dead(target):
# NOT pathlib.Path(...).stem: that strips a real suffix off names like
# `Lemonade and sd.cpp`, so a link to a destroyed page survives and the
# vault then fails its own link check.
t = target.strip()
if t.endswith(".md"):
t = t[:-3]
return t in stems or pathlib.PurePosixPath(t).name in stems
for line in lines:
found = [t for t in link_wl.findall(line) if dead(t[0])]
for m in link_md.finditer(line):
if dead(m.group(1)):
found.append(m.group(1))
if not found:
out.append(line)
continue
hits += len(found)
stripped = link_wl.sub(
lambda m: m.group(2) if (m.group(2) and not dead(m.group(1))) else "",
line)
stripped = link_md.sub(
lambda m: "" if dead(m.group(1)) else m.group(0), stripped)
# What is left of the line after the dead links are gone?
residue = re.sub(r"[\s\-*•·|:;,.\u2014\u2013]+", "", stripped)
if residue:
out.append(re.sub(r"\s+$", "", re.sub(r"^(\s*[-*]\s*)[\s:;,.\u2014-]+", r"\1", stripped)))
else:
lines_dropped += 1
if hits:
page.write_text("\n".join(out))
pages_cleaned += 1
links_dropped += hits
print(f"cleaned {pages_cleaned} surviving pages: {links_dropped} dead links dropped, "
f"{lines_dropped} lines left empty and deleted")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+93
View File
@@ -0,0 +1,93 @@
#!/usr/bin/env bash
# Download the frank-tts-engine payload: the two GGUF files the arm A engine
# (qwentts.cpp `tts-server`, dankclaw/voiceservice/Containerfile.engine) loads
# once at startup — Serveurperso/Qwen3-TTS-GGUF's **1.7 B base talker at
# Q4_K_M** and the **12 Hz tokenizer at Q8_0**. Public repo — no auth header,
# and never a token on the command line (world-readable in /proc/<pid>/cmdline).
#
# THE BASE VARIANT IS MANDATORY, NOT A PREFERENCE. The engine refuses every
# voice registration unless the loaded model reports type "base"
# (tools/tts-server.cpp, register_voice): on a customvoice or voicedesign talker
# the cloned-voice product does not exist. The 0.6 B talker (628,905,056 B for
# its Q4_K_M) also exists and is NOT the decided artifact — 1.7 B is what live
# output was measured with on 2026-10-18.
#
# Fresh downloads, no resume (the same rule the frank-zealot fetcher learned the
# hard way: `curl -C -` once produced an oversized file on a multi-GB payload);
# -f/--fail-with-body so an HTTP error is never a success; then exact byte count
# and sha256 against the Hub's own LFS oid. This repo IS LFS, so the sha values
# below are the pointer oids the Hub publishes — verified at fetch, not
# computed locally. Idempotent: a complete, correctly-hashed file is skipped,
# and `--verify-only` reports without downloading (that is the mode the deployer
# and the runbook use to answer "are the weights there?").
#
# Both files together are 1.4 GiB and the engine mmaps them: peak RSS measured
# 5.7 GiB with both resident. Nothing here is deleted on a swap — the previous
# talker stays on disk, because the rollback for a bad engine build is to point
# QWENTTS_MODEL at a different file, not to re-download 1.4 GiB under pressure.
#
# Usage:
# scripts/download-frank-tts-engine-qwentts.sh [DEST_DIR] [--verify-only]
# Default DEST_DIR is the host directory the frank-tts-engine unit mounts
# read-only at /models.
set -euo pipefail
DEST=""
VERIFY_ONLY=0
for arg in "$@"; do
case "$arg" in
--verify-only) VERIFY_ONLY=1 ;;
*) DEST="$arg" ;;
esac
done
DEST="${DEST:-/home/dank/models/Qwen3-TTS-GGUF}"
# repo | hub-name | dest-name | bytes | sha256 (Hub LFS oid, resolved 2026-10-18)
FILES=(
"Serveurperso/Qwen3-TTS-GGUF|qwen-talker-1.7b-base-Q4_K_M.gguf|qwen-talker-1.7b-base-Q4_K_M.gguf|1219245248|ea393ebaf2167ea23ce9fc18b093822851358a950d7075cd47ab4f6ce23e887d"
"Serveurperso/Qwen3-TTS-GGUF|qwen-tokenizer-12hz-Q8_0.gguf|qwen-tokenizer-12hz-Q8_0.gguf|291150624|1883beeed99348fc35e23dd225e9082f93f6f8c109330a33d935baa8acdbfd94"
)
mkdir -p "$DEST"
status=0
for entry in "${FILES[@]}"; do
IFS='|' read -r repo hubname destname size sha <<<"$entry"
url="https://huggingface.co/${repo}/resolve/main/${hubname}"
f="$DEST/$destname"
have=0
if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
if sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
have=1
fi
fi
if (( have )); then
echo "OK $destname"
continue
fi
if (( VERIFY_ONLY )); then
echo "MISSING $destname"
status=1
continue
fi
echo "FETCHING $destname (from $repo)"
tmp="$f.part"
curl -fsSL --fail-with-body -o "$tmp" "$url"
mv "$tmp" "$f"
if [[ "$(stat -c%s "$f")" != "$size" ]]; then
echo "SIZE FAIL $destname ($(stat -c%s "$f") != $size)"
status=1
continue
fi
if ! sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
echo "SHA FAIL $destname"
status=1
continue
fi
echo "VERIFIED $destname"
done
if (( status == 0 )); then
echo "DOWNLOAD_DONE $DEST"
else
echo "DOWNLOAD_FAILED $DEST" >&2
fi
exit "$status"
+106 -281
View File
@@ -1,306 +1,131 @@
# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving the
# PUBLISHER's 4-bit-codebook container of the official Qwen/Qwen3.8-Flash-Next
# (MODEL_FILE below, fetched by scripts/download-frank-archon-iq4r.sh) on
# BOTH R9700s (TP=2), dankserver :7770.
# 2026-10-15 PAYLOAD SWAP: the model moved from the in-house w4dn8 conversion
# of the Swift1.5 finetune to the engine author's published container
# qwen3.8-next-flash-fp8-iq4r-moe.rad (116,322 MiB, sha256-verified at fetch —
# the w4dn8 measured 116,416 MiB, so this shape carries over with 94 MiB to
# spare). NO argument changed: every number below is the 2026-10-13 shape,
# kept verbatim. The swap commit is the rollback (the w4dn8 payload stays on
# disk).
# frank-archon.env — radiance (StillDeadcode) on BOTH R9700s (TP=2),
# Dankserver :7770, publisher's 4-bit-codebook container of the official
# Qwen3.8-Flash-Next.
#
# SHAPE (2026-10-13): THREE decode slots x 200,000 tokens, fp8 (quantized) KV,
# with the per-card budget spent on resident experts: 25,600 MiB of slab +
# 5,120 MiB of pool = 30,720 MiB, inside the 30,969 MiB ceiling the engine's own
# budget table allows (card 32,624 MiB, settle floor 209 MiB). The pool is sized
# to the contract and nothing more (3 x 200,000 needs 5,063 MiB at fp8; 5,120
# predicts 608,027 tokens, 1.3 % slack), because every MiB taken from the slab
# is prefill and TTFT (2026-10-11 A/B: slab -14.8 % cost prefill -29 %, TTFT
# +25 % at the full window) while KV beyond the contract buys nothing.
# Consequence, deliberate: a FOURTH full-height conversation does not fit this
# pair, and the engine refuses it — it never truncates. See the KV block for the
# arithmetic and the measured dtype law. The w4dn8 payload is the swap
# rollback and keeps its trio under the w4dn8 runbook; the fp8-native payload
# that came before it was DELETED from disk 2026-10-15 (Frank's call, to free
# the volume for this download — rebuildable via the fetch + build scripts
# named in the fp8-native runbook).
#
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
# tag and rebuildable). Before that: tcclaviger vLLM fork (tag
# frank-archon-vllm-shapeG-20261003).
#
# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
# gfx1201, drives `podman run` directly from the digest-pinned
# stilldeadcode/radiance image (287 MiB; host needs only the amdgpu driver),
# and its qwen4exp plugin reads the modelopt FP8 encoding NATIVELY
# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
# THE 4-bit-codebook publisher shape is the DEPLOYED payload since
# 2026-10-15. The card's own run line (--host-pool-mib 12288,
# --expert-vs-cache-ratio 0.82, seqs 8, spec 3) is the publisher's
# suggestion on 32 GB cards; this file's shape is the fleet's measured one.
#
# PAYLOAD: /home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad — the
# official Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's
# own author (recipe qwen4exp-w4nl64-i8-hc8m: experts as 4-bit codes into a
# 16-level non-uniform codebook after a 128-point Walsh–Hadamard rotation,
# GPTQ + 3 coordinate-descent sweeps on a 10M-token calibration, ten
# bf16-protected experts per layer, int8 W8A8 trunk + lm_head, fp8 HC
# mixtures, PLE n-gram fp8 fixed-scale, MTP head, bf16 vision tower; card KL
# 0.0789 all-tokens / top-1 92.85 % vs bf16). Fetched byte-exact + sha256 by
# scripts/download-frank-archon-iq4r.sh; the recipe/LICENSE stay in the
# staging dir beside it as provenance. ROLLBACK: revert the swap commit — the
# in-house qwen4exp-swift15-w4dn8-i8-hc8m.rad (scripts/convert-swift15-w4dn8.sh,
# KLD-gated) stays on disk. Behind it, the fp8-native
# qwen4exp-fp8-native.rad was DELETED from disk 2026-10-15 (Frank's call,
# one-time rm, to free the volume for this payload); rebuild =
# scripts/download-frank-archon.sh + scripts/build-frank-archon-rad.sh — a
# NO-RECIPE rad-convert copy of the official 131-shard checkpoint under the
# descriptor law (no shipped plugin claims (qwen4exp, "fp8"), so the staging
# config.json drops quantization_config; per-weight encodings are data, not
# the descriptor).
#
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
# then engine.serve()) — during load the port refuses connections, so a
# bare curl /health gate is honest. /health == /ping: 200 JSON while
# serving, 503 only while stopping (core/server/admin.cpp:385).
# - Reasoning: --reasoning-format defaults to auto, emitting thinking in
# `reasoning_content` (config.cpp). Tool calls: the qwen3.8 dialect, from
# the chat format. Both need no parser flags.
# - Sampling: the OpenAI endpoint accepts temperature, top_p, top_k, min_p,
# typical_p, presence_penalty, frequency_penalty, repetition_penalty,
# repeat_last_n, chat_template_kwargs, reasoning_effort,
# enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
# models.yml's extraBody set is fully inside it. Unknown params are
# refused (400), which is why nothing unverified rides the wire.
# - MTP: the .rad carries the model's mtp.* head (the publisher's recipe
# names the mtp.fc_hidden/fc_embedding/draft_head rules; card default
# depth 3). Served at the depth named by FRANK_ARCHON_SPEC_TOKENS below —
# the depth this shape's decode ladder was measured at.
# - The fused TP all-reduce is declared over max-num-seqs x
# (1 + --num-speculative-tokens) rows; 3 x (1+2) = 9 rows, under the 25-row
# exactness bound of the wht6 wire.
# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
# 25558e07fc60d20328f1346f96f3c6e00845eb763f01fa, tag 1.2.0, commit
# e95d397) was benched 2026-10-06 on this exact shape (fp8-native rad,
# vram pair, spec 2, seqs 3, 43.6k greedy probe): decode 29.21 t/s and
# draft acceptance 0.7123 vs the 1.0.13 control's 30.08 and 0.7549.
# Acceptance on the greedy harness is deterministic (the control
# reproduced the recorded 0.755 exactly), so the drop is a real numerics
# change — the 1.2.0 MoE weight-rotation change (92b8aaf) is the named
# suspect; decode loss tracks the acceptance ratio (0.943 vs 0.945).
# The vram-pair translation itself is exact (1.2.0 boot ledger took
# vram_weights 25.46 GiB / vram_kv 3.46 GiB per card byte-identical to
# the 1.0.13 ledger at EVC 0.82). Re-test 1.2.0+ when the rotation change
# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of
# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this
# file already uses the pair, which BOTH pins accept (rollback symmetry).
FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
# The .rad container FILE (mounted under /models) and the state dir for the
# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
# .rad files; the raw HF download and its staging dir live beside it and
# are never mounted.
# ONE OWNER: shape, payload identity, eras, laws and every measured number live
# in serverAppConfigs-docs/LLM/frank-archon.md. This file carries the knobs plus
# only the reason a reader editing THIS line cannot recover from the page.
# Where the two disagree, this file wins on values; the boot ledger arbitrates
# the engine shape.
FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed
# 1.8.1 (2026-10-18), pinned by digest. Two CLI breaks sit between it and the
# 1.0.13 line this endpoint ran for two eras, and both are boot refusals, not
# warnings: --vram-weights-mib/--vram-kv-mib were REMOVED at 1.3.0 (the card is
# now split by the engine's budget resolver — see below), and the prefix cache
# lost its disk tier. Draft acceptance has to be re-scored on any pin past 1.2.0:
# 1.2.0's MoE weight rotation measured 0.7123 against the 0.7549 control, and the
# suspect is untangled (the page's Dead ends section names it).
# The .rad file mounts under /models; the tune cache mounts separately below.
# The model dir holds ONLY .rad files — the HF download and staging stay beside it.
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad
FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
# at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the
# cache must ride a host mount — serve script mounts it, see the runbook's
# "Tuning levers").
# Container identity is the SERVICE name (Frank's call, 2026-10-05): one
# name across systemctl and podman ps. The runtime changes; the name does
# not. The legacy list retires every predecessor identity that could hold
# the GPUs or the port.
# state dir's tune/ subdir carries rad-tune's kernel-selection cache; the engine
# reads $RADIANCE_HOME/tune/<machine>.tune and that defaults to /opt/radiance
# (inside the image), so the cache only survives on a host mount.
# Container identity is the SERVICE name: one name across systemctl and
# `podman ps`; the runtime changes, the name does not. The legacy list retires
# every predecessor identity that could hold the GPUs or the port.
FRANK_ARCHON_CONTAINER=frank-archon
FRANK_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
# Served model id — Protoss name (wire rename 2026-10-09). The container,
# unit, scripts and this file keep the frank-archon name above; only the
# alias on the wire moved. Radiance validates the requested model name, so
# every client (harness, bench scripts, probes) MUST send this value.
# Served id. Radiance VALIDATES the requested model name, so every client must
# send exactly this value (unlike llama.cpp and halogen, which ignore it).
FRANK_ARCHON_SERVED_NAME=frank-archon
FRANK_ARCHON_PORT=7770
# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
# repo's root chat_template.jinja, vendored here; the checkpoint's own
# template is baked into the .rad by rad-convert and this overrides it).
# Radiance derives the reasoning/tool-call reply format from it.
# Frank's chat-template call: froggeric's fixed Qwen3.8 template, vendored in
# the repo; it overrides the template baked into the .rad, and radiance derives
# the reasoning and tool-call reply format from it.
FRANK_ARCHON_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
# --- engine shape (the boot ledger arbitrates every number below) ---------
# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
# x8/x8 on root ports) made collectives legal on this host; radiance's TP
# wire is libr4d's own, not RCCL.
# TP=2 needs the 2026-09-20 board (x8/x8 on root ports): a collective cannot be
# dispatched on a card whose PCIe chain lacks atomics. Radiance's wire is
# libr4d's own, not RCCL.
FRANK_ARCHON_TP=2
# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
# engine default.
# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice). It
# stays EXACT for single-sequence decode and any step of <= 25 rows — see
# MAX_SEQS x (1 + SPEC_TOKENS) below.
FRANK_ARCHON_TP_WIRE=wht6
# Decode slots: THREE, so the served window is a CAPACITY CONTRACT — three
# concurrent conversations of --max-model-len fit the KV pool (see the KV
# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
# tcclaviger vLLM fork's torch.compile shape specialization on
# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
# (30.9-33.1 t/s, inside the +-3% noise floor) — slot count buys concurrency,
# not speed, so a slot costs KV, never decode.
# Cost of each extra slot: 110 MiB of per-slot recurrent state inside the KV
# pool, and the fused TP all-reduce is declared over seqs x (1 + spec) rows —
# 3 x 3 = 9 here, under the 25-row bound at which the wht6 wire stays EXACT
# (single-sequence decode and any step of <= 25 rows is bit-exact; only busier
# steps are lossy).
# The pool is sized to this contract and nothing more (see the KV block):
# 3 x 200,000 at fp8 = 5,063 MiB, pool 5,120, and the freed 1,792 MiB went into
# the resident expert slab. The trade is explicit and it is Frank's call:
# a FOURTH full-height conversation no longer fits (5,120 MiB of pool backs
# ~608k tokens; 4 x 200,000 needs 6,601), so a fourth 200k request is refused
# as too large to fit — it is NOT squeezed, paged, or truncated. Third-party
# traffic that wants four full-height sessions is a --vram-kv-mib change, not
# a code change.
FRANK_ARCHON_MAX_SEQS=3
# Served window. Native is 262,144; 200,000 is the publisher's production
# value (compose) and what models.yml's contextWindow must quote — a longer
# prompt is refused by the engine, never truncated. The engine treats
# `kv group tokens >= max-model-len` as a HARD limit ("ONE request longer
# than N tokens is accepted by the API and then refused as too large to ever
# fit — this is a hard limit, not a backing ratio"), so the pool is sized
# for seqs x maxlen, not for one sequence.
# Decode slots. A slot costs KV, never decode; the pool is sized to this
# contract and nothing more. At 1.8.1 the budget resolver caps the cache at
# 556,348 fp8 tokens — 2.8 sessions at 200,000 — so a THIRD full-height
# conversation answers 400 while /health and every shorter chat work fine
# (3 x 50k streams run at 77.6 t/s aggregate). Frank's trade: declare the two
# slots the engine can hold and let the freed ~1.1 GiB go to the expert plane,
# instead of advertising a third slot that only exists at shorter prompts.
FRANK_ARCHON_MAX_SEQS=2
# Served window. The engine treats `kv group tokens >= max-model-len` as a HARD
# limit, and models.yml contextWindow MUST quote this number (over-length is a
# refusal, not a truncation). The pool is sized for seqs x maxlen, not one
# sequence.
FRANK_ARCHON_MAX_MODEL_LEN=200000
FRANK_ARCHON_PLACEMENT=expert_tiered
# Pinned host pool for the expert rows that do not fit VRAM. vram_weights and
# this pool serve the per-rank resident set TOGETHER: at the fp8-KV pair
# (26071/3543) the ledger showed the VRAM slab filled to 259 KiB of slack,
# i.e. the pair was exactly sized. Cutting vram-weights by 3.8 GiB to pay for
# the bf16 KV pool SPILLS ~3.8 GiB/rank here, so the pool moves with it —
# under-sizing it is a boot failure (36864 refused the boot in the fp8 era).
# The boot ledger's coverage line remains the referee, not this arithmetic.
# Pinned host pool for expert rows that do not fit VRAM. Under the budget
# resolver this is ALSO the lever that decides which experts stay on the card:
# the card's leftover goes to the expert plane first and the KV cache takes the
# rest, so growing the pool moves experts off the card and shrinking it spills
# them onto VRAM the cache wanted. Under-sizing it is a boot failure (36,864
# refused a boot). The ledger's coverage line is the referee, not this file.
# Measured, not guessed: 45,568 (PR #30) moved host_pool to 45.09 GiB and left
# vram_kv byte-identical at 4.20 GiB / 556,348 tokens on both ranks — extra
# host RAM above what the expert plane's footprint needs buys no cache. Back at
# 44,928, the size that boots clean.
FRANK_ARCHON_HOST_POOL_MIB=44928
# VRAM left unclaimed per card (publisher default 96 MiB).
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
# floor is built from it.
FRANK_ARCHON_GPU_HEADROOM_MIB=96
# VRAM split of claimed memory per card, MiB (0 = derive from the card).
# THE CEILING, read off this engine's own VRAM-budget table (boot 2026-10-13,
# pair 24000/5120): card 32,624 MiB; budgets claimed 29,120; `3.42 GiB is
# neither budgeted nor accounted — the driver's context, the activation arena
# and --gpu-headroom-mib come out of it; anything past those is throughput on
# the floor`; and `card 0 settled at 2.01 GiB free against 209.19 MiB (the
# 96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)`.
# So the reserve above the ENGINE's floor is 2,058 - 209 = 1,849 MiB and the
# ceiling is 29,120 + 1,849 = 30,969 MiB. The 2026-10-11 bound of 29,120
# ("31,856 - 1,710 outside both budgets - keep 1 GiB free") was MY reserve, not
# the engine's; the pair below spends the difference and lands 458 MiB above
# the engine's floor. `weights + kv <= 30,969` is the gate.
# THE PRICE, and it is paid twice: on this engine idle KV VRAM is ALSO the
# expert cache's spillover (`mover: 2275 slots can be lent out of the KV
# cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <= 256 tokens`),
# so a bigger KV pool steals weight residency from the slab AND from the
# lendable cache. Priors on this card/model: stealing 2.14 GiB of weights for
# a bigger pool (MAX_MODEL_LEN 262144) cost 2.5% decode; the EVC ladder's
# other direction (0.82 -> 0.84, +2.5 GiB weights) cost 8.9%. The 2026-10-11
# exact-KV A/B measured the slab's own price (22208 vs the fp8-era 26071):
# prefill -29 %, TTFT at the full window +25 %, decode inside the noise band on
# 2 of 3 passes, draft acceptance UP 0.661 -> 0.740.
# 2026-10-13 (late): THREE slots, so the pool is cut to the contract (5,120)
# and every spare MiB goes to the slab: 22208 -> 24000 (the third slot's
# surplus KV), then 24000 -> 25600 (the card's settling slack). The direction
# the A/B data points: prefill is slab-limited, KV beyond the contract is not.
# The margin is thin by design (608,027 predicted tokens against the 600,000
# contract), so the boot ledger's `backs N sequences all at 200000 tokens at
# once` line is the referee for THIS pair and the concurrent probe is its
# regression test.
# VERDICT on the 25,600 arm (measured 2026-10-13, scripts/probe-frank-archon-kv.py):
# the threshold claim is FALSIFIED — the slab is a SLOPE at this end. A-cold
# 43.5k single-stream: 22,208 -> 3,397, 24,000 -> 3,290, 25,600 -> 3,584 t/s;
# the 3-way 197k arm's aggregate prefill 3,051 -> 3,589 t/s (wall
# 193.7 -> 164.5 s). So +1,600 MiB of slab = +8.9 % single-stream and
# +17.6 % concurrent, both outside the +-3 % noise floor. The 22,208/24,000
# pair landed inside noise of EACH OTHER, which is what made it read as a
# plateau; two points are not a curve.
# The cliff that remains is between 25,600 (3,584) and 26,071 (4,379, the
# 2026-10-11 fp8 control at seqs 3 / pool 3,543) — +22 % for +471 MiB. At THREE
# slots that is only reachable by breaking the ceiling (26,112 + 5,120 = 31,232
# vs 30,969, ~214 MiB above the engine's floor); at TWO slots it is free:
# pool 3,072 + slab 27,897 = exactly the ceiling, slab above the cliff.
FRANK_ARCHON_VRAM_WEIGHTS_MIB=25600
FRANK_ARCHON_VRAM_KV_MIB=5120
# KV cache QUANTIZED (fp8). This engine's flag takes ONLY bf16|fp8 — there is
# ONE kv_dtype for the whole cache, so "exact K, quantized V" (llama.cpp's
# --cache-type-k/--cache-type-v pair, which is what frank-zealot runs) is
# UNSERVABLE here: the rung below bf16 is fp8 on both halves.
# Measured per-rank cost, read out of the boot ledger's group table (fp8 boot
# 2026-10-08, bf16 boot 2026-10-11):
# kv_attn 13 layers, 4-token blocks, page 26.00 KiB (fp8) -> 52.00 KiB
# kv_qsa_bkey 13 layers, 4-token blocks, page 3.25 KiB at BOTH dtypes —
# the QSA block key is an index, not attention state, and does
# NOT follow --kv-cache-dtype
# => 7.3125 KiB/token fp8, 13.8125 KiB/token bf16: bf16 costs 1.89x fp8,
# not 2x, precisely because the bkey plane is dtype-fixed.
# The dtype-INDEPENDENT terms also live in the pool: kv_gdn_state 54 MiB x 2
# copies per sequence (+ conv/tail/ple ~= 110 MiB a sequence) and the prefix
# checkpoint region 8 x 55.99 = 448 MiB. So:
# pool_mib >= tokens x 7.3125 KiB + 110 x seqs + 448
# 3 x 200,000 at fp8 needs 4,285 + 330 + 448 = 5,063 MiB; the pool is 5,120,
# and it predicts (5120 - 330 - 448) x 1024 / 7.3125 = 608,027 tokens: the
# three-slot contract with 1.3 % slack, every MiB above 5,063 having been
# handed to the expert slab instead. The ledger line is the referee ("backs N
# sequences all at 200000 tokens at once") — a pair this tight is read off the
# boot, never assumed.
# At bf16 the same three slots cost 8,093 + 778 = 8,871 MiB, which would cut
# the slab to 20,249 MiB (-9 %); at FOUR slots bf16 costs 10,791 + 888 =
# 11,679 MiB and the slab falls to 17,441 (-33 %). fp8 keeps 22,208 at both
# slot counts. The 2026-10-11 bf16 shape (6,912 / seqs 2) backed 462,752
# tokens = 2 x 200,000; the historical fp8 pool of 3,543 gave 386,904 (1.93
# sessions), and the four-slot fp8 boot of 2026-10-13 19:21 backed 843,276.
# Secondary saving: the prefix-cache entry shrinks with the attention plane
# (one entry 110.50 KiB at bf16 -> 58.50 KiB at fp8, measured on the two boot
# ledgers' `kv tier transfer` lines), so the host tier below parks twice as
# many conversations as it did at bf16 — no re-size needed to park three slots.
# There is no VRAM pair any more. --vram-weights-mib/--vram-kv-mib were removed
# at 1.3.0: the engine takes capacity - already-held - GPU_HEADROOM - arena,
# puts the static (non-expert) weights off the top, then divides the rest between
# the expert plane and the KV cache — experts first when the cache has no idle
# blocks to lend, and never less than ONE MAX_MODEL_LEN sequence for the cache.
# What moves the split from here is HOST_POOL_MIB (which experts stay resident),
# MAX_MODEL_LEN/MAX_SEQS (the cache bound) and GPU_HEADROOM_MIB. The slab size
# the prefill-slope law cares about is a boot-ledger line now, not a knob: read
# it after every resize instead of asserting it here.
# fp8 is the ONLY rung below bf16 on this engine: one kv_dtype for the whole
# cache, so llama.cpp's exact-K/quantized-V split (frank-zealot) is unservable
# here. bf16 costs 1.89x fp8, not 2x — the QSA block-key plane is dtype-fixed.
FRANK_ARCHON_KV_DTYPE=fp8
# Spec decode: the checkpoint's own MTP head, served at depth 2 — the
# publisher's card ships depth 3 as its peak; this fleet's ladder was
# measured at 2 and the swap keeps every argument, so 2 stays (rows
# 3 x (1+2) = 9, inside the wht6 25-row exactness bound).
# The checkpoint's own MTP head at depth 2: this fleet's ladder was measured at
# 2, and 3 x (1+2) = 9 rows stays inside the wht6 25-row exactness bound.
FRANK_ARCHON_SPEC_TOKENS=2
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
# Prefix cache tiers (publisher pattern). The host tier is sized in ENTRIES,
# and one entry tracks the cache dtype (58.50 KiB at fp8, 117 KiB at bf16, per
# the ledger's `kv tier transfer` line): at fp8, 8192 MiB parks ~5.8 sessions
# of 200k (at bf16 the same tier parked ~2.9), so four live slots plus a full
# parked set fits with the tier as it stands — at RAM the host has (104 GiB
# available with the service loaded).
FRANK_ARCHON_PREFIX_HOST_MIB=8192
FRANK_ARCHON_PREFIX_DISK_MIB=65536
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
# Step sharing. A prefill chunk that shares a step with decoding sequences holds
# the device for the chunk's whole time, and every decode row in that step waits
# that long for its next token (the census: [[LLM Performance Diagnosis]] §9).
# `auto` leaves each decode row behind the chunk a row plus its draft window and
# keeps a share for whatever is waiting to be admitted; a number pins the chunk's
# cap; `off` is the pre-1.4 behaviour this endpoint ran for two eras, where the
# chunk took the step and admission never ran. Interleave inserts N decode-only
# steps after a chunk step taken beside decoders: it protects their cadence and
# holds a waiting request out of those steps, so it stays 0 until the post-migration
# decode-vs-prefill buckets say it is wanted.
FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto
FRANK_ARCHON_PREFILL_INTERLEAVE=0
# Prefix cache: pinned host memory only. The disk tier was removed outright
# (--prefix-cache-dir and --prefix-cache-disk-mib are refusals), so what used to
# be restored from NVMe is now re-prefilled. The tier is sized in ENTRIES and one
# entry tracks the cache dtype (58.50 KiB at fp8). Watch prefix_hit_rate after
# this change: a drop is extra prefill, and prefill is exactly the traffic that
# costs decode its cadence.
FRANK_ARCHON_PREFIX_CACHE_MIB=8192
# Frank's pin law: the PLE n-gram table serves from pinned host RAM the cards
# read. `ram` REFUSES the start if memlock is too small — the unit's
# LimitMEMLOCK=infinity is the other half of this law.
FRANK_ARCHON_NGRAM_PLACEMENT=ram
# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
# pages_limit (the current kernel exposes NO pages_max; the cap function
# takes the min over the knobs that exist). Below the plan the load dies
# mid-pin with an allocator error instead of this named refusal.
# THE CAP IS A RUNTIME KNOB — the kernel default is half of RAM, so every
# raise dies with a reboot. Measured the hard way: the 2026-10-05 manual raise
# to 52,428,800 pages (200 GiB) was gone after the 2026-10-12 23:17 POWER
# FAILURE, the cap fell back to 125,988 MiB, and the preflight refused the
# start 6,919 times in ~20 h while Restart=always re-ran it. The gate in
# scripts/ttm-preflight.sh now RAISES the knob itself before refusing (the
# root unit is privileged; the knob is a 0644 root module parameter), so the
# recovery is a boot, not a visit to the machine.
# TTM preflight. The pinned ledger rides TTM's shared pages, and the kernel
# RESETS pages_limit to half of RAM on every boot; below the plan the load dies
# mid-pin with an anonymous allocator error. ttm-preflight.sh raises the knob
# itself before refusing by name, so recovery is a boot, not a visit.
# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT
# per the bats ledger test; at the 44,928 pool that computes to 148,920 MiB.
FRANK_ARCHON_TTM_MIN_MIB=150000
FRANK_ARCHON_TTM_TARGET_MIB=204800
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
# multi-flag values as one string; the serve script expands it unquoted on
# purpose.
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
# this unquoted on purpose, so keep multi-flag values as one string.
FRANK_ARCHON_EXTRA_ARGS=
# The .rad's minimum size in MiB (deployer gate; a truncated fetch is a
# guaranteed long-timeout boot instead of a fast failure). The deployed
# publisher container measures 116,322 MiB (121,969,901,568 B); the
# in-house w4dn8 rollback measured 116,416 MiB. Floor sits just under both.
# Deployer gate: a truncated fetch must be a fast refusal, not a long-timeout
# boot. Floor sits under the live payload and under every rollback payload
# (see the page's Eras table for what is on disk and what needs a re-download).
FRANK_ARCHON_RAD_MIN_MIB=110000
+45 -172
View File
@@ -1,209 +1,82 @@
# frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
# Strix Halo iGPU (gfx1151), host port 7770.
#
# RUNTIME (2026-10-17, Frank): halogen-flash-server 0.16.2 — the
# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
# digest-pinned below) serving Quat3rnion/halogen-qwen3.8-flash-next-v2-
# abliterated — the NATIVE v2 conversion (2026-10-11) of orcarouter's pinned
# BF16 Qwen3.8-Flash-Next, abliterated — as a podman container under a root
# unit, driven by scripts/serve-frank-dragoon.sh. This CUTS OVER the
# 2026-10-06 Swift1.5-finetune shape; rollback = revert the swap commit and
# redeploy: the swift15 payload stays on disk at
# /home/dank/models/halogen-flashnext-v2/, the 27B-era payload beside it.
# ONE OWNER: payload identity, shape, eras, laws and every measured number live
# in serverAppConfigs-docs/LLM/frank-dragoon.md. This file carries the knobs
# plus only the reason a reader editing THIS line cannot recover from the page.
# Where the two disagree, this file wins on values; the boot ledger arbitrates
# the memory figures.
#
# MEASURED A/B, both boots on this box, identical battery traffic (engine
# tools with the service stopped, behavior probes against it, 2026-10-17):
# NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls and the
# red-PNG vision probe identical, needle hit at both depths of a 21.5k-token
# haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s,
# wikitext-2 PPL 3.968 vs 3.877 (the finetune's edge is gone: +2.3 %) and
# thinking tokens 1,476 vs 784 mean — the finetune compressed reasoning, the
# base thinks ~1.9x longer per answer.
#
# The A/B ran at the card's ctx/pool of 262,144. Afterwards Frank lowered ctx to
# 200,704 (RAM + the harness's compaction budget) and, on 2026-10-18, raised the
# pool to 2 x ctx = 401,408 so BOTH slots can carry a full-length request at
# once; the rates above are the ceiling-262,144 record and the production ledger
# is the ctx-200,704 one.
#
# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
# card it "will not load in transformers, vLLM, or llama.cpp" — and the v2
# .hgn loads ONLY in the flash-server line (the old halogen engine is for
# the 27B p1w4d file; the v2 quickstart pins flash-server). What the move
# buys (card + live probe evidence): MTP spec decode (53-57 t/s greedy,
# 3-10x serial), ~1,640 t/s prefill at 34k, ~72% fewer thinking tokens than
# the censored 27B (905 mean), the ABLITERATED base (refusals should NOT
# return), IMAGE INPUT (the v2 vision tower loads beside the checkpoint —
# red-PNG probe verified on the card; the 27B engine ignored its encoder),
# tool calls verified, and the native 262,144 context — served at 200,704 by
# choice from 2026-10-17 (see SHAPE: the ceiling is a RAM decision, not a
# capability limit).
#
# SHAPE — the knobs that are NOT free, with the arithmetic:
# HALOGEN_CTX=200704 is the PER-REQUEST ceiling and the per-conversation
# hard limit — a prompt over it is a 400 naming the limit, never a silent
# truncation. LOWERED from 262,144 on 2026-10-17 at Frank's direction (the
# checkpoint's native window is 262,144 and the engine serves anything
# below it): every position costs KV pool RAM and shrinks the prompt-cache
# entry (111 MiB at 262,144, ~85 MiB here). The 400-zone law is INVARIANT
# to the value — it says the harness window must sit ~45k UNDER the engine
# ceiling, because the harness clamps max_tokens to contextWindow − est −
# 4096, its chars/4 est UNDER-counts the rendered prompt by ~3 % on
# code/agent traffic, and compaction only fires at 85 % (measured 2026-10-07
# at the old 200,192 ceiling: est 152,938, server 157,388, over by 354).
# Harness contract (models.yml): contextWindow 155648 (= ceiling − 45056,
# the gap that absorbs the estimate error), maxTokens 45056.
# Compaction fires at 85 % = 132,301 est ≈ 136k measured — the 180k target
# of the 262,144 era is unreachable at this ceiling: the window itself is
# 155,648. Worst-case request: prompt + budget <= 155648 − 4096 + 3 %
# est-error ≈ 156.2k <= 200,704 — ~9.5× the 3 % error of margin (robust to a
# ~25 % under-count). Compaction rides on frank-zealot: input ≈ threshold −
# keepRecent(20k) ≈ 112k, inside its 200,704 slot.
# HALOGEN_KV_POOL_POSITIONS=401408 = SLOTS x CTX (2 x 200,704), the top of the
# legal band (the seam enforces ctx <= pool <= slots * ctx). Still ONE pool
# behind HALOGEN_KV_SLOTS=2 (ledger form: "2 slots over ONE <pool>-position KV
# pool, each request up to HALOGEN_CTX"), but sized so BOTH slots can hold a
# MAXIMUM-length request at the same time — the parity with frank-zealot's
# 401,408-position pool that Frank ordered on 2026-10-18. The PER-REQUEST
# ceiling does NOT move with it: HALOGEN_CTX stays 200,704, because the
# checkpoint's native window is 262,144 and the engine refuses to go past it
# without HALOGEN_ROPE_YARN (its own 400 text says so). ~28 KiB/position
# (FLAGS.md) ⇒ the DEPLOYED 2026-10-18 boot RESERVES 11.0 GiB (`reserving the
# KV pool, 401408 positions (about 11.0 GiB)`), where pool == ctx (200,704)
# cost 5.5, 262,144 cost 7.2 and the 2026-10-06 shape's 524,288 measured 14.6.
# The preflight formula bills 11.3 (block scratch + ~115 MiB/slot of O(1)
# state) — quote the RESERVATION. FLAGS.md law: "RAM the pool
# takes is RAM that table loses" — the 47.68 GiB n-gram table surrenders this
# much page cache, so the warm prompt-cache ratio is what to re-measure.
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
# drafter is NOT slot-locked on this engine (batching and speculation
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
# quickstart ships 2 slots WITH the head), so the second slot is
# concurrent capacity, not a speculation tax.
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
# mmap'd page cache beside the checkpoint, not a pinned charge — the boot
# ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
# weights 62.9 GiB locked (0.8 of them the vision tower) + KV 11.0 +
# working 5.7 (HALOGEN_MAX_TOK=8192) = 79.6 GiB engine-side (the boot
# ledger of the 401408-position deploy, 2026-10-18), with the 1.3 GiB
# prompt cache (12 entries x 111 MiB at ctx 200704) booked separately by
# the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host honest. Against
# the 121.2 GiB box that leaves ~20 GiB over the reserve (MemAvailable
# read 22 GiB right after the raise) for baseline co-tenancy
# (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot ledger is the
# referee — read it after deploy and re-size if the engine prints a
# clamp.
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
# turns the MTP head off for whole requests after low acceptance and
# HALVES decode as a result. Frank accepted the head always-on trade
# (temp 1.0 traffic costs acceptance; the card still ships the recipe).
#
# CLIENT CONTRACT deltas vs the 27B shape (harness side lives in
# ~/.omp/agent/models.yml):
# * SERVER-SIDE SAMPLER EXISTS NOW: HALOGEN_TEMPERATURE/TOP_P/TOP_K set
# the card's thinking recipe (1.0 / 0.95 / 20) as engine defaults, so
# an absent temperature is no longer greedy. models.yml extraBody
# carries the same values — consistent, not load-bearing.
# * HALOGEN_MAX_TOK=8192 is only the DEFAULT for requests that omit
# max_tokens; the harness sends at most its maxTokens 45056. Requests
# over HALOGEN_MAX_TOKENS_CAP (65536) are a 400 naming the cap. Cap and
# queue timeout are COUPLED (FLAGS.md table): 65536 at ~10 t/s serial
# worst case needs > 6550 s; 7200 honors it. Do not move one alone.
# * max_tokens INCLUDES thinking; a reasoning reply over a small budget
# returns empty content with finish_reason "length".
# * Prompt cache is ON by default (PROMPT_CACHE=2): a follow-up turn at
# 100k context re-prefills ~2 s instead of ~88 s cold. No pin needed —
# unlike the 27B engine, auto-sizing fits this box's ledger.
# * Thinking lands in reasoning_content (same as the 27B engine); the
# v2 chat template rejects reasoning_effort outside xhigh|medium|low,
# so the models.yml fold (high->xhigh) still applies.
# * Vision: OpenAI image_url content parts only (the fleet's standard
# contract). >=256 px probe images or the model answers "video".
# * Concurrency at depth is tight now: the KV pool is ONE 200,704-position
# budget behind 2 slots — exactly one maximum-length request fills it. Two
# conversations coexist only while they stay under ~100k each. The
# PER-REQUEST ceiling moved WITH the pool (262144 -> 200704), so the harness
# contract moves too: contextWindow 155648 = ceiling − 45k, never higher.
# * response_format json_object/json_schema unsupported in this line
# (no constrained decoding) — unchanged from the 27B engine.
# * The engine's token protocol has no auth and binds loopback INSIDE
# the container; only the frontend port (8731) is published. Same as
# before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
# amdgpu.noretry=0 (check at deploy).
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
# is what was qualified). 0.16.2 = the version the model card names as its
# quickstart pin; RepoDigest captured at pull on 2026-10-06.
# The .hgn format is the lock: it loads ONLY in the flash-server line, so the
# engine moves with the payload, never ahead of it.
# Image digest-pinned (a tag pull silently re-points; the digest is what was
# qualified — 0.16.2 is the version the model card names as its quickstart pin).
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
# Container identity: kept from the 27B era (unit ExecStop names it).
# Container identity kept from the 27B era: the unit's ExecStop names it.
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
# Old quadlet-era container identity; the serve script clears stale names.
# Predecessor identity the serve script clears if it is still holding the GPUs.
FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
# default); the engine port (8730) is never published.
# host:container — the frontend listens on 8731 inside; the engine's own port
# (8730) has no auth and is NEVER published.
FRANK_DRAGOON_PUBLISH=7770:8731
FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-v2-abliterated
# The checkpoint: Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated
# qwen38-flash-next-v2-abliterated.hgn — the NATIVE 2026-10-11 v2 conversion
# of orcarouter's pinned BF16 Qwen3.8-Flash-Next, abliterated. NOT the writer
# transplant this filename used to carry: 66,687,678,432 B where the swift15
# r2 measured 66,687,678,464 B — same filename family, different artifact.
# Hub LFS sha256 verified at fetch by scripts/download-frank-dragoon-v2abl.sh.
# Same filename family, DIFFERENT artifact: this is the native v2 conversion,
# 32 bytes smaller than the writer transplant that used to carry the name.
# Never infer the payload from the filename — this size and sha256 are the gate.
FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-abliterated.hgn
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678432
FRANK_DRAGOON_CHECKPOINT_SHA256=b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
# The n-gram lookup table: REQUIRED (the engine reads it beside the
# checkpoint; the card's quickstart names it explicitly). From
# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
# The card does NOT establish this stock table as matched to the NATIVE
# conversion and no source-matched table is published, so the engine's own
# tools are the referee: NIAH 18/18 and PPL 3.968 on it (2026-10-17). It is
# HARDLINKED from the swift15 payload dir (a hardlink is a regular file; the
# serve gate refuses symlinks), i.e. byte-identical to the file that era
# served.
# The n-gram table is REQUIRED (the engine reads it beside the checkpoint) and
# disk-backed: mmap'd page cache, never pinned. It is a stock peonist companion
# hardlinked from the previous payload dir — a hardlink is a regular file, and
# the serve gate refuses symlinks — with the engine's own NIAH/PPL verdicts on
# the page as the referee.
FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
FRANK_DRAGOON_NGRAM_BYTES=51200246144
FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
# The vision tower: OPTIONAL on the engine, ENABLED here (image input).
# From peonist-ai/halogen-qwen3.8-flash-next @ e053f488.
# Vision tower: optional on the engine, ENABLED here (image input).
FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
FRANK_DRAGOON_VISION_BYTES=897916416
FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
# The tokenizer is a FLAT DIRECTORY, $FRANK_DRAGOON_MODEL_DIR/tokenizer,
# mounted whole at /tokenizer. The two repos ship the same one; fetched
# fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to
# the 27B-era copy; the rest are plain git blobs — size-checked only). The
# serve gate and the deployer refuse a missing file or ANY symlink: an HF
# cache snapshot symlinks each file into blobs/, and those links dangle
# The tokenizer is a FLAT DIRECTORY ($FRANK_DRAGOON_MODEL_DIR/tokenizer) mounted
# whole. The serve gate and the deployer refuse a missing file or ANY symlink:
# an HF cache snapshot symlinks each file into blobs/ and those links dangle
# inside the container mount.
FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
# stays at the image default, which is what was qualified) ------------------
# Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
# scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
# model_ids for the :7770 backend MUST name this value.
# Served id. OpenWebUI api_configs model_ids for the :7770 backend MUST name it.
HALOGEN_MODEL_ID=frank-dragoon
# PER-REQUEST ceiling: a longer prompt is a 400 naming the limit, never a
# truncation. The harness window must sit ~45k UNDER this (the 400-zone law on
# the page) — a window equal to the ceiling makes every long request a 400.
HALOGEN_CTX=200704
# ONE pool behind both slots, sized so BOTH can carry a maximum-length request
# at once. Pool positions buy concurrency; the per-request window never moves
# with them. Legal band: ctx <= pool <= slots * ctx (the seam enforces it), and
# RAM the pool takes is RAM the n-gram table's page cache loses.
HALOGEN_KV_POOL_POSITIONS=401408
# Two concurrent conversations: the MTP drafter is not slot-locked on this
# engine, so the second slot is capacity, not a speculation tax.
HALOGEN_KV_SLOTS=2
HALOGEN_HOST_RESERVE_GIB=20
# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
# depth 2 is the documented "best all-round" default. HALOGEN_DRAFTER /
# HALOGEN_DFLASH_* from the 27B engine DO NOT EXIST here.
# The MTP head is INSIDE the checkpoint — do NOT set HALOGEN_MTP_HEAD.
# 0 is mandatory: the adaptive default disarms the head for whole requests after
# low acceptance and halves decode. Depth 2 is the card's best-all-round default.
HALOGEN_SPEC_ADAPT=0
# Coupled pair (FLAGS.md table): the cap bounds how long ONE request holds
# the GPU, the timeout bounds how long the NEXT client waits. Do not move
# one alone (see the client-contract note above).
# Coupled trio: MAX_TOK is only the default for requests that omit max_tokens;
# the cap bounds how long ONE request holds the GPU and the queue timeout bounds
# how long the NEXT client waits. Move all three together or none.
HALOGEN_MAX_TOK=8192
HALOGEN_MAX_TOKENS_CAP=65536
HALOGEN_QUEUE_TIMEOUT=7200
# Server-side sampler = the card's thinking recipe (the engine now HAS one;
# requests may still override per-call). Matches models.yml extraBody.
# The engine has a server-side sampler (an absent temperature is NOT greedy
# here); models.yml extraBody carries the same recipe.
HALOGEN_TEMPERATURE=1.0
HALOGEN_TOP_P=0.95
HALOGEN_TOP_K=20
# Card's belt-and-braces: the engine pins weights by default
# (HALOGEN_FLASH_PIN_TRUNK=1; only the retired w4b build honored 0).
# Weights stay pinned: the engine pins by default and only the retired w4b build
# honored HALOGEN_FLASH_PIN_TRUNK=0, which this payload rejects.
HALOGEN_WEIGHTS_LOCK=1
+147
View File
@@ -0,0 +1,147 @@
# frank-tts-engine.env — tunables for **frank-tts-engine**, the arm A TTS
# engine on Dankclaw (qwentts.cpp `tts-server`, container port **8880**, never a
# fleet endpoint). Read by the Quadlet unit
# (dankclaw/voiceservice/quadlet/frank-tts-engine.container) as its only
# EnvironmentFile=, which is why every path below is the value the CONTAINER
# sees. The host directories those mounts come from are named in
# dankclaw/voiceservice/quadlet/frank-tts-engine.container and
# scripts/download-frank-tts-engine-qwentts.sh.
#
# WHAT SITS HERE AND WHAT DOES NOT. `tts-server` reads its configuration from
# command-line flags only — it looks at no environment variable — so this file
# is consumed by the image's entrypoint
# (dankclaw/voiceservice/tts-engine-start.sh), which turns each key below into
# the matching flag. Every value has a default baked into that script, and the
# default equals the value below: this file exists so the decided shape is
# written down in the repo and one line can be changed without a rebuild, not
# because the engine would otherwise fail to boot.
#
# THE THREAD LAW (read this before reaching for CPUQuota=, CPUAffinity=,
# taskset or a systemd slice). There is **no `-t`/`--threads` flag** in this
# build: src/backend.h computes the CPU thread count as
# `std::thread::hardware_concurrency() / 2` (its own comment: GEMM shares SIMD
# units across hyperthreads). The engine therefore sizes its pools from what the
# kernel *reports*, not from what it is allowed to use. Pin the container to a
# cpuset and the thread count silently changes with it — different GEMM
# tiling, different timings, no log line saying so, and the measured numbers
# below stop describing the process you are running. Nothing in this fleet
# pins this container; that is a decision, not an oversight.
#
# THE REGISTRY LAW (why the unit has an ExecStartPost at all). Registrations live
# in memory only, so a restart loses them, and re-registration alone does NOT warm
# a voice — the warm pass has to synthesize something. Without it the façade
# answers 400 for every name until somebody asks for audio. So
# dankclaw/voiceservice/tts-engine-warm.sh (in the image at /app/warm-voices.sh)
# runs on every start and speaks a short sentence twice per voice. The penalty it
# removes is about 0.6 s, NOT the trial's 6.2-8.0 s: measured on the built
# container 2026-10-18 against an empty registry, the first synthesis cost
# 1,883 ms and the requests after it 1,226-1,283 ms. The hook runs INSIDE this
# container — `podman exec`, from the unit's ExecStartPost — so it reads /voices
# here and the keys at the bottom of this file are its knobs too.
#
# STREAMING IS NOT BUFFERED. The same seed gives audio that differs in 92.2 %
# of samples (-41.6 dB RMS) between `response_format=pcm` (the streaming path,
# codec chunks pushed as they decode) and `response_format=wav` (the buffered
# path). Nothing in this stack may compare the two, cache one against the
# other, or assume a seed pins output across `response_format`. frank-tts always
# asks for `wav` and re-encodes to mp3 itself; the warmer below also asks for
# `wav`, for that reason and because the streamed path would need a reader.
#
# THE ROUTE LIST, verbatim from tts-server at the pinned commit, so nobody
# debugs against a route that does not exist: GET /health, GET /v1/models,
# GET|POST /v1/audio/voices, DELETE /v1/audio/voices/{name},
# POST /v1/audio/speech (response_format wav or pcm). Everything else is a bare
# 404. Note two traps in that surface: `GET /v1/audio/voices` returns an empty
# JSON object rather than the voice list at this commit (the handler builds the
# array and never attaches it to the document), and a `POST /v1/audio/voices`
# carrying `spk_b64` without `rvq_b64` is rejected outright — it is `wav_b64` or
# BOTH of the pair.
# --- the two weights (see scripts/download-frank-tts-engine-qwentts.sh) -------
# Container paths; the host directory /home/dank/models/Qwen3-TTS-GGUF is
# mounted read-only at /models. 1,219,245,248 B + 291,150,624 B, fetched from
# https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF and byte-checked at fetch.
#
# The 1.7 B talker is the decided artifact (Frank, 2026-10-18). A 0.6 B talker
# exists at 628,905,056 B and is NOT what live output was measured with; the
# Q4_K_M quantization is the size that leaves the measured RTF margin intact on
# this box, and the tokenizer rides at Q8_0 because the codec is where clone
# fidelity is lost first.
#
# It has to be the **base** variant, not customvoice/voicedesign: the engine
# refuses any voice registration unless qt_model_type() reports "base"
# (tools/tts-server.cpp, the register_voice lambda). Cloned voices — the whole
# product — are impossible on the other two variants.
QWENTTS_MODEL=/models/qwen-talker-1.7b-base-Q4_K_M.gguf
QWENTTS_CODEC=/models/qwen-tokenizer-12hz-Q8_0.gguf
# What GET /v1/models reports, and what scripts/frank-tts.env's
# TTS_UPSTREAM_MODEL names. A model id, never a file name.
QWENTTS_ALIAS=frank-tts-engine
# 0.0.0.0 is required and is not an exposure decision: tts-server binds
# 127.0.0.1 by default, and inside its own netns that address is reachable by no
# one, not even the façade on the same bridge — which is why a host-loopback
# publish would not have helped either (netavark DNATs published ports only, and
# the unit publishes nothing at all). Binding 0.0.0.0 exposes the server to the
# one network it is attached to: the private voice bridge
# (systemd-frank-tts-engine, 10.89.5.0/24), whose only other member is frank-tts.
QWENTTS_HOST=0.0.0.0
QWENTTS_PORT=8880
# Language label applied when a request omits one. `auto` is upstream's default
# and what the trial ran; naming a language is a per-request field on
# /v1/audio/speech, so this is only the fallback.
QWENTTS_LANG=auto
# Serialised FIFO: one synthesis at a time, everyone else queued. Warm RTF on
# this box is 0.55-0.67 against real time 1.0, which is the entire margin a live
# stream has; two concurrent syntheses split the same cores and the margin is
# gone, with the added effect that the second caller's time-to-first-audio now
# includes the first one's whole utterance. The façade's own budget
# (SHIM_TIMEOUT=600, and open-notebook's ESPERANTO_TTS_TIMEOUT=660 sitting one
# round-trip outside it) is what absorbs the queue.
QWENTTS_MAX_BATCH=1
# --- the boot warmer (dankclaw/voiceservice/tts-engine-warm.sh) --------------
# These three are read by the warm hook, not by tts-server. They live here
# because the hook is exec-ed into this container and therefore inherits this
# same EnvironmentFile — one file, one restart, no second source of truth.
# Container path of the voice packs; the host directory is /home/dank/voices,
# mounted read-only by the unit and mounted (read-only, same source) by
# frank-tts as well. A directory counts as a voice exactly when it contains
# reference.wav, which is the same rule frank-tts applies.
VOICES_DIR=/voices
# How long the hook waits for /health before giving up. Model load dominates
# this: both GGUFs are read and the context is built before the HTTP thread
# binds, and a boot competes with every other unit on the box. 300 s is generous
# against a cold cache; the unit's TimeoutStartSec (900 s) is what bounds the
# whole pass, warming included.
HEALTH_WAIT=300
# The sentence each voice speaks to snapshot its codec state, and the text the
# second (warm-timing) pass repeats. Fixed on purpose: the cold=/warm= numbers in
# the journal are comparable across boots only if the input is the same. Kept
# short because warming a library is serial — two syntheses per voice, ~5 s each,
# inside the unit's start deadline.
# QUOTED, and the quoting means two different things to the two readers of this
# file, which is the trap here:
# * bash (`set -a; source`, how the repo's seams and any human read it) needs
# the quotes — an unquoted value with spaces is word-split and the second
# word is executed as a command.
# * podman's `--env-file` (how the unit actually reads it) does NOT strip
# quotes: measured 2026-10-18 against the image's pinned base, the value
# arrives as `"Warm up, one two three."` WITH them. Interpolated into the
# hook's JSON body that is `""Warm up…""`, the engine's parser rejects it,
# every voice stays cold and the boot still looks healthy.
# So the value stays quoted for bash and the hook strips one matched pair before
# it builds the request, then escapes what is left. Both readers now agree, and a
# sentence containing a quote or a backslash still arrives verbatim.
WARM_TEXT="Warm up, one two three."
# Per-request ceiling for the hook. A cold voice measures 6-8 s, so 120 s is
# ~15x margin: reaching it means the engine is wedged, and systemd is still
# waiting on this pass before it calls the unit started.
REQUEST_TIMEOUT=120
+28 -8
View File
@@ -30,23 +30,43 @@
#
# BACKEND SWITCH. TTS_BACKEND picks the shape and nothing else changes for the
# consumer:
# omni-http forward to an OpenAI-shaped upstream on the host (:8880),
# carrying the voice along as base64 reference_audio. The
# engine does the inference; the shim resolves the voice and
# re-encodes to mp3 when asked.
# omni-http forward to an OpenAI-shaped upstream, carrying the voice along
# as base64 reference_audio. The engine does the inference; the
# shim resolves the voice and re-encodes to mp3 when asked. The
# shipped engine (frank-tts-engine) speaks a voice it has
# REGISTERED on itself, from the same /home/dank/voices packs,
# and warms them at boot — see [[Frank TTS Engine]] for what that
# boot warm is for (a memory-only registry: without it every name
# is a 400) and what it saves (~0.6 s).
# llama-tts-cli run LLAMA_TTS_BIN once per request with
# --tts-speaker-file <reference.wav>, nice 19 so a narration
# never starves the text endpoints on this box. Needs
# LLAMA_TTS_BIN + LLAMA_TTS_MODEL filled in; LLAMA_TTS_MMPROJ
# only if that binary takes a projector.
# The upstream address is host.containers.internal — the gateway name a bridge
# container uses for the host. It is NOT the LAN IP and it is NOT
# 127.0.0.1: a bridge netns has neither.
# THE UPSTREAM ADDRESS. The engine is a container on the private bridge this
# unit also joins (dankclaw/voiceservice/quadlet/frank-tts-engine.network), so
# this is its container DNS name. It used to read
# `http://host.containers.internal:8880`, which is a shape that cannot work: a
# bridge container reaches the host's network namespace, netavark DNATs PUBLISHED
# ports only, and an engine on the host's loopback is published to nobody
# ([[Container Network Namespaces]] — the trap that 502'd hermes). Inert under
# TTS_BACKEND=llama-tts-cli, which needs no upstream at all.
VOICES_DIR=/voices
TTS_BACKEND=omni-http
TTS_UPSTREAM=http://host.containers.internal:8880
TTS_UPSTREAM=http://frank-tts-engine:8880
TTS_UPSTREAM_MODEL=voxcpm2
# THE DEFAULT VOICE — what a request that names NO voice speaks. The omp harness
# has no setting that feeds a voice to its speech device (`voice_id` is a
# per-call argument; `speech.voice` belongs to the TUI's Kokoro speak-aloud and
# never reaches an HTTP request), so a harness voiceover arrives voice-less and
# would otherwise be a 400 for every caller that does not name one. An UNKNOWN
# name still 400s with the list that exists — this covers the absent case only —
# and the journal marks such a line `(default)` so an unrequested voice never
# reads as a requested one. Any pack directory in /home/dank/voices is legal.
DEFAULT_VOICE=narrator1
# CLI-backend knobs, inert while TTS_BACKEND=omni-http. Empty means "not
# configured": the endpoint answers a 500 that says so instead of running
# `nice env -m ...` with an empty binary name.
+56 -125
View File
@@ -1,141 +1,72 @@
# frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) serving
# prism-ml/Ternary-Bonsai-2-9B-gguf at PQ2_0 **with the MTP head in the file**
# on the RTX 3090, :7771.
# frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) on the RTX 3090,
# :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file.
#
# 2026-10-18 (Frank): swap from the Bonsai-2-**27B** PQ2_0 to the 9B MTP export
# at THREE 262,144-token conversations (Frank: "if you can possibly fit a third
# context, do it"). The 9B bills HALF the KV of the 27B per token, which is what
# funds 3 x the native window on one 24,123 MiB card. Speculative decoding ships
# OFF — measured, not assumed; see the MTP law below, which is the whole story
# and the reason the publisher's "depth 2 works best" line does not transfer.
#
# THE RUNTIME IS A LOCK, NOT A PREFERENCE. The file is general.file_type 141
# carrying custom GGML type 142 tensors plus prism.hadamard.* metadata
# (normalized-Sylvester-Walsh transform); stock llama.cpp refuses type 142 at
# load and a loader that ignored the Hadamard metadata would dequant WRONG. On
# top of that lock, MTP beside a quantized V cache needs fork build >= b10776,
# so the vendor prebuilt moved from 10754 (commit 2459f68b5, the 27B era) to
# 10777 (commit e3de71522). The 10754 tree stays on disk as the rollback
# runtime for the 27B payload.
# ONE OWNER: shape, payload identity, eras, laws and every measured number
# live in serverAppConfigs-docs/LLM/frank-zealot.md. This file carries the
# knobs and only the reason a reader editing THIS line cannot recover
# elsewhere. Where the two disagree, this file wins on values.
LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server
# 2,675,410,720 B, Hub LFS sha256 c79fcdfaf6b7… (verified at fetch by
# scripts/download-frank-zealot-bonsai9.sh). This IS the MTP build: the draft
# head ships INSIDE the checkpoint, so MTP availability is a property of the
# FILE, never of a flag. The plain PQ2_0 sibling could not run it at any flag
# value; loading this file instead of that one is the only change that matters.
# The fork is a LOCK, not a preference: custom tensor type 142 + prism.hadamard.*
# metadata. Stock llama.cpp refuses the file; a loader ignoring the Hadamard
# metadata dequantizes wrong, silently.
MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf
# 624,229,664 B, Hub LFS sha256 e0cb6f69… — Q8_0 IS MANDATORY, not a preference:
# the publisher's card says the BF16 mmproj "can crash llama-server when several
# image requests run at once; Q8_0 is not affected", and this endpoint runs
# --parallel 3, i.e. exactly that shape. The projector's worst case lands AFTER
# the engine's own fit check, so it is never inside a projection number.
# Q8_0 IS MANDATORY: the publisher states the BF16 tower can crash llama-server
# under concurrent image requests, which is exactly --parallel 3.
MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
# Base alignment: Bonsai-2 is the censored BASE lineage. The 27B CRACK re-export
# (/home/dank/models/Bonsai2-CRACK/) is byte-identical geometry to the 27B it
# matched, NOT to this 9B file — if a refusal ever surfaces there is no
# one-line swap here; the 9B dealigned export would have to be fetched first.
#
# Served model id — the harness/family name. The SERVICE, unit, scripts and
# this env file keep the frank-zealot name; only the alias on the wire is the
# model id. Consumers that key on the served id: OpenWebUI (model row,
# access_grant, api_configs model_ids) and the harness models.yml.
# Served id — the harness/family name, not a fact about the payload.
# Id-keyed consumers: OpenWebUI (model row, access_grant, api_configs), models.yml.
ALIAS=frank-zealot
BIND_ADDR=0.0.0.0
LISTEN_PORT=7771
# CTX_SIZE is the POOL llama.cpp divides across slots: 786,432 / 3 = 262,144 per
# conversation — each slot at the model's TRAINED window, which is the point of
# the swap (the 27B era had to cut each conversation to 200,704 to fit two).
# MUST stay divisible by PARALLEL (the seam enforces it) and the harness
# contextWindow quotes the PER-SLOT number — a longer prompt is answered with a
# 400, never truncated. The ceiling cannot go past 3 x 262,144: that is the
# native wall (no YaRN in this path) and the arithmetic agrees — a 4th slot
# would push each below the compaction contract.
CTX_SIZE=786432
# CTX_SIZE is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
# conversation. MUST stay divisible by PARALLEL (the seam enforces it) and
# models.yml contextWindow quotes the PER-SLOT number — a longer prompt is a
# 400, never a truncation. The pool is bought, not granted: K+V at 25,088
# B/token is 14,406.00 MiB here and the MTP draft context bills its own f16 KV
# over the WHOLE pool (4,096 B/position ⇒ 2,352.00 MiB), so the old 786,432
# pool cannot carry a drafter at any KV dtype — 18,816 + 3,072 of cache alone
# against the 24,123 MiB the driver leaves addressable.
CTX_SIZE=602112
PARALLEL=3
BATCH_SIZE=4096
# Ubatch is a prefill knob (decode never moves) with NO interior optimum on this
# CUDA card up to 2048 — 2048 is the measured optimum (+4.6 % prefill). It has a
# second job here: it sets the compute-buffer size, and the compute buffers are
# what decide whether MTP fits (see the MTP law). 1024 is the documented MTP arm.
UBATCH_SIZE=2048
# KV law (header + boot ledger, both confirmed): qwen35 hybrid, 32 layers with
# qwen35.full_attention_interval = 4 ⇒ 8 full-attention + 24 Gated DeltaNet;
# 4 KV heads × 256 dim = 8,192 elements per token per cache. q8_0 bills 8,704 B
# per cache ⇒ 17,408 B/token = 17 KiB, so the 786,432 pool is 13,056.00 MiB —
# the ledger said exactly that. HALF the 27B's 34,816 B/token, which is the
# whole reason three native windows fit. f16 would bill 26,112 MiB at this pool
# (against a 24,123 MiB card) and dies at cudaMalloc; the 24 DeltaNet layers
# bill a fixed recurrent state per SLOT instead of per token — the PRODUCTION
# ledger prints "RS buffer size = 150.75 MiB (3 cells, 32 layers, 3 seqs)"
# (~50 MiB a cell), while the SPEC-ARMED arms print 452.25 MiB, because MTP
# depth 2 bills three seqs per slot (9 cells) — which is why pool arithmetic
# alone under-prices the shape.
CACHE_TYPE_K=q8_0
# The logical batch is VRAM- and throughput-neutral at a fixed ubatch (the law
# on [[frank-zealot]]); 1024×6 keeps it a clean multiple of the ubatch.
BATCH_SIZE=6144
# Ubatch is a prefill knob (decode never moves with it) and it sets the
# compute-buffer size, which decides whether a drafter fits. 1024 is the rung
# every measured MTP arm here ran on (213.5 t/s against 109.2 at 2048). Its
# VRAM credit is SHAPE-specific: the 786,432-pool arm census showed ~1,306 MiB
# back, this boot measured 2,089.88 MiB of target compute against 2,496.13 at
# ubatch 2048 — 406 MiB back, with the split moving (CUDA0 +257, CUDA_Host
# −663). The drafter is funded by CACHE_TYPE_V below, not by this knob.
UBATCH_SIZE=1024
# K exact, V in the 8-bit class: 16,384 + 8,704 = 25,088 B/token (8,192
# elements per token per cache × 2 B for f16, × 1.0625 for q8_0 — there is no
# true fp8 KV dtype in this build). That 7,680 B/token saved against exact V is
# what funds BOTH the third conversation and the drafter. Quantized V is
# illegal without --flash-attn, which the serve script pins. Rate law and the
# arithmetic: [[frank-zealot]] → Laws.
CACHE_TYPE_K=f16
CACHE_TYPE_V=q8_0
# SPECULATIVE DECODING: OFF, MEASURED. The head is in the file and the drafter
# is GOOD (acceptance 0.798/0.617 per position, mean accepted length 2.41) — it
# still loses, because the draft context bills its OWN f16 KV over the WHOLE
# pool: 4,096 B/position × 786,432 = 3,072.00 MiB, plus its compute buffers,
# 4,830 MiB all told. Booted arms, same card, same traffic (2026-10-18):
# 3 × 262,144 spec OFF ub 2048 → 17,876 used / 6,249 free, decode 167.2 t/s
# 3 × 262,144 spec ON ub 2048 → 22,706 used / 1,419 free, decode 109.2 t/s
# 3 × 262,144 spec ON ub 1024 → 21,400 used / 2,725 free, decode 213.5 t/s
# 2 × 262,144 spec ON ub 2048 → 17,178 used / 6,947 free, decode 213.2 t/s
# 1 × 262,144 spec ON ub 2048 → 11,652 used / 12,473 free, decode 214.2 t/s
# i.e. MTP is worth +28 % decode whenever the card has room, and at the 3-slot
# pool it costs 1.3 GiB of compute buffers that ubatch 2048 refuses to give up —
# the squeeze, not the drafter, is what halves decode. The arm that WOULD run
# MTP at three windows is SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024 (decode 213.5,
# 3-stream 250.8 aggregate) and pays prefill −17 % and 2.7 GiB of headroom on a
# card whose projector worst-case lands after the fit check. This endpoint's
# load is the fleet's COMPACTION HUB (prefill-bound, ~170k in) plus 2-3
# concurrent subagents, so the deployed arm takes the +17 % prefill
# (2,877 vs 2,395 t/s at 106,418 tokens) and the +12 % three-stream aggregate
# (280.2 vs 250.8) over single-stream speed, and keeps 6.2 GiB free.
# SPEC_TYPE=none strips the WHOLE spec set (--spec-type, every --spec-draft-*):
# flags move as a set or the arm measures a crash instead of an absence.
SPEC_TYPE=none
# Speculation ON (Frank, 2026-10-19): the head is in the file
# (qwen35.nextn_predict_layers = 1, blk.32.nextn.*) and rides the MAIN model's
# weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of
# draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for
# by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V
# beside MTP needs fork build >= b10776; LLAMA_BIN is 10777.
# SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and
# every --spec-draft-*); half a set has segfaulted at load.
SPEC_TYPE=draft-mtp
SPEC_DRAFT_N_MAX=2
SPEC_DRAFT_N_MIN=0
SPEC_DRAFT_P_MIN=0
# The card's sampler recipe lives in the checkpoint (general.sampling.* = temp
# 1.0 / top_p 0.95 / top_k 20 / min_p 0 / repeat_penalty 1.0 — confirmed in the
# engine's own /props defaults) and llama.cpp reads it itself. Presence penalty
# is the ONE recipe value with no GGUF metadata key, so it must reach the argv.
# PRESENCE_PENALTY="" is the documented opt-out.
# The sampler recipe lives in the checkpoint (general.sampling.*) and
# llama.cpp reads it itself. Presence penalty is the one recipe value with no
# GGUF metadata key, so it must reach argv. "" is the documented opt-out.
PRESENCE_PENALTY=1.0
NGL=99
# Boot ledger (2026-10-18 PRODUCTION boot under the unit, verbosity 4):
# 17,922 MiB used / 6,203 MiB free of 24,123 (nvidia-smi settled)
# CUDA0 model 2,036.71 MiB + CPU_Mapped 257.66 MiB
# CUDA0 KV 13,056.00 MiB (q8_0 K + q8_0 V, 786,432 positions)
# CUDA0 RS 150.75 MiB (3 cells, 32 layers, 3 seqs)
# compute 1,409.02 MiB CUDA + 1,088.87 MiB CUDA_Host (+2.84 MiB output buf)
# mmproj worst-case estimate 868.32 MiB (248.10 CUDA + 24.93 CPU compute
# meta) — charged AFTER the engine's own fit check, as always
# no clamp line, no cudaMalloc error
# Measured on this boot (/completion timings, greedy, ignore_eos, 512 tokens):
# 168.7 t/s decode (three runs 168.4-168.7) and ~300 t/s aggregate across
# THREE concurrent streams (102.8 each). The trial arm's record (167.2 decode,
# 280.2 aggregate, 2,877 t/s prefill at 106,418) stands; the 27B it replaces
# gave 76.6 / 109.4 / 691.7 at 166k. The FIRST request after a boot is a
# warmup artifact (37.7 t/s class) and a natural generation that stops early
# makes a wall-clock aggregate read 4x low - measure with ignore_eos and read
# the engine's own timings.
# Fallback rung if the margin is ever eaten (desktop, co-tenant): q4_0 KV on the
# 27B era measured 16,392 used / 7,733 free at 2 × 200,704; the same drop here
# frees ~6.5 GiB at three windows. UBATCH_SIZE=1024 is the cheaper lever
# (−1,306 MiB measured on this shape).
# Log verbosity threshold (llama.cpp -lv). 4 = debug: KV layout, buffer sizes
# and graph capture land in the journal, which is what the VRAM-fit forensics
# read. Levels above 4 flood the journal.
# 4 = debug: KV layout, buffer sizes and graph capture land in the journal, and
# that is what VRAM forensics read. Above 4 floods the journal.
VERBOSITY=4
# Serve /metrics. llama.cpp answers 501 without --metrics, and Dankclaw's
# Prometheus scrapes /metrics on every fleet endpoint — a 501 reads as DOWN, so
# this service has held the critical LlmEndpointDown alert for as long as the
# metric has history (verified 2026-10-18: the oldest retained sample of
# up{job="frank-zealot"}, 2026-10-11, is already 0; scrapeUrl
# http://host.containers.internal:7771/metrics, lastError "server returned HTTP
# status 501 Not Implemented"). 0 opts out and re-breaks the scrape.
# Without --metrics llama.cpp answers 501 on /metrics, which Prometheus reads
# as DOWN while /health stays 200. 0 opts out and re-breaks the scrape.
METRICS=1
+1 -1
View File
@@ -1,5 +1,5 @@
[Unit]
Description=llama.cpp (PrismML build 10777) Ternary-Bonsai-2-9B PQ2_0-MTP GGUF (qwen35 hybrid, 3x262144 slots over one 786432 pool, q8_0 K+V, MTP off, mmproj Q8_0 vision) on the RTX 3090 (dankclaw, frank-zealot)
Description=llama.cpp (PrismML build 10777) Ternary-Bonsai-2-9B PQ2_0-MTP GGUF (qwen35 hybrid, 2x262144 slots over one 524288 pool, unquantized f16 K+V, MTP off, mmproj Q8_0 vision) on the RTX 3090 (dankclaw, frank-zealot)
After=network-online.target
Wants=network-online.target
+211
View File
@@ -0,0 +1,211 @@
#!/usr/bin/env python3
"""Regenerate serverAppConfigs-docs/index.md from page frontmatter.
The root index is a locator (AGENTS.md -> *Documentation Policy*): one row per
page, one clause, no numbers. Hand-maintaining it is how rows go missing, and
the docs seam fails when a page is not linked from it, so it is generated:
* preamble everything above the first `## ` in the existing file is kept
verbatim (that is the curated statement of the vault's law)
* section from the page's directory, overridden by its frontmatter tags
(`endpoint` / `retired-endpoint` split the LLM pages)
* row text the page's frontmatter `description`, verbatim
A page with no description is a hard error rather than an empty cell, so the
generator cannot silently produce a silent index. Sections with no pages are
skipped. Deterministic: the same vault always yields the same bytes.
Sub-directory indices are hand-grouped (they carry curated prose), so they are
synced rather than regenerated: dead rows dropped, row text refreshed from the
page's description, uncovered siblings appended.
"""
import pathlib
import re
import sys
RESERVED = {"index.md", "log.md"}
SECTIONS = [
("Inference endpoints", "endpoint"),
("Retired endpoints", "retired-endpoint"),
]
DIRECTORY_SECTIONS = [
("Dankclaw", "Dankclaw services"),
("Dankserver", "Dankserver services"),
("FrankBook-Pro", "FrankBook-Pro"),
("Desktop", "Workstations"),
("Infrastructure", "Cross-host infrastructure"),
("Architecture", "Repo architecture and policy"),
("Operations", "Operations and incidents"),
("Agent", "Agent / harness"),
("Research", "Research and methodology"),
]
PIPELINE_SECTION = "Pipeline Parallelism (series)"
LLM_FALLBACK = "LLM technique and hardware references"
CHANGE_SECTION = "Change history"
ORDER = [t for t, _ in SECTIONS] + [t for _, t in DIRECTORY_SECTIONS] + \
[PIPELINE_SECTION, LLM_FALLBACK, CHANGE_SECTION]
def frontmatter_fields(text):
if not text.startswith("---"):
return {}
end = text.find("\n---", 3)
if end == -1:
return {}
raw = text[3:end]
fields = {}
for key in ("description", "type"):
m = re.search(rf"^{key}:\s*(.*)$", raw, re.M)
if m:
fields[key] = m.group(1).strip().strip("\"'")
tags = []
m = re.search(r"^tags:\s*\[(.*?)\]$", raw, re.M)
if m:
tags = [t.strip().strip("\"'") for t in m.group(1).split(",") if t.strip()]
elif re.search(r"^tags:\s*$", raw, re.M):
block = re.search(r"^tags:\s*\n((?:\s+-\s+.*\n?)+)", raw, re.M)
if block:
tags = [l.strip().lstrip("-").strip().strip("\"'")
for l in block.group(1).splitlines() if l.strip()]
fields["tags"] = tags
return fields
def section_for(page, vault, fields):
rel = page.relative_to(vault)
if len(rel.parts) == 1:
return CHANGE_SECTION if page.name == "log.md" else "Uncategorised"
top = rel.parts[0]
if top == "LLM":
if PIPELINE_SECTION_DIR in rel.parts:
return PIPELINE_SECTION
for title, tag in SECTIONS:
if tag in fields.get("tags", []):
return title
return LLM_FALLBACK
for prefix, title in DIRECTORY_SECTIONS:
if top == prefix:
return title
return "Uncategorised"
ROW = re.compile(r"^(\| )\[\[([^\]|]+)\]\]( \|)( .*?)( \|$)")
BULLET = re.compile(r"^(- )\[\[([^\]]+)\]\]( — )(.*?)$")
def sync_subindex(index: pathlib.Path, vault: pathlib.Path):
"""Keep a hand-grouped sub-index honest without erasing its grouping.
Rows are locators, so their text is the linked page's own description; a
row whose page is gone is a dead promise and is dropped; a direct sibling
with no row is appended under `## Pages` (that is what the link-integrity
seam checks). Curated headings and prose are untouched.
"""
here = index.parent
known = {p.stem for p in here.rglob("*.md")
if ".AGENTTRASH" not in str(p) and p.name != "index.md"}
lines = index.read_text().split("\n")
out, dropped, refreshed = [], 0, 0
for line in lines:
# A row whose first cell is empty is a description with no page: the link
# was deleted when the page was destroyed, so the row is a dead promise.
if re.match(r"^\|\s*\|", line.strip()):
dropped += 1
continue
for pat, wrap in ((ROW, "| [[{}]] |{} {} |"), (BULLET, "- [[{}]] — {}")):
m = pat.match(line)
if not m:
continue
stem = m.group(2).strip()
if stem not in known:
dropped += 1
line = None
break
cand = next((p for p in here.rglob(f"{stem}.md")
if ".AGENTTRASH" not in str(p)), None)
desc = frontmatter_fields(cand.read_text(errors="replace")).get("description") \
if cand else None
if desc:
current = m.group(4).strip()
if current != desc:
line = wrap.format(stem, " " if pat is ROW else "", desc) \
if pat is ROW else wrap.format(stem, desc)
refreshed += 1
break
if line is not None:
out.append(line)
text = "\n".join(out)
# Presence must be the wikilink itself: a bare stem is a substring of other
# page names (`frank-medium` lives inside `frank-medium-rocmfpx`), and a
# sibling matched that way silently never gets a row.
missing = sorted(
(p.stem, frontmatter_fields(p.read_text(errors="replace")).get("description", ""))
for p in here.glob("*.md")
if p.name != "index.md" and f"[[{p.stem}]]" not in text
)
if missing:
block = ["", "## Pages", "", "| Page | Owns |", "|---|---|"]
block += [f"| [[{stem}]] | {desc} |" for stem, desc in missing]
text = text.rstrip("\n") + "\n" + "\n".join(block) + "\n"
if text != index.read_text():
index.write_text(text)
return dropped, refreshed, len(missing)
PIPELINE_SECTION_DIR = "Pipeline Parallelism"
def main(argv):
vault = pathlib.Path(argv[1] if len(argv) > 1 else "serverAppConfigs-docs")
index = vault / "index.md"
if not index.is_file():
print(f"missing {index}")
return 2
existing = index.read_text()
preamble = existing.split("\n## ", 1)[0].rstrip("\n")
pages = [p for p in vault.rglob("*.md")
if ".AGENTTRASH" not in str(p) and ".obsidian" not in str(p)
and p.name != "index.md" and p != index]
buckets, errors = {}, []
for page in pages:
fields = frontmatter_fields(page.read_text(errors="replace"))
if page.name != "log.md":
desc = fields.get("description")
if not desc:
errors.append(str(page.relative_to(vault)))
continue
else:
desc = "Chronological incident and law log for the whole bundle."
buckets.setdefault(section_for(page, vault, fields), []).append((page.stem, desc))
if errors:
print("pages with no description (fix the frontmatter, not the index):")
for e in sorted(errors):
print(" " + e)
return 1
out = [preamble, ""]
for title in ORDER + sorted(set(buckets) - set(ORDER)):
rows = sorted(buckets.get(title, []), key=lambda r: r[0].lower())
if not rows:
continue
out += [f"## {title}", "", "| Page | Owns |", "|---|---|"]
out += [f"| [[{stem}]] | {desc} |" for stem, desc in rows]
out.append("")
out.append("<!-- generated by scripts/gen-vault-index.py; edit page frontmatter instead -->")
index.write_text("\n".join(out) + "\n")
touched = 0
for sub in sorted(vault.rglob("index.md")):
if sub == index or ".AGENTTRASH" in str(sub):
continue
dropped, refreshed, added = sync_subindex(sub, vault)
if dropped or refreshed or added:
touched += 1
print(f" {sub}: -{dropped} dead rows, {refreshed} refreshed, +{added} added")
print(f"{index}: {sum(len(v) for v in buckets.values())} rows in "
f"{len([t for t in ORDER if buckets.get(t)])} sections, {touched} sub-indices synced")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env python3
"""Port census: prove a merge/collapse lost no fact.
For every line that existed in the OLD set and not in the NEW set, check that
each of its significant tokens (numbers, long identifiers, paths) still occurs
somewhere in the NEW corpus. A line whose every token survives is a reworded
line; a line with an unheld token is a candidate fact loss and gets printed.
port-census.py <new-glob-dir> <old-file> [<old-file> ...]
Old files may be `git-rev:path` (read from git) or a path (e.g. a trash copy).
"""
import pathlib
import re
import subprocess
import sys
def toks(line):
out = []
for n in re.findall(r"\d[\d,]*(?:\.\d+)?", line):
if len(n.replace(",", "").split(".")[0]) >= 3 or (
"." in n and len(n.split(".")[-1]) >= 2 and len(n.replace(",", "")) >= 4
):
out.append(n)
# Identifiers, flags, paths and backticked spans are the facts worth
# tracking; ordinary prose words are not — they reword on every rewrite.
# A hyphen alone is not evidence of an identifier (`long-standing`), so a
# dashed word counts only with two hyphens (`frank-tts-engine`) or with a
# digit, dot, slash, colon or `--` inside it.
for w in re.findall(r"[A-Za-z][\w./:-]{7,}", line):
core = w.strip(".,;:!?\"')]")
if (core.startswith("--") or core.isupper()
or re.search(r"[\d_/:.]", core) or core.count("-") >= 2):
out.append(core)
out += [s.strip() for s in re.findall(r"`([^`]+)`", line) if len(s) > 3]
return out
def held(tok, corpus):
t = tok.replace(",", "")
alt = rf"\b{re.escape(t)}\b"
for c in corpus:
if tok in c or t in c:
return True
if re.search(alt, c.replace(",", "")):
return True
return False
def main(argv):
new_dir = pathlib.Path(argv[1])
def corpus_excluding(exclude):
"""Every vault page except the one under test.
The corpus is what must still hold a fact AFTER the file is gone, so a
file that lives inside the scanned directory has to be removed from it —
otherwise every token is "held" by the file itself and the census proves
nothing (it reports 0 losses for every page, which is the bug this
guards).
"""
out = []
for p in new_dir.rglob("*.md"):
s = str(p)
if ".AGENTTRASH" in s or ".obsidian" in s:
continue
if exclude is not None and p.resolve() == exclude:
continue
out.append(p.read_text(errors="replace"))
return out
lost = checked = 0
for spec in argv[2:]:
if spec.startswith("git-"):
ref, path = spec[4:].split(":", 1)
old = subprocess.run(["git", "show", f"{ref}:{path}"],
capture_output=True, text=True).stdout
label = f"{ref}:{path}"
exclude = None
else:
old = pathlib.Path(spec).read_text(errors="replace")
label = spec
exclude = pathlib.Path(spec).resolve()
blob = "\n".join(corpus_excluding(exclude))
for line in old.splitlines():
s = line.strip().strip("->*|# ").strip()
if len(s) < 30 or s.startswith("```"):
continue
checked += 1
missing = [t for t in toks(s) if not held(t, [blob])]
if missing:
lost += 1
print(f" {label}\n L: {s[:150]}\n unheld: {', '.join(missing[:6])}")
print(f"{checked} old lines checked, {lost} with unheld tokens")
return 1 if lost else 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env python3
"""Repair the two artifacts the destruction passes introduced.
1. The link stripper could eat the space after a `- ` marker (`-`X``) and leave a
dangling `see .` where a wikilink used to be.
2. The Eras collapse could swallow a table's HEADER row (it looked like just
another non-current row), leaving a separator with no header above it.
Usage: repair-damage.py [--apply]
"""
import pathlib
import re
import sys
VAULT = pathlib.Path("serverAppConfigs-docs")
ERAS_HEADER = "| Window | Payload | Runtime | Shape | Replaced because | Rollback |"
def main(argv):
apply = "--apply" in argv
marker = dangling = header = 0
for p in sorted(VAULT.rglob("*.md")):
s = str(p)
if ".AGENTTRASH" in s or ".obsidian" in s:
continue
lines = p.read_text(errors="replace").split("\n")
out = []
# Only a pointer whose target was deleted reads as `… see .` — never touch
# a line merely because it wraps on a comma. (An earlier version of this
# rule stripped the trailing comma off 660 wrapped prose lines, which
# `scripts/restore-punctuation.py` then had to undo from HEAD.)
for line in lines:
new = line
if re.match(r"^(\s*)[-*](?![-* ])[$`[(*_]", new):
new = re.sub(r"^(\s*)[-*]", r"\1- ", new)
marker += 1
if re.search(r"\s(?:see|also|details in|via|per)\s*\.?\s*$", new):
new = re.sub(r"\s(?:see|also|details in|via|per)\s*\.?\s*$", "", new)
dangling += 1
out.append(new)
# A separator row whose line above is not a header gets its header back.
fixed = []
for i, line in enumerate(out):
if re.match(r"^\|-+", line.strip()) and (not fixed or not fixed[-1].strip().startswith("|")):
fixed.append(ERAS_HEADER)
header += 1
fixed.append(line)
if apply and "".join(fixed) != "".join(lines):
p.write_text("\n".join(fixed))
print(f"{'APPLIED' if apply else 'dry run'}: marker spacing {marker}, "
f"dangling pointers {dangling}, restored table headers {header}")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env python3
"""Undo trailing-punctuation damage by restoring lines that match HEAD exactly
except for a stripped trailing `,` / `;` / removed trailing word.
The repair pass over-reached: its "dangling pointer" cleanup also matched every
wrapped prose line ending in a comma, and deleted the comma. A line whose text
is otherwise byte-identical to its HEAD version, differing only by trailing
punctuation, is not an edit — it is damage, so HEAD's line goes back.
Usage: restore-punctuation.py [--apply]
"""
import pathlib
import subprocess
import sys
REPO = pathlib.Path(".")
VAULT = REPO / "serverAppConfigs-docs"
def head_text(rel):
r = subprocess.run(["git", "show", f"HEAD:{rel}"], capture_output=True, text=True)
return r.stdout if r.returncode == 0 else None
def main(argv):
apply = "--apply" in argv
restored = 0
pages = 0
for p in sorted(VAULT.rglob("*.md")):
s = str(p)
if ".AGENTTRASH" in s or ".obsidian" in s:
continue
rel = s
head = head_text(rel)
if head is None:
continue
cur = p.read_text(errors="replace").split("\n")
# Key on (length, prefix) of the HEAD line minus trailing punctuation so a
# damaged line can find its original.
index = {}
for hl in head.split("\n"):
h = hl.rstrip()
if h.endswith((",", ";")) or True:
key = (len(h.rstrip(",;. ")), h.rstrip(",;. ")[:60])
index.setdefault(key, h)
touched = 0
for i, cl in enumerate(cur):
c = cl.rstrip()
if not c or c.endswith((".", ":", "|", "`", ")", "," , ";")):
continue
key = (len(c), c[:60])
hl = index.get(key)
if hl and hl != c and hl.startswith(c):
cur[i] = hl
touched += 1
if touched:
pages += 1
restored += touched
if apply:
p.write_text("\n".join(cur))
print(f"{'APPLIED' if apply else 'dry run'}: {restored} lines restored across {pages} pages")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+25 -26
View File
@@ -15,12 +15,13 @@
# log states every budget before anything is allocated (weights vs KV
# split, the expert host pool's coverage of the worst case, the PLE pin,
# the P2P answer per pair) — read it after every resize.
# Deployed shape (2026-10-13): THREE decode slots x 200,000 tokens, fp8
# (quantized) KV in a 5,120 MiB pool (contract 3 x 200,000 + 1.3 %) and a
# 25,600 MiB resident expert slab — 30,720 MiB of the 30,969 MiB the engine's
# own budget table makes claimable, spent on the slab because prefill is
# slab-limited. 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache host
# 8 GiB + disk 64 GiB, PLE pinned in RAM.
# Deployed shape (2026-10-19, radiance 1.8.1): THREE decode slots x 200,000
# tokens, fp8 KV, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache in
# pinned host RAM, PLE pinned in RAM. The per-card split between the expert slab
# and the KV pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair
# and the budget resolver derives it from headroom, the cache bound and the host
# pool, so the slab size and the pool's token count are LEDGER lines: after every
# boot read them out of the log and check the pool still backs 3 x 200,000.
#
# The root unit (scripts/frank-archon.service, User=dank,
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
@@ -56,13 +57,11 @@ set +a
: "${FRANK_ARCHON_PLACEMENT:?FRANK_ARCHON_PLACEMENT must be set}"
: "${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
: "${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
: "${FRANK_ARCHON_VRAM_WEIGHTS_MIB:?FRANK_ARCHON_VRAM_WEIGHTS_MIB must be set}"
: "${FRANK_ARCHON_VRAM_KV_MIB:?FRANK_ARCHON_VRAM_KV_MIB must be set}"
: "${FRANK_ARCHON_BUSY_PREFILL_CHUNK:?FRANK_ARCHON_BUSY_PREFILL_CHUNK must be set (auto|off|N)}"
: "${FRANK_ARCHON_KV_DTYPE:?FRANK_ARCHON_KV_DTYPE must be set}"
: "${FRANK_ARCHON_SPEC_TOKENS:?FRANK_ARCHON_SPEC_TOKENS must be set}"
: "${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
: "${FRANK_ARCHON_PREFIX_HOST_MIB:?FRANK_ARCHON_PREFIX_HOST_MIB must be set}"
: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
: "${FRANK_ARCHON_PREFIX_CACHE_MIB:?FRANK_ARCHON_PREFIX_CACHE_MIB must be set}"
: "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
: "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
: "${FRANK_ARCHON_TTM_TARGET_MIB:?FRANK_ARCHON_TTM_TARGET_MIB must be set}"
@@ -103,19 +102,12 @@ if [ ! -f "$FRANK_ARCHON_TEMPLATE" ]; then
exit 1
fi
mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
# The engine's create is NOT recursive: with only the state dir mounted, its
# "cannot create /data/kvcache/frank-archon: No such file or directory"
# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
# the full path on the host so the tier opens on first boot.
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
# rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
# the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
# $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
# the cache must ride a host mount or it dies with the --rm container
# that wrote it. The dir is pre-created for the same reason as kvcache
# (the engine's create is not recursive).
# that wrote it. The engine's create is not recursive, so the dir is pre-created.
if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
@@ -148,6 +140,15 @@ done
LOG=/home/dank/logs/frank_archon.log
mkdir -p "$(dirname "$LOG")"
# Interleave is opt-in: 0 is the engine's default and passing it anyway would put
# a flag in argv that changes nothing but the record. The sharing cap is always
# explicit because `auto` vs `off` is the difference between decode waiting one
# chunk and waiting the whole step.
PREFILL_ARGS=()
if [ "${FRANK_ARCHON_PREFILL_INTERLEAVE:-0}" -gt 0 ]; then
PREFILL_ARGS=(--prefill-interleave "$FRANK_ARCHON_PREFILL_INTERLEAVE")
fi
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
# --network host (the server binds the host's addresses directly, no NAT),
# --init (the engine releases the cards on SIGTERM), seccomp=unconfined
@@ -156,8 +157,9 @@ mkdir -p "$(dirname "$LOG")"
# makes for host memory), memlock unbounded (the pinned ledger — legal
# because the root unit grants LimitMEMLOCK=infinity to this dank process),
# /dev/kfd + /dev/dri (render/video groups are moot for a root container).
# The template rides a single-file mount; the .rad dir is ro; /data holds
# the prefix-cache disk tier.
# The template rides a single-file mount and the .rad dir is ro. The prefix cache
# has no disk tier any more, so the only host state this container writes is the
# rad-tune cache under /opt/radiance/share/radiance/tune.
podman run -d --name "$FRANK_ARCHON_CONTAINER" \
--network host \
--init \
@@ -165,7 +167,6 @@ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
--ulimit memlock=-1:-1 \
--device /dev/kfd --device /dev/dri \
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-v "$FRANK_ARCHON_STATE_DIR":/data \
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
"$FRANK_ARCHON_IMAGE" \
@@ -179,15 +180,13 @@ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
--placement "$FRANK_ARCHON_PLACEMENT" \
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
--prefix-cache-dir /data/kvcache/frank-archon \
--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
--prefix-cache-mib "$FRANK_ARCHON_PREFIX_CACHE_MIB" \
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
--busy-prefill-chunk "$FRANK_ARCHON_BUSY_PREFILL_CHUNK" \
"${PREFILL_ARGS[@]}" \
${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
> "$LOG" 2>&1
+79 -62
View File
@@ -3,33 +3,47 @@
# in the file) as frank-zealot on Dankclaw's RTX 3090 via llama.cpp (PrismML
# vendor build 10777, CUDA sm_86).
#
# 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export at THREE
# 262,144-token conversations (--ctx-size 786432 --parallel 3), q8_0 K + q8_0 V.
# The 9B bills 17,408 B/token against the 27B's 34,816 (8 full-attention layers
# instead of 16), and that halved rate is what funds three NATIVE windows where
# the 27B could only afford two cut to 200,704. Verbosity 4 ships so the
# VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in the
# journal.
# 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export, then the same
# evening TWO 262,144-token conversations (--ctx-size 524288 --parallel 2) over
# an UNQUANTIZED f16 KV cache.
#
# 2026-10-19 (Frank): THREE 200,704-token conversations over a 602,112 pool
# (--ctx-size 602112 --parallel 3), K left at the engine's own default (f16),
# V dropped to q8_0, batch 1024×6 / ubatch 1024, and the MTP drafter ARMED
# (--spec-type draft-mtp). The 9B bills 8,192 elements per token per cache (8
# full-attention layers instead of the 27B's 16), so f16 K is 16,384 B/token and
# q8_0 V is 8,704: 25,088 B/token = 14,406.00 MiB at the 602,112 pool. That
# 7,680 B/token against exact V is exactly what funds the third conversation
# AND a draft context that bills f16 KV over the whole pool. Verbosity 4 ships
# so the VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in
# the journal.
#
# Architecture (engine ledger, not the marketing page): qwen35 hybrid, 32
# layers with qwen35.full_attention_interval = 4 ⇒ 8 full-attention + 24 Gated
# DeltaNet, 4 KV heads × 256 dim, native 262,144. The KV pool scales with the 8
# full layers (13,056.00 MiB at the 786,432 pool, ledger-exact); the DeltaNet
# layers cost a fixed recurrent state per SLOT — 452.25 MiB for 3 cells, ~151
# MiB each. The runtime is a hard lock, not a preference: custom GGML type 142
# plus prism.hadamard.* metadata, which only the vendor build registers, and
# MTP beside quantized V needs build ≥ b10776 — so the prebuilt moved 10754 →
# 10777 with the payload. Vision comes from the publisher's Q8_0 projector (the
# BF16 one can crash the server on concurrent images, and this endpoint runs
# three slots), so --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is
# the documented text-only opt-out.
# full layers (14,406.00 MiB of f16 K + q8_0 V at the 602,112 pool); the
# DeltaNet layers cost a fixed recurrent state per SLOT — ~50.25 MiB a cell, so
# 150.75 MiB for three, and an armed depth-2 drafter triples that again. The
# runtime is a hard lock, not a preference: custom GGML type 142 plus
# prism.hadamard.* metadata, which only the vendor build registers — and the
# b10776 gate that moved the prebuilt 10754 → 10777 (MTP beside a quantized V)
# is LOAD-BEARING again on this shape, not moot as it was under exact KV.
# Vision comes from the publisher's Q8_0 projector (the BF16 one can crash the
# server on concurrent images, and this endpoint runs three slots), so --mmproj is
# what gives this endpoint eyes; MMPROJ_FILE="" is the documented text-only
# opt-out.
#
# Speculative decoding ships OFF, measured: the head loads and accepts well
# (0.798/0.617 per position, mean length 2.41) but the draft context bills its
# own f16 KV over the WHOLE pool — 4,096 B/position × 786,432 = 3,072 MiB plus
# buffers — and at ubatch 2048 that squeeze HALVES decode (167.2 → 109.2 t/s).
# frank-zealot.env carries the six-arm table and the arm that turns it back on
# (--ubatch-size 1024, decode 213.5 t/s) with its price.
# Speculative decoding ships ON (Frank, 2026-10-19). The head is IN the file and
# accepts well (0.798/0.617 per position, mean length 2.41, measured 2026-10-18);
# what made it lose that evening was the bill, not the drafts — the draft context
# is built against the target's WHOLE pool, so blk.32 adds 4,096 B/position
# (2,352.00 MiB at 602,112) plus 480.97 MiB of its own compute buffers on the
# card, and it triples the recurrent state per slot (452.25 MiB for three cells
# against 50.25 a cell with the drafter off). CACHE_TYPE_V=q8_0 is what pays
# that: 7,680 B/token cheaper than exact V, 4,410 MiB at this pool. The ubatch
# drop to 1024 contributes ~406 MiB here, NOT the ~1,306 the 786,432-pool census
# showed — that figure does not transfer between shapes. SPEC_TYPE=none is
# the rollback and strips the whole flag set.
#
# Sampling: the checkpoint carries the publisher's thinking recipe as GGUF
# metadata (general.sampling.* = temp 1.0 / top_p 0.95 / top_k 20 / min_p 0 /
@@ -56,43 +70,46 @@ fi
: "${ALIAS:=frank-zealot}"
: "${BIND_ADDR:=0.0.0.0}"
: "${LISTEN_PORT:=7771}"
# ctx-size is the POOL llama.cpp divides across slots: 786,432 / 3 = 262,144
# per conversation — every slot at the model's trained window. The harness
# contextWindow MUST quote the PER-SLOT number — a longer prompt is answered
# with a 400, never truncated.
: "${CTX_SIZE:=786432}"
# ctx-size is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
# conversation. The harness contextWindow MUST quote the PER-SLOT number — a
# longer prompt is answered with a 400, never truncated. The pool is priced by
# the KV bill, not by the trained window: at 25,088 B/token (f16 K + q8_0 V) the
# 602,112 pool costs 14,406.00 MiB and the armed drafter adds its own f16 KV over
# the whole pool (2,352.00 MiB). Exact V at this pool (18,816 MiB) or the old
# 786,432 pool at any dtype cannot boot inside the 24,123 MiB the driver leaves
# addressable.
: "${CTX_SIZE:=602112}"
: "${PARALLEL:=3}"
# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
# 2048 is the measured CUDA optimum (MiMo era, same arch/shape: +4.6 % prefill
# at 8.9k depth, decode flat, 616 MiB). The logical batch is VRAM- and
# throughput-neutral at a fixed ubatch. It ALSO sets the compute-buffer size,
# which is the term that decides whether an MTP draft context fits beside a
# 786,432-position pool: 1024 hands back 1,306 MiB and is the arm that runs the
# drafter at three slots (frank-zealot.env carries the table).
: "${BATCH_SIZE:=4096}"
: "${UBATCH_SIZE:=2048}"
# Quantized KV: q8_0 K + q8_0 V — the only pair that funds 3 × 262,144 on this
# card. 17 KiB/token at this geometry (8 full-attention layers × 4 KV heads ×
# 256 dim); 13,056.00 MiB at the 786,432 pool, ledger-exact. f16 would bill
# 26,112 MiB against a 24,123 MiB card and dies at cudaMalloc; q4_0 is the
# fallback rung. Explicit either way so the contract is visible in the argv,
# and --flash-attn stays required for quantized V (no true fp8 KV dtype exists
# in this build).
: "${CACHE_TYPE_K:=q8_0}"
# The logical batch is VRAM- and throughput-neutral at a fixed ubatch, so 6144
# (1024×6) is free; the ubatch is what sets the compute-buffer size, and 1024 is
# the rung every measured MTP arm here ran on (213.5 t/s at 1024 against 109.2
# at 2048) — worth ~406 MiB at THIS shape, not the ~1,306 the old pool showed.
: "${BATCH_SIZE:=6144}"
: "${UBATCH_SIZE:=1024}"
# Asymmetric KV: f16 K (the engine's own default, spelled out so the contract is
# visible in argv) + q8_0 V. 8,192 elements per token per cache (8 full-attention
# layers × 4 KV heads × 256 dim) ⇒ 16,384 B/token for K and 8,704 for V (q8_0 is
# 1.0625 B/element — llama.cpp has no true fp8 KV dtype), 25,088 B/token and
# 14,406.00 MiB at the 602,112 pool. That is the halfway door between exact KV
# (32,768 B/token, which costs the third conversation and the drafter both) and
# the fully quantized pair (17,408 B/token, which costs retrieval fidelity in the
# keys); --flash-attn stays pinned because quantized V is illegal without it.
: "${CACHE_TYPE_K:=f16}"
: "${CACHE_TYPE_V:=q8_0}"
# Speculative decoding defaults OFF, measured on this card and shape: the head
# IS in the checkpoint (qwen35.nextn_predict_layers = 1 — MTP is a property of
# the FILE, so the plain PQ2_0 sibling could not run it at any flag value), it
# loads, and it accepts well (mean accepted length 2.41) — but the draft context
# bills its own f16 KV over the WHOLE pool (4,096 B/position, 3,072 MiB here)
# and at ubatch 2048 that squeeze halves decode: 167.2 → 109.2 t/s. There is no
# --spec-draft-cache-type knob in this build to make the draft cheap, and no
# --spec-draft-model either: the draft rides the main model's weights. Card gate:
# quantized V alongside MTP needs fork build >= prism-b10776, which is why
# LLAMA_BIN moved to the 10777 prebuilt; on an older build the pair is illegal.
# SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024 is the documented MTP arm (213.5 t/s);
# SPEC_TYPE=none strips the whole set (flags move together).
: "${SPEC_TYPE:=none}"
# Speculative decoding defaults ON: the head IS in the checkpoint
# (qwen35.nextn_predict_layers = 1 — MTP is a property of the FILE, so the plain
# PQ2_0 sibling could not run it at any flag value) and the draft context rides
# the MAIN model's weights, so there is no --spec-draft-model. Its bill is the
# target's whole pool at 4,096 B/position plus ~0.45 GiB of buffers at this
# ubatch and a tripled recurrent state; CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024
# above are what pay for it. The draft cache dtype IS a further knob
# (--spec-draft-type-k / --spec-draft-type-v, both default f16 — verified in this
# build's --help) and stays at its default here. Card gate: quantized V alongside
# MTP needs fork build >= prism-b10776, which is why LLAMA_BIN is the 10777
# prebuilt; the type-142 loader lock keeps it pinned regardless.
# SPEC_TYPE=none is the rollback and strips the whole set (flags move together).
: "${SPEC_TYPE:=draft-mtp}"
: "${SPEC_DRAFT_N_MAX:=2}"
: "${SPEC_DRAFT_N_MIN:=0}"
: "${SPEC_DRAFT_P_MIN:=0}"
@@ -104,7 +121,7 @@ fi
: "${NGL:=99}"
# Log verbosity threshold (llama.cpp -lv). 3 is the engine default (info);
# 4 adds debug detail (KV layout, buffer sizes, graph capture) which is what
# the VRAM-fit forensics need — the q8_0 KV contract and the draft-KV bill are
# the VRAM-fit forensics need — the exact-KV size and the draft-KV bill are
# only provable from this log. Levels above 4 flood the journal.
: "${VERBOSITY:=4}"
# /metrics needs the flag: llama.cpp returns 501 unless the server starts with
@@ -169,11 +186,11 @@ fi
# open after boot; make sure it exists before the server needs it.
nvidia-modprobe -u -c=0 2>/dev/null || true
# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
# the q8_0 KV contract in frank-zealot.env is a real path — without this
# flag or that build option the engine silently falls back and the
# contract becomes a lie.
# Flash attention stays pinned on. Exact f16 KV runs fine with or without it,
# but the documented fallback rung (quantized V) is ILLEGAL without it, and the
# build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17) so that rung is a
# real path — dropping the flag here would leave the retreat route silently
# falling back to something the env file does not describe.
ARGS=(
--model "$MODEL_FILE"
--alias "$ALIAS"
+36
View File
@@ -138,6 +138,21 @@ for path in "${CHANGED[@]}"; do
add_test tests/docs_completeness.bats
add_test tests/deployment_scoping.bats
;;
# The voice stack: the arm A engine unit, the private bridge it owns,
# its image and its entrypoint. Like the open-notebook block above,
# this case MUST precede the dankclaw/*.container glob — that `*`
# crosses `/`, so without this case
# dankclaw/voiceservice/quadlet/frank-tts-engine.container is routed to
# the open-webui/pihole/searxng seams and its own seam never runs.
# The façade's seam rides along: the two share a directory, a network
# and the request path a consumer takes.
dankclaw/voiceservice/*)
add_test tests/frank_tts_engine.bats
add_test tests/frank_tts_unit.bats
add_test tests/deployment_flow.bats
add_test tests/deployment_scoping.bats
add_test tests/docs_completeness.bats
;;
dankclaw/*.container|dankclaw/*.network)
add_test tests/open_webui_quadlet.bats
add_test tests/pihole.bats
@@ -197,6 +212,19 @@ for path in "${CHANGED[@]}"; do
# as additions, with "no focused-test mapping".
.AGENTTRASH/*|*/.AGENTTRASH/*)
;;
# The two voice env files both reach a container through a Quadlet unit
# (EnvironmentFile=), so they select the voice seams plus the
# script-syntax seam. This case has to precede the generic
# scripts/*.service and scripts/* fallbacks below, or a change to
# scripts/frank-tts-engine.env would run nothing but bash -n. The
# warmer is NOT here: it ships inside the engine image and is routed
# with the rest of dankclaw/voiceservice/* above.
scripts/frank-tts-engine.env|scripts/frank-tts.env)
add_test tests/frank_tts_engine.bats
add_test tests/frank_tts_unit.bats
add_test tests/deployment_scoping.bats
add_test tests/scripts_syntax.bats
;;
# frank-archon is llama.cpp (the unsloth b11030-mix prebuilt) on the
# two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
# Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
@@ -229,6 +257,14 @@ for path in "${CHANGED[@]}"; do
add_test tests/scripts_syntax.bats
add_test tests/deployment_scoping.bats
;;
# The voice-pack builder is the only thing that enforces the 10-second
# source-material invariant and the artifact sizes the packs are pinned
# to, so it selects its own seam rather than landing on the generic
# scripts/* syntax fallback. This case has to precede that fallback.
scripts/build-voice-pack.sh|scripts/voice-packs.env)
add_test tests/voice_pack_unit.bats
add_test tests/scripts_syntax.bats
;;
scripts/*)
add_test tests/scripts_syntax.bats
;;
+123
View File
@@ -0,0 +1,123 @@
# voice-packs.env — tunables for the **frank-tts voice packs**: the three
# numbers per voice (source file, crop offset, crop length) plus the pinned
# engine facts that turn a crop into a pack. Read by
# `scripts/build-voice-pack.sh`, which is the only thing that consumes this
# file — the running service reads none of it, because the packs themselves are
# the interface (see scripts/frank-tts.env for what the shim resolves).
#
# WHAT IS TRACKED HERE AND WHAT IS NOT. Paths and numbers, never credentials,
# which is the same rule the other `scripts/*.env` files follow. The pack
# DIRECTORIES are host state under $VOICE_PACK_DIR and stay out of the repo; the
# crops that produce them are exactly the kind of decision worth a diff, so the
# offsets live here. If a crop changes, change it here and rebuild with
# --force — the manifest inside the pack records what was actually cut.
#
# THE OFFSETS ARE NOT ARBITRARY. Two rules pick them:
# 1. The source-material invariant: seconds 0-10 of any sample file are
# forbidden voice material. The builder refuses a crop that starts inside
# that region and asserts the same thing again from the manifest it just
# wrote, so the values below are checked, not trusted.
# 2. Speech density: the trial cut these from a silencedetect scan of the
# source (noise -35 dB, 0.6 s) and took the longest run of continuous
# speech at or past the scan window — a reference that starts mid-word or
# contains a page-turn of silence clones the artefacts, not the voice.
# serverAppConfigs-docs/Dankclaw/Voice Packs.md carries the recipe.
#
# WHY THE SHA256 LINES EXIST. A crop offset is a statement about one exact
# file: 98.739 s into a re-encode of the same audiobook is a different sentence
# and therefore a different voice. The builder hashes the source, compares it
# with the pinned digest when one is set, and writes the digest into the pack's
# manifest — so a pack can always be traced back to the bytes it was cut from.
# The three values below are the trial's digests (2026-10-18), verified against
# the files on disk.
#
# THE ENGINE BLOCK IS A PIN, NOT A DEFAULT. The speaker artifacts come from
# `qwen-codec` in the qwentts.cpp tree at $VOICE_PACK_ENGINE_COMMIT: it takes a
# 24 kHz WAV, writes the RVQ code stream next to it with the extension swapped,
# and with --talker also runs that talker's speaker encoder to write the
# x-vector .spk. The talker has to be a **Base** build — the customvoice and
# voicedesign GGUFs carry no speaker encoder and the extractor says so. The
# sizes the builder then verifies (8,192 B of .spk, 5,456-6,864 B of .rvq) are
# derived from the codec config in the GGUF, so an engine bump that changes them
# fails the build on purpose; see [[Voice Packs]] for the bump procedure.
#
# HOST PATHS, NOT CONTAINER PATHS. This runs on the host that holds the engine
# build (Dankclaw), outside any container, with no GPU and no fleet endpoint.
# The model paths below are where scripts/download-frank-tts-engine-qwentts.sh
# puts the GGUFs; until that fetch has run, the same two files sit in the trial
# tree at /home/dank/voice-audition/qwentts/models/ and the tool is only in its
# build directory. The SOURCE paths are the third thing to place: the three
# sample files live on the workstation today and have to be copied to
# $VOICE_PACK_SOURCE_DIR on Dankclaw before a build can read them.
# The packs root on the host: the directory frank-tts mounts read-only at
# /voices, one sub-directory per voice.
VOICE_PACK_DIR=/home/dank/voices
# The pack list. `--all` walks it in order; every name has to carry the four
# per-voice values below.
VOICE_PACKS="narrator1 narrator2 narrator3"
# Where the source recordings live on the engine host.
VOICE_PACK_SOURCE_DIR=/home/dank/voice-sources
# narrator1 — Randall Munroe "How To", read by Wil Wheaton. Longest speech run
# at or past the scan window: 98.739-118.636 s, 19.897 s, which packs to the
# 5,456 B floor of the .rvq window.
VOICE_PACK_NARRATOR1_SOURCE=${VOICE_PACK_SOURCE_DIR}/audiosample.mp3
VOICE_PACK_NARRATOR1_OFFSET=98.739
VOICE_PACK_NARRATOR1_DURATION=19.897
VOICE_PACK_NARRATOR1_SHA256=2299510467eb9143455d8c1e6a21601c3b451dd59c1dc99b99d4aacb54d1d850
# narrator2 — "Abyssal Archive" (Lokey). 25 s is the ceiling of the reference
# window and packs to exactly 6,864 B of .rvq.
VOICE_PACK_NARRATOR2_SOURCE=${VOICE_PACK_SOURCE_DIR}/audiosample2.m4b
VOICE_PACK_NARRATOR2_OFFSET=191.13
VOICE_PACK_NARRATOR2_DURATION=25
VOICE_PACK_NARRATOR2_SHA256=8e1e33317baf67b2cca67e4ce1b095ae477a18285c7373ec18ad013f2cc320e0
# narrator3 — "Children of Time" (Adrian Tchaikovsky, author-read).
VOICE_PACK_NARRATOR3_SOURCE=${VOICE_PACK_SOURCE_DIR}/audiosample3.m4b
VOICE_PACK_NARRATOR3_OFFSET=489.232
VOICE_PACK_NARRATOR3_DURATION=25
VOICE_PACK_NARRATOR3_SHA256=6875d2c1db421ab96ccbb68855fa9efd4e054297e94dd1b4345eb6a9869da945
# The pinned extractor and the two GGUFs it needs. $VOICE_PACK_TOOL is a build
# output, not a download: rebuild it at $VOICE_PACK_ENGINE_COMMIT after a bump
# rather than pointing this line at a newer tree.
VOICE_PACK_TOOL=/home/dank/voice-audition/qwentts/qwentts.cpp/build/qwen-codec
VOICE_PACK_CODEC_MODEL=/home/dank/models/Qwen3-TTS-GGUF/qwen-tokenizer-12hz-Q8_0.gguf
VOICE_PACK_TALKER_MODEL=/home/dank/models/Qwen3-TTS-GGUF/qwen-talker-1.7b-base-Q4_K_M.gguf
VOICE_PACK_ENGINE_REPO=https://github.com/ServeurpersoCom/qwentts.cpp
VOICE_PACK_ENGINE_COMMIT=5916bf7c79dbb4e66fc059cdc507a79a23a28bb4
# CPU only, and never in the way of the text endpoints on this box: ggml thread
# count and `nice -n`. The extractor picks the best backend it is given, so the
# device is stated rather than guessed.
VOICE_PACK_DEVICE=CPU
VOICE_PACK_THREADS=8
VOICE_PACK_NICE=19
# The reference window and the shape the packs are cut to. 24 kHz mono PCM is
# the codec's own input rate, so no resampling happens inside the engine.
VOICE_PACK_SAMPLE_RATE=24000
VOICE_PACK_MIN_SECONDS=18
VOICE_PACK_MAX_SECONDS=25
# Artifact arithmetic, all four values read out of the pinned tree: the codec
# runs at 12.5 Hz (1,920 samples per frame at 24 kHz), packs 16 codebooks at 11
# bits = 22 bytes per frame, and the Base talker's speaker encoder emits
# enc_dim 2048 f32 = 8,192 bytes. The .rvq window is 22 x 248 = 5,456 B up to
# 22 x 312 = 6,864 B, i.e. crops of 19.84-25 s: an 18 s crop is a legal
# reference.wav but too little code for the engine, and the builder says so
# before it runs anything.
VOICE_PACK_FRAME_RATE=12.5
VOICE_PACK_RVQ_BYTES_PER_FRAME=22
VOICE_PACK_SPK_BYTES=8192
VOICE_PACK_RVQ_MIN_BYTES=5456
VOICE_PACK_RVQ_MAX_BYTES=6864
# Where a --force rebuild puts the pack it replaced. Nothing is ever deleted;
# the retired directory stays readable beside its replacement until a human
# clears it.
VOICE_PACK_TRASH_DIRNAME=.AGENTTRASH
@@ -1,7 +1,7 @@
---
type: reference
title: Agent Docs and Edits
description: Two self-inflicted damage classes — structurally broken markdown after parallel agent edits, and botched hashline PUT/CUT repairs — with the audit checks and recovery order for each.
description: "Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each."
tags: [agents, docs, markdown, hashline, edit-recovery]
timestamp: 2026-10-18T00:00:00Z
---
@@ -21,7 +21,7 @@ Damage catalogue:
no longer frontmatter (title/tags/description become body text).
2. A rewrite **appended** instead of merged → duplicate headings and a stray
second `---` mid-file.
3. Literal model artifacts in the body: `<|channel>}<tool_call|>...`,
3. Literal model artifacts in the body: `<|channel>}<tool_call|>...`
`+- `, `- - `.
4. Wrapped bullets re-emitted with a `- ` prefix on every physical line.
5. Line-range replacement that dropped a sentence tail → dangling fragment,
@@ -46,7 +46,7 @@ Repair order (matters):
2. If it merely duplicated: **salvage unique content first** — cut the
one-of-a-kind section, paste it into the copy you keep, THEN delete the
duplicate ranges. Never delete first.
3. Re-add missing sentence tails (insert before the orphaned fragment),
3. Re-add missing sentence tails (insert before the orphaned fragment)
keeping list continuation lines indented.
4. Re-run all checks above.
@@ -57,7 +57,7 @@ safe, but its snapshot tag changes on every edit — re-read before each
subsequent edit. `.AGENTTRASH/` is gitignored (`.gitignore:25`), so NEW trash
stays local — but do not read that as "the repo has none of it": on 2026-10-18
`git ls-files | grep -c AGENTTRASH` returns **37** tracked paths
(`.AGENTTRASH/`, `scripts/.AGENTTRASH/`, `dankserver/.AGENTTRASH/`,
(`.AGENTTRASH/`, `scripts/.AGENTTRASH/`, `dankserver/.AGENTTRASH/`
`tests/.AGENTTRASH/`), because gitignore never untracks what was committed
before the rule. Trash is a temporary hold, not an archive.
@@ -1,7 +1,7 @@
---
type: reference
title: Command Center Dotfiles
description: The ~/.omp repo is itself the deployment — stow-equivalent packages, full-replica rsync transport, the pre-overwrite audit for non-blank hosts, deploy.sh mechanics, and the fish/sshpass transport gotchas.
description: "How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas."
tags: [dotfiles, omp, deploy, rsync, blankpad]
timestamp: 2026-10-05T00:00:00-07:00
---
+36 -38
View File
@@ -1,7 +1,7 @@
---
type: reference
title: OMP Harness
description: The omp agent harness as configured on this fleet — compaction chain arithmetic, per-model sampler routing via compat.extraBody, and the static-preamble/skill-library audits that keep session context lean.
description: "The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits."
tags: [omp, harness, compaction, models-yml, context, samplers]
timestamp: 2026-10-18T00:00:00Z
---
@@ -14,19 +14,19 @@ registry; single source of truth — there is no copy on the servers) and
this file is repo-managed; edits are harness-side only.
**Model ids are the StarCraft unit names (official 2026-10-08, on the wire
since 2026-10-09):** `frank-archon` (dankserver :7770, radiance),
`frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw
:7771, the 3090 compaction hub) — formerly `frank-archon`, `frank-dragoon`,
since 2026-10-09):** `frank-archon` (Dankserver), `frank-dragoon` and
`frank-zealot` (Dankclaw) — engine, port and card behind each id live on
[[frank-archon]], [[frank-dragoon]] and [[frank-zealot]] — formerly `frank-archon`, `frank-dragoon`
`frank-zealot`. The SERVICES keep the old names (units, scripts, env
files, runbook pages); pages written in the old era name the service,
the harness names the model. The wire now matches: each endpoint's
`/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI
picker/grants, the homepage docs and the raw API all use one id per model
(see [[OpenWebUI]] for the grant re-key and the [[frank-archon-radiance-swift15-w4dn8]]
note on radiance echoing the requested name).
(see [[OpenWebUI]] for the grant re-key and the [[frank-archon]]
note on the endpoint echoing the requested name).
Verified 2026-10-18: `/v1/models` on 192.168.0.235:7770, 192.168.0.143:7770 and
192.168.0.143:7771 each answer with the unit name (`frank-archon`,
192.168.0.143:7771 each answer with the unit name (`frank-archon`
`frank-dragoon`, `frank-zealot`).
## Compaction chain arithmetic (verify before ANY window change)
@@ -40,20 +40,21 @@ breaks compaction.
Where the contract lives:
- `~/.omp/agent/config.yml` → `compaction.remoteEndpoint` (target URL:
`http://192.168.0.143:7771/v1/chat/completions`, `config.yml:76`),
`methodOrder` (`shake → snapcompact → soft → handoff → remote`,
`http://192.168.0.143:7771/v1/chat/completions`, `config.yml:76`)
`methodOrder` (`shake → snapcompact → soft → handoff → remote`
`config.yml:77-82`), `thresholdPercent` (**95**, `config.yml:84`), idle
settings (`idleEnabled`, `idleThresholdTokens: 100000`)
- `~/.omp/agent/models.yml` → per-model `contextWindow`, `maxTokens`,
- `~/.omp/agent/models.yml` → per-model `contextWindow`, `maxTokens`
`compactionModel` references
- Implementation: omp plugin
`~/.omp/plugins/node_modules/@earendil-works/pi-coding-agent/dist/core/compaction/compaction.js`
The arithmetic (the coupled contract every endpoint's maxTokens comment
encodes). Recomputed from the live files 2026-10-18 — contextWindows
200,000 (archon) / 155,648 (dragoon) / 200,704 (zealot) at
`models.yml:48,131,206`; maxTokens 16,384 / 45,056 / 16,384 at
`models.yml:56,143,212`:
encodes). The authoritative `contextWindow`/`maxTokens` pair per model is on the
endpoint page ([[frank-archon]], [[frank-dragoon]], [[frank-zealot]]); what
follows is a **worked example** recomputed from the live files on 2026-10-18 —
a snapshot, not fleet state — with windows 200,000 / 155,648 / 200,704 and
maxTokens 16,384 / 45,056 / 16,384 (archon / dragoon / zealot):
1. **Source trigger** = `thresholdPercent × source contextWindow`; at the live
95 % that is 190,000 (archon), 147,866 (dragoon), 190,669 (zealot). The
@@ -103,16 +104,16 @@ input. Boot-shape OOM laddering is a different problem
## Per-model sampler config (`compat.extraBody`)
`models.yml` has **no per-model sampler keys**. The global knobs are
`config.yml`'s `temperature`, `topP`, `topK`, `minP`, `presencePenalty`,
`config.yml`'s `temperature`, `topP`, `topK`, `minP`, `presencePenalty`
`repetitionPenalty`, and a knob reaches the wire only when it is set — the
bundle guards each one with `!== void 0` (e.g. `minP!==void 0)e.min_p=t.minP`),
bundle guards each one with `!== void 0` (e.g. `minP!==void 0)e.min_p=t.minP`)
and this workstation's `config.yml` sets **none** of them. The 2026-10-05 note
that the unset default is `-1`/"not sent" is retired: there is no `-1`
sentinel, an unset key is simply absent. The per-model hook is
`compat.extraBody`:
**Why it works:** `applyOpenAIExtraBody` runs **last** in the
openai-completions builder — after `applyChatCompletionsCompatPolicy` — via
openai-completions builder — after `applyChatCompletionsCompatPolicy` —
`Object.assign(params, extraBody)`. So extraBody overrides global settings
and compat-policy output.
@@ -141,28 +142,25 @@ Use **wire names**, not OMP camelCase. Two traps:
2. **Never put `chat_template_kwargs` in `extraBody`** where a compat policy
also sets it — `Object.assign` is shallow and replaces the whole object,
silently dropping the `reasoning_effort` kwarg the compat policy set.
Top-level `preserve_thinking` is honoured by halogen. Live state 2026-10-18:
frank-archon's radiance entry carries `chat_template_kwargs:
{enable_thinking, preserve_thinking}` in `extraBody` (`models.yml:89-91`)
and nothing else sets that object at that endpoint, so the shallow-merge
hazard is latent rather than firing; the FreeToken-era "verify-and-prune at
the next swap" note is retired with that swap (2026-10-05,
`scripts/serve-frank-archon.sh:9-10`).
Top-level `preserve_thinking` is honoured by frank-dragoon
([[frank-dragoon]]). Live state 2026-10-18: frank-archon's `extraBody`
carries `chat_template_kwargs: {enable_thinking, preserve_thinking}`
(`models.yml:89-91`) and nothing else sets that object at that endpoint, so
the shallow-merge hazard is latent rather than firing.
**Gotcha (retired 2026-10-17): silent greedy decode.** Earlier flash-server
**Gotcha (retired 2026-10-17): silent greedy decode.** Earlier frank-dragoon
builds carried **no server-side sampler defaults**, so an absent
`temperature` decoded greedy while the endpoint looked configured. That era is
over: `HALOGEN_TEMPERATURE/TOP_P/TOP_K` now set the card's thinking recipe
(1.0 / 0.95 / 20) as engine defaults (`scripts/frank-dragoon.env:98-103`,
exported at `scripts/serve-frank-dragoon.sh:186-188`), so an absent temperature
is no longer greedy. frank-dragoon's `extraBody` still carries the same trio —
consistent, not load-bearing. Treat any older "do not drop it or you get
greedy" wording as history.
over — the endpoint now ships its own defaults ([[frank-dragoon]]), so an
absent temperature is no longer greedy. frank-dragoon's `extraBody` still
carries the same trio — consistent, not load-bearing. Treat any older "do not
drop it or you get greedy" wording as history.
**Reasoning-effort folding.** Qwen3.8's template accepts only
`{xhigh (default), medium, low}` for `reasoning_effort` and 400s on anything
else ("Unexpected reasoning effort high"). Every endpoint's
`reasoningEffortMap` folds all harness levels onto that set; a role whose
**Reasoning-effort folding.** The set `reasoning_effort` accepts is a property
of the endpoint, not of the harness: [[frank-dragoon]] and [[frank-zealot]]
raise on anything outside `xhigh|medium|low`, while [[frank-archon]] does not
gate the set. Every endpoint's `reasoningEffortMap` folds all harness levels
onto whichever set that endpoint ships; a role whose
level is unmapped fails on its FIRST request, and the failure names Jinja,
not the model.
@@ -193,7 +191,7 @@ assistant turn carrying a long `reasoning_content`, then compare
`usage.prompt_tokens` with the flag true vs false. A large delta means the
field changed template rendering; zero delta means it was ignored.
Cleanup: trash the probe profile and echo server (`profiles/.AGENTTRASH`),
Cleanup: trash the probe profile and echo server (`profiles/.AGENTTRASH`)
never `rm`.
## Static-preamble audit ("the context feels bloated")
@@ -207,7 +205,7 @@ What loads statically every session (measure each, don't guess):
```bash
python3 - <<'EOF'
import glob, re, os
for root in [os.path.expanduser("~/.omp/agent/managed-skills"),
for root in [os.path.expanduser("~/.omp/agent/managed-skills")
os.path.expanduser("~/.omp/agent/skills"), ".omp/skills"]:
n=tot=0
for sk in glob.glob(root+"/*/SKILL.md"):
@@ -307,4 +305,4 @@ standalone; fold go/no-go arithmetic skills into their probe skill
- [[Agent Docs and Edits]] — editing vault docs and hashline targets safely
- [[Command Center Dotfiles]] — the `~/.omp` repo deploy this config rides
- [[Llama Cpp Serving Sizing]] — the engine-side sizing the chain must fit
- [[LLM Bench Windows and Measurement]] — measuring the endpoints this config routes
- measuring the endpoints this config routes
+4 -15
View File
@@ -1,23 +1,12 @@
---
type: index
description: Pages about the agent harness itself — omp config, vault editing discipline, and the dotfiles repo that deploys the agent's own home.
tags: [agent, index]
timestamp: 2026-10-05T00:00:00-07:00
---
# Agent
Pages about the agent's own tooling and context — how omp is configured, how
docs get edited safely, and how the agent's home is deployed.
- [[OMP Harness]] — compaction chain arithmetic, per-model samplers via
`compat.extraBody`, preamble and skill-library audits
- [[Agent Docs and Edits]] — doc-structure audits after parallel edits;
hashline edit recovery
- [[Command Center Dotfiles]] — the `~/.omp` repo-as-deployment, rsync
transport, override packages
- [[OMP Harness]] — The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits.
- [[Agent Docs and Edits]] — Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each.
- [[Command Center Dotfiles]] — How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas.
## Related
- [[Testing Policy]] — the seams these pages' checks ride
- [LLM](../LLM/index.md) — the fleet these agents operate
- the fleet these agents operate
@@ -1,8 +1,7 @@
---
type: reference
title: Container Network Namespaces
description: Which local services are addressable from which Podman network namespace on
Dankclaw, and why bridge-network containers cannot dial host-netns loopback listeners.
description: "How Podman builds container networks here: the network inventory, the netns sharing mechanism, and the isolation limits."
tags:
- networking
- podman
@@ -48,7 +47,7 @@ network namespace**:
`Network=` values are read from the live Quadlet sources in
`/home/dank/.config/containers/systemd/*.container`; runtime network names on the
host at the same date were `podman`, `dankclaw_default`, `systemd-gitea`,
host at the same date were `podman`, `dankclaw_default`, `systemd-gitea`
`systemd-searxng`, `systemd-open-webui`.
Two consequences that are easy to miss:
@@ -84,7 +83,7 @@ podman exec <name> cat /proc/net/route # default route → gateway IP
podman exec <name> sh -c 'timeout 2 bash -c "exec 3<>/dev/tcp/10.89.1.1/8080"; echo rc=$?'
```
Verified outcomes of that recipe on 2026-09-02: from inside `hermes-dashboard`,
Verified outcomes of that recipe on 2026-09-02: from inside `hermes-dashboard`
`10.89.1.1:3000` (Gitea, published), `:8081` (SearXNG via Gluetun's netns,
published) and `:9119` (its own published port) connect; `:8080` and `:8787`
are **refused instantly** even though both listeners are bound to `0.0.0.0` in
@@ -1,7 +1,7 @@
---
type: reference
title: Deploy State Triage
description: Reading and reconciling the deploy-state markers — stale marker vs live container, pending-service lockout, interrupted/raced deploys — without restarting healthy heavy endpoints.
description: "Reading deploy-state markers against live containers and deciding, safely, whether to redeploy."
tags: [deployment, deploy-state, systemd, podman, triage]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -22,8 +22,10 @@ marker. So `marker != HEAD` means "not dispatched via deploy.sh", NOT
"behavior pending".
**Why it matters:** a plain `deploy.sh` run re-selects and restarts a service
whose marker lags. On llama.cpp/R9V endpoints that is a ~16–19 min reload of
a 100+ GiB payload — and frank-archon (dankserver:7770) is also the
whose marker lags. On the big-payload endpoints that is a full model reload
measured in tens of minutes — each endpoint's boot budget is on its own page
([[frank-archon]], [[frank-dragoon]], [[frank-zealot]]) — and frank-archon
(dankserver:7770) is also the
harness's own model backend, so a blind deploy is a self-inflicted outage for
possibly zero behavioral change.
@@ -34,7 +36,7 @@ possibly zero behavioral change.
2. Probe the target (ssh as `dank`, passwordless sudo present):
- `cat ~/.local/state/serverAppConfigs/deploy-state/<svc>`
- `git -C ~/serverAppConfigs rev-parse HEAD`
- `systemctl is-active/is-enabled <svc>`;
- `systemctl is-active/is-enabled <svc>`
`curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:<port>/health`
- the on-host env file already carries the new keys (it landed via the
ff-pull).
@@ -46,9 +48,8 @@ possibly zero behavioral change.
- `podman inspect <ctr> --format '{{.Args}}' | tr ' ' '\n' | grep -i <flag>`
- engine env: `podman exec <ctr> sh -c 'for f in /proc/[0-9]*/environ; do
tr "\0" "\n" <$f 2>/dev/null; done | grep -i <VAR>'`
- confirm the launcher maps each tunable to its flag
(`grep scripts/launch.sh`), e.g. `R9V_ENABLE_PREFIX_CACHING=1 →
--enable-prefix-caching`.
- confirm the launcher maps each tunable to its flag
(`grep scripts/launch.sh`).
**Decision:** live argv+env == target config AND `/health` 200 ⇒ behavior is
live — **do NOT restart**; reconcile the marker (below). Only if the live
@@ -118,8 +119,8 @@ each, kill stale ones by explicit PID.
A killed deploy leaves behind:
- the unit **stopped/failed** (`Restart=always` does NOT fire after an
explicit stop — start it yourself);
- the state marker **pending** (pre-deploy or in-flight revision);
explicit stop — start it yourself)
- the state marker **pending** (pre-deploy or in-flight revision)
- the e2e gate un-run (the deployer runs it last).
Reconcile the marker ONLY after the service itself is verified healthy — a
@@ -141,11 +142,11 @@ Boot failures must be read from the journal, not `podman logs`: serve scripts
numbers — which looks exactly like your change having no effect. Key on
container identity (`podman ps -a --format '{{.CreatedAt}}'`) and read the
cause from `journalctl --since "N min ago" | grep -a "<container-name>\["`.
See [[vLLM Boot Forensics and Config Interrogation]] for the full capture
See for the full capture
procedure.
## Related
- [[Deployment Flow]] — the deploy pipeline itself
- [[Quadlet Lifecycle]] — unit ownership and drift diagnosis
- [[vLLM Boot Forensics and Config Interrogation]] — journal capture
- journal capture
@@ -1,5 +1,5 @@
---
type: guide
type: howto
title: Deployment Flow
description: Service-scoped Git deployment for the two homelab hosts.
tags:
@@ -46,7 +46,7 @@ graph LR
./scripts/deploy.sh dankserver
./scripts/deploy.sh frankbook-pro --service qbittorrent # podman-machine target
./scripts/deploy.sh dankclaw --service searxng
./scripts/deploy.sh dankserver --service frank-medium
./scripts/deploy.sh dankserver --service frank-archon
./scripts/deploy.sh dankclaw --dry-run
```
@@ -62,13 +62,13 @@ service on the other host.
|---|---|---|
| Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
| `frank-dragoon` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
| `frank-zealot` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
| `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. |
| `frank-dragoon` | systemd + `systemctl` (root unit) | Relinks and restarts only the Dankclaw iGPU inference service; waits up to 20 minutes for `/health`. Shape, gates and boot budget: [[frank-dragoon]]. |
| `frank-zealot` | systemd + `systemctl` (root unit) | Relinks and restarts only the Dankclaw 3090 inference service; waits up to 20 minutes for `/health`. Shape, gates and boot budget: [[frank-zealot]]. |
| `frank-archon` | systemd + `systemctl` (root unit) | `deploy_llm_archon` plans from repo paths and gates on the port; it does NOT see host-side unit drift (see [[frank-archon]] → Deploy / verify). |
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
The shared chat template maps to the inference service on the selected target.
The `frank-npu` endpoint is retired (2026-09-11) and has no deploy handler;
The `frank-npu` endpoint is retired () and has no deploy handler
an unrelated large-model deployment never touches it.
## Caddy safety
@@ -125,6 +125,5 @@ routing. See [[Testing Policy]].
## See Also
- [[Testing Policy]]
- [LLM deployment](../LLM/deployment.md)
- [[Caddy]]
- [[SearXNG]]
@@ -1,6 +1,6 @@
---
type: reference
description: "`tests/e2e/` drives the **live fleet** through a real browser: fresh contexts"
description: "The Playwright suite that drives the live fleet through a real browser, its project set, and the FrankBook conditional."
tags:
- architecture
- e2e
@@ -49,7 +49,7 @@ user sees it; the `bats` seams assert config shape, not behavior.
*arr/qBittorrent links terminate on FrankBook-Pro, which sleeps when idle, and a
sleeping Mac mini must never fail an unrelated deploy. The project joins only
when `E2E_INCLUDE_FRANKBOOK=1` (set by `scripts/deploy.sh` when a frankbook
trigger path changed: `tests/e2e/specs/19-frankbook.spec.ts`, `frankbook-pro/**`,
trigger path changed: `tests/e2e/specs/19-frankbook.spec.ts`, `frankbook-pro/**`
`shared/homepage/services.json`, or the `dankclaw/Caddyfile` serving the arr
vhosts) or when an explicit `--project` argument names it. An `11-admin` edit
that touches the arr links must therefore be paired with a `19-frankbook` run
@@ -15,7 +15,7 @@ One Caddy fronts both hosts. It runs as a container on Dankclaw (192.168.0.143)
As of 2026-08-28 the front door is a two-tier chain: Caddy terminates TLS,
Anubis verifies the client (DENY known AI scrapers, CHALLENGE everyone
else), and accepted traffic returns to Caddy's internal listener on `:8016`,
else), and accepted traffic returns to Caddy's internal listener on `:8016`
which proxies every request whole to the Authentik server with the Host
header preserved; Authentik's embedded outpost routes on that Host against
each proxy provider's `external_host`, serves the login flow in place for
@@ -67,12 +67,12 @@ graph TB
```
Guarded (Anubis `:8923` → Caddy `:8016` → Authentik outpost → Caddy `:8017`):
homepage (`www.`), search, **grafana** (`dankclaw/Caddyfile:128-129`),
homepage (`www.`), search, **grafana** (`dankclaw/Caddyfile:128-129`)
prometheus (`:135-137`), loki (`:141-143`), seerr (`:148-150`), marreta
(`:155-160`), romm (`:167-169`), files (`:172-174`). Exempt / own-auth: `auth.`
(the IdP login itself), `ai.` (OpenWebUI, its own OIDC), gitea (git smart-HTTP
clients cannot solve JS challenges — `/info/refs` and `git-*-pack` pass
unchallenged), pihole (Caddy `basicauth`, `:167`), plex (plain HTTP),
unchallenged), pihole (Caddy `basicauth`, `:167`), plex (plain HTTP)
books/jellyfrank (own logins), `hermes.`/`hermesadmin.`, and the *arr vhosts
that terminate on FrankBook-Pro (`:254-281`, "exempt by construction: own
auth"). Grafana is **not** exempt even though it signs in through authentik as
@@ -104,13 +104,13 @@ Caddy holds one Let's Encrypt certificate per vhost, issued and renewed through
`Network=searxng.network` (runtime network `systemd-searxng`), while
`searxng.container`, `marreta.container` and `marreta-browser.container` join
the tunnel the modern way — `Network=gluetun.container`
(`dankclaw/searxng/quadlet/searxng.container:18`,
`dankclaw/marreta/quadlet/marreta.container:34`,
(`dankclaw/searxng/quadlet/searxng.container:18`
`dankclaw/marreta/quadlet/marreta.container:34`
`dankclaw/marreta/quadlet/marreta-browser.container:25`), the Quadlet
equivalent of the retired compose `network_mode: service:gluetun`. **Five**
containers share the VPN exit, not three, and Marreta's outbound fetches leave
through the same WireGuard tunnel as SearXNG's. Pi-hole is a different kind of
exception (`Network=slirp4netns`, `dankclaw/pihole/quadlet/pihole.container`);
exception (`Network=slirp4netns`, `dankclaw/pihole/quadlet/pihole.container`)
everything else on Dankclaw uses the normal LAN route. This matches KB Pattern
2 in [[Gluetun]].
See [[Container Network Namespaces]] for which addresses are reachable from that
@@ -1,7 +1,7 @@
---
type: reference
title: Quadlet Lifecycle
description: The full Quadlet discipline on this fleet — ownership auditing, compose-to-Quadlet cutover, drift/crash-loop diagnosis, the podman 5.x v1 generator quirks, log retrieval, and compose bring-up with ground truth.
description: "The Quadlet discipline: ownership audits, compose cutover, drift and crash-loop triage, generator quirks, log retrieval."
tags: [podman, quadlet, systemd, boot-persistence, compose, drift]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -36,14 +36,14 @@ done
## 2. Cutover: compose service → Quadlet unit
Checklist (precedents to copy:
`dankclaw/{caddy,gitea,searxng,open-webui,seerr}/quadlet/`,
`dankclaw/{caddy,gitea,searxng,open-webui,seerr}/quadlet/`
`dankclaw/observability/quadlet/`):
1. Unit file at `<host>/<svc>/quadlet/<name>.container`, mirroring compose
line for line: `Image=` (`localhost/` prefix for locally built images +
`Pull=never`), `ContainerName=` (Caddy and scrape configs address
containers by name), `Network=`, `User=0:0` when the image's own user
cannot write a rootless-owned bind mount, `Volume=` (`:ro` preserved),
cannot write a rootless-owned bind mount, `Volume=` (`:ro` preserved)
`Exec=` for compose `command:`, `PublishPort=`, health keys,
`[Service] Restart=always` + `RestartSec=10`, `[Install]
WantedBy=default.target`. **Never `systemctl --user enable` the generated
@@ -56,7 +56,7 @@ Checklist (precedents to copy:
`systemctl --user restart <name>.service` → `wait_for_health`; retire any
legacy hand-written unit into `~/.AGENTTRASH/` (a leftover
`~/.config/systemd/user/<svc>.service` that shells out to
`podman compose up` collides BY NAME with the generated unit);
`podman compose up` collides BY NAME with the generated unit)
`podman rm -f <container>` so the unit's container can take the name.
4. Planner: map `<host>/<svc>/**` → service in `scripts/deploy-plan.py`
AND add a `service_is_allowed` entry; keep `SERVICE_ORDER` dependencies
@@ -123,7 +123,7 @@ the restart policy faithfully resurrects it forever.
Quickest unblock: `podman compose up -d --force-recreate <svc>`. Durable fix
= the §2 cutover. **Verify the capability, not the status:** `podman exec
<name> python3 -c "import torch; print(torch.cuda.is_available())"`;
<name> python3 -c "import torch; print(torch.cuda.is_available())"`
`ss -ltn | grep <port>` for the actual bind addresses. ComfyUI incident:
2026-09-16, stale device-less container, 9,146 restarts. History repeats:
2026-08-26 Caddy (no unit), 2026-09-11 five observability services.
@@ -143,7 +143,7 @@ Quickest unblock: `podman compose up -d --force-recreate <svc>`. Durable fix
unhealthy but logs look fine, compare the baked
`{{.Config.Healthcheck.Test}}` array against the intended command before
believing the status.
- **Probe a new key before shipping:** drop a throwaway `probe.container`,
- **Probe a new key before shipping:** drop a throwaway `probe.container`
`daemon-reload`, `journalctl --user --since "1 min ago" -o cat | grep -i
"unsupported key"`, `systemctl --user cat probe.service` (shows the
generated ExecStart verbatim), then remove + reload.
@@ -1,7 +1,7 @@
---
type: reference
title: Stale Signal Audit
description: The audit battery for surfaces that rot without failing — unreachable probes, vacuous assertions, invalid checkers, gates that never run, and prose invalidated by a change. Four passes found real defects every time.
description: "The audit battery for surfaces that rot without failing: dead probes, vacuous assertions, gates that never run."
tags: [testing, observability, audit, signals, gates]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -34,8 +34,9 @@ port; a "probe" job scraping `/` (HTML → `unsupported Content-Type`); a
`/metrics` that is disabled (Gitea 404).
- A scrape cannot express "is this HTTP service answering?" — the right probe
is the **blackbox exporter** in probe mode (which also gives guarded vhosts
a real uptime signal, since Prometheus cannot solve an Anubis challenge).
is the **blackbox exporter** in probe mode
(`/probe?target=…&module=http_2xx`), which also gives guarded vhosts
a real uptime signal, since Prometheus cannot solve an Anubis challenge.
- **Fix in the direction of honesty, never of silence:** unreachable target →
mark the homepage card `wip: true` with the reason in `description` (the
checker skips `wip`; `wip` is the existing convention for "real service,
@@ -43,8 +44,8 @@ port; a "probe" job scraping `/` (HTML → `unsupported Content-Type`); a
database port on the LAN to make a badge green, and never delete a scrape
job — an absent scrape and a healthy one are indistinguishable.
- Alert rules must not depend on always-red signals — scope them to services
that can actually be healthy (`LlmEndpointDown` is scoped to `frank-*` for
exactly that reason).
that can actually be healthy; the `frank-*` scope `LlmEndpointDown` carries
is recorded on [[Prometheus]].
- A target's failure STRING can prove a fix: a `/metrics` scrape flipping from
`connection refused` to `unsupported Content-Type "text/html"` means the
service came up and only the scrape is wrong. Read the error, not just
@@ -54,4 +54,3 @@ Tests are written first. A failing test means the seam is not satisfied; the con
## See Also
- [[Deployment Flow]]
- [[index]]
@@ -1,6 +1,6 @@
---
type: howto
description: "Qwen-Image 2.1 workflows on Dankclaw: two proven txt2img workflows (art + legible text), the API recipe, and which models/LoRAs to add next."
description: "Proven Qwen-Image text-to-image workflows on Dankclaw: the recipes, the API call shape, and which models to add next."
tags:
- dankclaw
- comfyui
@@ -56,8 +56,8 @@ sampling), `sampler_name: euler`, `scheduler: simple`, `denoise: 1.0`, seed
fixed in the file (`1930` in the art workflow, `2026` in the text one) —
read from `dankclaw/comfyui/workflows/*.json` node `9`.
Node `class_type`s in both files: `CLIPTextEncode` ×2, `UNETLoader`,
`CLIPLoader`, `VAELoader`, `EmptySD3LatentImage`, `KSampler`, `VAEDecode`,
Node `class_type`s in both files: `CLIPTextEncode` ×2, `UNETLoader`
`CLIPLoader`, `VAELoader`, `EmptySD3LatentImage`, `KSampler`, `VAEDecode`
`SaveImage` (ids 3–11).
| Workflow | File | Shape | Steps | Made for |
@@ -113,9 +113,9 @@ only `models`, `input`, `output`, `custom_nodes` and the healthcheck file
Quantized preferred, sized for the 8060S iGPU (GTT-backed — `/system_stats`
reports 122.4 GB of device VRAM on `cuda:0`, `curl http://127.0.0.1:8188/system_stats`
2026-10-18; the RTX 3090 is frank-zealot's). All repos verified 2026-10-05 and
all four still resolve today (`huggingface.co/lightx2v/Qwen-Image-Lightning`,
`huggingface.co/unsloth/Qwen-Image-Edit-2511-GGUF`,
`huggingface.co/Comfy-Org/Qwen-Image-DiffSynth-ControlNets`,
all four still resolve today (`huggingface.co/lightx2v/Qwen-Image-Lightning`
`huggingface.co/unsloth/Qwen-Image-Edit-2511-GGUF`
`huggingface.co/Comfy-Org/Qwen-Image-DiffSynth-ControlNets`
`huggingface.co/QuantStack/Wan2.2-TI2V-5B-GGUF`, 2026-10-18). None are on disk
yet: `models/` holds no lora/, controlnet/ or Wan/Edit file (`ls -R` 2026-10-18).
@@ -126,7 +126,7 @@ yet: `models/` holds no lora/, controlnet/ or Wan/Edit file (`ls -R` 2026-10-18)
| 3 | **DiffSynth ControlNets** (structure control) | `Comfy-Org/Qwen-Image-DiffSynth-ControlNets` | `split_files/model_patches/` canny + depth | ~2–3 GB each | Pose/edge/depth-guided layouts — pairs with the legible-text workflow for exact poster composition. The `QwenImageDiffsynthControlnet` node is already in the running ComfyUI (`GET http://127.0.0.1:8188/object_info/QwenImageDiffsynthControlnet` → 200, 2026-10-18) |
| 4 | **Wan 2.2 TI2V-5B GGUF** (first video model) | `QuantStack/Wan2.2-TI2V-5B-GGUF` | Q4/Q5 GGUF (~6–8 GB) | ~6–8 GB | T2V+I2V in one dense 5B model; the gentlest on-ramp into video generation, runs in ComfyUI core (`wan` is in this build's `CLIPLoader` type enum — `GET http://127.0.0.1:8188/object_info/CLIPLoader` 2026-10-18) |
Placement: #1 → `models/lora/` (create it), #2 → `models/diffusion_models/`,
Placement: #1 → `models/lora/` (create it), #2 → `models/diffusion_models/`
#3 → `models/controlnet/` (create it), #4 → `models/diffusion_models/`.
Fetch with `scripts/download-frank-*.sh` discipline (`-f` + byte-count +
sha256 checks) or a plain `hf download` with the token in the environment —
+3 -3
View File
@@ -43,7 +43,7 @@ Dankclaw is an AMD Strix Halo APU (gfx1151). Standard PyTorch ROCm wheels do not
## GPU device access (rootless), and why the unit exists
The unit passes `/dev/kfd` and `/dev/dri` (`AddDevice=`,
The unit passes `/dev/kfd` and `/dev/dri` (`AddDevice=`
`dankclaw/comfyui/quadlet/comfyui.container:45-46`) and uses
`GroupAdd=keep-groups`. A literal `render` entry cannot work: rootless Podman
cannot map the group name into the user namespace. The companion udev rule
@@ -115,7 +115,7 @@ generated units.
The web UI responds on `/` (200 `text/html`, `curl http://127.0.0.1:8188/`
2026-10-18). ComfyUI also exposes `/system_stats`. On this APU, `/system_stats`
reports `device=cuda:0 Radeon 8060S Graphics` (type `cuda`, the ROCm device). The
health probe is the repo file `dankclaw/comfyui/comfyui-healthcheck.py`,
health probe is the repo file `dankclaw/comfyui/comfyui-healthcheck.py`
bind-mounted to `/usr/local/bin/comfyui-healthcheck.py` and run as `HealthCmd`
(`dankclaw/comfyui/quadlet/comfyui.container:52,57`); it GETs
`http://127.0.0.1:8188/` inside the container, and the container reports
@@ -135,5 +135,5 @@ bind-mounted to `/usr/local/bin/comfyui-healthcheck.py` and run as `HealthCmd`
- [[Dankclaw Stack]]
- [[ComfyUI ROCm and Vulkan]] — the live-verified GPU path research
- [[ComfyUI Qwen-Image Workflows]] — the Qwen-Image 2.1 payload, proven workflows, and recommended model/LoRA additions
- [[Lemonade and sd.cpp]] — the ggml-family alternative
- the ggml-family alternative
- Knowledge base: [[ComfyUI]] (tools)
@@ -13,11 +13,10 @@ timestamp: 2026-08-26
# Dankclaw Stack
Dankclaw (192.168.0.143) runs the AI, search, development, and observability
stack. Its LLM inference endpoint `frank-dragoon` (`:7770`) is a ROCmFPX
llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on
the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`,
FLM v1.0.1) serves sight+sound models. The completion endpoint
`frank-medium` (ROCmFPX llama.cpp, `:7770`) runs on Dankserver (192.168.0.235).
stack. Its LLM inference endpoints are bare-metal host services, not part of
the container stack; each has its own page — shape, payload, eras and wire
contract live there, nowhere else. See [[frank-dragoon]] and [[frank-zealot]]
(both on this host) and [[frank-archon]] (Dankserver, 192.168.0.235).
```mermaid
graph LR
@@ -28,9 +27,8 @@ graph LR
C --> F[ComfyUI]
C --> D[Grafana]
S --> V[Gluetun VPN]
O --> L[llama.cpp dankserver :7770]
O --> L[LLM endpoints]
O --> H[Hermes]
O --> E[frank-dragoon :7770]
```
## Services
@@ -48,18 +46,16 @@ graph LR
| Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] |
| Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — |
| Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] |
| frank-dragoon (host service) | — | `7770` | [[deployment]] |
| frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] |
| frank-medium (host service, Dankserver) | — | `7770` | [[frank-medium-rocmfpx]] |
| frank-dragoon (host service) | — | `7770` | [[frank-dragoon]] |
| frank-zealot (host service) | — | `7771` | [[frank-zealot]] |
| frank-archon (host service, Dankserver) | — | `7770` | [[frank-archon]] |
The LLM rows are bare-metal systemd services, not containers. This
repo owns their scripts, units, and knowledge base (see
[LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03):
`frank-dragoon` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
on the card's pinned `kingjones30/ROCmFPX` llama.cpp build;
`frank-npu` runs FLM v1.0.1 on the XDNA2 NPU;
`frank-medium` runs charlie12345/ROCmFPX llama.cpp serving
kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF on Dankserver.
The LLM rows are bare-metal systemd services, not containers, and this repo
owns their scripts, units and knowledge base (see
). Which payload and runtime sits on
each port is era state and lives on the endpoint page, not here. `frank-npu`
(the XDNA2 NPU endpoint) and `frank-medium` (the predecessor of `frank-archon`
on the Dankserver port) are both retired — see and
## Boot Persistence
@@ -75,7 +71,7 @@ Deploy the SearXNG trio with `scripts/deploy-searxng-quadlet.sh dankclaw`, open-
## See Also
- [[index]] — repository index
- repository index
- [[Network Architecture]] — routing and proxy
- [[Observability]] — Grafana, Prometheus, logs
- [LLM fleet knowledge base](../LLM/index.md) — the model-serving endpoints on this host
- the model-serving endpoints on this host
@@ -0,0 +1,133 @@
---
type: reference
description: "The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes."
tags:
- dankclaw
- tts
- voice
- quadlet
- cpu
timestamp: 2026-10-18
---
# Frank TTS Engine
`frank-tts-engine` is the **arm A text-to-speech engine** on Dankclaw: [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp)'s resident `tts-server` HTTP process, holding the Qwen3-TTS **1.7 B base talker (Q4_K_M)** and the **12 Hz tokenizer (Q8_0)** in RAM once and synthesizing from there. It listens on container port **8880** and is **deliberately not a fleet endpoint** — the endpoint consumers address is `frank-tts` on **7772**, which resolves voice names, re-encodes to mp3 and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page.
**Deployed 2026-10-18.** The chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified, 18 s), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit `active` with no drop-ins, container healthy 31 s after the build, **4.0 GiB** resident on a host that kept 17 GiB available, `3 voice(s) registered and warm` in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a **file**, not an argument (Troubleshooting), and `STEP 3/13 RUN useradd` is the first honest proof of the image's non-root user.
**Host:** Dankclaw (`192.168.0.143`), rootless Podman, user `dank` — **Port:** 8880, **never published** — **Network:** the private `systemd-frank-tts-engine` bridge (10.89.5.0/24) and no other — **Deployer:** `frank-tts-engine` — **Consumers:** `frank-tts` (7772) and, through it, every voice request in the fleet including [[Open Notebook]]'s podcast renders.
## Facts (verified 2026-10-18 against the pinned upstream commit, the Hub, and the CPU trial on this host)
| Item | Value |
|------|-------|
| Engine | `tts-server` from qwentts.cpp, pinned at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` (`dankclaw/voiceservice/Containerfile.engine`). The source is **not vendored**: the image clones that commit and its recorded `ggml` submodule, so the build needs outbound HTTPS and the `cfhash` label is what keeps an unchanged re-deploy offline |
| Build line | `cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF && cmake --build build -j 8` — the exact line the 2026-10-18 trial ran on this box. `GGML_BLAS=OFF` is **forced**: Dankclaw has `libopenblas.so.0` and no BLAS dev headers, so upstream's own `Dockerfile` and `buildcpu.sh` (both of which pass BLAS on) cannot be copied. No `-DGGML_NATIVE` is passed, and ggml defaults it on, so the binary is `-march=native` for the machine that built it: **build this image on the host that runs it** |
| Image | built locally as `franksdank/frank-tts-engine:latest`, two stages from one digest-pinned base `docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac` (index digest resolved 2026-10-18). Runtime stage: `libgomp1` (OpenMP is on), `curl` (the unit's `HealthCmd` runs as `/bin/sh -c`, and the warmer speaks HTTP), uid **10000**, `LD_LIBRARY_PATH=/app` because ggml builds shared by default and the link-time RPATH names the build tree |
| Talker | `qwen-talker-1.7b-base-Q4_K_M.gguf`, **1 219 245 248 B**, sha256 `ea393ebaf2167ea23ce9fc18b093822851358a950d7075cd47ab4f6ce23e887d` (the Hub's LFS pointer oid) |
| Tokenizer | `qwen-tokenizer-12hz-Q8_0.gguf`, **291 150 624 B**, sha256 `1883beeed99348fc35e23dd225e9082f93f6f8c109330a33d935baa8acdbfd94` |
| Source of both | `https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF`, fetched by `scripts/download-frank-tts-engine-qwentts.sh` into `/home/dank/models/Qwen3-TTS-GGUF`, mounted read-only at `/models`. It has to be the **base** variant: the engine refuses every voice registration unless the model reports type `base`, so on `customvoice`/`voicedesign` the cloned-voice product does not exist. A 0.6 B talker (628 905 056 B) exists and is **not** the decided artifact |
| Memory | **5.7 GiB peak RSS** with both files resident (2026-10-18 trial). No `LimitMEMORY`/`MemoryMax`: a limit would turn a busy moment into an OOM kill in the middle of an utterance |
| Concurrency | `--max-batch 1`: a serialised FIFO. Warm RTF on this box is 0.55–0.67 against real time 1.0, and that margin is the entire headroom a live stream has; two concurrent syntheses split the same cores and the second caller's first byte additionally waits for the first utterance to finish |
| Network | `systemd-frank-tts-engine` (10.89.5.0/24, `dankclaw/voiceservice/quadlet/frank-tts-engine.network`) and **nothing else**, with **no `PublishPort` at all**. `frank-tts` joins the same bridge and dials `http://frank-tts-engine:8880` |
| Voices | `/home/dank/voices` mounted read-only at `/voices`, and it is the **warmer** that reads it, not the server: `tts-server` receives reference audio as base64 in a registration body and keeps its registry in memory only |
| Health | `GET /health` → `{"status":"ok"}`, served by the HTTP thread rather than the synthesis queue, so a busy engine still reads healthy. Probed by the unit (`HealthCmd`, 30 s interval, 120 s start period) **inside the container's namespace**, because there is no published port to probe from outside |
| Routes | `GET /health`, `GET /v1/models`, `GET|POST /v1/audio/voices`, `DELETE /v1/audio/voices/{name}`, `POST /v1/audio/speech` (`response_format` = `wav` or `pcm`; **default `pcm`**). Everything else is a bare 404. `POST /v1/audio/speech` requires `input`; unknown `voice`, missing `input` and a bad `speed` are 400s, an empty generation is a 502 |
## Why this engine, and not the alternatives
The criterion Frank set on 2026-10-18 was **"whatever is better for live output"**, and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. RTF only has to stay under 1.0 with margin; what separates the arms is first-byte latency and **spread**. Measured on this host the same day (CPU; the full A/B is the section *"Decision: arm A for live output (2026-10-18)"* in , which is the source of these numbers):
| Measure | Resident `tts-server` (arm A, chosen) | Mainline per-request CLI path (arm B) |
|---|---|---|
| first audio byte | **86–451 ms** end to end; **94–112 ms** server-side | model load, per request |
| warm real-time factor while streaming | **0.55–0.67** | 0.93–0.99 |
| run-to-run spread | **±0.01** | ±0.25 |
| HTTP surface | serves `/v1/audio/speech` itself | none — façade CLI only |
The spread is the whole argument: at ±0.01 the engine finishes synthesizing 1.5–1.8× faster than the audio plays, every run; at ±0.25 some runs fall behind real time and a live listener hears the stream stall mid-sentence. The rollback path stays in the repo untouched — `TTS_BACKEND=llama-tts-cli` in `scripts/frank-tts.env` still selects the per-request CLI engine, and that file's header documents what it costs.
## Three laws this service lives under
**1. The registry law → the unit warms at boot.** The voice registry is **memory only**: a restart forgets every voice, and re-registering one does **not** warm it — only synthesizing does. A boot that registers nothing leaves the façade answering 400 for every name, which is why `frank-tts-engine.container` carries
```
ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh
TimeoutStartSec=900
```
and the warm pass runs on every start including boot. What it buys is registration (mandatory) plus a measurably smaller first request — and the size of that saving is **not** the number the trial published. **Re-measured on the deployed build, 2026-10-18,** against an empty registry (register, then time the synthesis): the first request for `narrator1` cost **1 883 ms**, the second **1 283 ms**, the third **1 226 ms** — about **0.6 s** of saved silence. A ~110-word request cost **9.4 s** and then **9.8 s**, so request time tracks the audio generated, not snapshot state. The trial's **6.2–8.0 s / first audio byte 5 530 ms / later requests 0.5 ms** ladder does **not** reproduce here; the 0.5 ms figure is an internal state-restore cost that was never an end-to-end request time, and the honest claim is the ~0.6 s above. The `-` prefix is load-bearing: with `Restart=always`, a warmer that exits non-zero would restart the container and crash-loop the endpoint forever. The hook is `podman exec` rather than a host script **because this container publishes nothing** — there is no address the host could dial — and a consequence worth knowing: an exec inherits the unit's `EnvironmentFile`, so the warmer's knobs (`VOICES_DIR`, `HEALTH_WAIT`, `WARM_TEXT`, `REQUEST_TIMEOUT`) live in `scripts/frank-tts-engine.env` beside the engine's own. The deployer runs the same hook a second time so a failed warm becomes a deploy result rather than only a journal line, and so the per-voice numbers land on the console:
```
frank-tts-engine-warm: voice narrator1 cold=1402ms warm=1812ms
frank-tts-engine-warm: 3 voice(s) registered and warm
```
**2. The thread law → nothing is ever pinned.** `tts-server` in this build has **no `-t`/`--threads` flag**: `src/backend.h` computes the CPU thread count as `std::thread::hardware_concurrency() / 2` (its comment: GEMM shares SIMD units across hyperthreads). Thread count therefore follows what the kernel *reports*, not what a cgroup *allows* — so `CPUQuota=`, `CPUAffinity=`, `taskset` or a slice silently changes the engine's GEMM tiling and its timings, with no log line. Nothing in this fleet pins this container. If a neighbour starves it, fix the neighbour.
**3. Streaming is not buffered.** At the same seed, `response_format=pcm` (chunked, codec chunks pushed as they decode) and `response_format=wav` (buffered) differ in **92.2 % of samples**, −41.6 dB RMS. Nothing may compare the two paths, cache one against the other, or assume a seed pins output across `response_format`. `frank-tts` always asks for `wav`; the warmer asks for `wav`; the test seam is forbidden from pinning either.
## The voice registry contract
A voice becomes known to the engine through `POST /v1/audio/voices`, and only these shapes are accepted:
- `{"name": …, "wav_b64": …}` — the reference clip travels as base64 WAV (the engine resamples to 24 kHz internally). **This is what the boot warmer sends.**
- `{"name": …, "spk_b64": …, "rvq_b64": …}` — the extracted embedding plus the reference codec codes. **`spk_b64` alone is rejected** (`src/tts-server.h:480`): it is `wav_b64` or *both* of the pair.
Whether a registered voice is *embedding-only* or *in-context-learning* is then decided by `ref_text`: `tools/tts-server.cpp:258` selects ICL only when **both** `ref_text` and the reference codes are present. Registering `{name, wav_b64}` with no `ref_text` therefore yields the **x-vector voice** — the shipped default (0.57–0.63 warm RTF against ICL's 0.67) and the only one this fleet can produce honestly, because the audition crops have no trustworthy transcript to put in `ref_text`. The ICL door stays open for batch work like the podcast render: a pack that wants it needs its text in the pack plus one key in the payload built by `dankclaw/voiceservice/tts-engine-warm.sh`.
The packs themselves are host state at `/home/dank/voices/<name>/reference.wav` (24 kHz mono PCM), alongside the `reference.spk` / `reference.rvq` extraction and a manifest. Both halves of the voice stack read that directory — `frank-tts` for its name list, the warmer for its registrations — and `tts-server` never opens it. Two naming facts, both from the source: pack names must be **lowercase**, because `tts_voice_name` lowercases every name engine-side while the façade matches the directory name exactly (the warmer refuses anything outside `[a-z0-9._-]`); and `GET /v1/audio/voices` at this commit returns an empty JSON object rather than the voice list (the handler builds the array and never attaches it to the document), so **use the façade's `/v1/voices` for "which voices exist"** and treat the engine's listing route as broken, not empty.
## Status visibility
The engine is scraped by **nothing in Prometheus** and has **no homepage card**, and both are shape decisions rather than gaps. It exposes no metrics endpoint; and since it publishes no port, there is no address the host's status checker (`scripts/homepage-status.py`, which dials `host:port`) or the `dankclaw_default` Prometheus container could reach. That absence is the point of the network shape, not a cost of it (see [[Container Network Namespaces]] for the netavark rule that makes a host-loopback publish useless to a peer container anyway). The pair's public signal is the endpoint in front of it: [[Frank TTS]] carries the card and is what a consumer can dial. Its up-state is visible in four places, and the runbook uses all of them:
- `systemctl --user status frank-tts-engine` — the unit, including whatever the warm hook logged on the way to `active`.
- `podman healthcheck run frank-tts-engine` — runs the unit's own `HealthCmd` against `/health` on demand and exits non-zero when the engine does not answer. This is also what `scripts/deploy-remote.sh` gates the deploy on (`wait_container_healthy`), precisely because there is no URL to gate on.
- `podman exec frank-tts-engine curl -fsS http://127.0.0.1:8880/health` — loopback is valid **inside** the container's namespace, which is where the address lives.
- The journal of the warm pass: `journalctl --user -u frank-tts-engine`, whose per-voice `cold=` / `warm=` lines are the only signal that voices are actually registered — a live HTTP listener with an empty registry still looks healthy.
The **façade** on 7772 is the pair's public surface and carries the probe. Note the honest limit there: a façade that answers `/health` does **not** prove the engine behind it is alive — it reports its backend and its voice list, not an upstream ping. That is exactly why the warm pass is part of the deploy and not only of the boot.
## Files this service owns
| Path | Role |
|---|---|
| `dankclaw/voiceservice/Containerfile.engine` | two-stage build of the pinned commit, proven cmake line, digest-pinned base, ships both scripts |
| `dankclaw/voiceservice/tts-engine-start.sh` | image entrypoint (`/app/start.sh`): env → flags, refuses to boot without weights |
| `dankclaw/voiceservice/tts-engine-warm.sh` | boot warmer (`/app/warm-voices.sh`): register from `reference.wav` (body via a temp file), speak a fixed sentence twice, report cold/warm |
| `dankclaw/voiceservice/quadlet/frank-tts-engine.container` | the unit: bridge only, no publish, env file, two read-only mounts, health probe, warm hook, `Restart=always` |
| `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the private 10.89.5.0/24 bridge, with the subnet registry |
| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law and the registry law in its header |
| `scripts/download-frank-tts-engine-qwentts.sh` | weight fetcher: no resume, byte count + LFS oid check, `--verify-only` |
| `scripts/deploy-remote.sh` → `deploy_frank_tts_engine` | weights gate → build → quadlet → restart → container health → warm |
| `tests/frank_tts_engine.bats` | the seam: generator output, entrypoint argv, warmer against a stub engine, planner and routing maps |
## Deploy record (host steps, in the order they ran)
Run 2026-10-18 as `./scripts/deploy.sh dankclaw --service frank-tts-engine`, then
`--service frank-tts`. Steps 1–6 are this service and the façade; only step 7 is
still open.
1. **Fetch the weights** — `scripts/download-frank-tts-engine-qwentts.sh` (1.4 GiB, byte + sha256 verified, idempotent; `--verify-only` is what the deployer calls).
2. **Deploy the engine** — `./scripts/deploy.sh dankclaw --service frank-tts-engine` (`--service` is the only way to name one service; `deploy-remote.sh` is the remote half and takes `<target> <git-revision> [service]...`, so never call it by hand with a bare service name). It builds the image (needs github.com once), installs the unit and the network file, gates on the container's health status, then warms.
3. **Confirm the endpoint is alive** — `podman exec frank-tts-engine curl -fsS http://127.0.0.1:8880/health` → `{"status":"ok"}`, and `systemctl --user status frank-tts-engine` showing the unit `active` with the warm pass behind it.
4. **Confirm the voices are known** — the warmer's journal lines name every pack, and once the façade is up `curl http://127.0.0.1:7772/v1/audio/voices` is the authoritative list (the façade has exactly three routes: `/health`, `/v1/audio/voices`, `/v1/audio/speech`).
5. **Measure the ladder instead of trusting the trial** — two identical requests for one voice back to back. The design expected 6.2-8.0 s then 0.5 ms; the deployed container gave **1,883 ms then 1,226-1,283 ms**, so the hook buys ~0.6 s and stays only for the registry reason (law 1). The retired numbers must not be quoted.
6. **Then the façade on 7772** (its own page), after the engine answered — that order is the whole point of step 6: publishing the façade in front of a dead engine would make every consumer fail at the first voice line.
7. **Open:** [[Open Notebook]] is still not installed, so nothing renders a podcast yet; the voice leg it waits on now exists.
## Troubleshooting
| Symptom | Meaning | Move |
|---|---|---|
| Unit restarts every 10 s, journal says `cannot read /models/…` | weights missing or the mount is empty | run the fetcher; re-deploy |
| Unit `active`, but every voice name returns 400 from the façade | the engine restarted and lost its in-memory registry, or the warm pass failed | `journalctl --user -u frank-tts-engine` for the warmer's lines; `podman exec frank-tts-engine /app/warm-voices.sh` |
| Every voice logs `registration failed (HTTP no response)`, with `/usr/bin/curl: Argument list too long` on the line above it | the hook handed the registration body to curl as one argument: the kernel caps a single argument at 131 072 bytes (`MAX_ARG_STRLEN`), and a shipped `reference.wav` base64's to about 1.27 MB | the body goes through a temp file and `--data-binary @file` (2026-10-18). Any pack over ~96 KB trips this, so no small fixture can see it — the seam ships a 300 KB pack for exactly that reason |
| Everything is ~2× slower after a systemd tweak | a cpuset/cap pin changed `hardware_concurrency()` and the engine resized its thread pools | remove the pin (law 2) |
| `502` from the façade with `frank-tts upstream request failed` | the façade cannot resolve or reach the engine | `podman network inspect systemd-frank-tts-engine` and `podman inspect frank-tts --format '{{json .NetworkSettings.Networks}}'` — the façade must be attached to the bridge; a `127.0.0.1:8880` or `host.containers.internal:8880` upstream in `scripts/frank-tts.env` is the old shape and cannot work |
| A voice is missing from the list | its directory has no `reference.wav`, or the name is not lowercase | fix the pack; both are refused loudly by design |
| `podman exec` in the deploy says `no such file or directory` for `/app/warm-voices.sh` | the image predates the warmer | re-deploy: since 2026-10-18 `build_compose_image` hashes every file the image `COPY`s, not only `Containerfile.engine`, so editing the warmer rebuilds it — see |
| The warm pass logs `registered, but the warm synthesis failed` for EVERY voice, while a hand-run `/app/warm-voices.sh` succeeds | `WARM_TEXT` arrived **with its quotes**. podman's `--env-file` does no shell parsing — it takes the literal remainder of the line — so the env-file reader got `"Warm up, one two three."` where `source` gives `Warm up, one two three.`; the quotes plus a backslash then make the synthesis body invalid JSON | the hook strips one matched pair and escapes what is left; prove the value with `podman exec frank-tts-engine env \| grep WARM_TEXT`. The seam pins both readers, because a test that only sources the env file cannot see this |
| A `502` for `/v1/voices` where the engine answers `200` on `/health` from the same boot | the engine restarted and its in-memory registry is empty again — `/health` is model-loaded, NOT voices-registered | re-run `podman exec frank-tts-engine /app/warm-voices.sh`; that is the same distinction the `frank-tts` healthcheck enforces |
+222
View File
@@ -0,0 +1,222 @@
---
type: reference
description: "The fleet's cloned-voice speech endpoint and the one voice surface consumers address; wired as the harness speech role."
tags:
- endpoint
- dankclaw
- tts
- voice
- quadlet
timestamp: 2026-10-19
---
# Frank TTS
`frank-tts` is the fleet's only speech endpoint: an OpenAI-shaped
`POST /v1/audio/speech` on **Dankclaw :7772** that answers `voice=<name>` with
audio in that cloned voice. Everything else in the voice stack sits behind it —
the engine ([[Frank TTS Engine]]) publishes no port on purpose, and consumers
never learn a path.
**Deployed 2026-10-18** — `./scripts/deploy.sh dankclaw --service frank-tts`
both units `active`, no drop-ins, and the homepage card's `wip` gate came off in
the same change. Probes on the deployed endpoint: `/health` answers
`{"status":"ok","backend":"omni-http","voices":["narrator1","narrator2","narrator3"]}`
and two different voices rendered the same sentence into **different** bytes
(24 kHz mono PCM16, 4.24 s, healthy RMS) — so name resolution is real, not a
pass-through of one default. The operator harness routes its `speech` role here;
see [[#The harness speech role]]. What still waits is [[Open Notebook]]: its
render needs the app installed, not this endpoint.
| | |
| --- | --- |
| Host | Dankclaw (`192.168.0.143`), rootless podman, user unit `frank-tts.service` |
| Port | **7772**, published (`PublishPort=7772:7772`) — the rest of the `frank-*` port family is in [[Dankclaw Stack]] |
| Routes | `GET /health`, `GET /v1/audio/voices`, `POST /v1/audio/speech`; everything else 404, and speech is POST-only |
| Image | `localhost/franksdank/frank-tts:latest`, built from `dankclaw/voiceservice/Containerfile`, `Pull=never` |
| Networks | `dankclaw_default` first (it owns the default route and the consumers), then `frank-tts-engine.network` (10.89.5.0/24, the bridge to the engine) |
| Voice packs | `/home/dank/voices` mounted read-only at `/voices` — host state, never repo state |
| Upstream | `http://frank-tts-engine:8880` (`scripts/frank-tts.env`, `TTS_UPSTREAM`) |
| Default voice | `DEFAULT_VOICE` (`scripts/frank-tts.env`) answers a request that names none; the journal marks that line `(default)`, and an **unknown** name is still a 400 |
| Rollback | `TTS_BACKEND=llama-tts-cli` runs a CLI binary per request and needs no upstream at all |
## Naming a voice
A voice **is** a directory name. `voice=narrator1` resolves to
`/voices/narrator1/reference.wav`, and that file — not a manifest, not a config
entry — is what makes the name real. omp's TTS device may send `voice_id`; the
shim accepts either. A call that sends **neither** — the harness's normal shape,
because no harness setting carries a voice for this transport — speaks
`$DEFAULT_VOICE`; empty default makes it a 400 again.
The usable shape is `^[a-z0-9][a-z0-9._-]*$`: lowercase, starting with a letter
or digit, then letters, digits, `.`, `_`, `-`, at most 64 characters. No slash,
no `..`, no space, no uppercase. The lowercase requirement is where two rules
meet: `app.py` would accept `[A-Za-z0-9…]`, but `tts-server` **lowercases
names**, so a pack called `Narrator1` resolves through the façade and not
through the engine. The boot warmer therefore refuses a mixed-case directory
loudly instead of letting it half-work.
Names are user-visible: an unknown voice comes back as
`400 unknown voice 'x'; available voices: …`, and that list is the honest
census. The podcast speakers are `narrator1`, `narrator2`, `narrator3` —
[[Open Notebook]] is configured with exactly those three, so renaming them
breaks episode rendering, not just speech. A personal voice gets the person's
name, lowercase.
## Voice packs
One directory per voice under `/home/dank/voices`:
```
reference.wav 24 kHz mono PCM, 18-25 s of clean single-speaker audio
reference.spk engine speaker embedding (8 192 B)
reference.rvq reference codec codes (22 B per 12.5 Hz frame)
manifest.json source file, offsets, engine, sha256
```
`reference.wav` is the engine-agnostic term the façade gates on; the other two
are what makes a voice reusable by this engine, and they come from the pack
builder ([[Voice Packs]]) — never by hand, never fabricated from the WAV. Two
laws apply to the
source material: offsets always start **past the first 10 seconds** of the
sample, and packs are host state that stays out of this repo. Because the
container runs as uid 10000, files must be world-readable (`644`/`755`), and the
directory must exist before the first deploy: rootless podman auto-creates a
missing bind source **root-owned**, which reads as "no voices" rather than as an
error.
Adding a voice to a running stack is a two-step thing — drop the pack, then
register **and warm** it with
`podman exec frank-tts-engine /app/warm-voices.sh` — because registering alone
does not warm it. The reason is the cold-voice law in [[Frank TTS Engine]], and
the procedure with its proof (the second request under a second) is the
`frank-tts-add-cloned-voice` skill.
## Why the upstream is a container name
`scripts/frank-tts.env` used to point at `http://host.containers.internal:8880`.
That cannot work, and it is the trap this shape exists to avoid: a bridge
container reaches the host's network namespace, netavark DNATs **published**
ports only, and an engine on the host's loopback is published to nobody — the
same shape that 502'd hermes ([[Container Network Namespaces]]). So both units
attach to one committed Quadlet network and the façade dials the engine's
container DNS name.
The consequence worth stating before an operator meets it: **the engine is not
reachable from the host at all** — not over the LAN, not over loopback. Its
health, voice registry and logs are observable only through
`podman exec frank-tts-engine …` and `systemctl --user status`. And the façade's
own `/health` answers 200 with the backend name and the voice list it resolved
without dialing the engine, so "façade healthy, engine down" is a real state: it
shows up as a failed synthesis, never as a down probe.
## Where the engine computes
CPU, and only CPU is compiled in: `Containerfile.engine` builds `qwentts.cpp` with
`-DGGML_VULKAN=OFF` written out explicitly, so the container carries `libggml-cpu` and no
other backend. The realistic accelerator is the gfx1151 iGPU behind that flag — plus
`/dev/dri` passthrough, the ComfyUI pattern — on the same payload and the same pinned
commit, and the intended shape is **co-tenancy**: the engine resident on the card beside
`frank-dragoon`, not a handover. Memory is not what decides that; shader contention is, and
the census plus both gates are in `report/tts-npu-spike-verdict-2026-10-19.md` §5.
**The Strix Halo NPU is not a door at all**: XDNA2 executes only precompiled AIE
partition programs, and AMD's own Linux runtime opens `/dev/accel/accel0`, succeeds at the
ioctls, then declines this model family's graph without emitting a single log line. The
measured record and the model-side contract live in `report/` at the repo root
(`tts-npu-spike-verdict-2026-10-19.md`).
## Files
| Path | What |
| --- | --- |
| `dankclaw/voiceservice/app.py` | the shim: stdlib only — resolves the voice, forwards, re-encodes mp3 |
| `dankclaw/voiceservice/Containerfile` | python 3.12-alpine pinned by digest + ffmpeg, uid 10000 |
| `dankclaw/voiceservice/quadlet/frank-tts.container` | the unit: two networks, `Restart=always`, `HealthCmd`, `[Install]` |
| `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the shared bridge, named after the engine unit and attached by both |
| `scripts/frank-tts.env` | backend, upstream, port, limits, and the default voice — paths and numbers, no credentials |
| `tests/frank_tts_unit.bats` | the shim against stub engines over real sockets |
| `tests/frank_tts_engine.bats` | the pair's wiring: generator output, planner, homepage, deployer |
## Deploying it
`./scripts/deploy.sh dankclaw --service frank-tts` (or `--service frank-tts-engine --service frank-tts` for the pair):
builds the façade image, installs **both** Quadlet units — including the
engine's `.network` file, because a referenced `Network=` file the generator
cannot find means the façade unit is not produced at all — creates
`/home/dank/voices` as the deploy user, restarts the unit (whose `Wants=` and
`After=` pull the engine and its warm pass ahead of it) and gates on podman's
recorded health. It never runs `systemctl --user enable`; boot persistence is
the `[Install]` section in each unit. Order matters in a full deploy:
[[Frank TTS Engine]] is planned before this, and both before [[Open Notebook]].
## Not scraped, and why
`shared/prometheus/` gets **no** frank-tts job, and that is a considered
absence. The façade exports no `/metrics` — `app.py` serves `/health` plus the
speech route only — and this Prometheus 3 install refuses non-metrics content
types: [[Prometheus]] records that the existing HTML probes (`comfyui`, the two
`services` targets) sit **down** with
`received unsupported Content-Type … and no fallback_scrape_protocol specified`.
A `/health` probe job would therefore be structurally never-green, and because
`LlmEndpointDown` is scoped across the whole `frank-*` job family (the rule set
lives on [[Prometheus]]), a permanently red `frank-tts` target would page on
Signal every few minutes for an endpoint that is fine. The engine cannot be
scraped at all: it publishes nothing. Status lives in `systemctl --user status`
`podman healthcheck run`, the two unit journals, and the homepage card, which
probes `127.0.0.1:7772` for real now — the `wip: down` gate came off with the
2026-10-18 deploy.
## The harness speech role
The operator harness reaches this endpoint through
`~/.omp/agent/models.yml` — provider `frank-tts (dankclaw)`
`baseUrl: http://192.168.0.143:7772/v1`, `auth: none`, and the runner api
`openai-speech` on the **model** row (naming a runner api at *provider* level
fails the whole file's validation and disables every custom provider, which is
how the fleet's chat endpoints disappear) — plus `config.yml`
`modelRoles.speech: frank-tts (dankclaw)/frank-tts`. Four laws, read out of the
harness bundle and measured the day of the flip:
- **The only voice the harness can send is the `voice_id` argument of the call.**
The transport builds `{model, input, response_format, voice?}` and adds `voice`
**only** when the caller passed `voice_id`; no setting feeds it.
`tts.localVoice` is the *local Kokoro backend's* voice (it is what answered
`voice=kokoro/bm_george, backend=local` in the control test), and
`speech.voice` is the TUI's **speak-aloud** voice — its enum is Kokoro's list,
and it never reaches an HTTP request. So a harness call arrives voice-less by
default, which is exactly why the endpoint grew `DEFAULT_VOICE`: the declared
pack answers a voice-less request and the journal marks that line `(default)`
while an **unknown** name still 400s with the list that exists.
- **`speech.voice` must stay a Kokoro name.** Setting it to a pack name does not
select a pack; it hands the vocalizer a value outside its enum. (The first
draft of this section said the opposite and was reverted the same day, after a
fresh session's call came back `unknown voice None`.)
- **There is no silent fallback.** An explicitly configured `modelRoles.speech`
yields exactly one candidate, so a dead 7772 fails the speech device with
*"No available speech model matches the speech role"* instead of quietly
degrading to Kokoro. That is the price of `auth: none` (which makes a dead
endpoint read as available) and it is the wanted shape: a narration job should
fail loudly rather than ship a voice Frank never asked for.
- **A session older than the models.yml row cannot see it.** `models.yml` loads
at process start while `config.yml` is read live, so a long-lived session
carrying the flipped role fails exactly like a dead endpoint while a fresh
process succeeds in the same minute. Measured 2026-10-18: the open session
failed twice (provider/id form, then the bare id), `local/kokoro` succeeded in
that same session, and a fresh `omp -p --tools tts` returned `Saved 180524
bytes (model=frank-tts (dankclaw)/frank-tts, voice=narrator1, codec=wav)`
while the façade logged the matching `speech voice=narrator1 format=wav
chars=51 -> 180524 bytes` from the podman bridge. Rule the stale registry out
before blaming the service.
## Troubleshooting
| Symptom | Cause |
| --- | --- |
| `400 unknown voice 'x'; available voices: …` | the name is not a pack: case, spelling, missing `reference.wav`, or an empty/root-owned `/home/dank/voices` |
| `(none: no voice pack with a reference.wav in /voices)` | the mount, not the voices: `podman inspect frank-tts --format '{{json .Mounts}}'`, then the directory owner |
| Endpoint answers, synthesis fails | the engine: `podman healthcheck run frank-tts-engine`, then `systemctl --user status frank-tts-engine.service` |
| First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
| The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
---
+2 -2
View File
@@ -1,5 +1,5 @@
---
type: configuration
type: reference
title: Gitea
description: Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service.
tags:
@@ -33,7 +33,7 @@ The containers share the private bridge network `systemd-gitea`. The quadlet net
| `dank/command-center` | second public repo on the instance (`GET http://127.0.0.1:3000/api/v1/repos/search` 2026-10-18) |
| `dank/vllmScripts` | retired 2026-08-22 — merged into `dank/serverAppConfigs` (history preserved) *(unverified 2026-10-18: the anonymous API search returns no match and `/dank/vllmScripts` is 404, so whether the repo still exists privately cannot be checked without a token)* |
The LLM fleet knowledge base: [serverAppConfigs-docs/LLM/index.md](../LLM/index.md)
The LLM fleet knowledge base:
## Management
@@ -30,7 +30,7 @@ entrypoint does not start the dashboard. The dashboard is a separate process, so
it runs in its own Quadlet unit
([`dankclaw/hermes/quadlet/hermes-dashboard.container`](../../dankclaw/hermes/quadlet/hermes-dashboard.container):
image `localhost/hermes-agent:latest`, `Pull=never`, `Exec=hermes dashboard
--host 0.0.0.0 --port 9119 --no-open --skip-build`, `Restart=always`,
--host 0.0.0.0 --port 9119 --no-open --skip-build`, `Restart=always`
`RestartSec=5`). All three Hermes containers mount the same `/data` bind mount.
Unlike its two siblings — `hermes.container` and `hermes-webui.container` both
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: Hermes-WebUI (Hermex server)
description: The hermes-webui container on Dankclaw — the API server that the Hermex iPhone app talks to, reachable over an SSH tunnel, from the whole LAN (it binds all interfaces), and from https://hermes.franksdank.cloud (internal-only vhost).
description: "The iPhone companion app for Hermes on Dankclaw and the ways it is reached from the LAN."
tags:
- hermes
- hermes-webui
@@ -40,7 +40,7 @@ from another LAN host: `/health` → 200, `/` → 302) — **Vhost:**
## Why a container, not a host process
The gateway's state files are owned by the container-mapped uid (`109999`,
The gateway's state files are owned by the container-mapped uid (`109999`
from rootless Podman's `1 → 100000/65536` subuid range — `/etc/subuid` →
`dank:100000:65536`, and the agent's runtime uid is 10000, so 100000 + 9999 =
109999; `ls -ld /home/dank/Containers/hermes/data` → uid 109999, 2026-10-18). A fresh rootless
@@ -82,12 +82,12 @@ the sibling dashboard on `9119` is not.
- Dedicated `tunnel` account on Dankclaw (`getent passwd tunnel` → uid 994,
shell `/usr/sbin/nologin`, 2026-10-18; password locked). Authentication is key-only
*(unverified 2026-10-18: `/home/tunnel/.ssh/authorized_keys` is not readable
- (unverified 2026-10-18: `/home/tunnel/.ssh/authorized_keys` is not readable
without `sudo`)*; the account can do nothing except
authenticate — `ForceCommand /usr/sbin/nologin` kills any session channel
while the transport stays up for port forwarding.
- sshd drop-in `/etc/ssh/sshd_config.d/99-tunnel.conf` (exists, 118 B, 2026-08-31)
scopes `AllowTcpForwarding yes` to that user only, plus `PermitTTY no`,
scopes `AllowTcpForwarding yes` to that user only, plus `PermitTTY no`
`X11Forwarding no` and `ForceCommand /usr/sbin/nologin` — all four lines are
present under `Match User tunnel` (`cat` 2026-10-18).
- Key options on the authorized key pin the blast radius:
@@ -130,7 +130,7 @@ only if inactive (it is never restarted on deploy, so in-flight browser
sessions survive), re-tags the image alias, then restarts
`hermes.service`, `hermes-dashboard.service` and
`hermes-webui.service` in the **user** manager of `dank` (linger must be on for
them to come up at boot — `loginctl show-user dank -p Linger` → `yes`,
them to come up at boot — `loginctl show-user dank -p Linger` → `yes`
2026-10-18), so gateway/dashboard/webui restarts stay isolated from
inference deploys.
Before first deploy, set a strong `HERMES_WEBUI_PASSWORD` in the live
+16 -15
View File
@@ -18,7 +18,7 @@ bindings, 2026-10-18). Note the **container name drift**: the unit is
`podman run --name systemd-%N …` (`systemctl --user cat hermes.service`), so
`podman ps` shows the gateway as **`systemd-hermes`** — the repo unit sets no
`ContainerName=` key even though its header comment says "container name:
hermes". The user-facing web surface is [[Hermes-WebUI]] on `:8787`, reached via
hermes". The user-facing web surface is [[Hermes-WebUI]] on `:8787`, reached
`hermes.franksdank.cloud` — internal-only by DNS absence (no public Porkbun A
record; the name resolves from the Pi-hole hosts file,
`dankclaw/pihole/addn-hosts:39`) — **Docs:**
@@ -37,7 +37,7 @@ designed wiring; the remedy for both halves is in the log line
**Persistent state:** `/home/dank/Containers/hermes/data` on Dankclaw, bind-mounted
at `/data` in all three Hermes containers (gateway, webui, dashboard —
`hermes.container:36`, `hermes-webui.container:46`,
`hermes.container:36`, `hermes-webui.container:46`
`hermes-dashboard.container:24`). Each
container additionally gets an anonymous volume at `/opt/data`, which is
re-created empty on every container recreation — anything that lives only there
@@ -78,9 +78,9 @@ is not durable. The tracked, sanitized configuration baseline is
`/data/config.yaml` is not readable without `sudo`):
`dankclaw-large` reaches `http://192.168.0.143:7770/v1` and exposes
`frank-dragoon` — verified from the host with `curl
http://127.0.0.1:7770/v1/models` → single id `frank-dragoon`, `max_model_len
200704` (2026-10-18; write `127.0.0.1`, not `localhost`, because that port is
a rootless pasta publish and is IPv4-only);
http://127.0.0.1:7770/v1/models` → a single served id (2026-10-18; write
`127.0.0.1`, not `localhost`, because that port is a rootless pasta publish
and is IPv4-only)
`dankserver-small` reaches `http://192.168.0.235:7770/v1` and, **as of
2026-10-18, that endpoint serves only `frank-archon`** (`GET
http://192.168.0.235:7770/v1/models` → `[{"id":"frank-archon"}]`). The name
@@ -88,7 +88,8 @@ is not durable. The tracked, sanitized configuration baseline is
2026-09-15 — is retired-era: nothing on that endpoint answers to it today, so
if the live `config.yaml` still names `frank-medium` the provider is dead
weight and requests will 404 on a nonexistent model. Diff the live config
before trusting either name.
before trusting either name. Which payload sits behind each port, and the
served id it answers to, is on [[frank-dragoon]] and [[frank-archon]].
- The live `config.yaml` contains secrets and user state and is not committed;
the tracked example is the sanitized backup of its non-secret settings.
@@ -114,7 +115,7 @@ per session; the browser itself lives in a dedicated daemon container, the
digest-pinned headless-shell unit `chrome-devtools-protocol.container` (host
network, loopback-only bind on 127.0.0.1:9222 — `ss -ltnp` 2026-10-18 shows
`127.0.0.1:9222 headless-shell` and nothing on `0.0.0.0`; image
`docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5…`,
`docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5…`
`dankclaw/hermes/quadlet/chrome-devtools-protocol.container:28`). Both consumers — the Signal
gateway and the webui server — run in the host network namespace (`podman
inspect` → `net=host` for both, 2026-10-18), so they can
@@ -227,9 +228,10 @@ resp = call_llm(task="title_generation", messages=[...], max_tokens=64,
What actually breaks, and the fix:
- **`response_format` (json_object / json_schema) is a hard 400 on every
local backend here** — neither FreeToken nor halogen implements constrained
decoding. Config cannot override it: `call_llm` merges the task's
- **`response_format` (json_object / json_schema) is a hard 400 on the local
backends this task was written against** — what each endpoint accepts on the
wire is on [[frank-dragoon]] / [[frank-zealot]] / [[frank-archon]]. Config
cannot override it: `call_llm` merges the task's
`extra_body` first and then `update()`s the caller's, so the hardcoded one
wins. Only temperature and max_tokens have reactive retry paths.
- **Fix: move that one task to the Responses transport**, which never
@@ -246,9 +248,8 @@ What actually breaks, and the fix:
effort: none # mandatory, see below
```
Before copying this: as of 2026-10-18 `frank-zealot` is served on **Dankclaw
`:7771`**, and `http://192.168.0.235:7770/v1` — the endpoint this provider
name is recorded against — answers only `frank-archon`. The provider/model
Before copying this: which id is served on which port is endpoint state —
[[frank-zealot]] and [[frank-archon]]. The provider/model
pairing above is the shape that was landed in 2026; re-derive it from the live
config before reusing it.
@@ -281,7 +282,7 @@ Auth contract (verified live 2026-08-31, re-checked 2026-10-18): `POST /api/auth
on match, 401 on miss. No username field. `/health` answers 200
unauthenticated (`curl http://127.0.0.1:8787/health` → 200 with
`{"status":"ok", …}` today) and `/` answers 302 (`curl -o /dev/null -w
%{http_code} http://127.0.0.1:8787/` → 302 today) — **never verify auth via
%{http_code} http://127.0.0.1:8787/` → 302 today) — **never verify auth
those**; probe
login with a deliberately wrong password (expect 401) and the real one
(expect 200). Details on the webui surface: [[Hermes-WebUI]].
@@ -310,7 +311,7 @@ login with a deliberately wrong password (expect 401) and the real one
instead of loopback" — live bind `0.0.0.0:8787`, `ss -ltnp` 2026-10-18;
off-network ingress is SSH tunnel only —
user `tunnel`, port **22** (`sshd -T` → `port 22`, 2026-10-18), key-only auth
*(unverified 2026-10-18: the per-user `Match` block needs `sudo` to read)*,
- (unverified 2026-10-18: the per-user `Match` block needs `sudo` to read)*
forward remote end
localhost:8787).
+11 -11
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Marreta on Dankclaw
description: The content-cleaning proxy (manualdousuario/marreta, Laravel/serversideup fpm-nginx) deployed as a rootless Quadlet pair that shares SearXNG's Gluetun netns for egress and sits behind the Anubis + Authentik guard chain at marreta.franksdank.cloud.
description: "The content-cleaning proxy SearXNG borrows egress from: a quadlet pair in the VPN netns behind the guard chain."
tags:
- dankclaw
- marreta
@@ -19,7 +19,7 @@ returns a stripped, readable version of the page — no paywall chrome, no
tracking, no cookie walls. It is a Laravel 13 app on the
`serversideup/php:8.4-fpm-nginx` image (`composer.json` requires
`php: ^8.4`, `laravel/framework: ^13.0`, `filament/filament: ^5.7` —
<https://raw.githubusercontent.com/manualdousuario/marreta/main/composer.json>),
<https://raw.githubusercontent.com/manualdousuario/marreta/main/composer.json>)
with an optional headless-browser companion for JavaScript-heavy pages.
**Host:** Dankclaw — **vhost:** `marreta.franksdank.cloud` (Caddy, guard-chained)
@@ -31,7 +31,7 @@ with an optional headless-browser companion for JavaScript-heavy pages.
- Fetches a target URL with a cascade of strategies: plain HTTP (curl), a
headless browser (Lightpanda) for JS pages, and the Internet Archive
(Wayback) as a last resort — the three `app/Services/Fetch/*FetchStrategy.php`
files upstream (`BrowserFetchStrategy`, `CurlFetchStrategy`,
files upstream (`BrowserFetchStrategy`, `CurlFetchStrategy`
`WaybackFetchStrategy`, dispatched by `app/Services/FetchManager.php`).
- Strips paywall/consent/tracking markup and re-renders clean HTML.
- Ships a Filament admin panel at `/admin` (login = `ADMIN_EMAIL` /
@@ -69,7 +69,7 @@ graph LR
| `marreta-browser.service` | `dankclaw/marreta/quadlet/marreta-browser.container` | `docker.io/lightpanda/browser:nightly` (`:26`) | `gluetun.container` (`:25`) | `9222` (CDP) |
Both units are boot-persistent (`WantedBy=default.target`) and self-healing
(`Restart=always`, `RestartSec=10` — `marreta.container:62-69`,
(`Restart=always`, `RestartSec=10` — `marreta.container:62-69`
`marreta-browser.container:30-34`), per the repo's Boot Persistence rule. Live
2026-10-18 both report `systemctl --user is-enabled` → `generated` and
`is-active` → `active`, and `podman inspect` reports marreta **healthy**
@@ -87,14 +87,14 @@ The fix is a single env var, verified against the serversideup source:
- The image sets `ENV NGINX_HTTP_PORT="8080"` and the site template is
`listen ${NGINX_HTTP_PORT} default_server;`.
- The image does **not** bake `conf.d/*.conf` (it `rm -rf`s them at build);
- The image does **not** bake `conf.d/*.conf` (it `rm -rf`s them at build)
the entrypoint `10-init-webserver-config.sh` runs `envsubst` over every
`NGINX_*` variable **at container start**.
- So `Environment=NGINX_HTTP_PORT=8085` in the quadlet relocates the listen
port off the collision with no custom build.
Marreta then publishes **nothing** of its own — the netns owner (Gluetun)
publishes `8082:8085` on its behalf (`dankclaw/searxng/quadlet/gluetun.container:32`),
publishes `8082:8085` on its behalf (`dankclaw/searxng/quadlet/gluetun.container:32`)
exactly as it publishes `8081:8080` for SearXNG. Caddy dials
`192.168.0.143:8082` (`dankclaw/Caddyfile:419-422`). Live env confirms the
relocation: `podman inspect marreta` carries `NGINX_HTTP_PORT=8085`
@@ -127,7 +127,7 @@ on the mechanism: `BrowserFetchStrategy.php` only calls
(`dankclaw/Caddyfile:159-163`).
2. **Auth decision** — Anubis dials Caddy's `http://:8016`, which proxies
whole-request to `authentik-server:9000` (`dankclaw/Caddyfile:22-24,94`).
3. **Post-auth routing** — authenticated traffic returns to `http://:8017`,
3. **Post-auth routing** — authenticated traffic returns to `http://:8017`
where `@marreta host marreta.franksdank.cloud` routes to
`reverse_proxy 192.168.0.143:8082` (`dankclaw/Caddyfile:419-422`).
@@ -164,7 +164,7 @@ Marreta ships `trustProxies(at: '172.16.0.0/12,10.0.0.0/8,192.168.0.0/16,127.0.0
— that is the `TRUSTED_PROXIES` fallback in upstream `bootstrap/app.php`, and
this container sets no `TRUSTED_PROXIES`, so the default is what runs
(`podman inspect marreta` 2026-10-18) — and forces `https` when `APP_URL`
starts with `https://` (`app/Providers/AppServiceProvider.php:23-24`,
starts with `https://` (`app/Providers/AppServiceProvider.php:23-24`
`URL::forceScheme('https')`). With
`APP_URL=https://marreta.franksdank.cloud` (`dankclaw/marreta/quadlet/marreta.container:41`) the app honors the forwarded
scheme and client IP through the whole chain, so there is no redirect loop
@@ -234,7 +234,7 @@ netns down and would otherwise orphan it).
Deployed via `deploy.sh dankclaw --service gluetun --service marreta --service
caddy` (gluetun first to recreate the tunnel with the new `8082:8085` publish,
which also cycles SearXNG's netns ref). Authentik `ProxyProvider` `marreta`
registered (pk 13, `external_host=https://marreta.franksdank.cloud/`,
registered (pk 13, `external_host=https://marreta.franksdank.cloud/`
`internal_host=http://caddy:8017`). Results:
- Unauth edge → Anubis challenge page (not 502). TLS cert issued via DNS-01.
@@ -245,14 +245,14 @@ registered (pk 13, `external_host=https://marreta.franksdank.cloud/`,
(`GET 127.0.0.1:9222/json/version` → `Lightpanda/1.0`).
- SearXNG unaffected: `:8081` → 200 after the tunnel recreate.
> **Gotcha hit during deploy:** the first seed used a bare-base64 `APP_KEY`,
> **Gotcha hit during deploy:** the first seed used a bare-base64 `APP_KEY`
> which 500s every app route (see Secrets above). Fixed by re-keying with the
> `base64:` prefix before any encrypted data was persisted.
## Citations
- Marreta upstream: <https://github.com/manualdousuario/marreta> (README,
`compose.yml`, `bootstrap/app.php`, `app/Services/Fetch/BrowserFetchStrategy.php`,
`compose.yml`, `bootstrap/app.php`, `app/Services/Fetch/BrowserFetchStrategy.php`
`lang/` directory listing).
- serversideup/php fpm-nginx port mechanism: `serversideup/docker-php`
`src/variations/fpm-nginx/Dockerfile` (`NGINX_HTTP_PORT="8080"`) and
@@ -1,6 +1,6 @@
---
type: reference
description: "open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out — script written by frank-zealot, voices rendered by frank-tts, no cloud provider anywhere in the path."
description: "The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house."
tags:
- dankclaw
- open-notebook
@@ -13,7 +13,7 @@ timestamp: 2026-10-18
open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook)) is the notebook layer that turns existing notes and sources into a **spoken episode**: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by **frank-zealot**, the voices are rendered by **frank-tts**, and nothing leaves the LAN.
**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Deploying it is a deliberate next step, not a pending one.
**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Re-censused 2026-10-19 and unchanged: no `*notebook*` unit in `systemctl --user list-unit-files`, nothing installed under `~/.config/containers/systemd/` by that name, no container in `podman ps` — the repo's three quadlet files have never been installed on the host. Deploying it is a deliberate next step, not a pending one.
**Host:** Dankclaw (rootless Podman, user `dank`) — **Ports:** 8502 UI and 5056 API, published on the LAN, **no vhost and no Caddy route** — **Deployer:** `open-notebook` (one service, two units) — **Docs:** https://github.com/lfnovo/open-notebook
@@ -59,9 +59,9 @@ podman exec open-notebook curl -sS http://frank-tts:7772/health
podman exec open-notebook curl -sS http://host.containers.internal:7771/v1/models
```
**The blocker to state plainly:** frank-tts is built, named and tested but **not deployed** — nothing answers on 7772 (`ss -ltn`, Dankclaw, 2026-10-18; [[Local TTS and Voice Cloning]] records the same census). open-notebook deployed today writes scripts and fails at the first voice line. frank-tts ships first, or podcast generation waits.
**The blocker moved; it did not vanish.** frank-tts and frank-tts-engine are **deployed** (2026-10-18) and answer on 7772, so the voice leg works end to end — `podman exec open-notebook curl -sS http://frank-tts:7772/health` is the check to run the day the app exists. What still blocks a render is open-notebook itself: not installed on this host yet. When it is, the two provider rows above are the only wiring, and the seeded speaker profiles carrying OpenAI voice names are the first trap.
**Why frank-zealot and not frank-dragoon:** podcast scriptwriting asks the model for a JSON-schema response (with a plain-JSON fallback). llama.cpp's OpenAI-compatible endpoint is a known-good target for that; frank-dragoon's halogen runtime has **unverified** structured-output behaviour, so it is the unproven alternative, not the fallback. `ESPERANTO_LLM_TIMEOUT=180` (the app's own default) covers an outline or transcript call on zealot comfortably; upstream's note is to keep it under 600 so a slow call fails with a clear error instead of a UI timeout.
**Why frank-zealot and not frank-dragoon:** podcast scriptwriting asks the model for a JSON-schema response (with a plain-JSON fallback). The decision rests on each endpoint's documented request surface ([[frank-zealot]], [[frank-dragoon]]): zealot's is a known-good target for that call shape, dragoon's structured-output behaviour is **unverified**, so it is the unproven alternative, not the fallback. `ESPERANTO_LLM_TIMEOUT=180` (the app's own default) covers an outline or transcript call on zealot comfortably; upstream's note is to keep it under 600 so a slow call fails with a clear error instead of a UI timeout.
## Speakers and voices: 1–4, and the IDs are free text
+66 -34
View File
@@ -39,28 +39,26 @@ As of 2026-10-07 the picker exposes **exactly one model: `frank-zealot`** *(unve
`frank-dragoon`/`frank-zealot`/`frank-archon`, per the cards in
`shared/homepage/services.json`)*.
| Connection (`openai.api_configs` idx) | URL as recorded 2026-10-09 | Enabled | `model_ids` | Runtime | Host |
|---|---|---|---|---|---|
| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | halogen flash-server (Strix Halo iGPU) | Dankclaw |
| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | radiance TP=2 (2× R9700) | Dankserver |
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | llama.cpp CUDA (RTX 3090; production shape 3×262,144 per `scripts/frank-zealot.env`) | Dankclaw |
| Connection (`openai.api_configs` idx) | URL as recorded 2026-10-09 | Enabled | `model_ids` | Endpoint page |
|---|---|---|---|---|
| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | [[frank-dragoon]] |
| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | [[frank-archon]] |
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | [[frank-zealot]] |
The URLs in the first column are the DB's recorded values and predate the 2026-08-22
bridge move; on this network they are wrong — see the reachability gotcha below, which
is the corrected law. Note also that the 2026-10-18 **bench window** on `:7771`
closed the same day (production unit stopped 03:38 per the repo Benchmarking
Policy while candidate arms held the port): the candidate — Ternary-Bonsai-2-**9B**
PQ2_0 with the MTP head, 3 × 262,144 — was **deployed as production**, so the
picker row below was re-keyed to it (see the next paragraph).
is the corrected law. The picker row for `frank-zealot` was re-keyed on 2026-10-18 after
that day's payload swap (see [[frank-zealot]] for what was deployed and when).
The `model` table carries exactly two decoration rows, `frank-zealot` and
`frank-dragoon`. Both the context length and the description live **inside
JSON columns** — `params` holds `{"context": {"length": …}}` and `meta` holds
`{"description": …}`; there is no `context_length` or `description` column, so
`sqlite3` one-liners against those names fail. Re-keyed 2026-10-18:
`frank-zealot` → 262,144 with the Bonsai-2-9B description, and `frank-dragoon`
→ 200,704 (its `params` had kept the pre-lowering 262,144 window from
2026-10-17 while its own description already said 200,704). These rows are
`sqlite3` one-liners against those names fail. `params.context.length` is a copy
of the endpoint's per-slot window, so the authoritative number lives on the
endpoint page ([[frank-zealot]], [[frank-dragoon]]) and the row must follow it —
the 2026-10-18 re-key existed because `frank-dragoon`'s row still carried the
window its payload had before that day's swap. These rows are
live DB state, so a payload swap must re-key the database too, not only the
repo. There is no `sqlite3` binary on Dankclaw — use `python3 -c` with the
stdlib `sqlite3` module, and take a `Connection.backup()` snapshot first (the
@@ -104,16 +102,16 @@ Two gotchas make the wiring non-obvious:
The Ollama connection is disabled (`ollama.enable=false`).
> All three endpoints are bare-metal systemd services managed by this
> repository. Model swaps, context sizing, and spec tuning happen there. See
> the [LLM knowledge base](../LLM/index.md). Disabling a connection here only
> hides it from this picker — the endpoint itself keeps serving every other
> client (the harness, Hermes, API consumers).
> repository; their shape, swaps and tuning live on
> [[frank-dragoon]] / [[frank-zealot]] / [[frank-archon]]. Disabling a
> connection here only hides it from this picker — the endpoint itself keeps
> serving every other client (the harness, Hermes, API consumers).
## Container introspection (DB and backend python)
- Working dir for imports: `/app/backend`; package name `open_webui`
(underscore), NOT `openwebui`. `os.environ["WEBUI_SECRET_KEY"]` MUST be set
BEFORE importing any `open_webui` module (import-time hard requirement);
BEFORE importing any `open_webui` module (import-time hard requirement)
the secret lives at `/app/backend/.webui_secret_key`.
- **The `sqlite3` CLI is NOT in the container** — use Python's `sqlite3`
module (`podman exec open-webui python3 -c "import sqlite3;
@@ -124,8 +122,8 @@ The Ollama connection is disabled (`ollama.enable=false`).
pydantic `ModelModel` instances (attribute access, not `.get()`), with NO
`status`/`enabled` attribute — use `is_active` from the DB.
- **API auth for verification:** `create_token` lives in
`open_webui.utils.auth` (NOT `open_webui.utils.utils`);
`get_verified_user` requires an `id` claim (not just `sub`);
`open_webui.utils.auth` (NOT `open_webui.utils.utils`)
`get_verified_user` requires an `id` claim (not just `sub`)
`SESSION_SECRET = WEBUI_SECRET_KEY`. Mint an admin token inside the
container:
@@ -143,14 +141,14 @@ The Ollama connection is disabled (`ollama.enable=false`).
models. Response key is `items`, not `data`.
2. `POST /api/chat/completions` per model with a trivial prompt ("Reply
with exactly one word: pong") → each returns content `pong`. Reasoning
models need a generous budget: frank-zealot (Qwen3.5 template) returns
empty content + `finish_reason: "length"` at `max_tokens 200` (the
thinking block eats it) and answers `pong` at 1500.
endpoints need a generous budget: thinking consumes `max_tokens`, so a short
one returns empty content + `finish_reason: "length"` (see
[[frank-zealot]]); `frank-zealot` answers `pong` at 1500.
3. A 500 "Server Connection Error" → `podman logs open-webui` for
`ClientConnectorError`: a network-reachability problem (host LAN IP vs
`host.containers.internal`), not a model problem.
**Durability:** the model table + connections are DB-only (not git-managed);
**Durability:** the model table + connections are DB-only (not git-managed)
a fresh data volume loses them — including the public access grant above.
For reproducibility, add an idempotent first-start seed script or document
the setup in the service doc.
@@ -158,7 +156,7 @@ the setup in the service doc.
## API access (auth-gated)
Every `/api/*` endpoint requires an authenticated session (SSO login JWT)
or a per-user API key (`auth.enable_api_keys=true`; keys are minted per
or a per-user API key (`auth.enable_api_keys=true`; keys are minted
account under Profile → API Keys). Anonymous web visitors get the `pending`
role (`ui.default_user_role=pending`), which cannot chat. Verified
2026-10-07 from loopback `:3005` and over the public vhost:
@@ -168,13 +166,12 @@ bogus bearer token); only `/api/config`, `/api/version` and `/health`
`POST /api/chat/completions` all return **401** with no key (also with a bogus bearer
token); only `/api/config`, `/api/version` and `/health` answer publicly, and none of
them expose the model list. An external program needs an account + API key — there is
no keyless access. Re-verified 2026-10-18 from loopback: `/health` → `{"status":true}`,
no keyless access. Re-verified 2026-10-18 from loopback: `/health` → `{"status":true}`
`/api/version` → 0.11.4, `/api/v1/models/list`, `/api/v1/models` and
`POST /api/chat/completions` → 401; `/api/config` additionally publishes
`features.enable_login_form=false`, `features.enable_signup=false` and
`oauth.providers.oidc="authentik"` — the SSO-only posture, visible from outside.
## Web search
Chat web search is served by [[SearXNG]] on this host, wired in the quadlet
@@ -188,7 +185,7 @@ Chat web search is served by [[SearXNG]] on this host, wired in the quadlet
The Anubis→Authentik guard chain is a browser mechanism (proof-of-work +
session flow); a plain HTTP client cannot solve it, and the internal port
bypasses Caddy entirely.
- **Limiter:** SearXNG caps non-HTML (JSON) requests at 4/IP/hour (`API_MAX`,
- **Limiter:** SearXNG caps non-HTML (JSON) requests at 4/IP/hour (`API_MAX`
a hardcoded Python constant — no config knob). The `pass_ip` list is checked
FIRST and short-circuits the whole botdetection chain, and OpenWebUI's bridge
egress (`10.89.x`) is in `pass_ip`, so the cap never applies. Verified: an
@@ -251,7 +248,6 @@ browser-driven and only touches the two exempt vhosts (`ai.` and `auth.`).
restore the password form: set `ui.enable_login_form` back to `true` in the
Admin Panel (or DB) and drop the two env vars from the quadlet.
## Boot persistence (Quadlet)
The unit lives at `dankclaw/open-webui/quadlet/open-webui.container`. On the host it installs to `~/.config/containers/systemd/open-webui.container`. The generator turns it into `open-webui.service`. Because the unit sets `[Install] WantedBy=default.target` and user linger is on, it starts at boot. `systemctl --user is-enabled open-webui.service` reports `generated` (the quadlet boot-start state).
@@ -292,6 +288,41 @@ carries base64 image data or a URL.
Failure modes: connection refused ⇒ OpenWebUI or ComfyUI is down; timeouts
on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
## Text-to-speech (frank-tts)
TTS is the fleet's one OpenAI-compatible speech endpoint —
`http://host.containers.internal:7772/v1` (`audio.tts.openai.api_base_url`
in the config DB, 2026-10-18), model `frank-tts`, voice `narrator1`. That
port is the rootless quadlet `frank-tts` (this repo:
`dankclaw/voiceservice/quadlet/frank-tts.container`
`scripts/frank-tts.env`), and its shim resolves `voice=<name>` against
`/home/dank/voices/<name>/reference.wav` — a name without a reference is a
400 listing the voices that exist, never a silent fallback.
**The 29-voice `af_* … bm_fable` list is NOT frank-tts.** That enum belongs
to **Kokoro.js running in the browser**: the deployed front end (0.11.4)
ships an in-page Kokoro implementation (chunk carrying a frozen voice table
`{af_heart:{name:"Heart",…}}` and `_validate_voice`, model
`onnx-community/Kokoro-82M-v1.0-ONNX`, plus a personal *Kokoro.js dtype*
setting). Its engine select takes `""` (Default) or `browser-kokoro`; when
`browser-kokoro` is chosen the page synthesizes locally and rejects any
voice outside its fixed pack — so `narrator1` fails with
`Voice: "narrator1" not found. Should be one of: af_heart, …` **without a
single request reaching the server**. Personal audio settings are
browser: no `user.settings` row and no container env carries an audio
override, yet the toast still appears ⇒ the choice lives in that browser's
own storage. Fix in the UI: Settings → Audio → TTS Engine → **Default**,
which hands synthesis to the admin config (frank-tts); picking `af_heart`
instead keeps Kokoro, and Kokoro cannot clone — fixed pack only.
Verified 2026-10-18 from inside the container with the exact OpenWebUI body
(`{"model":"frank-tts","input":…,"voice":"narrator1",
"response_format":"mp3","speed":1.0}`) → **200, `audio/mpeg`, 7,629 B**,
while `podman logs open-webui` recorded **zero** `audio/speech` requests for
the failing click — that contrast is the instrument: a server-path failure
leaves a log line and a frank-tts error, a `browser-kokoro` failure leaves
neither.
## Retired instances
- The compose `open-webui` service (port 3000) was never run: 3000 is Gitea's HTTP port, so it could not bind.
@@ -302,8 +333,8 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
- The image tracks upstream `main`; pin a digest in the quadlet unit if reproducible rollback matters more than tracking. (The old "CPU-only host" framing is stale: this APU runs the gfx1151 ROCm build — see [[ComfyUI]].)
- The default admin account is created on first login; secure it immediately.
- If the backend is loading (3–8 minute model load), OpenWebUI shows connection errors until the model is ready.
- The on-host LLM services (Dankclaw `:7770`, `:7771`) are reached via the **host LAN IP `192.168.0.143`**, which is exactly what the deployed unit does (`open-webui.container:32`). The older "use `host.containers.internal`" advice is retired for this container: it is a slirp4netns/pasta-only name and the unit has been on the pinned bridge `systemd-open-webui` since 2026-08-22, where that name does not resolve (`open-webui.network:15-19`). Both endpoints are published on all interfaces today (`ss -ltn` → `*:7770`, `0.0.0.0:7771`; `curl http://192.168.0.143:7771/health` → 200, 2026-10-18). The Dankserver backend is a different host, reached via its LAN IP `192.168.0.235:7770`.
- If the backend is loading, OpenWebUI shows connection errors until the endpoint is ready; each endpoint's boot budget is on its own page ([[frank-dragoon]], [[frank-zealot]], [[frank-archon]]).
- Repeated from *The LLM backends*: the on-host endpoints are reached via the **host LAN IP**, never `host.containers.internal`.
## History
@@ -321,7 +352,7 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three
id-keyed objects had to move in the same window as the alias flip:
`model.id`+`name` (`frank-zealot` → `frank-zealot`, `params`
`{"context":{"length":200704}}` and the `meta.description` kept),
`{"context":{"length":200704}}` and the `meta.description` kept)
`access_grant` (`resource_type='model'`, `resource_id`, principal
`user`/`*`, permission `read`), and `openai.api_configs` per-backend
`model_ids` (index 2 = `:7771` is the enabled family backend; indices 0/1
@@ -349,6 +380,7 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
## See Also
- [[Dankclaw Stack]]
- [[Frank TTS]] — the speech endpoint behind *Text-to-speech (frank-tts)*
- [[Hermes]] — the agent OpenWebUI can talk to
- [[SearXNG]] — the search engine on this host
- [LLM knowledge base](../LLM/index.md) — the vLLM and llama.cpp endpoints
- [[frank-dragoon]] / [[frank-zealot]] / [[frank-archon]] — the endpoints behind the picker
+1 -1
View File
@@ -88,7 +88,7 @@ never enter this repo. See [[Gitea]] and [[Authentik]].
## Boot persistence (Quadlet)
Both units carry `Restart=always` (`gitea-postgres.container:28`,
Both units carry `Restart=always` (`gitea-postgres.container:28`
`authentik-postgres.container:38`) and `[Install] WantedBy=default.target`
(`gitea-postgres.container:31`, `authentik-postgres.container:41-42`), are installed
under `~/.config/containers/systemd/`, and report `generated` / `active` with
@@ -143,7 +143,7 @@ engines:
disabled: false
```
Key per-engine keys (each overwrites the engine-module default; see
Key per-engine keys (each overwrites the engine-module default
`settings_engines.html`):
| Key | Effect |
@@ -164,7 +164,7 @@ Key per-engine keys (each overwrites the engine-module default; see
*different* category. When the primary `google` gets CAPTCHA'd it suspends, but
`google2` still returns results, and vice-versa.
**Parser caveat (read before fiddling with `parsers`):** the modern `google`,
**Parser caveat (read before fiddling with `parsers`):** the modern `google`
`duckduckgo`, and `brave` engines ship a **custom `response()`** with hardcoded
XPath and *ignore* the generic `parsers`/`p_parsers` table. So `parsers:
[scrapy_html]` under `name: google` mostly does nothing. To make Google
@@ -376,14 +376,14 @@ done
**2. `/config` REST endpoint** lists every engine + enabled/disabled +
categories — an inventory of what is live. It answers 200 on this instance today
(`curl http://127.0.0.1:8081/config`, 2026-10-18; each entry carries `enabled`,
(`curl http://127.0.0.1:8081/config`, 2026-10-18; each entry carries `enabled`
`categories`, `timeout`, `shortcut`, and it is the source of the engine counts on
[[SearXNG]]). `?engines=<name>` isolates a single
provider; `/debug` adds engine diagnostics.
**3. `check.searx.space`** — the hosted official checker probes every engine of
your instance with a **headless Firefox** (catches Cloudflare/CAPTCHA blocks a
plain HTTP probe misses) and posts results on searx.space. Add your instance via
plain HTTP probe misses) and posts results on searx.space. Add your instance
a GitHub issue in `searxng/searx-instances`, then allow `check.searx.space` IPs
to reach the SearXNG port.
@@ -417,10 +417,10 @@ misleading).
already replaced — checked 2026-10-18; see [[SearXNG]]). It is
also immune to the shared-VPN-exit-IP blocking this page is about: it
authenticates, so reputation of the exit IP is irrelevant.
5. **Verify first** — `/json` + `?engines=google` and the searx-checker to see
5. **Verify first** — `/json` + `?engines=google` and the searx-checker to
exactly which engines are down and why. Confirm the running SearXNG version
(check whether `parsers`/`rate_limit` exist in the pinned build).
6. **Use conservative request tuning** — Dankclaw now allows 5 seconds per
6. **Use conservative request tuning** — Dankclaw now allows 5 seconds
upstream request (`shared/searxng/settings.yml:26`), caps it at 15 seconds
(`:27`), and retries once (`:32`). This addresses
modest VPN latency without multiplying 429/CAPTCHA traffic. Adjust only
+16 -17
View File
@@ -108,7 +108,7 @@ engines:
disabled: true
```
The block above is the repo file (`shared/searxng/settings.yml`, 21 engine stanzas);
The block above is the repo file (`shared/searxng/settings.yml`, 21 engine stanzas)
the deployed render at `/opt/searxng/config/settings.yml` differs only in the baked
`secret_key`, `base_url` and `KAGI_API_KEY` values, and the running engine set is
censused below.
@@ -154,7 +154,7 @@ what keeps the `trusted_proxies` entry meaningful.
## Unblocking Google & Search Providers
The current request policy allows a modest amount of VPN latency (`5.0s` per
The current request policy allows a modest amount of VPN latency (`5.0s`
request, `15.0s` maximum — `shared/searxng/settings.yml:26-27`) while limiting
retries to one (`:32`). This can recover slow
providers, but it cannot fix an upstream CAPTCHA or 429 caused by the shared
@@ -169,10 +169,10 @@ upstream request volume. SearXNG **suspends** offending engines — which is wha
up as "Google went unavailable". The default suspension is shorter than the folklore:
see the table in [[SearXNG Providers]]. This is upstream, not local.
Fixes (cheapest first): tune the `outgoing` block (`request_timeout`,
Fixes (cheapest first): tune the `outgoing` block (`request_timeout`
`retry_on_http_error`, `suspended_times`), duplicate the `google` engine under a
fresh `name` so a CAPTCHA ban on one leaves the other returning results, and —
if Google still re-blocks — route the Google engines at a clean-IP proxy via
if Google still re-blocks — route the Google engines at a clean-IP proxy
per-engine `proxies` in `settings.yml`. See [[SearXNG Providers]] for the full
root cause, the suspend-model table, the `engines:` config, and the
`/json` verification recipe.
@@ -212,17 +212,16 @@ root cause, the suspend-model table, the `engines:` config, and the
(`searxng.container:21`), and that is one of the variables SearXNG substitutes at
load, so both paths carry the same LAN-IP value. Flagged, not fixed here: it is a
config change, out of scope for docs.
- Keep the valkey cache container on the network SearXNG can actually reach — see
- Keep the valkey cache container on the network SearXNG can actually reach —
[[Valkey]] Gotchas: SearXNG itself has no network of its own (it borrows Gluetun's
netns), so the network that matters is the one Gluetun and `searxng-valkey` share.
## Kagi (paid API engine)
Added 2026-09-26 (user-ordered) and still enabled: the running build ships a native
`kagi` engine module (`searx/engines/kagi.py` at the pinned commit `12f8b6515`),
`kagi` engine module (`searx/engines/kagi.py` at the pinned commit `12f8b6515`)
which sets `Authorization: Bearer <key>` (`kagi.py:103`) and issues
`POST https://kagi.com/api/v1/search` (`base_url` + `/api/v1/search`, `kagi.py:76,104,131`),
`POST https://kagi.com/api/v1/search` (`base_url` + `/api/v1/search`, `kagi.py:76,104,131`)
and declares `require_api_key: True` (`kagi.py:63`).
- **Enabled**: `weight: 2` alongside bing/duckduckgo, categories
@@ -231,10 +230,10 @@ and declares `require_api_key: True` (`kagi.py:63`).
`enabled: true`, categories `general, web` (2026-10-18).
- **Cost control**: `paging: false` — exactly one API credit per aggregated
query; `timeout: 8.0` so a slow paid query is not wasted (scrapers use 5.0)
(`shared/searxng/settings.yml:74-77`; upstream paging default is `True`,
(`shared/searxng/settings.yml:74-77`; upstream paging default is `True`
`kagi.py:67`, so the `false` here is a real override).
- **Key handling**: the repo file keeps `api_key: "${KAGI_API_KEY}"`;
`deploy-remote.sh` bakes the real key into the live settings.yml via
- **Key handling**: the repo file keeps `api_key: "${KAGI_API_KEY}"`
`deploy-remote.sh` bakes the real key into the live settings.yml
`scripts/searxng-render-settings.py`, reading `/opt/searxng/.env`
(owner `dank`, mode 0600 — `ls -l /opt/searxng/.env` 2026-10-18; its key names are
the WireGuard set plus `SEARXNG_SECRET`, `SEARXNG_BASE_URL`, `KAGI_API_KEY`).
@@ -262,15 +261,15 @@ and declares `require_api_key: True` (`kagi.py:63`).
- 262 engines in the build, **75 enabled**, categories of the enabled set led by
`other` (12), `it` (11), `general` (9), `videos` (8), `stock images` (7).
- Enabled and matching the policy page: `bing`, `duckduckgo`, `wikipedia`, `kagi`,
`github`, `mdn`, `stackoverflow`, `askubuntu`, `superuser`, `docker hub`, `pypi`,
`arxiv`, `pubmed`, `semantic scholar`, `reuters`, `bing news`, `duckduckgo news`,
`wikinews`, `bing images`, `duckduckgo images/videos/news`, `wikicommons.*`,
- Enabled and matching the policy page: `bing`, `duckduckgo`, `wikipedia`, `kagi`
`github`, `mdn`, `stackoverflow`, `askubuntu`, `superuser`, `docker hub`, `pypi`
`arxiv`, `pubmed`, `semantic scholar`, `reuters`, `bing news`, `duckduckgo news`
`wikinews`, `bing images`, `duckduckgo images/videos/news`, `wikicommons.*`
`youtube`, `vimeo`, `dailymotion`, `bing videos`.
- Disabled, as configured: `google`, `google cse`, `google news`, `google scholar`,
- Disabled, as configured: `google`, `google cse`, `google news`, `google scholar`
`brave`, `brave.images`, `brave.news`, `brave.videos`, `qwant`.
- The three `startpage` stanzas in `shared/searxng/settings.yml:113-124` target
engines that **no longer exist in this build** — `/config` lists no `startpage`,
engines that **no longer exist in this build** — `/config` lists no `startpage`
`startpage images` or `startpage news`, so those overrides are no-ops kept as
intent. *(labelled 2026-10-18)*
+3 -3
View File
@@ -1,6 +1,6 @@
---
type: reference
description: "Seerr is the unified request manager for the *arr stack (the maintained `seerr-team/seerr` successor to the archived Overseerr/Jellyseerr line)."
description: "The request manager for the *arr stack — the maintained successor to the archived Overseerr line."
tags:
- dankclaw
- seerr
@@ -40,8 +40,8 @@ the next outpost reload), and the missing app bounced post-login to
authentik's own user UI instead of Seerr. Both backfilled live: the three
standard flows copied from a known-good provider, plus an application
(`seerr-guard`) linked to the provider. Verified by the hardened SSO sweep
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik]] for the two
provisioning rules this incident established.
(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik Outpost Reference]] for the
two provisioning rules this incident established.
## *arr wiring
@@ -1,7 +1,7 @@
---
type: configuration
type: reference
title: Dankclaw Swap Configuration
description: The host swapfile on Dankclaw — 64 GiB at /swap.img on ext4, fstab-driven via the generated swap.img.swap unit. Covers why it was raised from 8 GiB, the safe resize ordering that avoids an OOM on a host whose MemAvailable is inflated by pinned model weights, and the swappiness caveat.
description: "The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat."
tags:
- dankclaw
- host
@@ -36,14 +36,14 @@ it is not invisible.
## Why 64 GiB
Dankclaw is shared. The 2026-09-12 decision measured `frank-dragoon` alone holding
**103.5 GiB of the 121.2 GiB** host (`68.0` weights + `14.4` KV pool + `21.1`
working), leaving roughly **7.1 GiB** for the then-28 co-tenant containers whose
working set was ~4.3 GiB. That is a decision-time reading, not today's: on
**103.5 GiB of the 121.2 GiB** host, leaving roughly **7.1 GiB** for the then-28
co-tenant containers whose working set was ~4.3 GiB. How that total is split
between weights, KV pool and working memory is the engine's own ledger — it lives
on [[frank-dragoon]]. That is a decision-time reading, not today's: on
2026-10-18 the host reports **91 GiB used / 29 GiB available of 121 GiB**
(`free -h`), 30 containers are up (`podman ps -q | wc -l`), and the
`frank-dragoon-halogen` container accounts for **70.33 GiB**
(`podman stats --no-stream`, same day) on the current flash-server payload —
a different model and KV shape than the one measured in September.
(`podman stats --no-stream`, same day).
The engine itself warned at the time that when the headroom runs out,
frank-dragoon and its neighbours "can stall for minutes at a time at full CPU with
no disk activity and no output". Swap is the valve that turns an OOM kill into slow
@@ -59,7 +59,7 @@ like this, **never `swapoff` before the replacement is online.** The old 8 GiB
held 4 GiB of live pages, and `MemAvailable` read 73 GiB while only ~5 GiB was
genuinely free — the pinned model weights are counted as reclaimable page
cache, so the headline number is a lie by roughly the size of the model
(see [[halogen-flash-server]]). Swapping off into that apparent space, with
(see [[frank-dragoon]]). Swapping off into that apparent space, with
nothing else to receive the pages, is how you invite the OOM killer.
The ordering that avoids it — create and enable the new swap **first**, so the
@@ -113,10 +113,10 @@ frank-dragoon `health=200` throughout.
## Caveats
- **Swap cannot help the pinned weights.** frank-dragoon's ~68 GiB of weights are
held deliberately; paging them is a stall, not a relief. More swap does not
make the doubled KV pool affordable — see the reserve/pool ceiling in
[[halogen-flash-server]].
- **Swap cannot help the pinned weights.** frank-dragoon's weights are pinned
deliberately; paging them is a stall, not a relief. More swap does not make a
larger KV pool affordable — see the reserve/pool ceiling in
[[frank-dragoon]].
- **`vm.swappiness=60` is untested against this workload.** Under real
pressure the kernel will happily page anonymous memory, and frank-dragoon's KV
pool and working set *are* anonymous. Swapping the KV pool would be far worse
@@ -144,13 +144,13 @@ frank-dragoon `health=200` throughout.
The previous 8 GiB `/swap.img` (dated 2026-07-21) was staged, not deleted, at
`dankclaw:~/.AGENTTRASH/20260912-swapfile-8g/swap.img` (8.1 GiB). **That staging
directory no longer exists** — `ls /home/dank/.AGENTTRASH` on 2026-10-18 lists only
`20261005-frank-small-ubatch-ab`, `20261006-frank-large-tuning`,
`20261007-openwebui-modelpicker`, `20261010-fleet-rename`, `20261017-bonsai-scratch`,
`20261005-frank-small-ubatch-ab`, `20261006-frank-large-tuning`
`20261007-openwebui-modelpicker`, `20261010-fleet-rename`, `20261017-bonsai-scratch`
`20261017-prism-build`, `20261017-v2abl-trial`, and there is no `/root/.AGENTTRASH`.
The old swapfile has therefore been purged; nothing on the host still holds it.
## See Also
- [[halogen-flash-server]] — the memory model that motivated this
- [[frank-dragoon]] — the memory model that motivated this
- [[Dankclaw System Specs]] — the Strix Halo box
- [[Dankclaw Improvements]] — other host-level changes
@@ -0,0 +1,261 @@
---
type: reference
description: "How a voice pack is built: the files every pack must carry, the source-material gate, adding a voice, engine-bump drift."
tags:
- dankclaw
- frank-tts
- voices
- tts
timestamp: 2026-10-18
---
# Voice Packs
A **voice pack** is the unit of identity for **frank-tts**: one directory under
`/home/dank/voices`, named by the voice, carrying the reference audio that makes
the name real plus the pinned engine's own speaker artifacts and a provenance
record. `voice=narrator1` on `POST /v1/audio/speech` resolves to
`/home/dank/voices/narrator1/reference.wav` and to nothing else — a name without
that file is a 400 that lists the names that do exist, because a silent fallback
voice is how a wrong-voice endpoint stays wrong for weeks.
**Host:** Dankclaw (192.168.0.143) — **Pack root:** `/home/dank/voices`, host
state that stays out of the repo, mounted read-only at `/voices` by the façade
unit — **Builder:** `scripts/build-voice-pack.sh` with tunables in
`scripts/voice-packs.env` — **Seam:** `tests/voice_pack_unit.bats` — **Engine:**
arm A `qwentts.cpp`, decided in .
Packs are **world-readable on purpose** (`drwxr-xr-x`, owner `dank`): the façade
container runs as uid 10000 and reads the mount from outside its own uid map.
## The required files
| File | What it is | Who reads it |
|------|------------|--------------|
| `reference.wav` | the crop: 24 kHz mono `pcm_s16le`, **18–25 s** of clean single-speaker audio | the façade (it is the existence gate for the name) and the engine's boot warmer, which base64s it and registers the voice with no transcript |
| `reference.spk` | the engine's x-vector: **8,192 B** = `enc_dim` 2048 × f32 | the engine, for the ICL/`--ref-spk` path |
| `reference.rvq` | the engine's RVQ code stream: **5,456–6,864 B**, 16 codebooks × T frames packed at 11 bits = 22 B per 12.5 Hz frame | the engine, for the ICL/`--ref-rvq` path |
| `manifest.json` | provenance: source path and sha256, crop start/end offsets, sample rate, duration, the tool and engine commit that made the artifacts, and the `voice` name | humans, and the builder itself on its second gate assertion |
`qwen-codec.log` rides along with the extractor's own record of the run; the
façade ignores anything it does not name and never deletes anything.
Division of enforcement, which is deliberate and not a gap: the **façade** gates
on `reference.wav` because that is the only engine-agnostic term (see the
voice-pack law in `scripts/frank-tts.env`); the **builder** is what refuses to
emit a pack without its engine artifacts; the **engine** registers a voice from
the WAV alone because its own `POST /v1/audio/voices` refuses a lone embedding —
it wants `wav_b64` or **both** `spk_b64` and `rvq_b64`. So the artifacts are in
the pack to make ICL a per-voice opt-in, not to boot it.
## How a pack is built
```bash
# everything except the extractor: gate, crop, probe, plan, the manifest it would write
scripts/build-voice-pack.sh --dry-run narrator1 /home/dank/voices
# the real thing, on the host that holds the engine build
scripts/build-voice-pack.sh narrator1 /home/dank/voices
```
Crops come from `scripts/voice-packs.env` — per voice: source path, offset,
duration, and the source's pinned sha256 — and any of them can be overridden on
the command line (`--source`, `--offset`, `--duration`, `--sha256`, `--tool`).
`--all` walks `VOICE_PACKS`. The builder assumes no GPU, no container, no fleet
endpoint and no network: `ffmpeg` cuts the crop, the pinned extractor makes the
artifacts, and both run `nice -n 19` so a build never competes with the text
endpoints on that box.
The extractor is `qwen-codec` in the `qwentts.cpp` tree at
`VOICE_PACK_ENGINE_COMMIT` (`5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`). Its
usage text, read out of that tree and not guessed:
```
Usage: qwen-codec --model <gguf> [-i <input>] [--talker <gguf>] [--format <fmt>]
Required:
--model <gguf> Codec GGUF (qwen-tokenizer-12hz-*.gguf)
Optional:
--device <name> Compute device: CUDA0, Vulkan0, MTL0, CPU...
(default: best available)
-i <path> Input. WAV -> encode, .rvq -> decode
--talker <gguf> Talker GGUF (Base only). Encode also extracts the speaker
embedding and writes it next to the .rvq as a .spk file
--format <fmt> WAV output format: wav16, wav24, wav32 (default: wav16)
Output is auto-named next to input : clip.wav -> clip.rvq, clip.rvq -> clip.wav.
```
Three consequences shape the builder. **There is no `-o`**, so the pack's WAV is
staged under its final stem and the artifacts land beside it with nothing to
rename. **`--talker` must be a _Base_ talker** — the customvoice and voicedesign
GGUFs carry no speaker encoder and the tool says `has no speaker encoder (Base
only)`. **Encode truncates to the hop boundary**, so the `.rvq` size is arithmetic,
not chance: `22 × floor(samples / 1920)` bytes, which is where the recorded
5,456–6,864 B window comes from and why the **effective crop window is
19.84–25 s** even though the WAV law says 18–25 s. A crop shorter than that is
refused before the extractor runs, with the number in the message.
The build is staged and published, never assembled in place: `reference.wav` is
the file the façade reads, so it moves into position **last**, after the
artifacts and the manifest. A failed extractor therefore leaves a pack directory
that is not a voice. Nothing is ever deleted — an existing pack is refused unless
`--force`, and `--force` moves the whole old directory to
`/home/dank/voices/.AGENTTRASH/<voice>-<UTC>/` and builds fresh beside it.
Measured on the shipped `narrator1` crop (2026-10-18, `--dry-run` against the
real source): 98.739–118.636 s → `955,378 B` of 24 kHz mono `pcm_s16le`, 19.897
s, planned `.rvq` 5,456 B — the same byte counts the trial produced by hand, and
the extractor itself costs 1.8–2.1 s and 1.3–1.5 GiB peak per voice.
## The 10-second gate
Project-wide source-material invariant: **the first 10 seconds of any sample
file used to generate voices is forbidden material.** An offset that starts
inside it is refused with exit 1 and a message that names the invariant, before
a directory is created:
```
ERROR: source-material invariant violated: the crop must start past the first
10 s of the source file, and 'narrator1' asks for 4.5 s into … — nothing was
written.
```
The same gate is then asserted **from the manifest** that was just written —
`.crop.offset_seconds >= .invariant.forbidden_source_seconds`, plus the recorded
sha256 against the source it claims — so a pack cannot reach disk carrying a
forbidden crop even if the offsets in the env file are edited into nonsense.
`10` is not a knob: `scripts/voice-packs.env` carries the offsets, the sample
rate, the windows and the engine facts, but the forbidden region is a constant in
the builder. `tests/voice_pack_unit.bats` pins both halves (a refusal that
creates nothing, and one legal crop that builds the whole pack) and applies the
invariant to every crop in the tracked pack list.
## Adding a fourth voice
1. Pick the source file and hash it: `sha256sum /path/to/sample.wav`. A crop
offset is a claim about one exact file, so the digest goes in the env with it.
2. Find a speech-dense stretch **past 10 s**. The trial's recipe, applied once:
`ffmpeg -ss 60 -t 600 -i SRC -af silencedetect=noise=-35dB:d=0.6 -f null -`
build the runs of non-silence, keep runs ≥ 18 s, take the longest (earliest
on ties), and cap at 25 s. Aim for 20–25 s: shorter than 19.84 s and the
`.rvq` falls under the floor.
3. Add the four lines to `scripts/voice-packs.env` and the name to
`VOICE_PACKS`:
```
VOICE_PACKS="narrator1 narrator2 narrator3 narrator4"
VOICE_PACK_NARRATOR4_SOURCE=/home/dank/voice-sources/audiosample4.m4b
VOICE_PACK_NARRATOR4_OFFSET=142.31
VOICE_PACK_NARRATOR4_DURATION=24
VOICE_PACK_NARRATOR4_SHA256=<64 hex>
```
The name must be lowercase `[a-z0-9._-]`: the façade matches the directory
name verbatim while `tts-server` lowercases names internally, so an uppercase
name would be reported one way by the façade and another by the engine. The
builder refuses one anyway.
4. `scripts/build-voice-pack.sh --dry-run narrator4 /home/dank/voices`, read the
plan, then drop `--dry-run`.
5. `./tests/run.sh` (the pack-list invariant test covers the new row for free).
6. Register it with the running engine. The engine's voice registry is
memory-only and warm-on-boot, but a restart re-decodes **every** voice (6–8 s
each), so adding one pack uses the hook instead:
```bash
podman exec frank-tts-engine /app/warm-voices.sh
curl -s localhost:7772/v1/audio/voices # the façade's list, not the engine's
```
The hook walks `/voices` and registers **every** pack that has a
`reference.wav`, on every run — the engine replaces a name it already holds
(`tools/tts-server.cpp:148` at the pinned commit: *"Re-registering a name
replaces the previous entry."*) — then synthesizes twice per voice and logs
`voice <name> cold=…ms warm=…ms` to the unit's journal. The two numbers should
be close; see law 1 of [[Frank TTS Engine]] for what they measure. The rootless
unit restart — `systemctl --user restart frank-tts-engine.service` — is for when
the unit or its env changed, not for a new pack.
## When the extractor's output size changes on an engine bump
The builder treats the two artifact sizes as properties of the pinned engine, so
an engine bump that changes them **fails the build**: `reference.spk is N B,
expected 8192 B` or `reference.rvq is N B, expected …`. That is the failure being
loud, not a bug. Work through it in order:
1. Re-read the tool's own numbers rather than trusting this page: `qwen-codec`
prints `SpeakerEncoder] Loaded: enc_dim=…` and `Pipeline] Ready: hop … , …
codebooks @ … Hz` on every run, and the pinned tree's
`tools/qwen-codec.cpp` documents the packing. `enc_dim × 4` is the `.spk`
`codebooks × code_bits / 8` bytes per frame × frames is the `.rvq`.
2. Decide whether the new number is intended. A changed `enc_dim` means the old
`.spk` files are unreadable by the new engine, so **every** pack must be
rebuilt, not just the failing one.
3. Update the four arithmetic lines in `scripts/voice-packs.env`
(`VOICE_PACK_SPK_BYTES`, `VOICE_PACK_RVQ_MIN_BYTES`, `VOICE_PACK_RVQ_MAX_BYTES`
and `VOICE_PACK_RVQ_BYTES_PER_FRAME` if the codebook packing changed) **and**
`VOICE_PACK_ENGINE_COMMIT`, `VOICE_PACK_TOOL`, `VOICE_PACK_CODEC_MODEL`
`VOICE_PACK_TALKER_MODEL` to the new build. An env diff that moves the sizes
without moving the commit is the mistake to avoid.
4. Rebuild all three packs with `--force` (the retired directories keep the old
bytes readable beside their replacements until a human clears them), then
re-run the warm hook: `podman exec frank-tts-engine /app/warm-voices.sh`. That
hook registers **every** pack it finds on every run, and the engine replaces a
name it already holds — `tools/tts-server.cpp:148` at the pinned commit says so
in its own words: *"Re-registering a name replaces the previous entry."* So the
rebuilt `reference.wav` is what the running engine speaks afterwards, and the
hook's cold/warm pass is what re-snapshots it. A restart
(`systemctl --user restart frank-tts-engine.service`) is what an ENGINE bump
needs — the registry is memory-only and lives in that process — and the unit
runs the same hook on every start anyway, so a restart is never worse, only
slower.
5. `./tests/voice_pack_unit.bats` — the stub in that seam reproduces the packing
law, so it needs the same arithmetic update as the env file. The seam's
engine-bump test is exactly this scenario with a wrong `.spk` size, and its
assertion — no `reference.wav`, no manifest — is the property that matters.
## Host state (census 2026-10-18, read-only)
- `/home/dank/voices/` holds `narrator1`, `narrator2`, `narrator3`, each with
`reference.wav`, a `reference16k.wav`, and a `manifest.json` **in the trial's
own schema** (`kind: voice-reference-pack`, a silence-scan record). They carry
**no** `reference.spk` / `reference.rvq`: the trial's artifacts live in its own
work directory (`/home/dank/voice-audition/qwentts/voices/`, and only for
narrator1 and narrator2). The builder's first real run is therefore a
migration, and it should be a `--force` run per voice so the old directories
are retired rather than left half-updated.
- The three source files (`/home/blank/audiosample.mp3`
`audiosample2.m4b`, `audiosample3.m4b`, digests pinned in
`scripts/voice-packs.env`) live on the workstation, not on Dankclaw; the
builder reads them from `$VOICE_PACK_SOURCE_DIR=/home/dank/voice-sources`, so
they have to be copied there once before a build.
- `VOICE_PACK_TOOL` points at the trial's build tree
(`/home/dank/voice-audition/qwentts/qwentts.cpp/build/qwen-codec`, checked out
at exactly `5916bf7c`), and the two GGUFs live at
`/home/dank/models/Qwen3-TTS-GGUF/`, the engine slice's fetch target — that
path and its fetch state belong to [[Frank TTS Engine]].
- Whether the façade (7772) and the engine (8880) are up and serving is deploy
state on [[Frank TTS]] / [[Frank TTS Engine]] — read it there, not from this
census.
## Known gaps, on purpose
- **No transcript in the manifest.** ICL wants a `ref_text` that matches the
crop, and the trial's machine transcripts came back wrong ("Demon Souls" for
*Dark Souls*). Transcribing on the host is not a fix, so the pack carries none
and the engine registers the WAV-only voice, which is what its boot warmer
does anyway. When ICL becomes a per-voice opt-in, the curated transcript
belongs in this manifest — a human-supplied field, not something the builder
can derive.
- **No 16 kHz variant.** The packs the trial left behind carry
`reference16k.wav` for the `llama-tts-cli` rollback engine; arm A is native
24 kHz, so the builder emits one reference. `--force` retires the whole
directory including that variant, so if the rollback engine is ever revived:
`ffmpeg -i reference.wav -ar 16000 -c:a pcm_s16le reference16k.wav`.
- **No loudness pass.** The trial's pipeline also measured and normalised
(two-pass `loudnorm`, `astats`, `ebur128`, spectrograms) to a `-16 LUFS` spec.
None of that is in the pack law, which asks for 24 kHz mono inside the
duration window, so the builder does only that; a future engine that is
loudness-sensitive should get the pass as an explicit, documented step.
+1 -1
View File
@@ -80,7 +80,7 @@ image's own nginx (the desktop), so the content server runs on internal 8090.
**Auth model (calibre 9.x):** users live in a SQLite userdb whose `pw` column
is **plaintext** — the file is mode 600. The single admin account is
bootstrapped once by the deploy handler
(`deploy_calibre_server` in `scripts/deploy-remote.sh`) via
(`deploy_calibre_server` in `scripts/deploy-remote.sh`)
`calibre-server --manage-users -- add`, the password piped on stdin with
`printf` (a trailing newline from `echo` lands in the stored password and
breaks every login). The desktop's basic auth (`CUSTOM_USER`/`CUSTOM_PASSWORD`)
+15 -8
View File
@@ -83,7 +83,7 @@ The `iv` edition is no longer available for arm32/armv6. Use `dj` for maximum co
## Volumes
Live deployment: the whole LUKS array is served at URL `/w` via
Live deployment: the whole LUKS array is served at URL `/w`
`Volume=/mnt/entropy:/w:ro` in the quadlet unit ([[Entropy Volume]]) —
read-only **at the mount level**, so no containerized consumer can write into
the array through this path regardless of account permissions. The ac edition
@@ -111,7 +111,7 @@ convention; see [[#Command-line interface (ac edition)]] above.
```
`${COPAY_ACCOUNT}`/`${COPAY_PASSWORD}` are **never** written into the unit:
they live in the host-local `/home/dank/Containers/copyparty/env` (0600),
they live in the host-local `/home/dank/Containers/copyparty/env` (0600)
loaded via `[Service] EnvironmentFile=` so systemd expands them into
`Exec=-a ${COPAY_ACCOUNT}:${COPAY_PASSWORD}` at start. The password still
lands in the container argv — visible to anyone who can run
@@ -143,13 +143,13 @@ Copyparty v1.20.9+ supports WOPI (Web Application Open Platform Interface) for e
## Security Advisories
| Version | CVE | Issue |
|---------|-----|-------|
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | FTP upload outside volume |
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | dirkey/filekey confusion |
| v1.20.9 | CVE-2026-27948 / GHSA-62cr-6wp5-q43h | XSS via file upload |
| Version | Advisory | CVE | Issue |
|---------|----------|-----|-------|
| v1.20.19 | GHSA-phv8-wgjp-g4p9 | CVE-2026-104102 | FTP upload outside the defined volumes (limited path traversal) |
| v1.20.17 | GHSA-x5pq-m9p8-f4vx | CVE-2026-70657 | dirkey/filekey confusion |
| v1.20.9 | GHSA-62cr-6wp5-q43h | CVE-2026-27948 | reflected XSS via the `?setck=` URL parameter (not the upload path — the separate upload/DOM-XSS advisory is GHSA-m2jw-cj8v-937r) |
Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`.
Upgrade to v1.20.19+ to cover all known advisories. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`. Each row's patched version is the advisory's own `first_patched_version`.
## Gotchas
@@ -174,3 +174,10 @@ Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters
- [[Dankserver Stack]]
- [[Caddy]] — TLS termination and authentication in front
- [[ARR-Stack]] — Copyparty as a viewer alongside automated download management
## Citations
- GitHub Security Advisories, `9001/copyparty` —
https://github.com/9001/copyparty/security/advisories (accessed 2026-10-19)
each row's patched version read from `first_patched_version` in the
`https://api.github.com/advisories/<GHSA-id>` response.
@@ -1,6 +1,6 @@
---
type: reference
description: "Dankserver (192.168.0.235) runs the media and library stack."
description: "Service inventory and routing overview for the media and library host."
tags:
- dankserver
timestamp: 2026-09-03
@@ -54,5 +54,5 @@ The following services ran on Dankserver and are now retired from this stack:
## See Also
- [[index]] — repository index
- repository index
- [[Network Architecture]] — routing and proxy
+4 -4
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Jellyfin
description: Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and hardware-transcode status.
description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status."
tags: [jellyfin, dankserver, media-server]
timestamp: 2026-09-03T00:00:00Z
---
@@ -14,7 +14,7 @@ serving music, movies, and shows over LAN. It replaced the earlier research
brief; it runs alongside [[Plex]], which remains the primary server for some
clients.
**Host:** Dankserver (192.168.0.235) — **Access:** `http://192.168.0.235:8096`,
**Host:** Dankserver (192.168.0.235) — **Access:** `http://192.168.0.235:8096`
public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
[[Authentik]] outpost (post-auth route proxies `localhost:8096`).
@@ -67,7 +67,7 @@ user to verify the password.
- LDAP provider `Jellyfin LDAP` (Base DN `dc=franksdank,dc=cloud`, bind/search
mode `direct`) + application `Jellyfin`.
- LDAP outpost `Jellyfin LDAP Outpost` → container `authentik-outpost-ldap`
(quadlet `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`),
(quadlet `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`)
publishing `389`/`636` on the Dankclaw host for cross-host binds.
- Service account `svc-jellyfin-ldap` (DN
`cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud`).
@@ -80,7 +80,7 @@ user to verify the password.
**Jellyfin side** (plugin config, set via admin → Plugins → LDAP-Auth):
- Server `192.168.0.143:389` (plain LDAP over the trusted LAN), Base DN
`dc=franksdank,dc=cloud`, bind user = the service-account DN, search
attributes `uid,cn,mail,displayName`, uid attr `uid`, username attr `cn`,
attributes `uid,cn,mail,displayName`, uid attr `uid`, username attr `cn`
`CreateUsersFromLdap=true`, all folders enabled.
**Gotcha — the bind credential is a PASSWORD, not a token.** The LDAP bind is
+2 -2
View File
@@ -30,7 +30,7 @@ public traffic can be fronted by Caddy like every other vhost.
| Identity | fleet rootless model — container root maps to the `dank` user (1000:1000); the deployer chowns the data root on first cutover |
| Port | `32400/tcp` published on `0.0.0.0` — zero client reconfiguration |
| Network | `dankserver_appnet`, alias `plex`; Tautulli reaches it at `http://plex:32400` |
| GPU | none by decision — CPU decode only; the two R9700s stay dedicated to `frank-medium` |
| GPU | none by decision — CPU decode only; the two R9700s stay dedicated to [[frank-archon]] |
| Deploy | `./scripts/deploy.sh dankserver --service plex` |
| Docs | https://support.plex.tv |
@@ -60,7 +60,7 @@ root runs as the `dank` user, and the deployer adopts the data root with
## Integration points
- **Caddy:** Plex is exempt from the Anubis bot challenge
(plain-HTTP legacy clients cannot solve JS challenges) — see
(plain-HTTP legacy clients cannot solve JS challenges) —
[[Anubis]] "Exempt by construction".
- **Tautulli** ([[Tautulli]]) is the companion: it polls PMS on the appnet
for statistics and notifications and makes no sense without Plex.
@@ -1,6 +1,6 @@
---
type: reference
description: "This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping workflow for RomM on Dankserver."
description: "Metadata providers, cover art, ROM naming and the scraping workflow behind the RomM library."
tags:
- dankserver
- romm
+2 -2
View File
@@ -32,7 +32,7 @@ Anubis → Authentik chain ([[Caddy]] routes it post-auth to
## Architecture
Current RomM releases require MariaDB plus Redis (or Valkey) for the task
queue; this deployment runs **redis** (`REDIS_HOST=redis` in the live unit),
queue; this deployment runs **redis** (`REDIS_HOST=redis` in the live unit)
not valkey.
```mermaid
@@ -114,7 +114,7 @@ scan:
## Deployment (Quadlet)
Since 2026-09-01 the trio runs as repo-tracked quadlet units, deployed with
`./scripts/deploy.sh dankserver --service romm` (any of `romm`,
`./scripts/deploy.sh dankserver --service romm` (any of `romm`
`romm-mariadb`, `romm-redis` triggers the full ordered cutover):
- Units: `dankserver/romm/quadlet/{romm,romm-mariadb,romm-redis}.container`
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: MariaDB (RomM Backend)
description: MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and backup path.
description: "The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path."
tags: [mariadb, romm, dankserver]
timestamp: 2026-09-03T00:00:00Z
---
+1 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Redis (RomM Cache / Queue)
description: Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup path.
description: "The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path."
tags: [redis, romm, dankserver]
timestamp: 2026-09-03T00:00:00Z
---
@@ -1,7 +1,7 @@
---
type: reference
title: Noctalia and Niri
description: The noctalia v5 desktop shell on niri (workstation + blankpad) — config layers, remote IPC over ssh, custom-palette recolor, offline template rendering, and output rotation/pinning.
description: "The noctalia shell on niri across workstations: config layers, remote IPC, custom palettes, output rotation and pinning."
tags: [desktop, noctalia, niri, blankpad, workstation, ghostty]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -45,14 +45,14 @@ Verified: `msg status` returns live JSON state; unknown panel ids return an
error listing all valid panel ids (clipboard, control-center, launcher,
polkit, session, setup-wizard, test, tray-drawer, wallpaper).
Verified v5 msg verb grammar (keybinds-safe): `panel-toggle <id>`,
`session lock|suspend|lock-and-suspend|logout|reboot|shutdown`,
`volume-up [step]`, `volume-down`, `volume-mute`, `mic-mute`,
Verified v5 msg verb grammar (keybinds-safe): `panel-toggle <id>`
`session lock|suspend|lock-and-suspend|logout|reboot|shutdown`
`volume-up [step]`, `volume-down`, `volume-mute`, `mic-mute`
`media next|previous|playPause` (graceful no-op, rc 0, with no active
player), `brightness-up [target] [step]`, `brightness-down`,
`settings-open [context]`, `settings-toggle [context]`, `config-reload`,
`wallpaper-get [connector]`, `wallpaper-set <path>`, `wallpaper-next`,
`color-scheme-get|set`, `theme-mode-get`, `templates-apply`,
player), `brightness-up [target] [step]`, `brightness-down`
`settings-open [context]`, `settings-toggle [context]`, `config-reload`
`wallpaper-get [connector]`, `wallpaper-set <path>`, `wallpaper-next`
`color-scheme-get|set`, `theme-mode-get`, `templates-apply`
`screenshot-fullscreen`.
**Wallpaper directory semantics:** empty/unset `[wallpaper] directory`
@@ -81,9 +81,9 @@ Custom palettes live at `~/.config/noctalia/palettes/<Name>.json`
synced. Missing/invalid file → silent fallback to the builtin palette.
Palette JSON shape = the community file's shape: `dark`/`light` maps of
`mPrimary`, `mOnPrimary`, `mSecondary`, `mTertiary`, `mError`, `mSurface`,
`mPrimary`, `mOnPrimary`, `mSecondary`, `mTertiary`, `mError`, `mSurface`
`mSurfaceVariant`, `mOutline`, `mShadow`, `mHover` plus a `terminal` block
(`foreground`, `background`, `normal.{black..white}`, `bright.{…}`,
(`foreground`, `background`, `normal.{black..white}`, `bright.{…}`
`cursor`, `cursorText`, `selectionFg`, `selectionBg`). NOTE: `noctalia theme
-o` emits M3 snake_case tokens (`on_primary_container`, …) — that is the
*generator* format, not this one. Copy an existing community JSON and mutate
@@ -114,7 +114,7 @@ Procedure:
`noctalia msg color-scheme-set custom "Gruber Red"` → `ok`; confirm with
`color-scheme-get`.
5. Re-render every configured template: `noctalia msg templates-apply`.
Regenerates `~/.config/ghostty/themes/noctalia`,
Regenerates `~/.config/ghostty/themes/noctalia`
`~/.config/gtk-{3,4}.0/noctalia.css`, kcolorscheme, foot, wezterm,
starship, etc. Symlinked-into-dotfiles paths show up as modified files
worth committing.
@@ -216,7 +216,7 @@ Verified on blankpad, niri 26.04 (two PM1601 externals + eDP-1 laptop panel).
scale, transform, logical pos/size) and `niri --version`. Output **names are
connector names** (`eDP-1`, `DP-2`, `HDMI-A-1`), not EDID model strings —
the EDID strings printed before them are decoration. On blankpad the live
config splits via `include "./cfg/*.kdl"` from `~/.config/niri/config.kdl`,
config splits via `include "./cfg/*.kdl"` from `~/.config/niri/config.kdl`
and `cfg/display.kdl` is a **symlink into the dotfiles repo** — edit the
repo path (that is what gets committed); check with `ls -ld`.
@@ -1,7 +1,7 @@
---
type: reference
title: "*arr Stack"
description: "The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) — FrankBook-Pro only, every container on the gluetun WireGuard netns (all egress VPN): topology, wiring, auth posture, and gotchas."
description: "The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas."
tags: [arr-stack, prowlarr, sonarr, radarr, lidarr, readarr, qbittorrent, frankbook-pro, dankserver]
timestamp: 2026-09-04T00:00:00Z
---
@@ -38,7 +38,7 @@ live call at a time instead of guessing payload shapes:
- `'Base Url' must not be empty` when keys were camelCase → **PascalCase is
required** (`BaseUrl`, `ApiKey`) — schema keys, not the JSON wire keys.
- `DefaultMetadataProfileId must be > 0` → fetch real ids from
`GET /api/v1/metadataprofile` (rows like `(1,'Standard')`);
`GET /api/v1/metadataprofile` (rows like `(1,'Standard')`)
`GET /api/v1/qualitydefinition` returns the quality CATALOG — ids ≠
profiles.
- `targetType null` → configContract not registered in that build (this
@@ -87,7 +87,7 @@ settings values live in `prowlarr.db` → `Applications.Settings` JSON (the
pipeline hides a miss and you conclude "empty field" when the grep itself
failed.
4. Decision framing: when posture = gated-with-zero-users, present it as a
decision menu (None behind the front-door wall vs External/authentikarr),
decision menu (None behind the front-door wall vs External/authentikarr)
not a silent change — first-account-wins means whoever logs in first owns
the instance.
+7 -7
View File
@@ -7,10 +7,10 @@ Gluetun WireGuard netns. The host overview lives at [[FrankBook-Pro]]
## *arr Stack
- [[ARR-Stack]] — the full *arr stack: indexers, managers, and the download client
- [[sonarr]] — TV show management
- [[radarr]] — movie management
- [[lidarr]] — music management
- [[readarr]] — ebook/audiobook management
- [[prowlarr]] — indexer management
- [[qbittorrent]] — torrent download client
- [[ARR-Stack]] — The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas.
- [[sonarr]] — Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
- [[radarr]] — Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
- [[lidarr]] — Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
- [[readarr]] — The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client.
- [[prowlarr]] — Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas.
- [[qbittorrent]] — The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe.
@@ -1,7 +1,7 @@
---
type: reference
title: Prowlarr
description: Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and verified live gotchas.
description: "Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas."
tags: [prowlarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
@@ -70,7 +70,7 @@ http://prowlarr:9696/{indexerId}/download?link=...&file=... (torrent handoff)
The global API key is in `config.xml` (`<ApiKey>`); per-application keys are
only used by Prowlarr's own sync calls, NOT by the feeds. The arr download
clients authenticate to qBittorrent with the permanent WebUI password stored
in `/home/dank/.qbpass` (mode 600, owner `dank`) on Dankserver — see
in `/home/dank/.qbpass` (mode 600, owner `dank`) on Dankserver —
[[qBittorrent]] for why that password must be pinned in the conf file.
**Readarr caveat:** Readarr is abandoned upstream; its qBittorrent client
@@ -97,13 +97,13 @@ successor; the indexer row itself is healthy.
- Categories in the indexer row must be a subset of what the feed's `caps`
advertises; extras log "does not support the following categories" and
some feeds reject them.
- **A freshly created indexer row defaults to all-off.** If `enableRss`,
- **A freshly created indexer row defaults to all-off.** If `enableRss`
`enableAutomaticSearch`, or `enableInteractiveSearch` aren't set, the
app logs `Searching indexers for [X]. 0 active indexers` (Lidarr,
verified). Search proves TV + music chains 2026-08-30: Sonarr returned
76 live releases for an anime term, Lidarr 18 for YOASOBI – THE BOOK,
all via the Prowlarr-Nyaa feed.
- The v1 API collection is `/api/v1/applications` (not `/api/v1/app`);
- The v1 API collection is `/api/v1/applications` (not `/api/v1/app`)
GET responses carry `fields: []` — real settings values live in
`prowlarr.db` → `Applications.Settings` JSON (read-only inspection).
- Instance name/`InstanceName=Prowlarr` in `config.xml` does NOT prefix the
@@ -1,7 +1,7 @@
---
type: reference
title: qBittorrent
description: qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns, PBKDF2 WebUI credential recipe.
description: "The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe."
tags: [qbittorrent, arr-stack, frankbook-pro, gluetun]
timestamp: 2026-09-03T00:00:00Z
---
@@ -13,7 +13,7 @@ qBittorrent is the torrent download client for the *arr stack. The stack runs on
not on Dankserver. It receives download requests from Sonarr, Radarr, Lidarr,
and Readarr via category tagging.
**Host:** FrankBook-Pro (podman machine) — **Access:** WebUI on port `8080`,
**Host:** FrankBook-Pro (podman machine) — **Access:** WebUI on port `8080`
published by the [[Gluetun]] unit because qBittorrent shares that container's
network namespace (a netns-sharer publishes no ports of its own).
@@ -121,7 +121,7 @@ Mac shell export `PATH="/usr/local/bin:/opt/homebrew/bin:$PATH"` for
`podman machine ssh`.
**Access note (2026-09-21):** the Mac's login user is **dank**@192.168.0.112
(NOT core — `core` is the podman-machine VM user, reached only via
(NOT core — `core` is the podman-machine VM user, reached only
`podman machine ssh`). blankpad has key auth for `dank@` (bootstrapped
2026-09-21) and a `frankbook-pro` alias in `~/.ssh/config`. macOS sshd has a
low MaxAuthTries: when password-bootstrapping use `-o
+17 -9
View File
@@ -1,18 +1,19 @@
---
type: reference
title: Readarr
description: Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB download client.
description: "The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client."
tags: [readarr, arr-stack, frankbook-pro]
timestamp: 2026-09-03T00:00:00Z
---
# Readarr
> **RETIRED UPSTREAM (May 2026).** The Servarr Team retired Readarr: its
> metadata became unusable and the community migration to Open Library stalled.
> There is no upstream development. Alternatives: the Bookshelf community fork,
> or the Calibre content server. This instance still runs on a pinned image
> tag; see below.
> **RETIRED UPSTREAM (announced 2025-06-27).** The Servarr Team retired
> Readarr: its metadata became unusable and the community migration to Open
> Library stalled. There is no upstream development — the GitHub repository is
> archived and its last push is 2025-06-27. Alternatives: the Bookshelf
> community fork, or the Calibre content server. This instance still runs on a
> pinned image tag; see below.
Readarr manages ebooks and audiobooks in the *arr stack. The stack runs on
**FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
@@ -69,9 +70,9 @@ Restart=always
## Gotchas
- Readarr is abandoned upstream (retired May 2026; last line was 0.4.x before
the final tags). The pinned `amd64-1.0.980` tag exists precisely because
`:latest` does not exist for this image on LSIO.
- Readarr is abandoned upstream (retired 2025-06-27; last line was 0.4.x
before the final tags). The pinned `amd64-1.0.980` tag exists precisely
because `:latest` does not exist for this image on LSIO.
- Calibre integration is folder-based, not API-based. Readarr drops files into
the books root which [[Calibre-Web]] serves.
- Not all indexers carry all formats (epub, mobi, pdf). Goodreads/StoryGraph
@@ -81,3 +82,10 @@ Restart=always
- [[ARR-Stack]] — full *arr stack documentation
- [[Calibre-Web]] — ebook web frontend that serves Readarr downloads
## Citations
- Servarr wiki, *Readarr* (Retired) — "Announcement: Retirement of Readarr
2025-06-27" — https://wiki.servarr.com/readarr (accessed 2026-10-19)
- `Readarr/Readarr` on GitHub: `archived: true`, last push 2025-06-27 —
https://api.github.com/repos/Readarr/Readarr (accessed 2026-10-19)
@@ -1,6 +1,6 @@
---
type: reference
description: "Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to the host-side Signal relay — which sends it through..."
description: "Receives firing rules from Prometheus, groups and dedupes them, and forwards each group to the host-side Signal relay."
tags:
- infrastructure
timestamp: 2026-09-17
@@ -93,7 +93,7 @@ and `hermesadmin.` (app-enforced passwords, no public DNS for the first,
(`dankclaw/Caddyfile:241-251`), and the FrankBook-Pro *arr/qBittorrent vhosts
(`dankclaw/Caddyfile:260-352`). There is also no vhost at all for ComfyUI, so
it is exempt by absence — and therefore unauthenticated on the LAN at
`comfyui.franksdank.cloud:8188`. The guarded set is exactly `www.`, `search.`,
`comfyui.franksdank.cloud:8188`. The guarded set is exactly `www.`, `search.`
`grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `romm.`, `files.`
(`dankclaw/Caddyfile:283-423`).
@@ -140,7 +140,7 @@ it is exempt by absence — and therefore unauthenticated on the LAN at
instead of allowed, the browser's fetch throws and it renders its own
error page (iOS/macOS browsers phrase it as "the request failed and the
interceptors did not return an alternative response") instead of the
login flow. Verify with a JS-capable client end-to-end — mint via
login flow. Verify with a JS-capable client end-to-end — mint
`make-challenge`, solve, submit with browser-exact query params — before
declaring a challenge chain healthy.
7. **Single-file bind mount + git pull = stale policy.** The policy is
@@ -158,7 +158,7 @@ it is exempt by absence — and therefore unauthenticated on the LAN at
answers any request lacking `X-Real-Ip` with a 500 whose log line reads
`[misconfiguration] X-Real-Ip header is not set`. The real chain always
sets it (every guarded Caddy listener carries `header_up X-Real-Ip
{remote_host}` — the bats seam enforces the line), so users never see
{remote_host}` — the bats seam enforces the line), so users never
this. But any direct probe of `:8923` (scratch container, curl, a new
reverse-proxy hop) MUST set the header or it looks like a policy failure.
Scratch-verify with `-H "X-Real-Ip: 192.168.0.20"`.
@@ -0,0 +1,230 @@
---
type: reference
description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for the homelab's Authentik SSO."
tags:
- infrastructure
- authentik
timestamp: 2026-10-19
---
# Authentik Outpost Reference
Configuration lookup for the [[Authentik]] deployment: the live proxy-provider
inventory behind the guarded vhosts, the provisioning rules the embedded outpost
enforces, this fork's OAuth2 relying-party quirks, the LDAP provider objects, the
self-serve user-settings flow, and the live database shape. What the service is,
how it is deployed, wired and operated is [[Authentik]]; the three core
containers are [[Authentik Processes]].
## Live proxy-provider inventory
Live provider inventory (read from the `authentik-postgres` container with
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
members in `authentik_outposts_outpost_providers`), all with
`internal_host = http://caddy:8017`, all carrying the **same three flow
UUIDs** (`default-authentication-flow`
`default-provider-authorization-implicit-consent`
`default-provider-invalidation-flow`), and each backed by an Application
(the `slug` column below):
| Provider pk | name | external_host | app slug |
|---|---|---|---|
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
the lockstep rule in [[Authentik]] holds today. Two corrections against the 2026-09-02 table:
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
not proxy providers, so the proxy outpost does not deserialize them: pk 9
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
below rejects), and pk 14 `Open WebUI`
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
`external_host` (the runbook-recipe style) and bare origins both intercept
correctly — the outpost matches on hostname.
## Provisioning rules for proxy providers
Two provisioning rules, each verified live by an outage:
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
null wedges the embedded outpost's config fetch: the Rust side rejects the
whole config JSON (serde: `invalid type: null, expected a formatted UUID
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
forever, and **everything behind the outpost dies at the next outpost
reload** — including hosts whose providers were fine (verified 2026-09-02
when three providers provisioned without any flows took down every guarded
vhost after a server restart). Backfill all three flows (copy from pk 1:
`default-authentication-flow`
`default-provider-authorization-implicit-consent` (the implicit-consent
authorization flow, `default-invalidation-flow`) and the retry loop
recovers on its next 3-second cycle. The silent killer: while the outpost
process keeps its last good in-memory config, everything works — the
outage only surfaces on restart/reload.
2. **Every proxy provider needs an attached Application.** The outpost
resolves the post-login redirect through the provider's application; a
provider with no app completes the login (authentik session established)
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
the app. No error is logged anywhere — the SSO "works" and the app never
appears. Create the Application with `provider_id` set (the UI does this
automatically; raw API/ORM drives must do it explicitly).
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
implementation diverges from upstream in ways that bit us; all verified live:
- **Application row required.** `OAuth2Provider.objects.create()` does not
create the single-table-inheritance `Application` row; without it the flow
executor can never complete an authorize request. Create both rows.
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
contains `"authorization_code"` (enum value with underscore), the token
endpoint rejects every exchange with `invalid_request`.
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
`parse_token_request` see no `grant_type` → `unsupported_grant_type`
POST-body client secrets are not accepted — authenticate the confidential
client with `Authorization: Basic base64(client_id:client_secret)`.
- **Provider resolution is by `client_id` query param** on the global
`/application/o/authorize/` route; per-slug authorize paths do not exist in
this build. Token/userinfo live at `/application/o/token/` and
`/application/o/userinfo/?api=1`.
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
becomes the *expression source* instead of the context; every call raises a
swallowed `PropertyMappingExpressionException` and the claim is skipped.
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
provider FK (linkage lives in the `ProviderPropertyMapping` through model)
so `filter(provider=...)` always yields an empty queryset and tokens asked
to carry scopes get zero. Net effect: userinfo/introspection answers with
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
build until the evaluator is patched or upgraded. Clients consuming this
provider must key identity off `sub`.
- Scope mappings for this provider exist — `grafana-email-scope` and
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
the evaluator bug is fixed; no config change is needed then.
## LDAP provider objects (provisioned via `ak shell` ORM)
The LDAP outpost's own deployment — unit, image digest, published ports, token
env var — is in [[Authentik]]. The objects it serves:
Read from the live database (read-only `SELECT`s against the
`authentik-postgres` container, 2026-10-18):
| Object | Value |
|---|---|
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — an earlier revision's "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — an earlier revision recorded `intent=verification`; the live value is `app_password` |
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (an earlier revision said slug `jellyfin`, which does not exist) |
Hard-won rules (the first is now contested by the live database):
- **Token intent: contested, do not trust either value blindly.** An earlier
revision asserted that a bind token with `intent=app_password` is rejected by
the flow's password stage with `Invalid password` and only
`intent=verification` passes. The live token for `svc-jellyfin-ldap` is
`app_password` (`authentik_core_token`, 2026-10-18), so either the rule is
wrong or the current Jellyfin bind path is not that token. Nothing observable
settles it today: the LDAP outpost's retained log holds only its 5-minute
`initialized direct binder` / `Update providers` config cycles and no bind
activity *(unverified 2026-10-18: no bind was performed — reproducing it
requires the bind secret)*. Re-verify with a real bind before changing the
token's intent either way.
- **Search needs `search_full_directory`.** Without the RBAC grant the
outpost returns only the bind user itself — a `(memberOf=…)` filter then
matches zero entries and the plugin logs `Found no users matching X`.
The permission is object-level on the provider; grant it via an RBAC role
and retest.
- `ak shell` import path for the provider model in 2026.8:
`from authentik.providers.ldap.models import LDAPProvider`.
- (unverified 2026-10-18: importing it outside a configured Django shell fails
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
auth service for this check)*
## Self-serve user-settings flow (password change)
Users change their own account details through the `default-user-settings-flow`
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
target of the homepage footer's "account" link. It is a `stage_configuration`
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
user.
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
insertion order):
| order | stage | type | what it does |
|---|---|---|---|
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
**Password change is a prompt field, not a stage.** The `UserWriteStage`
writes whatever the preceding `PromptStage` collected: for the key `password`
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
is a `password` field on the `default-user-settings` prompt
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
`validate_password` runs `authenticate()` against the backends to check the
*current* password; it never sets a new one. (A `PasswordStage` was added on
2026-09-25 and removed the same day for exactly this reason.)
Two ORM gotchas: `Flow.stages.add()` fails on this through model
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
MTI), so create the `FlowStageBinding` row directly with an explicit `order`
and a prompt field is a `Prompt` row linked through `PromptStageField`
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
**A blank password must not set an empty password.** `set_password("")` would
make the account log in with no password, so a validation policy
(`default-user-settings-no-empty-password`, bound to the prompt
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
is `""` and allows it when set. Verified with the `PolicyEvaluator`
(empty → deny, set → allow, absent → allow) and end-to-end by driving
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
invalid).
## Live database shape
For live DB truth prefer the postgres container, and it works without any
credential — the local socket is trusted, so a read-only audit is just
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
before guessing: 2026.x renamed both tables and columns, and the models are
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
three `*_flow_id` columns; the proxy table is
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
table in 2026.8), the LDAP table is
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`
`authentik_core_application` attaches to a provider through `provider_id`
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
is the same story from the ORM side: `type` lives in the subclass tables.
Never select secret columns (`client_secret`, `key`, `password`) into a log.
## Cross-references
- [[Authentik]] — what the service is, its deployment, wiring, operations and
failure modes
- [[Authentik Processes]] — the postgres, server and worker containers
- [[Caddy]] — the reverse proxy carrying the `:8016`/`:8017` auth-chain listeners
- [[Jellyfin]] — consumes the LDAP outpost via its LDAP-Auth plugin
- [[Grafana]] — guarded vhost *and* OAuth2 relying party of this provider
@@ -0,0 +1,76 @@
---
type: reference
description: "The three core Authentik containers — database, server and worker — with their units, images, roles and probes."
tags:
- infrastructure
- authentik
- postgres
timestamp: 2026-10-19
---
# Authentik Processes
The three core containers of the [[Authentik]] stack on Dankclaw. The two
outpost units are not covered here — they live on [[Authentik]] and
[[Authentik Outpost Reference]]. Server and worker run the same image; the
`command` selects the role (`server` / `worker`). Data persists under
`/home/dank/Containers/authentik/` as bind mounts (repo policy — no named
volumes), moved from `/home/dank/authentik/` in the 2026-09-01 quadlet
standardization; the retired stubs this page replaces recorded the pre-move
paths `/home/dank/authentik/data` (server and worker) and
`/home/dank/authentik/postgres` (database).
## authentik-postgres
The PostgreSQL 16 database for Authentik — the only stateful store in the stack
(users, flows, sessions, events). Sessions, caching, and WebSockets moved to
PostgreSQL in 2025.10 (Redis removed).
- **Image:** `docker.io/library/postgres:16-alpine`
- **Data:** `/home/dank/Containers/authentik/postgres` (bind mount; no named
volumes)
- **Health:** `pg_isready -d authentik -U authentik`
- **Retired wording:** the stub called this a "Compose service … in
`dankclaw/compose.yaml`"; the compose stack is gone and the unit is a quadlet.
- The side-by-side with `gitea-postgres` (network, published ports, secret
source, boot persistence) is owned by [[Postgres]].
## authentik-server
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
It also serves the login page at `https://auth.franksdank.cloud`.
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container`
(`authentik-server.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: server`)
- **Port:** `9000` (published) — the outpost, the admin API and the login UI all
live here
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
worker)
- **Health:** recorded by the retired stub as
`http://localhost:9000/-/health/`; superseded — [[Authentik]] → Operations
owns the live probe.
- **Retired wording:** the stub described "the Caddy `forward_auth` vhosts"
proxying here; the chain Caddy actually runs is [[Authentik]] → Caddy
integration.
## authentik-worker
The Authentik worker — background tasks: event processing, provider
synchronization, scheduled flows, and certificate renewal.
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container`
(`authentik-worker.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: worker`)
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns
the data dir)
- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
server)
- **No port** — the worker has no HTTP endpoint.
## Cross-references
- [[Authentik]] — what the service is, its deployment, wiring, operations and
failure modes
- [[Authentik Outpost Reference]] — provider, flow and token configuration
- [[Postgres]] — both PostgreSQL clusters on Dankclaw, side by side
+46 -235
View File
@@ -15,26 +15,33 @@ application-level authentication with its proxy outpost. Not wrapped: Gitea,
OpenWebUI, Jellyfin (LDAP outpost), the *arr stack, Pi-hole (basicauth edge) —
they handle their own session. Grafana is the exception that is both: it is a
guarded vhost (`grafana-proxy`, pk 8, locked to Authentik SSO) *and* an OAuth2
relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
`dankclaw/Caddyfile:374-377`; live provider table below).
relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`
`dankclaw/Caddyfile:374-377`; live provider table in [[Authentik Outpost Reference]]).
- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`;
the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`,
- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`
the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`
2026-10-18)
- **Host:** dankclaw (`192.168.0.143`)
- **Name:** `auth.franksdank.cloud`
- **Stack: five quadlet units, no compose.** `authentik-postgres`,
`authentik-server`, `authentik-worker` (`deploy_authentik`,
- **Stack: five quadlet units, no compose.** `authentik-postgres`
`authentik-server`, `authentik-worker` (`deploy_authentik`
`scripts/deploy-remote.sh:980-1004`) plus two outposts — `authentik-outpost`
(the Rust **proxy** outpost that actually serves `/outpost.goauthentik.io/*`,
(the Rust **proxy** outpost that actually serves `/outpost.goauthentik.io/*`
`deploy_authentik_outpost`, `scripts/deploy-remote.sh:1006-1018`) and
`authentik-outpost-ldap` (`deploy_authentik_outpost_ldap`,
`authentik-outpost-ldap` (`deploy_authentik_outpost_ldap`
`scripts/deploy-remote.sh:1020-1032`). All five report `generated` +
`active`, `Restart=always`, `WantedBy=default.target`, with linger on
(`systemctl --user`, 2026-10-18). The "3 compose services" wording is the
pre-quadlet era, retired.
- **LDAP outpost:** serves LDAP for Jellyfin — see the section below.
**Split off from this page:** [[Authentik Processes]] owns the per-container
facts for `authentik-postgres`, `authentik-server` and `authentik-worker`
[[Authentik Outpost Reference]] owns the provider/flow/token lookup material —
the live proxy-provider inventory, the proxy-provider provisioning rules, the
OAuth2 relying-party quirks, the LDAP provider objects, the user-settings flow
design, and the live database shape.
## Why Authentik
The comparison that chose it (against Authelia, Keycloak, Zitadel, Ory) was
@@ -100,23 +107,10 @@ contract:
The chain terminates there — nothing dials back into Authentik from
`:8017`.
## The three compose services
## The stack's processes
| Service | Role | Image |
|---|---|---|
| `authentik-postgres` | PostgreSQL 16 (users, flows, sessions, events) | `docker.io/library/postgres:16-alpine` |
| `authentik-server` | Core + admin API + the embedded Caddy outpost (port 9000) | `ghcr.io/goauthentik/server:2026.8.3` |
| `authentik-worker` | Background tasks (event processing, provider sync, flows) | `ghcr.io/goauthentik/server:2026.8.3` |
- **Postgres** is the only stateful store. Sessions, caching, and WebSockets
moved to PostgreSQL in 2025.10 (Redis removed).
- **Server** and **worker** run the same image; the `command` selects the role
(`server` / `worker`).
- The worker runs as `0:0` (rootless: container root maps to the host user that
owns the data dir).
- Data persists under `/home/dank/Containers/authentik/{data,postgres}`
(bind mounts, repo policy — no named volumes; moved from `/home/dank/authentik/`
in the 2026-09-01 quadlet standardization).
Per-container facts — unit path, image and role, port, run-as, data directory,
health probe — are owned by [[Authentik Processes]].
## Caddy integration
@@ -142,110 +136,15 @@ routing table, and the Authentik proxy-provider list must be the same set of
hosts. A guarded host with no provider 404s at login time; a provider with no
DNS record breaks off-LAN clients.
Live provider inventory (read from the `authentik-postgres` container with
read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
members in `authentik_outposts_outpost_providers`), all with
`internal_host = http://caddy:8017`, all carrying the **same three flow
UUIDs** (`default-authentication-flow`,
`default-provider-authorization-implicit-consent`,
`default-provider-invalidation-flow`), and each backed by an Application
(the `slug` column below):
| Provider pk | name | external_host | app slug |
|---|---|---|---|
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
the lockstep rule holds today. Two corrections against the 2026-09-02 table:
**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
not proxy providers, so the proxy outpost does not deserialize them: pk 9
`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
below rejects), and pk 14 `Open WebUI`
(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
`external_host` (the runbook-recipe style) and bare origins both intercept
correctly — the outpost matches on hostname.
Two provisioning rules, each verified live by an outage:
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
null wedges the embedded outpost's config fetch: the Rust side rejects the
whole config JSON (serde: `invalid type: null, expected a formatted UUID
string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
forever, and **everything behind the outpost dies at the next outpost
reload** — including hosts whose providers were fine (verified 2026-09-02
when three providers provisioned without any flows took down every guarded
vhost after a server restart). Backfill all three flows (copy from pk 1:
`default-authentication-flow`,
`default-provider-authorization-implicit-consent` (the implicit-consent
authorization flow, `default-invalidation-flow`) and the retry loop
recovers on its next 3-second cycle. The silent killer: while the outpost
process keeps its last good in-memory config, everything works — the
outage only surfaces on restart/reload.
2. **Every proxy provider needs an attached Application.** The outpost
resolves the post-login redirect through the provider's application; a
provider with no app completes the login (authentik session established)
but bounces to authentik's own user UI (`/if/user/#/library`) instead of
the app. No error is logged anywhere — the SSO "works" and the app never
appears. Create the Application with `provider_id` set (the UI does this
automatically; raw API/ORM drives must do it explicitly).
## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
implementation diverges from upstream in ways that bit us; all verified live:
- **Application row required.** `OAuth2Provider.objects.create()` does not
create the single-table-inheritance `Application` row; without it the flow
executor can never complete an authorize request. Create both rows.
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
contains `"authorization_code"` (enum value with underscore), the token
endpoint rejects every exchange with `invalid_request`.
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
`parse_token_request` see no `grant_type` → `unsupported_grant_type`;
POST-body client secrets are not accepted — authenticate the confidential
client with `Authorization: Basic base64(client_id:client_secret)`.
- **Provider resolution is by `client_id` query param** on the global
`/application/o/authorize/` route; per-slug authorize paths do not exist in
this build. Token/userinfo live at `/application/o/token/` and
`/application/o/userinfo/?api=1`.
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
`PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
`BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
becomes the *expression source* instead of the context; every call raises a
swallowed `PropertyMappingExpressionException` and the claim is skipped.
Scope-grant filtering has the same class of bug: `ScopeMapping` has no
provider FK (linkage lives in the `ProviderPropertyMapping` through model),
so `filter(provider=...)` always yields an empty queryset and tokens asked
to carry scopes get zero. Net effect: userinfo/introspection answers with
`{sub}` only — email/groups/entitlements claims cannot be delivered by this
build until the evaluator is patched or upgraded. Clients consuming this
provider must key identity off `sub`.
- Scope mappings for this provider exist — `grafana-email-scope` and
`grafana-profile-scope` are both rows in `authentik_core_propertymapping`
(read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
the evaluator bug is fixed; no config change is needed then.
The live proxy-provider inventory (the per-vhost `external_host` → app mapping)
the two provisioning rules the outpost enforces, and this fork's OAuth2
relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
## LDAP outpost (Jellyfin auth, live 2026-09-01)
The second outpost — `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`
→ `authentik-outpost-ldap.service`, one of the five units in this stack —
serves LDAP for Jellyfin's LDAP-Auth plugin (`enabled`=generated, `active`,
serves LDAP for Jellyfin's LDAP-Auth plugin (`enabled`=generated, `active`
`Restart=always`, container up, 2026-10-18). The image is pinned by its **amd64
single-platform digest**
`ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9`
@@ -260,51 +159,16 @@ rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect
.digest'` and pin that digest.
- Ports **389** (ldap) and **636** (ldaps) publish on the LAN
(`PublishPort=389:3389`, `636:6636`,
(`PublishPort=389:3389`, `636:6636`
`dankclaw/authentik/quadlet/authentik-outpost-ldap.container:30-31`; `ss -ltnp`
shows `0.0.0.0:389` and `0.0.0.0:636`, 2026-10-18) and Dankserver can reach
them — a bare TCP connect from Dankserver to `192.168.0.143:389` succeeds
(2026-10-18). The outpost token travels in `AUTHENTIK_LDAP_OUTPOST_TOKEN`
from the host `dankclaw/.env` (git-ignored).
### Authentik-side objects (provisioned via `ak shell` ORM)
Read from the live database (read-only `SELECT`s against the
`authentik-postgres` container, 2026-10-18):
| Object | Value |
|---|---|
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — the page's earlier "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — the page previously recorded `intent=verification`; the live value is `app_password` |
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (the page previously said slug `jellyfin`, which does not exist) |
Hard-won rules (the first is now contested by the live database):
- **Token intent: contested, do not trust either value blindly.** This page
long asserted that a bind token with `intent=app_password` is rejected by the
flow's password stage with `Invalid password` and only `intent=verification`
passes. The live token for `svc-jellyfin-ldap` is `app_password`
(`authentik_core_token`, 2026-10-18), so either the rule is wrong or the
current Jellyfin bind path is not that token. Nothing observable settles it
today: the LDAP outpost's retained log holds only its 5-minute
`initialized direct binder` / `Update providers` config cycles and no bind
activity *(unverified 2026-10-18: no bind was performed — reproducing it
requires the bind secret)*. Re-verify with a real bind before changing the
token's intent either way.
- **Search needs `search_full_directory`.** Without the RBAC grant the
outpost returns only the bind user itself — a `(memberOf=…)` filter then
matches zero entries and the plugin logs `Found no users matching X`.
The permission is object-level on the provider; grant it via an RBAC role
and retest.
- `ak shell` import path for the provider model in 2026.8:
`from authentik.providers.ldap.models import LDAPProvider`.
*(unverified 2026-10-18: importing it outside a configured Django shell fails
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
auth service for this check)*
The objects behind it — LDAP provider, bind DN, service account and bind token,
RBAC grant, groups, application slug — and the hard-won bind rules are in
[[Authentik Outpost Reference]].
## Configuration (secrets)
@@ -332,7 +196,7 @@ every outpost session (see the ORM playbook).
core quadlets, starts postgres health-gated (`wait_container_healthy
authentik-postgres 120`), then the server (300 s health wait) and the worker.
The outposts are separate handlers: `--service authentik-outpost` and
`--service authentik-outpost-ldap` (`scripts/deploy-remote.sh:1006-1032`,
`--service authentik-outpost-ldap` (`scripts/deploy-remote.sh:1006-1032`
dispatch at `:1579-1588`).
2. Open `https://auth.franksdank.cloud` — the initial setup wizard runs. It
creates the superuser (email + password) and generates the DB schema.
@@ -340,12 +204,13 @@ every outpost session (see the ORM playbook).
Applications** → add an application + a **Proxy** provider whose
`external_host` is exactly the browser-facing origin
(`https://<host>`), `internal_host = http://caddy:8017`, and **all three**
flows set (see the provisioning pitfall above). Attach the provider to the
flows set (see the provisioning rules in
[[Authentik Outpost Reference]]). Attach the provider to the
embedded outpost.
4. For apps with native OIDC: add an **OAuth2/OpenID Connect** provider, create
the app with the correct redirect URI, and paste the `client_id` /
`client_secret` into the app's SSO settings. The relying parties that exist
today are `grafana` (pk 9), `Seerr OIDC` (pk 12) and `Open WebUI` (pk 14);
today are `grafana` (pk 9), `Seerr OIDC` (pk 12) and `Open WebUI` (pk 14)
**Gitea has no provider row at all** (live provider table, 2026-10-18) — it
authenticates itself and stays off the chain by design
(`dankclaw/Caddyfile:110-120`).
@@ -373,53 +238,12 @@ every outpost session (see the ORM playbook).
the LDAP section) — bumping only the tag would leave the LDAP outpost behind.
Authentik migrates the schema on startup.
## Self-serve user-settings flow (password change)
Users change their own account details through the `default-user-settings-flow`
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
target of the homepage footer's "account" link. It is a `stage_configuration`
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
user.
Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
insertion order):
| order | stage | type | what it does |
|---|---|---|---|
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
**Password change is a prompt field, not a stage.** The `UserWriteStage`
writes whatever the preceding `PromptStage` collected: for the key `password`
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
is a `password` field on the `default-user-settings` prompt
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
`validate_password` runs `authenticate()` against the backends to check the
*current* password; it never sets a new one. (A `PasswordStage` was added on
2026-09-25 and removed the same day for exactly this reason.)
Two ORM gotchas: `Flow.stages.add()` fails on this through model
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
and a prompt field is a `Prompt` row linked through `PromptStageField`
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
**A blank password must not set an empty password.** `set_password("")` would
make the account log in with no password, so a validation policy
(`default-user-settings-no-empty-password`, bound to the prompt via
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
is `""` and allows it when set. Verified with the `PolicyEvaluator`
(empty → deny, set → allow, absent → allow) and end-to-end by driving
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
invalid).
## Outpost claim cache (`proxysession`) — 403 after a group add
The embedded outpost does not re-evaluate group membership per request. Each
completed authorize writes one row to `authentik_providers_proxy_proxysession`
whose `session_data` JSON holds the claims (`groups` included) that get served
for that browser session until the row expires (observed TTLs 1–24 h per
for that browser session until the row expires (observed TTLs 1–24 h
row). A session authorized **before** a user was added to a group keeps
serving the old group list — Caddy-side RBAC header gates then 403 despite
correct membership, and no authentik event is generated (the 403 happens at
@@ -457,7 +281,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
(the inventory table carries both bare origins and trailing slashes;
routing matches on hostname, verified 2026-09-02).
`internal_host = http://caddy:8017`, all three flows set, and an
Application attached — the two provisioning rules above.
Application attached — the two provisioning rules in [[Authentik Outpost Reference]].
2. **Route flip in ONE commit/deploy**: the vhost becomes
`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
inside the `http://:8017` listener add `@<svc> host <vhost>` +
@@ -485,7 +309,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
000 (connection refused) — the post-auth listener lives in the caddy
container's netns; probe inside it or grep the config. Non-browser clients
cannot solve the JS challenge — before flipping, audit service-to-service
traffic: ingestion/push paths must target internal addresses (`loki:3100`),
traffic: ingestion/push paths must target internal addresses (`loki:3100`)
never a public hostname of a guarded service. Exempt-by-construction
categories stay unguarded: the IdP itself (`auth.`), git smart-HTTP
endpoints, services with their own complete auth — document the exemption
@@ -512,31 +336,18 @@ EOF
`sed -n '/objects imported/,$p'` to drop the noise *(unverified 2026-10-18:
the exact banner text and the 229-object count — no `ak shell` was driven
against the live auth service for this audit)*.
- Import paths (each wrong guess costs a failed exec): `User`, `Group`,
- Import paths (each wrong guess costs a failed exec): `User`, `Group`
`Application`, `ApplicationEntitlement`, `PropertyMapping` →
`authentik.core.models`; `ProxyProvider` →
`authentik.providers.proxy.models` (NOT `authentik.outposts.models`);
`LDAPProvider` → `authentik.providers.ldap.models`;
`authentik.providers.proxy.models` (NOT `authentik.outposts.models`)
`LDAPProvider` → `authentik.providers.ldap.models`
`authentik.core.models.application` does NOT exist as an import path; there
is no `authentik.policies.group` module in this build.
*(unverified 2026-10-18: these module paths were not re-executed — importing
- (unverified 2026-10-18: these module paths were not re-executed — importing
outside a configured Django shell fails with `ImproperlyConfigured`)*
- For live DB truth prefer the postgres container, and it works without any
credential — the local socket is trusted, so a read-only audit is just
`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
before guessing: 2026.x renamed both tables and columns, and the models are
MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
three `*_flow_id` columns; the proxy table is
`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
table in 2026.8), the LDAP table is
`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
`authentik_core_application` attaches to a provider through `provider_id`
(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
is the same story from the ORM side: `type` lives in the subclass tables.
Never select secret columns (`client_secret`, `key`, `password`) into a log.
- The live database's table and column names (the 2026.x renames, the MTI parent
keys, the read-only `psql` audit recipe) are reference lookup:
[[Authentik Outpost Reference]].
## User provisioning playbook (user-level account)
@@ -565,7 +376,7 @@ downstream —
`category: user` panels from `services.json`.
- `openwebui_users` / `openwebui_admins` → ApplicationEntitlements
**"Open WebUI Users"** and **"Open WebUI Admins"**, both rows attached to the
`Open WebUI` application (`authentik_core_applicationentitlement`,
`Open WebUI` application (`authentik_core_applicationentitlement`
2026-10-18), consumed by the "Open WebUI roles" property mapping → `roles`
claim *(unverified 2026-10-18: the group→entitlement binding and the mapping
expression were not read; the mapping itself is not visible under that name
@@ -581,7 +392,7 @@ downstream —
artem 16, kirby 17, ry 18, nic 19, debo 20, freya 21, mitch 22, eric 23 —
and every one carries the `<user>@franksdank.cloud` address the template
prescribes. Creation dates (2026-10-07 → 2026-10-15) were not re-checked
*(unverified 2026-10-18)*. Template: `username` +
- (unverified 2026-10-18)*. Template: `username` +
`<user>@franksdank.cloud` + display name + `is_active=True` + the password
set via `set_password`, membership in `Non-Admin Users` + `jellyfin_users` +
`openwebui_users`. `alex` was recreated from scratch on 2026-10-08 into this
@@ -589,7 +400,7 @@ downstream —
the row carried no MFA device, token, entitlement, or proxy session, and
events survive a user delete because `Event.user` is a denormalized JSON
field, not a FK).
- EXCLUDE from a user-level account: `homepage-admins`, `jellyfin_admins`,
- EXCLUDE from a user-level account: `homepage-admins`, `jellyfin_admins`
`openwebui_admins`, `authentik Admins`, `authentik Read-only`.
- Other user-category vhosts (SearXNG, Gitea, Calibre-Web, RomM, Copyparty,
Plex, Seerr, Marreta) need no per-user group: any authenticated user
@@ -618,8 +429,8 @@ More field gotchas (each cost a failed exec): `ApplicationEntitlement.applicatio
can be null (guard with `getattr`); `LDAPProvider` has
`search_mode`/`bind_mode`/`base_dn` and lacks `search_base`/`of_groups`/
`organization` — the backing table
(`authentik_providers_ldap_ldapprovider`) has exactly `provider_ptr_id`,
`base_dn`, `certificate_id`, `tls_server_name`, `gid_start_number`,
(`authentik_providers_ldap_ldapprovider`) has exactly `provider_ptr_id`
`base_dn`, `certificate_id`, `tls_server_name`, `gid_start_number`
`uid_start_number`, `search_mode`, `bind_mode`, `mfa_support`
(`information_schema.columns`, 2026-10-18); `OAuth2Provider` has no `.scopes`
(scopes are linked through `ProviderPropertyMapping`, which is why
@@ -673,5 +484,5 @@ podman run --rm --network container:<container> --entrypoint /bin/busybox \
- [Jellyfin](../Dankserver/Jellyfin.md) — consumes the LDAP outpost via its LDAP-Auth plugin
- [Network Architecture](../Architecture/Network%20Architecture.md) — the flat naming scheme
- Research: `.docaudit/research/auth.md` — the full provider comparison
*(the file is gone from the repo and from the host checkout as of
- (the file is gone from the repo and from the host checkout as of
2026-10-18)*
@@ -47,7 +47,7 @@ torrent.
WIREGUARD_PERSISTENT_KEEPALIVE_INTERVAL=25s
The private key is hashed into the VM env file and never enters the repo.
NAT-PMP inbound forwarding is available on Proton (gluetun supports it via
NAT-PMP inbound forwarding is available on Proton (gluetun supports it
`FIREWALL_INPUT_PORTS` + `NATPMP=true`) — deliberately deferred: the e2e
gate passes outbound-only, and swarm peers needing an inbound port is a
seeding-era concern, not a grab-only one.
@@ -86,7 +86,7 @@ are unaffected.
got `Unauthenticated`); the live conf was patched and
`scripts/deploy-remote.sh` fixed to write the `_PBKDF2` key. Gotcha
(live, 2026-09-01): after qB 5.x ran, its conf rewrite left a hash line
4.6.7 rejects (`invalid credentials` for every login, including the arrs);
4.6.7 rejects (`invalid credentials` for every login, including the arrs)
rewriting the same `@ByteArray` line fresh (same password, 100k
iterations) restored login. Any future version flip: regenerate the line.
- `HostHeaderValidation=false` + `ServerDomains=*`: the socat forwarder
@@ -98,7 +98,7 @@ are unaffected.
path (no auto-TMM), so the default itself points at `Finished Torrents`:
every grab lands there regardless of client behavior; human review moves
files onward; nothing writes to the entropy volume.
- Categories seeded: `Finished Torrents`, `lidarr`, `radarr`, `readarr`,
- Categories seeded: `Finished Torrents`, `lidarr`, `radarr`, `readarr`
`tv-sonarr` (all → `Finished Torrents`).
- **Download clients** (gateway `10.88.0.1:8080`, user `admin`): all five test
green against live qB — sonarr (`tv-sonarr`), radarr, lidarr, readarr
@@ -109,12 +109,12 @@ are unaffected.
as "Authentication failure" — that kills runtime auth too, not just API
creates; the fix was pinning qB itself (below), after which readarr tests
200 like the rest.
- **qB pinned to 4.6.7** (`Image=lscr.io/linuxserver/qbittorrent:4.6.7`,
- **qB pinned to 4.6.7** (`Image=lscr.io/linuxserver/qbittorrent:4.6.7`
not `latest`): `latest` resolved to 5.x, whose `204` login breaks
Readarr's frozen client; 4.6.7 answers the legacy `200` + `"Ok."` that
every client accepts. Pinned in
`frankbook-pro/qbittorrent/quadlet/qbittorrent.container`.
- **Prowlarr Applications** push to sonarr/radarr/lidarr (`fullSync`,
- **Prowlarr Applications** push to sonarr/radarr/lidarr (`fullSync`
gateway ports). Readarr deliberately omitted: prowlarr's indexer-push
test to readarr's `/api/v1/indexer/test` 500s (EOL schema drift).
Legacy `Prowlarr-*` indexer rows carried in the migrated dankserver DBs
@@ -46,7 +46,7 @@ The Prometheus datasource is declared in git, not clicked into `grafana.db`:
address Caddy proxies to, never the loopback-published port — and carries
`timeInterval: 15s` so Grafana's rate windows cannot be shorter than the
scrape interval (`prometheus.yml:5`). The `llm-fleet` dashboard is provisioned
alongside it by `dashboards/provider.yml` (provider `fleet`,
alongside it by `dashboards/provider.yml` (provider `fleet`
`updateIntervalSeconds: 30`, `allowUiUpdates: true`).
A Loki datasource is still **not** provisioned — the tree contains exactly one
@@ -69,8 +69,8 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
client_id `grafana-21ca190cde75d60f`, global authorize/token/userinfo paths,
Basic client auth, form-encoded token body, and the fork's userinfo-claims
bug that leaves identity at the `sub` hash only) is in
[[Authentik]] → "OAuth2 relying-party quirks". Consequences for Grafana:
The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`,
[[Authentik Outpost Reference]] → "OAuth2 relying-party quirks". Consequences for Grafana:
The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`
`ORG_ID=1` and **no** role-attribute mapping (`grafana.container:52-55`), so
users land as Viewer with `login = <sub hex>`; no email/groups/role claims
reach Grafana until the evaluator bug is fixed — do not write role mappings
@@ -79,7 +79,7 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
the host without the database or an admin password. *(unverified 2026-10-18)*)
- **Never point this unit's `EnvironmentFile` at the repo-root `.env`.** That
file is a gluetun env file and it carries `HOME=/root`; systemd imports it
for the service, podman then looks for its configuration under `/root`,
for the service, podman then looks for its configuration under `/root`
cannot read it, and the unit dies at start with
`stat /root/.config: permission denied`. The unit reads
`/home/dank/Containers/grafana/env`, which the deployer seeds with exactly
@@ -1,7 +1,7 @@
---
type: reference
title: Homepage Status Probes
description: The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage procedure, local render check, deploy, and the e2e gate interactions (asleep Mac mini, raw-IP matrix).
description: "How probe targets are chosen, the three false-down shapes, the triage procedure, and the e2e gate interactions."
tags: [homepage, status, probes, e2e, homepage-status]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -51,7 +51,7 @@ See [[Container Host Reachability]] for the netns rules.
published nothing → structurally unprobeable from the host).
3. Probe from the **correct vantage**: from a bridge-net container the host
is its gateway IP (`/proc/net/route`, e.g. `10.89.1.1` on
`dankclaw_default`) — *not* `host.containers.internal` (`169.254.1.2`),
`dankclaw_default`) — *not* `host.containers.internal` (`169.254.1.2`)
which belongs to the default `podman` net (`10.88.0.0/16`). Probe with
bash `/dev/tcp` (no `python3`/`ip` in these images); escape loop vars so
they expand **inside** `podman exec bash -c`.
@@ -83,7 +83,7 @@ python3 -m http.server 8899 --bind 127.0.0.1
```
Then in the browser: load `http://127.0.0.1:8899/index.html`, **wait ~1.5 s**
(panels render after the async fetch; checking immediately returns 0 panels),
(panels render after the async fetch; checking immediately returns 0 panels)
extract via `tab.evaluate` — LLM cards render `.desc-role`/`.desc-spec`/
`.desc-inputs`/`.desc-use` from the card's `details` object, other cards a
plain `.desc`; `getComputedStyle(el).color` confirms tones. Screenshot for
@@ -1,6 +1,6 @@
---
type: reference
description: "A password-protected web site on Dankclaw lists the services that are up, with a short description of each and a link to both the service and its usage documentation."
description: "The password-protected service list on Dankclaw: what each card shows and where its data comes from."
tags:
- infrastructure
- homepage
@@ -22,7 +22,7 @@ The homepage is live on Dankclaw. `./scripts/deploy.sh dankclaw` copies the stat
**Docs page shipped (2026-09-17):** `/docs` serves a static user documentation
page (`shared/homepage/docs.html`) — access walkthrough, the running-model
guide, media how-tos, and troubleshooting. Per-service `/docs/<svc>/` pages
remain a separate thread. The three LLM cards (`frank-dragoon`, `frank-medium`,
remain a separate thread. The three LLM cards (`frank-dragoon`, `frank-medium`
`frank-zealot`) moved from admin-only to the **user** page with live model
descriptions and a `docs` link to `/docs#models`.
@@ -59,7 +59,7 @@ without which the picker showed "No results found" for every family account
"equivalent to Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier
intelligence)", frank-archon "equivalent to Claude Opus 4.8 / OpenAI-Terra
(same era)", frank-zealot "equivalent to last year's frontier level
(2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic);
(2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic)
pinned in `homepage_config.bats`.
**Protoss rename + Model Information component (2026-10-08):** the model
@@ -91,7 +91,7 @@ same renames, api-id mapping, supervision and compaction notes.
app card now carries hex-clipped art beside its name (`services.json` `art`
field, files under `img/apps/`): 19 brand marks tinted amber from the
Simple Icons CDN plus 19 hand-improvised monogram SVGs (amber hex + VT323
initials) for apps with no mark — Copyparty, Marreta, Hermes (+dashboard),
initials) for apps with no mark — Copyparty, Marreta, Hermes (+dashboard)
node-exporter, Gluetun, OpenWebUI, Calibre-Web, RomM, Seerr, Loki, Alloy,
Alertmanager, Valkey, Prowlarr, Readarr, Lidarr, Tautulli, ComfyUI. Model
cards render in the order zealot → dragoon → archon (registry order is the
@@ -143,8 +143,8 @@ per-app colors: Copyparty gold, Marreta brick, Grafana orange, Loki
indigo, Neo4j network, Alloy violet, Promtail pink, Alertmanager red,
Prometheus orange, Node Exporter teal, cAdvisor sky, Guake yellow,
Authentik blue, Seer green radar, RomM a purple GameBoy. New per-service
fields: `art_bg` (art backdrop — OpenWebUI's dark logo needs `#f5f5f5`),
`warn` on OpenWebUI (searches ride the Seerr→SearXNG path),
fields: `art_bg` (art backdrop — OpenWebUI's dark logo needs `#f5f5f5`)
`warn` on OpenWebUI (searches ride the Seerr→SearXNG path)
`model.compaction` (the amber "set this as compaction model" callout
frank-zealot carries). frank-dragoon's card portrait is the SC2 unit
portrait (fandom wiki `Dragoon_portrait.jpg`, webp→jpeg). The whole
@@ -159,10 +159,9 @@ Sonarr, Radarr, Lidarr, Readarr, qBittorrent) is out of the family docs
entirely — admin tooling; Seerr is presented as the front door for
requests. Each fleet model gained a Hugging Face source link: card field
`model.hf` rendered as the cyan `model on huggingface ↗` line under the
role, and the docs `Model:` bullets link the checkpoint names (zealot →
HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive, dragoon →
Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated, archon →
Qwen/Qwen3.8-Flash-Next-FP8).
role, and the docs `Model:` bullets link the checkpoint names — which
checkpoint each endpoint runs is payload state on
[[frank-dragoon]], [[frank-archon]] and [[frank-zealot]].
**Navy Protoss console, maximal cornering, font swap (2026-10-08, sixth
wave):** the amber CRT palette is retired. The page is now deep space
@@ -176,7 +175,7 @@ titles are chamfered plates with the diamond glyph inside the plate.
Fonts: body is **Share Tech Mono** (legible retro terminal), headings
are **Rajdhani 600** (the StarCraft UI face) — VT323 is retired; all
three pages load the pair from Google Fonts. Four tiles that had
reverted to initials got pictograms: Calibre-Web an open book (orange),
reverted to initials got pictograms: Calibre-Web an open book (orange)
RomM a purple GameBoy, Copyparty a blue folder with upload arrow,
Marreta a crumbling brick wall with a red crack (`img/apps/app-*.svg`).
frank-dragoon's card accent is phase-crystal teal (`#5fe0d0`), matching
@@ -198,7 +197,7 @@ state — the only animation left is the speaking-voice pulse. The result
is flat navy panels on crisp 1px borders: same Protoss direction, half
the pixels.
**Model ids official (2026-10-08):** the card ids `frank-zealot`,
**Model ids official (2026-10-08):** the card ids `frank-zealot`
`frank-dragoon`, `frank-archon` are now the harness's official model ids
in `models.yml`/`config.yml` (see [[OMP Harness]]); the
systemd services keep the `frank-archon`/`frank-dragoon`/
@@ -206,7 +205,7 @@ systemd services keep the `frank-archon`/`frank-dragoon`/
one name per model.
**Api ids renamed on the wire (2026-10-09):** the promise in
`docs.html` is kept — the served model ids are now `frank-zealot`,
`docs.html` is kept — the served model ids are now `frank-zealot`
`frank-dragoon`, `frank-archon`, so the docs page no longer carries
`api id:` annotations, and the curl/opencode/Pi examples send
`frank-zealot`. The systemd services keep the old names. `docs.html`
@@ -217,7 +216,7 @@ ids, which did not change.
clip-paths replaced with `border-radius` (the clipped corners read as
gaps in the 1px border); section-title diamonds moved from absolute
positioning to a fixed-width inline glyph (aligned, wrap-safe); closed
WIP items grey out (dim type, desaturated art, muted border via
WIP items grey out (dim type, desaturated art, muted border
`:has(.badge.wip-closed)`); the Dragoon tagline is one spliced track
(`sfx/dragoon-tagline.ogg` — loudnorm-matched, silence-trimmed, 350 ms
beat; the two SC2 source oggs are trashed); privacy notes became
@@ -235,11 +234,10 @@ house-DNS advice; the bullet now just says the names resolve on the
home network.
**Archon card runs the fleet's own model (2026-10-08):** the
`frank-archon` homepage card and docs Model bullet now say the payload
is our in-house GPTQ-w4 quant of Swift1.5-Qwen3.8-Flash-Next and link
our HF publication (`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`)
in place of the FP8-era base-model link; capacity line says three
submissions (`--max-num-seqs 3`).
`frank-archon` homepage card and docs Model bullet swapped the FP8-era
base-model link for our own HF publication, and the capacity line for the
endpoint's own concurrency figure; the payload, its publication and the slot
count all live on [[frank-archon]].
## Design decision — 2026-08-22
@@ -188,7 +188,7 @@ sudo systemctl restart systemd-resolved && resolvectl flush-caches
```
The `~` (route-only) prefix is load-bearing: it routes only those domains to
Pi-hole. Verify: `getent hosts www.franksdank.cloud` → `192.168.0.143`;
Pi-hole. Verify: `getent hosts www.franksdank.cloud` → `192.168.0.143`
`resolvectl status` shows `DNS Domain: ~franksdank.cloud ~franksdank.casa`.
**Layer 2 — browser DoH (the bypass: "curl works, browser times out").** If
@@ -236,7 +236,7 @@ Endpoints (all POST, JSON body):
`.content.records[]` as some docs claim — parse with
`jq -r '.records[] | [.id,.name,.type,.content,(.ttl|tostring)] | @tsv'`.
- Delete: `/dns/delete/{domain}/{id}` (body carries apikey+secretapikey).
- Edit: `/dns/edit/{domain}/{id}` — pass the full record (`id`, `name`,
- Edit: `/dns/edit/{domain}/{id}` — pass the full record (`id`, `name`
`type`, `content`, `ttl`, `prio`).
Quirks that bite (verified live 2026-08-31):
+1 -1
View File
@@ -55,7 +55,7 @@ The [[Alloy]] agent on Dankclaw is wired to ship container and journal logs to `
The service answers and the API works; there is simply no data in it. Measured on 2026-10-18:
- `curl -s http://127.0.0.1:3100/loki/api/v1/labels` → `{"status":"success"}` with no `data` key (no label names exist), and a 7-day `query_range` on `{job="systemd-journal"}` returns `"result":[]`.
- `du -sh /opt/loki/data` → a few hundred KB of directory scaffolding (`wal`,
- `du -sh /opt/loki/data` → a few hundred KB of directory scaffolding (`wal`
`chunks`, `compactor`, `tsdb-shipper-active`, `tsdb-shipper-cache`) —
`find /opt/loki/data -type f` counted **8 files**, all WAL segments,
cluster-seed and delete-request bookkeeping, **no chunk objects**
@@ -1,7 +1,7 @@
---
type: reference
title: Observability Provisioning
description: How Prometheus config/rules and Grafana datasources/dashboards are provisioned as code in serverAppConfigs — the config triad, deploy copiers, planner mappings, seam tests, entry points for each change type, and API verification.
description: "Monitoring config as code: the config triad, deploy copiers, planner mappings, and where each change type lands."
tags: [observability, prometheus, grafana, provisioning, deployment]
timestamp: 2026-10-05T00:00:00-07:00
---
@@ -21,7 +21,7 @@ committed.
| `shared/prometheus/rules/llm-fleet.yml` | alert rules + recording rules |
| `shared/grafana/provisioning/` | datasources + dashboards (provisioned tree) |
Targets and datasources reference **container names** (`grafana:3000`,
Targets and datasources reference **container names** (`grafana:3000`
`loki:3100` — `shared/prometheus/prometheus.yml:52-61`), never
loopback-published ports; `alloy` is **not scraped at all** (no job in the
config — its `127.0.0.1:12345` UI is only probed by the homepage timer,
@@ -47,7 +47,7 @@ from `deploy_observability`:
- `copy_grafana_provisioning` → `/opt/grafana/provisioning/` (`:286-300`)
`scripts/deploy-plan.py` maps `shared/prometheus/**` to `prometheus` (plus
`alertmanager` when the changed file is `alertmanager.yml`, `:251-256`),
`alertmanager` when the changed file is `alertmanager.yml`, `:251-256`)
`shared/loki/**` to `loki` and `shared/alloy/**` to `alloy` (`:257-260`), and
`shared/grafana/**` to `grafana` (`:261-264`); changes under
`dankclaw/observability/` select the touched unit and its dependency consumer
@@ -71,9 +71,9 @@ HTTP probe, not a metrics endpoint.
**Add an alert/recording rule** — edit `shared/prometheus/rules/llm-fleet.yml`.
Current delivered set: **4 alert rules + 2 recording rules** —
`LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`,
`LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`
`LlmEndpointDown`, plus `job:llm_prefix_cache_reuse:rate5m` and
`job:llm_spec_accept_length:rate5m` (`shared/prometheus/rules/llm-fleet.yml:32-110`),
`job:llm_spec_accept_length:rate5m` (`shared/prometheus/rules/llm-fleet.yml:32-110`)
confirmed live: `curl -s http://127.0.0.1:9090/api/v1/rules` (2026-10-18)
reports 4 alerting + 2 recording. Rules reload with the prometheus service.
@@ -101,7 +101,7 @@ curl -s -u admin:<pw> http://127.0.0.1:3002/api/datasources | python3 -m json.to
The two Prometheus queries were run on 2026-10-18 and are what every number on
this page comes from: 4 alerting + 2 recording rules, and `up` for
`prometheus`, `node-dankclaw`, `node-dankserver`, `grafana`, `loki`,
`prometheus`, `node-dankclaw`, `node-dankserver`, `grafana`, `loki`
`frank-zealot`, `frank-dragoon`, `frank-archon` with the four structural downs
listed in [[Prometheus]]. The Grafana query needs the local admin password and
was not run here; the filesystem side (`ls /opt/grafana/provisioning/*`) is
@@ -114,12 +114,12 @@ table (refused = outage, 404/HTML = config defect).
## Current delivered state (delivery chain live 2026-09-17, re-verified 2026-10-18)
Prometheus → Alertmanager (`dankclaw/observability/quadlet/alertmanager.container`,
Prometheus → Alertmanager (`dankclaw/observability/quadlet/alertmanager.container`
loopback `:9093`) → host-side Signal relay (`scripts/signal-relay.py`, user
unit, `192.168.0.143:18093` — the LAN IP, the one bind a container can reach
via `host.containers.internal`) → signal-cli, addressed by number
`+12062476336` (`scripts/signal-relay.py:36-43`). Every hop re-checked
2026-10-18: `ss -ltnp` shows `python3` on `192.168.0.143:18093`,
2026-10-18: `ss -ltnp` shows `python3` on `192.168.0.143:18093`
`curl -s http://192.168.0.143:18093/health` → `ok`, and
`journalctl --user -u signal-relay` ends with successful `sent to +12062476336:`
lines for `LlmEndpointDown`, `LlmRequestsQueueing`, `LlmSpecDecodeOff` and
@@ -130,13 +130,14 @@ the blackbox exporter. See [[Prometheus]] → Targets.
**A llama.cpp endpoint is invisible unless it opts in.** Until 2026-10-18,
`up{job="frank-zealot"}` was 0 for the entire retained history of the metric —
the oldest sample, 2026-10-11, is already 0 — while the service itself answered
`/health` 200 the whole time. Cause: llama.cpp answers **501 Not Implemented**
on `/metrics` unless the server is started with `--metrics`, so the scrape was
refused at the protocol layer and `LlmEndpointDown` (critical) stayed firing:
`/health` 200 the whole time. Cause: llama.cpp only exports `/metrics` when it
is started with the flag, so the scrape was refused at the protocol layer (the
flag and its opt-out are on [[frank-zealot]]) and `LlmEndpointDown` (critical)
stayed firing:
the relay re-sent it on the 4 h `repeat_interval` for days (`journalctl --user
-u signal-relay` — firing sends at 01:37, 05:42, 09:47, 13:52, 17:57 and 22:02
on 2026-10-12). It finally resolved at `2026-10-18T01:53:21-07:00`, after the
`METRICS=1` knob added to `scripts/frank-zealot.env` that day; the target
metrics opt-in landed that day; the target
reports up since (`curl -s http://127.0.0.1:9090/api/v1/targets` 2026-10-18).
How many payload generations it spanned is not readable from any host
artifact. *(unverified 2026-10-18)*
@@ -147,11 +148,12 @@ state lives in Prometheus' process, not on disk — a restart re-stamps
`activeAt` on a long-standing alert, so a "fresh" alert after a reboot is not
evidence of a fresh fault. (Alertmanager by contrast **does** persist:
`--storage.path=/alertmanager` on a host volume keeps silences and the
notification log — `dankclaw/observability/quadlet/alertmanager.container:39-42`,
notification log — `dankclaw/observability/quadlet/alertmanager.container:39-42`
`/opt/alertmanager/data`.)
Remaining known downs are the structural ones above (HTML-scraped probes, Gitea
metrics, ComfyUI), and halogen serves `/metrics` natively, which is why
metrics, ComfyUI), and `frank-dragoon` exports `/metrics` natively
([[frank-dragoon]]), which is why
`frank-dragoon` has always been green. The active-alert count is a
moment-in-time value and should never be documented as a constant:
`/api/v1/alerts` was empty at the 2026-10-18 morning census, held one firing
@@ -167,8 +169,8 @@ arm (`systemctl is-active frank-zealot` → `failed` after
`stop-sigterm timed out. Killing`, which is llama.cpp ignoring SIGTERM past
`TimeoutStopSec`, the known shutdown shape for these engines). So
`up{job="frank-zealot"} == 1` that day measured the trial, not the production
shape in `scripts/frank-zealot.env` (Ternary-Bonsai-27B-PQ2_0, `CTX_SIZE=401408
PARALLEL=2` = 2 × 200,704 per slot, q8_0 K+V), which remains the repo contract.
shape in `scripts/frank-zealot.env` — that shape is [[frank-zealot]]'s to state,
and it is the repo contract.
The unit stayed `enabled`; `./scripts/deploy.sh dankclaw --service frank-zealot`
reinstalls and starts it, restoring boot persistence and self-healing. Never
quote a candidate arm's flags or VRAM as fleet state — the process and its shape
@@ -1,7 +1,7 @@
---
type: reference
title: "Observability on Dankclaw"
description: "The Dankclaw observability stack — Prometheus, Grafana, Loki, Alloy, node-exporter and Alertmanager (Signal delivery): what runs, how it is deployed, how it is exposed, and what is still unconfigured."
description: "The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured."
tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw]
timestamp: 2026-09-11
---
@@ -51,13 +51,13 @@ Dankserver runs one member of this family, node-exporter, as its own unit
## Exposure
The five HTTP surfaces publish on `127.0.0.1` only — `ss -ltnp` (2026-10-18)
shows `127.0.0.1:9090`, `127.0.0.1:9093`, `127.0.0.1:3002`, `127.0.0.1:3100`,
shows `127.0.0.1:9090`, `127.0.0.1:9093`, `127.0.0.1:3002`, `127.0.0.1:3100`
`127.0.0.1:12345` and no wildcard — because each of them is unauthenticated
inside, and several carry lifecycle or query APIs. Browser access goes through
Caddy: each vhost (`dankclaw/Caddyfile:128-145`) proxies to the guard
`anubis:8923`, and the post-auth handlers route by container name on
`dankclaw_default` — `loki.franksdank.cloud` → `loki:3100` (`:364-366`),
`grafana.franksdank.cloud` → `grafana:3000` (`:374-376`),
`dankclaw_default` — `loki.franksdank.cloud` → `loki:3100` (`:364-366`)
`grafana.franksdank.cloud` → `grafana:3000` (`:374-376`)
`prometheus.franksdank.cloud` → `prometheus:9090` (`:379-381`).
Alertmanager has no vhost at all. node-exporter is the exception that must live
in the host network namespace, and it binds exactly two addresses rather than
@@ -70,13 +70,13 @@ and the podman bridge subnet `10.89.0.0/16` before the DROP
## Deployment
`./scripts/deploy.sh dankclaw --service <name>` runs `deploy_observability`,
`./scripts/deploy.sh dankclaw --service <name>` runs `deploy_observability`
which installs the unit, copies that service's config
(`copy_prometheus_config`, `copy_alertmanager_config`, `copy_loki_config`,
(`copy_prometheus_config`, `copy_alertmanager_config`, `copy_loki_config`
`copy_alloy_config`, `copy_grafana_provisioning`, plus
`install_signal_relay` with the alertmanager deploy), then waits on the loopback
health endpoint (`/prometheus`: `/-/healthy`, `/alertmanager`: `/-/healthy`,
`/grafana`: `/api/health`, `/loki`: `/ready`, `/alloy`: `/-/healthy`,
health endpoint (`/prometheus`: `/-/healthy`, `/alertmanager`: `/-/healthy`
`/grafana`: `/api/health`, `/loki`: `/ready`, `/alloy`: `/-/healthy`
`/node-exporter`: `/metrics`). Startup order follows the data dependencies —
`grafana` after `prometheus`, `alloy` after `loki`
(`grafana.container:32-33`, `alloy.container:38-39`). See
@@ -1,6 +1,6 @@
---
type: reference
description: "Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints, and feeds the provisioned Grafana datasource. Down targets today are structural, not outages."
description: "Scrapes each service's metrics endpoint and feeds the provisioned Grafana datasource; down targets are structural."
tags:
- infrastructure
- prometheus
@@ -25,7 +25,7 @@ Authentik; post-auth the Caddy vhost routes straight to `prometheus:9090`) —
| Config | `/opt/prometheus/config/prometheus.yml`, installed from `shared/prometheus/prometheus.yml` on every deploy, together with `/opt/prometheus/config/rules/llm-fleet.yml` from `shared/prometheus/rules/llm-fleet.yml` (`scripts/deploy-remote.sh:219-227`) |
| Data | `/opt/prometheus/data` (TSDB, `prometheus.container:52`) |
| Retention | `--storage.tsdb.retention.time=30d` (`prometheus.container:49`) |
| Alerting | **4 alert rules + 2 recording rules** (`shared/prometheus/rules/llm-fleet.yml`: alerts `LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`, `LlmEndpointDown`; recordings `job:llm_prefix_cache_reuse:rate5m`, `job:llm_spec_accept_length:rate5m`), confirmed against the running server: `curl -s http://127.0.0.1:9090/api/v1/rules` (2026-10-18) lists 4 alerting + 2 recording. Evaluated **and delivered** since 2026-09-17: the `alerting:` block ships to `alertmanager:9093` (`prometheus.yml:19-22`) → Signal relay → signal-cli (see [[Alertmanager]]) |
| Alerting | **4 alert rules + 2 recording rules** (`shared/prometheus/rules/llm-fleet.yml`: alerts `LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`, `LlmEndpointDown`; recordings `job:llm_prefix_cache_reuse:rate5m`, `job:llm_spec_accept_length:rate5m`), confirmed against the running server: `curl -s http://127.0.0.1:9090/api/v1/rules` (2026-10-18) lists 4 alerting + 2 recording. `LlmEndpointDown` is scoped `up{job=~"frank-.*"} == 0`, so only the `frank-*` LLM jobs can page on it — any other job that is structurally never-green must stay out of that family ([[Stale Signal Audit]]). Evaluated **and delivered** since 2026-09-17: the `alerting:` block ships to `alertmanager:9093` (`prometheus.yml:19-22`) → Signal relay → signal-cli (see [[Alertmanager]]) |
## Targets
@@ -37,10 +37,10 @@ Authentik; post-auth the Caddy vhost routes straight to `prometheus:9090`) —
| `node-dankclaw` | `host.containers.internal:9100` | host metrics from [[Node-Exporter]] — a net=host bind, so via the gateway (the LAN address is unreachable from this bridge container) |
| `node-dankserver` | `192.168.0.235:9100` | host metrics from [[Node-Exporter]] — a different machine, so the LAN address works |
| `grafana`, `loki` | `grafana:3000`, `loki:3100` | container names on the `dankclaw_default` network; both serve real `/metrics` |
| `frank-archon` | `192.168.0.235:7770` | the LLM fleet — the radiance engine since 2026-10-05, exporting vLLM-compatible names plus `radiance:*` (`prometheus.yml:122-131`); a different machine, so the LAN address works |
| `frank-dragoon`, `frank-zealot` | `host.containers.internal:7770`, `:7771` | the LLM fleet — both export the `llamacpp:` namespace (halogen included: `curl -s http://127.0.0.1:7770/metrics` → `llamacpp:prompt_tokens_total`, `text/plain; version=0.0.4`, 2026-10-18). They are addressed through the gateway because they are **same-host** — `frank-zealot` is a host-process bind, `frank-dragoon` a container publishing `0.0.0.0:7770->8731`, not a net=host pair (`prometheus.yml:96-99`, `:108-120`) |
| `frank-archon` | `192.168.0.235:7770` | the LLM fleet — a different machine, so the LAN address works (`prometheus.yml:122-131`); engine, metric families and their meaning live on [[frank-archon]] |
| `frank-dragoon`, `frank-zealot` | `host.containers.internal:7770`, `:7771` | the LLM fleet. They are addressed through the gateway because they are **same-host** — a host-process bind and a published container port are equally invisible to a bridge container at the host's LAN address (`prometheus.yml:96-99`, `:108-120`); what each endpoint exports is on [[frank-dragoon]] and [[frank-zealot]] |
The `frank-medium` job and its `vllm:` namespace are **retired**: that endpoint left the fleet, `frank-archon` briefly had no exporter at all (the FreeToken-era job scraped 404s forever) and re-joined on radiance the same day, 2026-10-05 (`prometheus.yml:101-107`). No `frank-medium` job exists in the config or in `/api/v1/targets` (2026-10-18).
The `frank-medium` job is **retired**: no `frank-medium` job exists in the config or in `/api/v1/targets` (2026-10-18), and the endpoint itself left the fleet — its history is on [[frank-archon]], which holds the port today.
**Three jobs read down today, and they are not outages** (re-audited against `curl -s http://127.0.0.1:9090/api/v1/targets`, 2026-10-18; first audited 2026-09-16):
@@ -53,15 +53,14 @@ The `frank-medium` job and its `vllm:` namespace are **retired**: that endpoint
Gitea's exporter stays off until `[metrics] ENABLED = true` in its own config (no metrics env is set in the container, and `curl -s http://127.0.0.1:3000/metrics` → 404, 2026-10-18); ComfyUI and the two `services` probes are HTML pages, which Prometheus 3 refuses as metrics. Everything else is green: `prometheus`, `node-dankclaw`, `node-dankserver`, `grafana`, `loki`, `frank-zealot`, `frank-dragoon`, `frank-archon`.
**`frank-zealot` reads up since 2026-10-18, and did not before.** llama.cpp answers **501 Not Implemented** on `/metrics` unless started with `--metrics`; the endpoint had no such flag, so `up{job="frank-zealot"}` was 0 across the whole retained history (the oldest sample, 2026-10-11, is already 0) while the service answered `/health` 200 the whole time. The `METRICS=1` knob in `scripts/frank-zealot.env` is the fix, and the target is up as measured 2026-10-18. Read `lastError` before believing an outage: a target can read down beside a healthy service.
**`frank-zealot` reads up since 2026-10-18, and did not before.** llama.cpp only answers `/metrics` when started with `--metrics`, so `up{job="frank-zealot"}` was 0 across the whole retained history (the oldest sample, 2026-10-11, is already 0) while the service answered `/health` 200 the whole time; the flag and its opt-out are on [[frank-zealot]]. Read `lastError` before believing an outage: a target can read down beside a healthy service.
Alert state is a live property, not a page fact: `curl -s http://127.0.0.1:9090/api/v1/alerts` returned no alerts at the 2026-10-18 morning census, one firing `LlmPrefixCacheCollapse` (info, `frank-zealot`, `activeAt` 11:10:48Z) minutes later, and none again by 11:56Z.
A scrape cannot express "is this HTTP service answering?" — that is what a
probe is for, and the remedy is the **blackbox exporter** in probe mode
(`/probe?target=…&module=http_2xx`), which also gives the guarded vhosts a real
uptime signal since Prometheus cannot solve an Anubis challenge itself. Until it
is deployed, these three stay red for a structural reason: deleting them would
The remedy for a probe-shaped question is the **blackbox exporter**; the
classification rule and the honesty law for it live in [[Stale Signal Audit]].
Until the exporter is deployed, these three stay red for a structural reason:
deleting them would
hide a real service, and fixing them by publishing databases or enabling metrics
would trade a wrong signal for a real exposure. Targets are `host:port`, never
URLs.
@@ -1,24 +0,0 @@
---
type: reference
description: "The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /"
tags:
- infrastructure
- authentik
- postgres
timestamp: 2026-08-28
---
# authentik-postgres
The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /
OIDC provider. It is the only stateful store in the stack (users, flows,
sessions, events). Redis was removed in Authentik 2025.10; sessions and caching
now live here.
- **Compose service:** `authentik-postgres` (in `dankclaw/compose.yaml`)
- **Image:** `docker.io/library/postgres:16-alpine`
- **Data:** `/home/dank/authentik/postgres` (bind mount; no named volumes)
- **Health:** `pg_isready -d authentik -U authentik`
See [Authentik.md](Authentik.md) for the full architecture, setup, backups
(`pg_dump`), and failure modes.
@@ -1,24 +0,0 @@
---
type: reference
description: "The Authentik server — the core, the admin API, and the embedded Caddy outpost."
tags:
- infrastructure
- authentik
timestamp: 2026-09-16
---
# authentik-server
The Authentik server — the core, the admin API, and the embedded Caddy outpost.
This is the service the Caddy `forward_auth` vhosts proxy to
(`authentik-server:9000`). It also serves the login page at
`https://auth.franksdank.cloud`.
- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container` (`authentik-server.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `server`)
- **Port:** `9000` (published; the outpost + admin + login all live here)
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the worker)
- **Health:** `http://localhost:9000/-/health/`
See [Authentik.md](Authentik.md) for the full architecture, the Caddy
forward_auth pattern, first-time setup, and failure modes.
@@ -1,23 +0,0 @@
---
type: reference
description: "The Authentik worker — background tasks: event processing, provider"
tags:
- infrastructure
- authentik
timestamp: 2026-09-16
---
# authentik-worker
The Authentik worker — background tasks: event processing, provider
synchronization, scheduled flows, and certificate renewal. It runs the same
image as the server; the `command: worker` selects the role.
- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container` (`authentik-worker.service`)
- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `worker`)
- **Runs as:** `0:0` (rootless: container root maps to the host user that owns the data dir)
- **Data:** `/home/dank/authentik/data` (bind mount, shared with the server)
- **No port** — the worker has no HTTP endpoint.
See [Authentik.md](Authentik.md) for the full architecture, setup, and failure
modes.
@@ -14,7 +14,7 @@ timestamp: 2026-09-03
DFlash and DSpark are the two newest speculative decoding methods. They draft many tokens in one parallel pass instead of one token at a time. This reduces decode latency on memory-bound workloads.
DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both have draft models for the Qwen3.6 models that Dankserver and Dankclaw run.
DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both publish draft models for the Qwen3.5/Qwen3.6 families (table below); what any of them pairs with on this fleet is an era on [[frank-dragoon]] or [[frank-zealot]].
## How DFlash Works
@@ -100,7 +100,7 @@ The DFlash drafter proposes, then ngram-mod runs as an additional implementation
`--spec-type` takes a comma list, and the gate is `ctx_dft` in
`common/speculative.cpp` (`common_speculative_init_result`): **draftless**
types (the ngram family — simple, map-k, map-k4v, mod, cache) are added
unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`
`draft-dspark`, `draft-eagle3`) all require a draft CONTEXT. Consequences:
- A chain may hold ONE context-backed drafter plus any number of ngram types.
@@ -115,12 +115,12 @@ unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
- Internal order is fixed (ngram types first, then drafters); CLI order is
irrelevant.
- **A legal chain can still lose on a recurrent model.** `draft-mtp,ngram-mod`
on the qwen35 hybrid (frank-dragoon era) measured −15 % prose / −7 %
block-copy (2026-10-03): adding the ngram layer logs `recurrent state
rollback is not compatible with 'ngram-mod' - disabling rollback support`,
and MTP verification depends on that rollback, so acceptance drops on ALL
traffic even where the matcher fires. Measure chains per architecture,
never by analogy.
on the qwen35 hybrid loses on both traffic shapes (2026-10-03): adding the
ngram layer logs `recurrent state rollback is not compatible with
'ngram-mod' - disabling rollback support`, and MTP verification depends on
that rollback, so acceptance drops on ALL traffic even where the matcher
fires. The measured size of each loss is [[frank-dragoon]]'s (Laws). Measure
chains per architecture, never by analogy.
- ngram-mod prints NO startup banner — the load log shows only the drafter's
`adding speculative implementation` line. Never conclude "chain not
engaged" from the load log; identify behaviourally (`draft_n` > 0 only
@@ -198,7 +198,10 @@ Compatible MTP-only draft GGUFs on Hugging Face:
Speculative decoding is lossless under rejection sampling: the target verifies every draft token. A draft head from the base model works with a derivative target; wrong proposals just lower the acceptance rate. The accepted MTP draft lengths are 2-3 tokens; the "MTP Defaults Are a Trap" writeup (thefrontierlab.ai) documents the tuning pitfalls.
On Dankclaw, `draft-dflash` remains the benchmarked winner (20.2 t/s at 4k vs 16.9 for DFlash+ngram and 8.7 for ngram-only). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
On the mid-August 2026 Dankclaw shape `draft-dflash` was the benchmarked winner
at 4k — 20.2 t/s, against 16.9 with ngram layered on and 8.7 ngram-only; those
three figures are that retired llama.cpp era's property ([[frank-dragoon]] owns
the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see ).
## Draft Models
@@ -213,13 +216,12 @@ DFlash draft model sizes for the models on these servers:
The DFlash draft is not a standalone model. It must pair with its target model. The model cards recommend block size 8 for higher concurrency and 16 for single-stream throughput.
## Deployment on Dankserver (NVIDIA RTX 3090)
## DFlash on an RTX 3090 (Ampere) — the archived vLLM path
The Dankserver llama.cpp deployment (`frank-zealot`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-zealot` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
DFlash was tried on the 3090 line, lost there to the ngram chain, and was never
kept deployed; that card's payloads, shapes and rates are [[frank-zealot]]'s
(its Eras rows are the llama.cpp arms the draft was tested against). What
transfers is the Ampere arithmetic — compute capability 8.6, so:
- FP8 KV cache is valid on CUDA 12.
- FP8 weights compute through a fallback path; no native FP8 tensor cores.
@@ -248,11 +250,15 @@ docker run --rm --name vllm-qwen36-dflash \
--gpu-memory-utilization 0.95
```
## Deployment on Dankclaw (AMD Strix Halo gfx1151)
## DFlash and DSpark on Strix Halo (gfx1151) — what killed them
`frank-dragoon` on Dankclaw ran the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec was the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. This page is a historical record of that era (mid-August 2026); for the current deployment see [[kingjones-strix-rocmfp4]].
History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only.
Both context-backed drafters died the same way on the iGPU: under parallel load.
DSpark took `ROCm error: an illegal memory access` inside the shared draft
context with two concurrent 16k prompts; DFlash corrupted multi-slot into
premature EOS and was retired first. The timeline, the crash class, the
"concurrency is the trigger, not size" finding and the `PARALLEL=1` workaround
are on ; the shape they were measured on is
[[frank-dragoon]]'s mid-August 2026 era.
## A/B protocol on a live llama.cpp endpoint
@@ -297,7 +303,7 @@ Protocol:
Read the right counters — per request `timings.draft_n` / `draft_n_accepted` /
`predicted_per_second`; cumulative `/metrics`
`llamacpp:spec_decode_num_drafts_total` / `..._accepted_tokens_total`;
`llamacpp:spec_decode_num_drafts_total` / `..._accepted_tokens_total`
per-task `draft acceptance = ... (N accepted / M generated)`. `draft_n == 0`
while tokens flow = the proposer is NOT engaged, whatever the flags say. A
zero-draft config is a useful negative control: it proves the other config's
@@ -315,7 +321,7 @@ not switch without re-measuring".
The A/B vehicle for ANY single-GPU llama.cpp knob (batch/ubatch, KV type,
slot split, quant rung — not just spec flags) is the transient systemd
drop-in procedure in [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]].
drop-in procedure in [[frank-dragoon]] and [[frank-zealot]].
## Is it worth it? (arithmetic before deployment)
@@ -359,24 +365,26 @@ pay for K sequential forwards.
Trained heads are target-specific: EAGLE/DFlash consume the TARGET's hidden
states, so a dense-27B DFlash2 is useless for a 512-expert MoE trunk.
**Step 4 — acceptance reality checks.** Fleet datum: a trained z-lab DFlash2
at depth 7 accepted ~41 % on its matched target (`draft-dflash 7 → 43.5
tok/s`); treat 40–60 % as the honest band for a matched pair. Cheapest proxy
for `p` at temp 0 when both models are greedy and share a tokenizer: greedy
continuation agreement on identical prompts, two API calls, no weights moved.
llama.cpp can also synthesize the projection without any draft artifact:
**Step 4 — acceptance reality checks.** Treat 40–60 % as the honest band for a
matched pair; the arm that anchors the bottom of it on this fleet is the
depth-7 `draft-dflash` row of 's spec table
(accepted share, rate and VRAM for every depth are in that table). Cheapest
proxy for `p` at temp 0 when both models are greedy and share a tokenizer:
greedy continuation agreement on identical prompts, two API calls, no weights
moved. llama.cpp can also synthesize the projection without any draft artifact:
`--spec-synth-rates P0,P1,...` / `--spec-synth-len L`. Speculation pays IN
PROPORTION TO HOW SLOW THE TARGET IS — the same 41 % gave 2.3x on an
18.8 tok/s layer-split target and a loss on a 25 tok/s one; fast targets are
usually sparse MoE, the worst case for drafting.
PROPORTION TO HOW SLOW THE TARGET IS — that band's midpoint was worth 2.3× on
the slow two-card layer-split target and a LOSS on the 25 tok/s target of the
worked example above; fast targets are usually sparse MoE, the worst case for
drafting.
**Step 5 — check the attachment surface BEFORE the arithmetic.** Closed
engines refuse external drafters by design: halogen-flash-server's
`DRAFTER_CAP` gates `dflash2` on the loaded checkpoint and an unavailable
name is a deliberate 400 (a silent downgrade would make comparisons lie);
name is a deliberate 400 (a silent downgrade would make comparisons lie)
there is no device routing in its loopback token protocol, so a drafter can
never sit on another card. Pure llama.cpp DOES route the draft:
`--spec-draft-device/-devd` (default follows `--device`), `-ngld`,
`--spec-draft-device/-devd` (default follows `--device`), `-ngld`
`-otd`, `-ctkd/-ctvd`, `--spec-draft-n-max` (default 3), `--spec-draft-p-min`.
One process sees only the devices its backends were built with — CUDA+Vulkan
is the combination that shows a CUDA card AND an AMD iGPU (HIP+CUDA cannot
@@ -403,20 +411,16 @@ measurement that would settle it.
- Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
- DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
- The old `--swap-space` flag no longer exists in current vLLM. Use `--kv-offloading-size`.
-The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by .
- `--speculative-config` is the preferred CLI. Legacy `--spec-method`, `--spec-model`, and `--spec-tokens` flags still exist.
- Use `rejection_sample_method` default `strict` for lossless output.
## See Also
- [[Quantization by Hardware Architecture]]
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
- [[Running Qwen3.6 27B FP8 on Dankclaw]]
- [[Dankclaw System Specs]]
- [[Dankserver System Specs]]
- [[VLLM Overview]]
- [[deployment]]
- [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in
- [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic
- [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands
- [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]] — the single-GPU env drop-in A/B vehicle
- where the drafter's KV actually lands
- [[frank-dragoon]] and [[frank-zealot]] — the single-GPU env drop-in A/B vehicle
@@ -17,66 +17,59 @@ Dates: researched 2026-08. Hardware is verified on the live machine unless
marked "verify first". Re-verified 2026-08-15/16; corrections marked below.
**Live today (2026-10-18).** The era described below — llama.cpp on the 8060S
iGPU behind :7770 — is retired. Dankclaw now serves
[[frank-dragoon-flash-server-v2-native]] (halogen-flash-server 0.16.2, port
7770) and [[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML fork on
the RTX 3090, port 7771 — drift today: that unit is `failed` since 2026-10-18
03:40 and a hand-run 9B-MTP process answers the port), and boots `iommu=pt` since 2026-10-18
([[dankclaw-iommu-translation-off]]). The numbers below are the measurements of
the era they were taken in; the `ttm` ceiling and the NVMe scheduler rows were
re-checked 2026-10-18 and still hold.
iGPU — is retired; what runs on this box now (both endpoints, their engines and
ports) is [[frank-dragoon]]'s and [[frank-zealot]]'s pages, and both run under their
repo units today — no drift open as of 2026-10-19 (`frank-zealot` active since
2026-10-18 21:06, the deployer's 2-slot reslice; `frank-dragoon` active). The OS facts
are this page's: `iommu=pt` since 2026-10-18 ([[dankclaw-iommu-translation-off]]), and
the `ttm` ceiling and NVMe scheduler rows below were re-checked 2026-10-18 and
still hold. Every benchmark number below is a measurement of the era it was
taken in.
## Corrections (re-verified 2026-08-15/16)
- **Backend**: that era's production was **Vulkan0**, not ROCm. Measured on this box
(b10433): Vulkan 25.13 t/s vs ROCm 24.33 t/s decode at 16k; at 262k ctx
Vulkan decodes 48% faster (5.68 vs 3.83 t/s). ROCm wins prefill (410 vs
302 t/s @64k). The "HIP out-performs Vulkan for decode" claim did not
hold on the 8060S. See [[frank-dragoon-benchmark]].
- **Backend**: that era's production was **Vulkan0**, not ROCm. On this box ROCm
wins prefill and Vulkan wins decode, and the decode gap widens with context:
the 64k A/B is §4, the 262k A/B is
§High-Context Backend Comparison. The "HIP
out-performs Vulkan for decode" claim did not hold on the 8060S.
- **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
does not exist on this machine (checked 2026-08-15). GPU DPM is already
`performance`, forced level `auto`. This item is N/A.
- **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). The law
still holds on `frank-zealot` (`CACHE_TYPE_K=q8_0`, `CACHE_TYPE_V=q8_0` —
`scripts/frank-zealot.env:55-56`).
- **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). What
`frank-zealot` runs now is its own Current shape — this row is not it.
- **Spec**: DFlash/DSpark are retired. That era's production was `--spec-default`
(ngram-mod) with draft-mtp first. See [[qwen3.8-spec-matrix]]. Today `frank-zealot`
is configured with speculation **off** (`SPEC_TYPE=none` — `scripts/frank-zealot.env:67`).
- **TTM**: raised to 114 GiB on 2026-08-25, set live (no reboot) and
persisted in `/etc/modprobe.d/ttm.conf`. The effective VRAM ceiling that
once held the service at ~15 GiB is gone.
(ngram-mod) with draft-mtp first — timeline on ; the
spec setting each endpoint runs today is on its own page.
- **TTM**: the `pages_limit` raise that removed the ~15 GiB ceiling is recorded
once on [[Dankclaw System Specs]] (Memory).
## Verified Memory Picture (measured 2026-08-11)
| Figure | Value | Source |
| ------ | ----- | ------ |
| OS physical RAM | 30.5 GiB | `/proc/meminfo` |
| TTM pages_limit | 114 GiB (29,883,536 x 4KiB) — raised 2026-08-25 | `/sys/module/ttm/parameters/pages_limit` |
| ROCm VRAM total | 96.0 GiB | `rocm-smi --showmeminfo vram` under the then firmware VGM carve |
| ROCm VRAM used | 76.2 GiB | `rocm-smi` while serving 3x200700 |
That 2026-08 reading is the firmware-VGM era: the BIOS carved ~96 GiB as
"dedicated" iGPU VRAM, so the kernel saw only ~30 GiB (the same split is
recorded in [[ling-3.0-flash-trial]] §Why Vulkan + GTT). The OS now accounts
**121 GiB** (`MemTotal 127108080 kB`, 2026-10-18) and `rocm-smi
--showmeminfo vram` reports **2.0 GiB** (2,147,483,648 B) of dedicated VRAM —
the carve is gone and the GPU budget is the GTT aperture, not the SMI figure
([[Dankclaw System Specs]]).
The TTM row above is still live: `pages_limit = 29883536` (114 GiB) and
`/etc/modprobe.d/ttm.conf` still carries the `options ttm` line (both read
2026-10-18).
recorded in §Why Vulkan + GTT; the 76.2 GiB row is an
era [[frank-dragoon]] load reading). Every live number on this box — `MemTotal`
the ~114 GiB GTT aperture that is the real GPU budget, the 2.0 GiB dedicated
carve and the `ttm` page cap — belongs to [[Dankclaw System Specs]], re-read
2026-10-18.
This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
limits documented in [[frank-dragoon-benchmark]].
limits documented in [[frank-dragoon]].
## 1. Self-Hosted Agentic Tools
All of these connect to the local OpenAI-compatible endpoint
(`http://127.0.0.1:7770/v1`). Today that port is **halogen-flash-server**, not a
llama.cpp server, and the address must be the IPv4 literal: `curl
http://localhost:7770/v1/models` returns nothing while
`http://127.0.0.1:7770/v1/models` answers 200 (checked 2026-10-18 — the rootless
podman publish is IPv4-only). None need NVIDIA.
All of these connect to the local OpenAI-compatible endpoint on
`:7770` (which engine owns that port is [[frank-dragoon]]'s to say), and the
address must be the IPv4 literal: `curl http://localhost:7770/v1/models` returns
nothing while `http://127.0.0.1:7770/v1/models` answers 200 (checked 2026-10-18
— the rootless podman publish is IPv4-only). None need NVIDIA.
| Tool | Use | Endpoint config | Priority |
| ---- | --- | --------------- | -------- |
@@ -90,12 +83,13 @@ podman publish is IPv4-only). None need NVIDIA.
| Qdrant | Vector store for RAG | Standalone, CPU-native | Low |
The pragmatic first installs are Open WebUI, Cline, and Aider. They all work
with the endpoint on 7770 and need no extra GPU. The 7770 endpoint today is
halogen-flash-server, not llama.cpp.
with the endpoint on 7770 and need no extra GPU.
## 2. Inference Runtime Tuning (llama.cpp on gfx1151)
Verified against the running service and llama.cpp source.
Verified against that era's running service and llama.cpp source; the era is
retired ([[frank-dragoon]]), so this table is history and a method, not a live
TODO list.
| Change | Recommendation | Priority |
| ------ | -------------- | -------- |
@@ -104,8 +98,8 @@ Verified against the running service and llama.cpp source.
| Flash attention | Leave `--flash-attn auto`. It helps prompt processing, not decode. | Keep |
| Load mode | Start with mmap (default). Enable `--mlock` only if page-fault stutter appears. | Optional |
| Threads | Use `--threads 12 --threads-batch 16 --threads-http 1` for 16 Zen5 cores + 3 slots. Frees cores for the ROCm driver. | Medium |
| DFlash n_max | Current 8 (block 16) is high. Try `--spec-draft-n-max 5`; it may reduce rejection cost at the same throughput. | Verify |
| ngram layering | Measured slower than DFlash alone at 4k (16.9 vs 20.2 t/s). Not enabled. | Keep off |
| DFlash n_max | Depth 8 (block 16) was high; a shallower draft cuts rejection cost at the same throughput. Sweep depth per the A/B protocol in [[DFlash DSpark Speculative Decoding]]. | Verify |
| ngram layering | Layering ngram on the DFlash draft measured slower than the draft alone at 4k — figures on [[DFlash DSpark Speculative Decoding]]. | Keep off |
## 3. Server / OS / Hardware Settings
@@ -113,7 +107,7 @@ All require root and a reboot for the modprobe change.
| Setting | Action | Effect | Priority |
| ------- | ------ | ------ | -------- |
| TTM GPU pool | `/etc/modprobe.d/ttm.conf` → `options ttm pages_limit=29883536 page_pool_size=29883536` (applied 2026-08-25, set live then persisted) | Lifts the 15 GiB cap to 114 GiB; the full 262k KV + Q8 AGENT payload fit in one slot | Done |
| TTM GPU pool | Raised 2026-08-25; the knob, its value and its persistence file are [[Dankclaw System Specs]]'s (Memory row) | Lifted the ~15 GiB kernel-page ceiling so a full 262k KV plus a ~30 GB payload fits one slot | Done |
| Power profile | **N/A on this machine** (2026-08-15; re-checked 2026-10-18: no `power_profile` under `/sys/class/drm/card0/device/` or `card1/device/`); GPU DPM already `performance` | none | ~~Medium~~ |
| NVMe scheduler | Already `none` (`[none] mq-deadline` in `/sys/block/nvme0n1/queue/scheduler`, re-read 2026-10-18) | No change | Done |
| BIOS VGM | Set Variable Graphics Memory / UMA to "High" in the EVO-X2 BIOS | Max dedicated VRAM for the GPU carve-out; verify in the live BIOS | Verify |
@@ -130,16 +124,14 @@ All require root and a reboot for the modprobe change.
## Sources
- [[frank-dragoon-benchmark]] — measured backend/spec numbers
- [[deployment]] — deploy and update process
- [[frank-dragoon]] — measured backend/spec numbers
- deploy and update process
- AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS)
- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`,
- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`
`common/arg.cpp`, `common/common.h`
- NotebookCheck Ryzen AI MAX+ 395 analysis
## See Also
- [[frank-dragoon-benchmark]]
- [[frank-dragoon]]
- [[DFlash DSpark Speculative Decoding]]
- [[deployment]]
- [[index]]
@@ -10,7 +10,7 @@ timestamp: 2026-09-03
## Server Overview
Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[ciru-activefpx-promptforge]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon-flash-server-v2-native]] (halogen-flash-server 0.16.2 on the iGPU, port 7770) and [[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[frank-dragoon]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon]] (halogen-flash-server 0.16.2 on the iGPU, port 7770) and [[frank-zealot]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
## Hardware Specifications
### CPU
@@ -32,7 +32,7 @@ Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp
| Device ID | 0x1586 |
| Architecture | gfx1151 (Strix Halo) |
| GPU-Visible Memory | ~114 GiB GTT aperture (the budget; `mem_info_gtt_total` = 122,402,963,456 B read 2026-10-18; rocm-smi's smaller SVM carve-out is not the budget) |
| Physical RAM | 121 GiB visible to the OS (`free -g` total 121, read 2026-10-18) on a 128 GB part — 8 × 16 GB LPDDR5 ([[ling-3.0-flash-trial]] §hardware). The 64 GiB this page recorded as "corrected 2026-08-17" (from 32 GB) is stale |
| Physical RAM | 121 GiB visible to the OS (`free -g` total 121, read 2026-10-18) on a 128 GB part — 8 × 16 GB LPDDR5 ( §hardware). The 64 GiB this page recorded as "corrected 2026-08-17" (from 32 GB) is stale |
### NPU
@@ -44,9 +44,9 @@ Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp
| Driver | amdxdna 0.7.0, in-tree — bound since 2026-10-18 (`lspci -k -s c6:00.1` → `Kernel driver in use: amdxdna`; journal 2026-10-18 01:25:14 `Initialized amdxdna_accel_driver 0.7.0 for 0000:c6:00.1 on minor 0`) |
| Firmware | 1.1.2.65 (`/usr/lib/firmware/amdnpu/17f0_11/npu_7.sbin` → `npu.sbin.1.1.2.65.zst`, loaded at boot 2026-10-18 01:25:13) |
| Runtime | None installed. FastFlowLM 0.9.45 + XRT (installed 2026-08-16) was **retired 2026-09-11** with the `amd_iommu=off` decision — see [[dankclaw-iommu-translation-off]]; the package remains at `/home/dank/GitPackages/fastflowlm_0.9.45_ubuntu26.04_amd64.deb` |
| Status | Not in service — no NPU endpoint is deployed. The device itself is usable since the 2026-10-18 `iommu=pt` boot (`/dev/accel/accel0` present, `crw-rw---- root:render 261,0`); Gitea issue #3 tracks enablement. See [[FastFlowLM NPU Inference]] |
| Status | Not in service — no NPU endpoint is deployed. The device itself is usable since the 2026-10-18 `iommu=pt` boot (`/dev/accel/accel0` present, `crw-rw---- root:render 261,0`); Gitea issue #3 tracks enablement. See |
The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name now serves **Ternary-Bonsai-2-27B-PQ2_0** on the RTX 3090 via the llama.cpp PrismML fork (see [[frank-zealot-llama-prism-bonsai2-pq20]]); the Gemma4-12B QAT rung it once pointed at is retired history ([[frank-zealot-gemma4-12b]], 2026-09-17 → 10-01).
The NPU endpoint is retired (2026-09-11; ). The `frank-zealot` name now serves **Ternary-Bonsai-2-27B-PQ2_0** on the RTX 3090 via the llama.cpp PrismML fork (see ); the Gemma4-12B QAT rung it once pointed at is retired history (, 2026-09-17 → 10-01).
### ROCm
@@ -66,6 +66,7 @@ The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name
| System RAM | 121 GiB (shared CPU + GPU; `free -g` total 121, `MemTotal 127108080 kB`, 2026-10-18) |
| GPU-Visible (ROCm) | ~114 GiB GTT aperture (`mem_info_gtt_total` 122,402,963,456 B — the budget) |
| Swap | 64 GB `/swap.img` (`swapon`, 2026-10-18; `free` reports 63 GiB) |
| TTM page cap | `ttm.pages_limit = 29883536` (114 GiB of 4 KiB pages) and `page_pool_size=29883536`, persisted in `/etc/modprobe.d/ttm.conf`, applied live 2026-08-25 and re-read 2026-10-18 (`/sys/module/ttm/parameters/pages_limit`). The kernel default is half of RAM, so in the firmware-VGM era (~30 GiB visible) the cap was ~15 GiB and a 28 GB payload plus a 262k KV cache could not load — this is the row every other page should cite |
| /dev/shm | 61 GiB tmpfs (half of RAM, `df /dev/shm`, 2026-10-18) |
### Storage
@@ -79,7 +80,7 @@ The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name
| Interface | Type | Speed | IP | Notes |
|-----------|------|-------|----|-------|
| eno1 | onboard | 2500 Mb/s (2.5 GbE) | 192.168.0.143 (DHCP) | Carried the RPC path to Dankserver during the 2026-08-17 pipeline-parallelism trial; no RPC listener runs today (`ss -ltn` shows no :50052, 2026-10-18). Default route via 192.168.0.1. |
| enx00e04cc2e1e5 | USB-C to Ethernet adapter | 1000 Mb/s | 192.168.0.116 | UNPLUGGED 2026-08-17. While present, the route to 192.168.0.235 went through this adapter (lower route metric), so all RPC traffic was capped at 1 GbE. See [[Pipeline Parallelism/Ethernet Bandwidth Math]] section 0. |
| enx00e04cc2e1e5 | USB-C to Ethernet adapter | 1000 Mb/s | 192.168.0.116 | UNPLUGGED 2026-08-17. While present, the route to 192.168.0.235 went through this adapter (lower route metric), so all RPC traffic was capped at 1 GbE. See section 0. |
| docker0 | bridge | - | 172.17.0.1 | Usually down. |
The L2 gateway/switch at 192.168.0.1 (MAC f0:4b:8a:65:c5:40) is at
@@ -95,21 +96,19 @@ cable between claw and server is required for 10G.
| Kernel | 7.0.0-38-generic (`uname -r`, 2026-10-18) |
| Boot line | `iommu=pt` since 2026-10-18 (`/proc/cmdline`, no `amd_iommu` token); eras before: `amd_iommu=off` 2026-09-11 → 10-18, IOMMU default-on before that — [[dankclaw-iommu-translation-off]] |
| Python | 3.14.4 (system, `python3 --version`, 2026-10-18). The 3.12.13 CIRU venv left with that runtime (retired 2026-08-22) |
| CIRU runtime | ciru-ai/ROCmFPX — **retired 2026-08-22** ([[ciru-activefpx-promptforge]]); not the serving runtime |
| CIRU runtime | ciru-ai/ROCmFPX — **retired 2026-08-22** ([[frank-dragoon]]); not the serving runtime |
| uv | 0.12.8 (`~/.local/bin/uv --version`, 2026-10-18) |
## Runtime Relevance
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
- The GPU budget is the GTT aperture: ~114 GiB (`mem_info_gtt_total` in `/sys/class/drm/card1/device/` = 122,402,963,456 B; `card1` is the amdgpu iGPU, `PCI_ID=1002:1586`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.16.2** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon-flash-server-v2-native]]) — and `frank-zealot` on 7771 ([[frank-zealot-llama-prism-bonsai2-pq20]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon-gufo-qwen38-27b]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.16.2** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon]]) — and `frank-zealot` on 7771 ([[frank-zealot]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
- The gufo-era note that the active runtime links against the host ROCm (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH belongs to that retired source build. The serving runtimes do not depend on the host ROCm that way: halogen runs inside its digest-pinned image, and `frank-zealot` is a CUDA build on the 3090. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) was never in the serving path either.
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
- 424 GB free on `/` (`df -h`, 2026-10-18) is ample for model storage.
## See Also
- [[ciru-activefpx-promptforge]]
- [[frank-dragoon]]
- [[Shared Memory Architecture Dankclaw]]
- [[frank-redo-2026-08-17]]
- [[FastFlowLM NPU Inference]]

Some files were not shown because too many files have changed in this diff Show More