Compare commits
26
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1d082a7f1d | ||
|
|
5c8ac208a0 | ||
|
|
dbc2c67bcb | ||
|
|
1eb8b2199a | ||
|
|
0e4bb327da | ||
|
|
e030486ce6 | ||
|
|
d89ae23195 | ||
|
|
114dd7a93e | ||
|
|
e2393bacad | ||
|
|
ff11c37ee0 | ||
|
|
0fe23c5233 | ||
|
|
716eefef5d | ||
|
|
8816a9d498 | ||
|
|
032229c1e2 | ||
|
|
09b0badaf5 | ||
|
|
0e7936f94e | ||
|
|
f206ddfde5 | ||
|
|
bf29518a15 | ||
|
|
169dbe2f38 | ||
|
|
4a6d5b8191 | ||
|
|
49c4e198fd | ||
|
|
31266fb7f3 | ||
|
|
06a9cb60eb | ||
|
|
e7841bf9d6 | ||
|
|
c0759492e6 | ||
|
|
695a5f3281 |
No files matched your search
@@ -26,8 +26,8 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
|||||||
| Target | Service | Runtime | Model | Port |
|
| Target | Service | Runtime | Model | Port |
|
||||||
|--------|---------|---------|-------|------|
|
|--------|---------|---------|-------|------|
|
||||||
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
|
||||||
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool (= 2 × ctx) at `HALOGEN_CTX=200704`** (pool RAISED from 200,704 on 2026-10-18 so BOTH slots can hold a maximum-length request at once; the parity play was frank-zealot's then-401,408 total in-flight context — zealot moved to 786,432 the same evening and dragoon's pool was NOT raised again, because every position is page cache the n-gram table loses; ~10.8 GiB reserved where 200,704 cost 5.5, 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; the ceiling does NOT move with it — 262,144 is the native wall and the engine refuses more without `HALOGEN_ROPE_YARN`), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770|
|
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool (= 2 × ctx) at `HALOGEN_CTX=200704`** (pool RAISED from 200,704 and DEPLOYED 2026-10-18 so BOTH slots can hold a maximum-length request at once; the parity play was frank-zealot's then-401,408 total in-flight context — zealot moved to 786,432 the same evening and dragoon's pool was NOT raised again, because every position is page cache the n-gram table loses; **11.0 GiB reserved, measured on that deploy** (the preflight formula bills 11.3) where 200,704 cost 5.5, 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; the ceiling does NOT move with it — 262,144 is the native wall and the engine refuses more without `HALOGEN_ROPE_YARN`), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770|
|
||||||
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10777**, commit `e3de71522`; root systemd unit, relinked — the ONLY loader for this payload, and since 2026-10-18 also the minimum build that pairs MTP with a quantized V cache) | `prism-ml/Ternary-Bonsai-2-9B-gguf` **PQ2_0 carrying the MTP head in the file** (2,675,410,720 B, Hub-LFS sha256 `c79fcdfa…` verified at fetch by `scripts/download-frank-zealot-bonsai9.sh`; `qwen35` hybrid: `block_count 33` = **32 trunk layers** (8 full-attention + 24 Gated DeltaNet at `full_attention_interval 4`) + the `blk.32` MTP block, 4 KV heads × 256 dim, native 262,144; `general.file_type 141` with **202 custom-type PQ2_0 tensors of 442** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (624,229,664 B, sha256 `e0cb6f69…`; **Q8_0 is the publisher's requirement** for concurrent image requests — the BF16 tower can crash the server, and this endpoint runs three slots) for **vision**, on the RTX 3090, **THREE 262,144-token slots** (`--ctx-size 786432 --parallel 3` = every conversation at the trained window), **q8_0 K + q8_0 V** under flash attention (17 KiB/token over the 8 full layers = 13,056 MiB pool, ledger-exact; **f16 KV needs 26,112 MiB and cannot boot this card**; +452 MiB of recurrent state for the three DeltaNet cells), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF — measured, not because the head is absent**: the draft context bills its own f16 KV over the WHOLE pool (3,072 MiB) and at ubatch 2048 that squeeze halves decode (167.2 → 109.2 t/s); the documented MTP arm is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s. Boot 17,876 used / 6,249 free; 167.2 t/s single-stream decode, 280.2 t/s aggregate across three concurrent streams, 2,877 t/s prefill at 106,418. Cutover 2026-10-18 from Ternary-Bonsai-2-**27B** PQ2_0 at 2 × 200,704 (payload + build 10754 tree on disk as rollback). | 7771 |
|
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10777**, commit `e3de71522`; root systemd unit, relinked — the ONLY loader for this payload, and since 2026-10-18 also the minimum build that pairs MTP with a quantized V cache) | `prism-ml/Ternary-Bonsai-2-9B-gguf` **PQ2_0 carrying the MTP head in the file** (2,675,410,720 B, Hub-LFS sha256 `c79fcdfa…` verified at fetch by `scripts/download-frank-zealot-bonsai9.sh`; `qwen35` hybrid: `block_count 33` = **32 trunk layers** (8 full-attention + 24 Gated DeltaNet at `full_attention_interval 4`) + the `blk.32` MTP block, 4 KV heads × 256 dim, native 262,144; `general.file_type 141` with **202 custom-type PQ2_0 tensors of 442** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (624,229,664 B, sha256 `e0cb6f69…`; **Q8_0 is the publisher's requirement** for concurrent image requests — the BF16 tower can crash the server, and this endpoint runs three slots) for **vision**, on the RTX 3090, **THREE 262,144-token slots** (`--ctx-size 786432 --parallel 3` = every conversation at the trained window), **q8_0 K + q8_0 V** under flash attention (17 KiB/token over the 8 full layers = 13,056 MiB pool, ledger-exact; **f16 KV needs 26,112 MiB and cannot boot this card**; +151 MiB of recurrent state for the three DeltaNet cells — the SPEC-ARMED arms print 452 MiB, because MTP depth 2 bills three seqs per slot), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF — measured, not because the head is absent**: the draft context bills its own f16 KV over the WHOLE pool (3,072 MiB) and at ubatch 2048 that squeeze halves decode (167.2 → 109.2 t/s); the documented MTP arm is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s. Production boot 17,922 used / 6,203 free; **168.7 t/s** single-stream greedy decode (three runs, 168.4–168.7) and **~300 t/s** aggregate across three concurrent streams (102.8 each); 2,877 t/s prefill at 106,418 was the trial arm. Cutover 2026-10-18 from Ternary-Bonsai-2-**27B** PQ2_0 at 2 × 200,704 (payload + build 10754 tree on disk as rollback). | 7771 |
|
||||||
|
|
||||||
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
||||||
the [runbook index](#runbook-index). The full swap history lives in
|
the [runbook index](#runbook-index). The full swap history lives in
|
||||||
@@ -48,17 +48,37 @@ the [runbook index](#runbook-index). The full swap history lives in
|
|||||||
pages (`serverAppConfigs-docs/log.md` carries the wave plan and results)
|
pages (`serverAppConfigs-docs/log.md` carries the wave plan and results)
|
||||||
- `tests/` — bats suite + Playwright e2e
|
- `tests/` — bats suite + Playwright e2e
|
||||||
|
|
||||||
## Git Workflow
|
## Git Workflow — trunk-based (fleet law, aligned 2026-10-18)
|
||||||
|
|
||||||
All edits flow through git. Never edit files on a remote machine directly.
|
All edits flow through git. Never edit files on a remote machine directly.
|
||||||
|
|
||||||
1. **Source** the live config from the target machine
|
`main` is the only long-lived branch: no `develop`, no release branches, no
|
||||||
2. **Commit** the fetched version here (redact secrets into `.env`)
|
back-merges. Frank is the only dev; this repo and `dank/command-center` follow
|
||||||
3. **Edit** locally
|
the same regime.
|
||||||
4. **Test** — `./tests/run.sh` must pass
|
|
||||||
5. **Commit and push** — origin is Gitea on Dankclaw
|
1. **Branch fresh** — `git fetch`, then branch off the current `origin/main`
|
||||||
6. **Pull** on the target (`deploy.sh` does this)
|
(`<type>/<slug>`, type ∈ feat|fix|docs|chore). If main moves under you,
|
||||||
7. **Deploy** — `scripts/deploy.sh` plans and dispatches only changed services
|
merge it into your branch — never rebase.
|
||||||
|
2. **Source** the live config from the target machine
|
||||||
|
3. **Commit** the fetched version here (redact secrets into `.env`)
|
||||||
|
4. **Edit** locally
|
||||||
|
5. **Test** — `./tests/run.sh` must pass
|
||||||
|
6. **Push the branch and open a PR against `main`** — origin is Gitea on
|
||||||
|
Dankclaw. Merge with `--no-ff`; **never merge a PR without an explicit,
|
||||||
|
current OK from Frank.** **The moment the PR is open, open it in Frank's
|
||||||
|
browser** (`xdg-open <PR-URL>` — a Firefox tab on the workstation); never
|
||||||
|
make him hunt for it in Gitea. Releases: tag `vX.Y.Z` straight on `main`.
|
||||||
|
7. **Pull** on the target (`deploy.sh` does this)
|
||||||
|
8. **Deploy** — `scripts/deploy.sh` plans and dispatches only changed services
|
||||||
|
|
||||||
|
**Attribution footer (issues and PRs)** — every issue or PR body you open MUST end with:
|
||||||
|
|
||||||
|
```
|
||||||
|
---
|
||||||
|
cwd: <working directory> · session: <omp session id>
|
||||||
|
```
|
||||||
|
|
||||||
|
Session id from `/status`; `cwd` is that session's own working directory — not the build checkout, when they differ (note the build worktree in the body instead). Machine-made artifacts stay attributable to the box and session that spawned them.
|
||||||
|
|
||||||
## Deployment
|
## Deployment
|
||||||
|
|
||||||
@@ -231,8 +251,12 @@ archive.
|
|||||||
**STOP the production service BEFORE benchmarking** — these servers serve
|
**STOP the production service BEFORE benchmarking** — these servers serve
|
||||||
real users, and a bench beside a live service starves the GPU and corrupts
|
real users, and a bench beside a live service starves the GPU and corrupts
|
||||||
results. Use the runtime's own harness: `llama-bench` for the llama.cpp
|
results. Use the runtime's own harness: `llama-bench` for the llama.cpp
|
||||||
endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-dragoon and frank-zealot were measured. Rolling your own bench
|
endpoints; a live llama.cpp server (frank-zealot) also reports honest
|
||||||
requires asking first.
|
per-request numbers in its `/completion` `timings` object (prefill/decode
|
||||||
|
rates, `draft_n`, `draft_n_accepted`), which is how frank-zealot was measured.
|
||||||
|
halogen has NO `/completion` path at all — frank-dragoon's rates come from
|
||||||
|
wall-clock around `/v1/chat/completions` plus the boot ledger. Rolling your own
|
||||||
|
bench requires asking first.
|
||||||
Scratch ports are not used on LLM hosts — a
|
Scratch ports are not used on LLM hosts — a
|
||||||
test process cannot share the single GPU anyway; validate against the service
|
test process cannot share the single GPU anyway; validate against the service
|
||||||
itself.
|
itself.
|
||||||
@@ -250,9 +274,13 @@ real text made it 14 % *slower*. Use it for shape, not for A/B. For an A/B,
|
|||||||
replay **real** text through the live server and read the `Server-Timing`
|
replay **real** text through the live server and read the `Server-Timing`
|
||||||
header — but note the unit rides in the **name** half (`ttft;dur=402.485`),
|
header — but note the unit rides in the **name** half (`ttft;dur=402.485`),
|
||||||
so splitting on `=` alone keys the dict `ttft;dur` and every lookup silently
|
so splitting on `=` alone keys the dict `ttft;dur` and every lookup silently
|
||||||
returns nothing. Generate **at least 128 tokens** for the measurement: at 8
|
returns nothing. **That header is a llama.cpp feature**: halogen flash-server
|
||||||
tokens, `inter_token` is first-token noise and the run-to-run spread swamps
|
0.16.2 answers `/v1/chat/completions` with only `date`, `server` and
|
||||||
every real difference.
|
`content-*` (verified 2026-10-18), and `/completion`, `/ping`, `/props` and
|
||||||
|
`/tokenize` are all 404 on it — on frank-dragoon the A/B instrument is
|
||||||
|
wall-clock around the POST plus `usage.prompt_tokens`. Generate **at least 128
|
||||||
|
tokens** for the measurement: at 8 tokens, `inter_token` is first-token noise
|
||||||
|
and the run-to-run spread swamps every real difference.
|
||||||
|
|
||||||
This covers **any GPU experiment, not just benchmarks** — including a bare
|
This covers **any GPU experiment, not just benchmarks** — including a bare
|
||||||
2-rank RCCL/`torch.distributed` probe. On 2026-09-16 one such probe was run
|
2-rank RCCL/`torch.distributed` probe. On 2026-09-16 one such probe was run
|
||||||
@@ -407,10 +435,10 @@ MAXIMUM-length conversations DO coexist. The PER-REQUEST ceiling stays
|
|||||||
`HALOGEN_CTX=200704`: the checkpoint's native window is 262,144 and the engine's
|
`HALOGEN_CTX=200704`: the checkpoint's native window is 262,144 and the engine's
|
||||||
own 400 text refuses to exceed it without `HALOGEN_ROPE_YARN`, so "more context"
|
own 400 text refuses to exceed it without `HALOGEN_ROPE_YARN`, so "more context"
|
||||||
bought here is CONCURRENCY, not a longer single conversation. ~28 KiB/position ⇒
|
bought here is CONCURRENCY, not a longer single conversation. ~28 KiB/position ⇒
|
||||||
the ledger reserves "about 10.8 GiB", where 200,704 cost 5.5, 262,144 cost 7.2
|
the 2026-10-18 deploy RESERVES **11.0 GiB** (`reserving the KV pool, 401408
|
||||||
and the 2026-10-06 shape's 524,288 measured 14.6; the engine's preflight FORMULA
|
positions (about 11.0 GiB)`; the preflight formula bills 11.3 with block scratch
|
||||||
bills slightly more than the reservation (block scratch + ~115 MiB/slot of O(1)
|
+ ~115 MiB/slot of O(1) state), where 200,704 cost 5.5, 262,144 cost 7.2 and
|
||||||
state) — quote the reservation.
|
the 2026-10-06 shape's 524,288 measured 14.6 — quote the reservation.
|
||||||
`HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
`HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
||||||
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
||||||
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
|
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
|
||||||
@@ -428,10 +456,12 @@ stays mandatory. Thinking tokens COUNT against the budget (empty content +
|
|||||||
"video".
|
"video".
|
||||||
|
|
||||||
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger): weights
|
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger): weights
|
||||||
62.9 (trunk 62.11 + vision 0.84) + KV ~10.8 + working 5.7
|
62.9 (trunk 62.11 + vision 0.84) + KV **11.0** + working 5.7
|
||||||
(`3.8 + 8.4 × MAX_TOK/16384`) ≈ **79.4 GiB in all**, the ~1.0 GiB prompt cache
|
(`3.8 + 8.4 × MAX_TOK/16384`) = **79.6 GiB in all** (boot ledger, the
|
||||||
booked separately by the ledger, + `HALOGEN_HOST_RESERVE_GIB=20` → ~22 GiB
|
401,408-position deploy of 2026-10-18), the **1.3 GiB** prompt cache (12 entries
|
||||||
margin over baseline co-tenancy (the pool-equals-ctx boot printed 74.1 GiB with
|
× 111 MiB at ctx 200704) booked separately by the ledger, + `HALOGEN_HOST_
|
||||||
|
RESERVE_GIB=20` → ~20 GiB margin over baseline co-tenancy (`MemAvailable` read
|
||||||
|
22 GiB right after the raise; the pool-equals-ctx boot printed 74.1 GiB with
|
||||||
~30 GiB of margin). The 51.2 GB n-gram table rides as mmap'd page cache beside
|
~30 GiB of margin). The 51.2 GB n-gram table rides as mmap'd page cache beside
|
||||||
the checkpoint (ledger: `mapped 51.2 GB, NOT pinned (Pin::None)`) — which is why
|
the checkpoint (ledger: `mapped 51.2 GB, NOT pinned (Pin::None)`) — which is why
|
||||||
every pool position is a double cost: **RAM the pool takes is RAM that table
|
every pool position is a double cost: **RAM the pool takes is RAM that table
|
||||||
@@ -494,9 +524,10 @@ so **q8_0 K + q8_0 V bills 17,408 B/token ≈ 17 KiB** = **13,056.00 MiB** at th
|
|||||||
786,432 pool (ledger-exact) — exactly HALF the 27B's 34,816 B/token, which is
|
786,432 pool (ledger-exact) — exactly HALF the 27B's 34,816 B/token, which is
|
||||||
the whole reason three native windows fit where the 27B afforded only two
|
the whole reason three native windows fit where the 27B afforded only two
|
||||||
shortened ones. **f16 K would bill 26,112 MiB and cannot boot on a 24,123 MiB
|
shortened ones. **f16 K would bill 26,112 MiB and cannot boot on a 24,123 MiB
|
||||||
card.** The 24 DeltaNet layers bill a fixed recurrent state per SLOT —
|
card.** The 24 DeltaNet layers bill a fixed recurrent state per SLOT — the
|
||||||
`RS buffer size = 452.25 MiB (3 cells, 32 layers, 3 seqs)` ≈ 150 MiB each — so
|
production ledger prints `RS buffer size = 150.75 MiB (3 cells, 32 layers,
|
||||||
pool arithmetic alone under-prices the shape.
|
3 seqs)`, while the SPEC-ARMED arms print 452.25 MiB (MTP depth 2 bills three
|
||||||
|
seqs per slot, so 9 cells) — so pool arithmetic alone under-prices the shape.
|
||||||
|
|
||||||
**The shape (2026-10-18):** `--ctx-size 786432 --parallel 3` = THREE
|
**The shape (2026-10-18):** `--ctx-size 786432 --parallel 3` = THREE
|
||||||
conversations of 262,144, every one at the model's TRAINED window (Frank: "if
|
conversations of 262,144, every one at the model's TRAINED window (Frank: "if
|
||||||
@@ -1168,6 +1199,26 @@ linked runbooks.
|
|||||||
is broken". Diagnose with `lastError` from `/api/v1/targets`, not the alert list,
|
is broken". Diagnose with `lastError` from `/api/v1/targets`, not the alert list,
|
||||||
and note Alertmanager has no persistence: a reboot re-stamps `startsAt` on an
|
and note Alertmanager has no persistence: a reboot re-stamps `startsAt` on an
|
||||||
alert that has been firing for weeks.
|
alert that has been firing for weeks.
|
||||||
|
- **A leftover bench drop-in silently hijacks the next deploy, and no health
|
||||||
|
gate sees it.** The sanctioned bench-window method — a transient
|
||||||
|
`Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env` drop-in, the arm env
|
||||||
|
`source`ing the repo env and then overriding it — outlives the bench unless
|
||||||
|
it is retired. On 2026-10-18 a leftover `frank-zealot.service.d/ptq1.conf` —
|
||||||
|
the arm Frank had ordered at 02:36 and measured from 03:14 — made the cutover
|
||||||
|
deploy boot a **different model at a different shape** (3 slots of 133,888,
|
||||||
|
not 3 × 262,144) while `/health` answered 200, `/v1/models` echoed the alias
|
||||||
|
(a flag, not a fact) and the Playwright gate passed 33/33. After deploying any
|
||||||
|
recently-benched LLM service: `systemctl show <svc> -p DropInPaths` (must be
|
||||||
|
empty), then `tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot
|
||||||
|
ledger — and retire drop-ins to `~/.AGENTTRASH/<date>-<topic>/` +
|
||||||
|
`daemon-reload`.
|
||||||
|
- **A bench verdict that never reaches the repo is indistinguishable from a
|
||||||
|
rogue arm.** Same window: PTQ1_0 served production for 24 minutes with real
|
||||||
|
numbers (~48 t/s decode, 801 t/s prefill at 141k), but the figures lived only
|
||||||
|
in `/tmp` env files and journald, so the next agent on the host read the
|
||||||
|
surviving drop-in as an unfinished experiment and reported a hijack. Write the
|
||||||
|
measurement into the runbook and the arm script's header in the session that
|
||||||
|
takes it — `/tmp` and the journal are not records.
|
||||||
|
|
||||||
**Ops**
|
**Ops**
|
||||||
|
|
||||||
|
|||||||
@@ -79,3 +79,23 @@ MARRETA_APP_KEY=changeme
|
|||||||
# account on first boot from these values.
|
# account on first boot from these values.
|
||||||
MARRETA_ADMIN_EMAIL=admin@franksdank.cloud
|
MARRETA_ADMIN_EMAIL=admin@franksdank.cloud
|
||||||
MARRETA_ADMIN_PASSWORD=changeme
|
MARRETA_ADMIN_PASSWORD=changeme
|
||||||
|
|
||||||
|
# open-notebook (podcast generator; internal-only, no vhost).
|
||||||
|
# deploy-remote.sh seeds /home/dank/Containers/open-notebook/env from these
|
||||||
|
# values (only if that file does not already exist) and refuses to start the
|
||||||
|
# units while any of them is blank: a blank OPEN_NOTEBOOK_PASSWORD means
|
||||||
|
# authentication OFF, and a blank encryption key means no provider credential
|
||||||
|
# can be saved. The seed writes SURREAL_PASS too — the spelling the `surreal`
|
||||||
|
# CLI reads — from SURREAL_PASSWORD, so the app and the database cannot be
|
||||||
|
# given different values.
|
||||||
|
# OPEN_NOTEBOOK_ENCRYPTION_KEY: encrypts the stored AI provider keys at rest.
|
||||||
|
# Generate: openssl rand -hex 32
|
||||||
|
# Changing it after credentials are saved makes them undecryptable.
|
||||||
|
OPEN_NOTEBOOK_ENCRYPTION_KEY=changeme
|
||||||
|
# UI/API password. Every API route — including the episode-audio download the
|
||||||
|
# app serves without a session — is open to the LAN while this is unset.
|
||||||
|
OPEN_NOTEBOOK_PASSWORD=changeme
|
||||||
|
# SurrealDB root password (the root user NAME comes from
|
||||||
|
# scripts/open-notebook.env, so this file holds only the secret).
|
||||||
|
# Generate: openssl rand -base64 32
|
||||||
|
SURREAL_PASSWORD=changeme
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
# Quadlet unit: open-notebook's SurrealDB (rootless, boot-persistent).
|
||||||
|
#
|
||||||
|
# Install path on dankclaw: ~/.config/containers/systemd/open-notebook-surrealdb.container
|
||||||
|
# Unit name: open-notebook-surrealdb.service (container name: open-notebook-surrealdb)
|
||||||
|
#
|
||||||
|
# WHAT IT IS. open-notebook's only datastore: notebooks, notes, sources,
|
||||||
|
# provider credentials and podcast episodes all live here. The app talks to it
|
||||||
|
# over the WebSocket RPC on the private bridge (SURREAL_URL in
|
||||||
|
# scripts/open-notebook.env), which is why the container NAME is part of the
|
||||||
|
# contract — renaming it breaks the app's database URL, not just this file.
|
||||||
|
#
|
||||||
|
# IMAGE. `surrealdb/surrealdb:v2` pinned by manifest-list digest, resolved from
|
||||||
|
# the registry on 2026-10-18 (that digest is SurrealDB 2.7.0, image built
|
||||||
|
# 2026-09-23; `v2` itself is a MOVING tag, so the digest — not the tag — is what
|
||||||
|
# makes this reproducible). Upstream open-notebook's own compose file pins the
|
||||||
|
# same `v2` tag, and its Dockerfile copies /surreal out of the same image for
|
||||||
|
# the retired single-container variant, so v2 is the version this app is built
|
||||||
|
# against. Pull=never + the documented pull step (docs: [[Open Notebook]]):
|
||||||
|
#
|
||||||
|
# podman pull docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
|
||||||
|
#
|
||||||
|
# NO PUBLISHED PORT. 8000 is deliberately not published: the app reaches it by
|
||||||
|
# name on systemd-open-notebook, and a published 8000 would put the RPC with
|
||||||
|
# the root credentials on every interface (upstream publishes 127.0.0.1:8000
|
||||||
|
# "purely for local debugging" — a `podman exec` probe is enough here). The
|
||||||
|
# binary binds 0.0.0.0:8000 INSIDE its own namespace via the image's
|
||||||
|
# SURREAL_BIND env; without that image default it would bind loopback only and
|
||||||
|
# the app could never connect, so no --bind flag belongs here.
|
||||||
|
#
|
||||||
|
# CREDENTIALS, AND WHY THEY ARE NOT IN THE COMMAND LINE. The `surreal start`
|
||||||
|
# CLI reads its options from the environment (`--username` is [env:
|
||||||
|
# SURREAL_USER=], `--password` is [env: SURREAL_PASS=] — verified against the
|
||||||
|
# pinned image's own `start --help`), so the credentials arrive through the two
|
||||||
|
# EnvironmentFile lines and the argv stays clean. Passing `--pass` instead —
|
||||||
|
# what upstream's compose does — would put the database password in
|
||||||
|
# /proc/<pid>/cmdline, readable by any `ps` on the host. Note the two
|
||||||
|
# spellings: the DATABASE reads SURREAL_PASS, the APP reads SURREAL_PASSWORD,
|
||||||
|
# and scripts/open-notebook.env declares both because they must hold the same
|
||||||
|
# value; the deployer writes that pair when it seeds the host secret file.
|
||||||
|
#
|
||||||
|
# NO USER= OVERRIDE. The image runs as uid 65532 (Chainguard glibc-dynamic
|
||||||
|
# base). With a NAMED volume that works unmodified: podman creates the volume's
|
||||||
|
# backing directory owned by the container's user (measured with this digest —
|
||||||
|
# the host directory lands on the mapped subuid and the store starts clean),
|
||||||
|
# which is exactly why upstream's `user: root` line is not repeated here: that
|
||||||
|
# line is commented in upstream as being "required for bind mounts", and this
|
||||||
|
# unit has none. The consequence is that the DB files are NOT readable by dank
|
||||||
|
# without `podman unshare` — accepted for a datastore nobody edits by hand;
|
||||||
|
# episode AUDIO is a host directory for the opposite reason (see the app unit).
|
||||||
|
#
|
||||||
|
# HEALTH. The base image is distroless: no shell, no curl, no wget, so no
|
||||||
|
# CMD-SHELL probe and no wget/curl probe is possible here (same class of unit
|
||||||
|
# as loki, whose probe had to move into the image). SurrealDB ships its own
|
||||||
|
# probe instead — `surreal is-ready` connects to ws://localhost:8000 by default
|
||||||
|
# and needs no credentials — verified against this digest: it prints `OK` and
|
||||||
|
# exits 0 while the server is up. Argument-only HealthCmd, like every unit on
|
||||||
|
# this host, because the quadlet generator splits it on spaces.
|
||||||
|
|
||||||
|
[Unit]
|
||||||
|
Description=SurrealDB (open-notebook datastore)
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Container]
|
||||||
|
ContainerName=open-notebook-surrealdb
|
||||||
|
Image=docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
|
||||||
|
Pull=never
|
||||||
|
Network=open-notebook.network
|
||||||
|
EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env
|
||||||
|
EnvironmentFile=/home/dank/Containers/open-notebook/env
|
||||||
|
Exec=start --log info rocksdb:/mydata/mydatabase.db
|
||||||
|
Volume=open-notebook-db:/mydata
|
||||||
|
HealthCmd=/surreal is-ready
|
||||||
|
HealthInterval=30s
|
||||||
|
HealthTimeout=10s
|
||||||
|
HealthRetries=3
|
||||||
|
HealthStartPeriod=20s
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Restart=always
|
||||||
|
RestartSec=5
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# Quadlet unit: open-notebook (rootless, boot-persistent).
|
||||||
|
#
|
||||||
|
# Install path on dankclaw: ~/.config/containers/systemd/open-notebook.container
|
||||||
|
# Unit name: open-notebook.service (container name: open-notebook)
|
||||||
|
#
|
||||||
|
# WHAT IT IS. The fleet's podcast generator: notes and sources go in, a
|
||||||
|
# two-voice (up to four) episode MP3 comes out. One image runs THREE processes
|
||||||
|
# under supervisord — the FastAPI API on 5055, the background worker that
|
||||||
|
# writes and speaks the episode, and the Next.js UI on 8502 — so this one unit
|
||||||
|
# owns the whole app, and the frontend refuses to start until the API answers
|
||||||
|
# (the image's own wait-for-api.sh). Runbook: [[Open Notebook]].
|
||||||
|
#
|
||||||
|
# IMAGE. `lfnovo/open_notebook:v1-latest` pinned by manifest-list digest,
|
||||||
|
# resolved from the registry on 2026-10-18; it is the tag upstream's own
|
||||||
|
# docker-compose.yml pins for the multi-container shape. Do NOT reach for the
|
||||||
|
# `-single` tags: that variant bundles SurrealDB inside the app image, is
|
||||||
|
# deprecated upstream, and would give this host two databases. Pull=never, so
|
||||||
|
# the pin is fetched by the documented one-time step (docs: [[Open Notebook]]),
|
||||||
|
# which the deployer also runs when the image is absent:
|
||||||
|
#
|
||||||
|
# podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
|
||||||
|
#
|
||||||
|
# NETWORKS, in this order on purpose. The generated run line keeps file order,
|
||||||
|
# so the DEFAULT ROUTE rides dankclaw_default: that is the network where this
|
||||||
|
# host has measured that a bridge container can reach a host-namespace
|
||||||
|
# listener through the gateway name (the Prometheus scrape of frank-zealot is
|
||||||
|
# the proof — see [[Container Network Namespaces]]), and frank-tts lives there
|
||||||
|
# too, so the TTS credential can be `http://frank-tts:7772/v1`. The private
|
||||||
|
# systemd-open-notebook bridge is second and carries only the database; its
|
||||||
|
# traffic is on-link, so losing the default route costs it nothing.
|
||||||
|
#
|
||||||
|
# PUBLISHED PORTS, and the 5055 collision. Upstream publishes 8502 and 5055,
|
||||||
|
# and 5055 is exactly the port seerr already holds on this host
|
||||||
|
# (`PublishPort=127.0.0.1:5055:5055`; `ss -ltn` on Dankclaw 2026-10-18 shows
|
||||||
|
# 127.0.0.1:5055 taken). So the API leaves the container on host port 5056.
|
||||||
|
# That forces one setting: the frontend auto-detects the API as
|
||||||
|
# `<host-you-opened>:5055`, so API_URL in scripts/open-notebook.env must name
|
||||||
|
# the remapped port explicitly, or the UI loads and every call fails. Both
|
||||||
|
# ports are published on all interfaces because the consumer is a browser on
|
||||||
|
# the LAN — the fleet's browser-facing services (open-webui, comfyui, gitea)
|
||||||
|
# bind the same way, while loopback-only is reserved for services whose user
|
||||||
|
# path is the guard chain. This service has NO vhost and no Caddy route: it is
|
||||||
|
# internal-only until it is guarded, and its own password
|
||||||
|
# (OPEN_NOTEBOOK_PASSWORD) is what stands in front of it. That password is not
|
||||||
|
# decoration: with it unset the API answers everyone on the LAN, including the
|
||||||
|
# unauthenticated episode-audio download.
|
||||||
|
#
|
||||||
|
# VOLUMES. `/app/data` is a named volume: uploads, the LangGraph SQLite
|
||||||
|
# checkpoints and the extraction caches the image keeps under it
|
||||||
|
# (UV_CACHE_DIR, HF_HOME) are app state nobody browses. The ONE exception is
|
||||||
|
# finished audio, which is bind-mounted out of the container at
|
||||||
|
# /home/dank/media/podcasts. That mount is interim and reversible: episode
|
||||||
|
# destination is an open human-only decision (Gitea issue #7, label
|
||||||
|
# `human-only` — agents must not work it), and until it lands, episodes are
|
||||||
|
# ordinary files on the host. DATA_FOLDER is hard-coded `./data` in the app
|
||||||
|
# (WORKDIR /app) and the DB stores audio paths RELATIVE to data/podcasts, so
|
||||||
|
# covering that one subdirectory is enough and removing the line changes
|
||||||
|
# nothing else. The image declares no USER, so the app runs as container root,
|
||||||
|
# which in this host's rootless mapping (0 -> 1000, measured on this host —
|
||||||
|
# see [[Seerr]]) writes those episode files as dank.
|
||||||
|
#
|
||||||
|
# HEALTH. `GET /health` on the API returns {"status":"healthy"} and is in the
|
||||||
|
# password middleware's excluded-path list, so the probe stays honest after the
|
||||||
|
# password is set (both read from api/main.py upstream). The image installs
|
||||||
|
# curl and ffmpeg in the runtime layer, so the probe needs no shell — kept
|
||||||
|
# argument-only for the generator's space-splitting. Note what this probe does
|
||||||
|
# NOT prove: the route returns a static body, so it says the API serves, not
|
||||||
|
# that the database answers. The DB has its own probe in its unit, and the
|
||||||
|
# worker's failure to reach the database shows up as failed jobs, not as an
|
||||||
|
# unhealthy container.
|
||||||
|
|
||||||
|
[Unit]
|
||||||
|
Description=open-notebook (podcast generator: API, worker, web UI)
|
||||||
|
After=network-online.target open-notebook-surrealdb.service
|
||||||
|
Wants=network-online.target open-notebook-surrealdb.service
|
||||||
|
|
||||||
|
[Container]
|
||||||
|
ContainerName=open-notebook
|
||||||
|
Image=docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
|
||||||
|
Pull=never
|
||||||
|
Network=dankclaw_default
|
||||||
|
Network=open-notebook.network
|
||||||
|
PublishPort=0.0.0.0:8502:8502
|
||||||
|
PublishPort=0.0.0.0:5056:5055
|
||||||
|
EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env
|
||||||
|
EnvironmentFile=/home/dank/Containers/open-notebook/env
|
||||||
|
Volume=open-notebook-data:/app/data
|
||||||
|
Volume=/home/dank/media/podcasts:/app/data/podcasts
|
||||||
|
HealthCmd=curl -fsS http://127.0.0.1:5055/health
|
||||||
|
HealthInterval=30s
|
||||||
|
HealthTimeout=10s
|
||||||
|
HealthRetries=3
|
||||||
|
HealthStartPeriod=60s
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Restart=always
|
||||||
|
RestartSec=10
|
||||||
|
TimeoutStartSec=900
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
# Quadlet network: private bridge for the open-notebook pair (rootless).
|
||||||
|
#
|
||||||
|
# Install path on dankclaw: ~/.config/containers/systemd/open-notebook.network
|
||||||
|
# Unit name: open-notebook-network.service (podman network name:
|
||||||
|
# systemd-open-notebook — quadlet prefixes "systemd-", so probe the runtime
|
||||||
|
# name, never the filename; see [[Container Network Namespaces]]).
|
||||||
|
#
|
||||||
|
# WHY A PRIVATE BRIDGE. The app reaches its database by container name
|
||||||
|
# (SURREAL_URL=ws://open-notebook-surrealdb:8000/rpc), and container DNS names
|
||||||
|
# resolve only on the network that owns them. Putting SurrealDB here instead of
|
||||||
|
# on dankclaw_default keeps the datastore reachable ONLY to the container that
|
||||||
|
# owns it: 8000 is not published (an unpublished port is refused outright from
|
||||||
|
# every other namespace anyway), and nothing else on the fleet-wide network can
|
||||||
|
# dial it with the root credentials. Same reasoning as the gitea and authentik
|
||||||
|
# database pairs.
|
||||||
|
#
|
||||||
|
# Kernel bridge, not slirp4netns: the 2026-08-22 open-webui incident (a dead
|
||||||
|
# per-container userspace helper left a container that accepted TCP and reset
|
||||||
|
# every request for two days, logging nothing). A bridge has no such helper.
|
||||||
|
#
|
||||||
|
# Subnet registry on dankclaw (rootless):
|
||||||
|
# 10.88.0.0/16 default "podman" bridge
|
||||||
|
# 10.89.0.0/24 gitea (systemd-gitea)
|
||||||
|
# 10.89.1.0/24 compose stack (dankclaw_default)
|
||||||
|
# 10.89.2.0/24 searxng (systemd-searxng)
|
||||||
|
# 10.89.3.0/24 open-webui (systemd-open-webui)
|
||||||
|
# 10.89.4.0/24 open-notebook (systemd-open-notebook) <- this file
|
||||||
|
# Pinned so the container IPs stay deterministic across reboots, like the
|
||||||
|
# other pinned nets; 10.89.4.0/24 was the next unclaimed /24 in this registry.
|
||||||
|
|
||||||
|
[Network]
|
||||||
|
Driver=bridge
|
||||||
|
Subnet=10.89.4.0/24
|
||||||
|
Gateway=10.89.4.1
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
# frank-tts — the cloned-voice speech façade (dankclaw/voiceservice/app.py).
|
||||||
|
#
|
||||||
|
# Standard library only, so there is nothing to pip-install and nothing to
|
||||||
|
# compile: the image is a python runtime plus ffmpeg (the one binary the app
|
||||||
|
# shells out to, for response_format=mp3) plus one COPY of app.py. Rebuild to
|
||||||
|
# ship a new app.py.
|
||||||
|
#
|
||||||
|
# Build on dankclaw (deploy-remote.sh build_compose_image frank-tts does this,
|
||||||
|
# including --format docker, which the fleet's other local builds also pass):
|
||||||
|
# podman build --format docker -t franksdank/frank-tts:latest \
|
||||||
|
# -f dankclaw/voiceservice/Containerfile .
|
||||||
|
#
|
||||||
|
# BASE PINNED BY DIGEST, like every other image this fleet runs from a
|
||||||
|
# registry: a tag moves under a rebuild and a voice endpoint that changes
|
||||||
|
# python or ffmpeg mid-week is a debug session nobody asked for. The digest is
|
||||||
|
# the multi-arch INDEX of python:3.12-alpine as published 2026-10-18 (amd64 +
|
||||||
|
# arm64 on linux); Dankclaw resolves the amd64 entry.
|
||||||
|
#
|
||||||
|
# WHY 3.12: the fleet's ceiling for this app is python <= 3.12 (single file,
|
||||||
|
# no build step); the alpine flavour also keeps the image in the tens of MB
|
||||||
|
# and ships busybox `wget`, which is what the quadlet unit's HealthCmd runs —
|
||||||
|
# so no HEALTHCHECK belongs here (the probe lives in the unit, as in
|
||||||
|
# alertmanager/prometheus; only distroless bases need it baked in).
|
||||||
|
FROM docker.io/library/python:3.12-alpine@sha256:1b668429b3511ab407d8e00648891631b0b1a4d7e15e3ca70f38ab5b91ad4ab4
|
||||||
|
|
||||||
|
# ffmpeg: mp3 output. The app always asks the upstream for wav and re-encodes
|
||||||
|
# on the way out, so ffmpeg is the only external program in the picture.
|
||||||
|
RUN apk add --no-cache ffmpeg
|
||||||
|
|
||||||
|
# Unbuffered stdout: the unit's journal must show the "listening on" line (and
|
||||||
|
# any backend error) the moment it happens, not when the buffer fills.
|
||||||
|
ENV PYTHONUNBUFFERED=1
|
||||||
|
|
||||||
|
COPY dankclaw/voiceservice/app.py /app/app.py
|
||||||
|
|
||||||
|
# Non-root. The voice packs are bind-mounted read-only, so this user only ever
|
||||||
|
# reads them — they must be world-readable (the pack builder writes 644/755).
|
||||||
|
# The one thing it writes is its per-request temp dir under /tmp.
|
||||||
|
RUN adduser -D -u 10000 -h /app -s /sbin/nologin shim
|
||||||
|
USER shim
|
||||||
|
|
||||||
|
EXPOSE 7772
|
||||||
|
CMD ["python3", "/app/app.py"]
|
||||||
Executable
+413
@@ -0,0 +1,413 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""frank-tts — the fleet's OpenAI-compatible speech façade (component: the shim).
|
||||||
|
|
||||||
|
One local endpoint that answers ``voice=<name>`` with audio in a cloned voice.
|
||||||
|
Consumers (the omp harness, OpenWebUI, open-notebook) speak the OpenAI
|
||||||
|
``POST /v1/audio/speech`` shape and never learn a file path: a voice is a
|
||||||
|
directory under ``$VOICES_DIR`` and ``reference.wav`` inside it is what makes
|
||||||
|
the name real. A name that has no reference is not a voice — the shim answers
|
||||||
|
400 with the list that is, instead of silently narrating in a fallback voice
|
||||||
|
(silent fallback is how a wrong-voice endpoint stays wrong for weeks).
|
||||||
|
|
||||||
|
Two backends sit behind the one surface, chosen by ``TTS_BACKEND``:
|
||||||
|
|
||||||
|
``omni-http``
|
||||||
|
Forward to an OpenAI-shaped upstream TTS server (``$TTS_UPSTREAM``) and
|
||||||
|
carry the cloned voice with the request as ``reference_audio`` (base64 of
|
||||||
|
``reference.wav``). The upstream does the inference; this process only
|
||||||
|
resolves the voice and, when asked, re-encodes to mp3 with ``ffmpeg``.
|
||||||
|
|
||||||
|
``llama-tts-cli``
|
||||||
|
Run a local binary per request (``$LLAMA_TTS_BIN``), hand it the
|
||||||
|
reference with ``--tts-speaker-file``, read the WAV it writes. ``nice -n
|
||||||
|
19`` because this box also serves text and a voice request must never
|
||||||
|
starve it.
|
||||||
|
|
||||||
|
Everything comes from the environment, read at REQUEST time (no
|
||||||
|
import-time-only globals): the same module can be imported with an injected
|
||||||
|
``Config`` or run with ``python3 app.py`` and the unit's env file.
|
||||||
|
|
||||||
|
Runtime law: single process, standard library only, no build step. Audio
|
||||||
|
bytes never reach the log — request lines and error text only.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
|
import urllib.request
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
WAV_CONTENT_TYPE = "audio/wav"
|
||||||
|
MP3_CONTENT_TYPE = "audio/mpeg"
|
||||||
|
|
||||||
|
# A voice name is a plain directory name. Slashes and `..` are refused here so
|
||||||
|
# a client-supplied name can never escape $VOICES_DIR.
|
||||||
|
VOICE_NAME_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}")
|
||||||
|
|
||||||
|
|
||||||
|
class ShimError(Exception):
|
||||||
|
"""A failure with the HTTP status the client should see."""
|
||||||
|
|
||||||
|
def __init__(self, status, message):
|
||||||
|
super().__init__(message)
|
||||||
|
self.status = status
|
||||||
|
self.message = message
|
||||||
|
|
||||||
|
|
||||||
|
class Config:
|
||||||
|
"""Every knob of the shim. Constructed from the environment per request by
|
||||||
|
default, or injected into ``create_server`` for tests and embedding."""
|
||||||
|
|
||||||
|
def __init__(self, voices_dir, backend, upstream, upstream_model, llama_bin,
|
||||||
|
llama_model, llama_mmproj, llama_lang, llama_threads, port,
|
||||||
|
max_chars, timeout):
|
||||||
|
self.voices_dir = Path(voices_dir)
|
||||||
|
self.backend = backend
|
||||||
|
self.upstream = upstream
|
||||||
|
self.upstream_model = upstream_model
|
||||||
|
self.llama_bin = llama_bin
|
||||||
|
self.llama_model = llama_model
|
||||||
|
self.llama_mmproj = llama_mmproj
|
||||||
|
self.llama_lang = llama_lang
|
||||||
|
self.llama_threads = llama_threads
|
||||||
|
self.port = port
|
||||||
|
self.max_chars = max_chars
|
||||||
|
self.timeout = timeout
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_env(cls, environ=None):
|
||||||
|
env = os.environ if environ is None else environ
|
||||||
|
return cls(
|
||||||
|
voices_dir=env.get("VOICES_DIR", "/voices"),
|
||||||
|
backend=(env.get("TTS_BACKEND") or "omni-http").strip().lower(),
|
||||||
|
upstream=(env.get("TTS_UPSTREAM") or "http://host.containers.internal:8880").rstrip("/"),
|
||||||
|
upstream_model=env.get("TTS_UPSTREAM_MODEL") or "voxcpm2",
|
||||||
|
llama_bin=env.get("LLAMA_TTS_BIN") or "",
|
||||||
|
llama_model=env.get("LLAMA_TTS_MODEL") or "",
|
||||||
|
llama_mmproj=env.get("LLAMA_TTS_MMPROJ") or "",
|
||||||
|
llama_lang=env.get("LLAMA_TTS_LANG") or "english",
|
||||||
|
llama_threads=_int(env, "LLAMA_TTS_THREADS", 8),
|
||||||
|
port=_int(env, "SHIM_PORT", 7772),
|
||||||
|
max_chars=_int(env, "SHIM_MAX_CHARS", 15000),
|
||||||
|
timeout=float(_int(env, "SHIM_TIMEOUT", 600)),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _int(env, name, default):
|
||||||
|
try:
|
||||||
|
return int(str(env.get(name, "")).strip())
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
|
def list_voices(voices_dir):
|
||||||
|
"""Voice names in ``voices_dir``: directories that hold a ``reference.wav``.
|
||||||
|
|
||||||
|
A missing or unreadable directory is an empty fleet, not an error — the
|
||||||
|
health endpoint must still answer so the unit reports its state.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
entries = sorted(p.name for p in voices_dir.iterdir()
|
||||||
|
if p.is_dir() and (p / "reference.wav").is_file())
|
||||||
|
except OSError:
|
||||||
|
return []
|
||||||
|
return entries
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_voice(voices_dir, voice):
|
||||||
|
"""``$VOICES_DIR/<voice>/reference.wav`` for a valid name, else ``None``."""
|
||||||
|
if not isinstance(voice, str) or not VOICE_NAME_RE.fullmatch(voice):
|
||||||
|
return None
|
||||||
|
reference = voices_dir / voice / "reference.wav"
|
||||||
|
return reference if reference.is_file() else None
|
||||||
|
|
||||||
|
|
||||||
|
def to_mp3(wav_bytes, timeout):
|
||||||
|
"""Re-encode WAV bytes to mp3 with the image's ``ffmpeg`` (pipes, no temp
|
||||||
|
file: the audio never lands on disk to be left behind)."""
|
||||||
|
argv = ["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", "pipe:0",
|
||||||
|
"-f", "mp3", "pipe:1"]
|
||||||
|
try:
|
||||||
|
proc = subprocess.run(argv, input=wav_bytes, capture_output=True,
|
||||||
|
timeout=timeout)
|
||||||
|
except FileNotFoundError:
|
||||||
|
raise ShimError(500, "response_format=mp3 needs the ffmpeg binary, which is not on PATH")
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
raise ShimError(502, "ffmpeg mp3 conversion timed out after %gs" % timeout)
|
||||||
|
if proc.returncode != 0 or not proc.stdout:
|
||||||
|
raise ShimError(502, "ffmpeg mp3 conversion failed (exit %s): %s"
|
||||||
|
% (proc.returncode, _tail(proc.stderr)))
|
||||||
|
return proc.stdout
|
||||||
|
|
||||||
|
|
||||||
|
def _tail(raw):
|
||||||
|
"""Last line of a subprocess' stderr, for an error message. Text only —
|
||||||
|
audio bytes are never echoed, and never logged."""
|
||||||
|
text = (raw or b"").decode("utf-8", "replace").strip()
|
||||||
|
return text.splitlines()[-1][:300] if text else "no output"
|
||||||
|
|
||||||
|
|
||||||
|
def _error(message, voices=None):
|
||||||
|
body = {"error": {"message": message, "type": "invalid_request_error"}}
|
||||||
|
if voices is not None:
|
||||||
|
body["error"]["available_voices"] = voices
|
||||||
|
return body
|
||||||
|
|
||||||
|
|
||||||
|
class SpeechHandler(BaseHTTPRequestHandler):
|
||||||
|
protocol_version = "HTTP/1.1"
|
||||||
|
server_version = "frank-tts"
|
||||||
|
sys_version = ""
|
||||||
|
|
||||||
|
# -- plumbing ----------------------------------------------------------
|
||||||
|
def config(self):
|
||||||
|
return getattr(self.server, "config", None) or Config.from_env()
|
||||||
|
|
||||||
|
def log_message(self, fmt, *args):
|
||||||
|
# One prefix for the journal. Request lines and outcomes only: a
|
||||||
|
# synthesized text, a base64 reference voice and the audio bytes
|
||||||
|
# itself never reach stderr, in success or in failure.
|
||||||
|
sys.stderr.write("frank-tts: %s - %s\n"
|
||||||
|
% (self.address_string(), fmt % args))
|
||||||
|
|
||||||
|
def log_error(self, fmt, *args):
|
||||||
|
# The base class routes refused requests through a separate sink; keep
|
||||||
|
# them on the same line format so the journal reads uniformly.
|
||||||
|
self.log_message(fmt, *args)
|
||||||
|
|
||||||
|
def _json(self, status, obj, head=False):
|
||||||
|
self._bytes(status, json.dumps(obj).encode("utf-8"), "application/json", head)
|
||||||
|
|
||||||
|
def _bytes(self, status, data, content_type, head=False):
|
||||||
|
self.send_response(status)
|
||||||
|
self.send_header("Content-Type", content_type)
|
||||||
|
self.send_header("Content-Length", str(len(data)))
|
||||||
|
self.end_headers()
|
||||||
|
if not head:
|
||||||
|
self.wfile.write(data)
|
||||||
|
|
||||||
|
def _drain(self):
|
||||||
|
"""Read and discard a body this route will not answer. HTTP/1.1
|
||||||
|
keep-alive makes an unread body the NEXT request line, so a client
|
||||||
|
that reuses the connection would see a parse error it never sent."""
|
||||||
|
if getattr(self, "_body_read", False):
|
||||||
|
return # already consumed; a second read would block the response
|
||||||
|
try:
|
||||||
|
length = int(self.headers.get("Content-Length") or 0)
|
||||||
|
except ValueError:
|
||||||
|
return
|
||||||
|
if length > 0:
|
||||||
|
self.rfile.read(length)
|
||||||
|
|
||||||
|
def _body(self):
|
||||||
|
"""Parse the JSON request body. Never returned to a log or an error."""
|
||||||
|
try:
|
||||||
|
length = int(self.headers.get("Content-Length") or 0)
|
||||||
|
except ValueError:
|
||||||
|
raise ValueError("Content-Length is not a number")
|
||||||
|
raw = self.rfile.read(length) if length > 0 else b""
|
||||||
|
self._body_read = True
|
||||||
|
if not raw:
|
||||||
|
raise ValueError("expected a JSON body")
|
||||||
|
parsed = json.loads(raw.decode("utf-8"))
|
||||||
|
if not isinstance(parsed, dict):
|
||||||
|
raise ValueError("expected a JSON object")
|
||||||
|
return parsed
|
||||||
|
|
||||||
|
# -- routes ------------------------------------------------------------
|
||||||
|
def do_GET(self, head=False):
|
||||||
|
# HEAD shares the route table: `wget --spider` (the fleet's probe
|
||||||
|
# shape) asks for HEAD, and a 501 there would read as a dead service.
|
||||||
|
path = urllib.parse.urlparse(self.path).path
|
||||||
|
cfg = self.config()
|
||||||
|
if path == "/health":
|
||||||
|
self._json(200, {"status": "ok", "backend": cfg.backend,
|
||||||
|
"voices": list_voices(cfg.voices_dir)}, head)
|
||||||
|
elif path == "/v1/audio/voices":
|
||||||
|
self._json(200, list_voices(cfg.voices_dir), head)
|
||||||
|
else:
|
||||||
|
self._json(404, _error("no such endpoint: %s" % path), head)
|
||||||
|
|
||||||
|
def do_HEAD(self):
|
||||||
|
self.do_GET(head=True)
|
||||||
|
|
||||||
|
def do_POST(self):
|
||||||
|
path = urllib.parse.urlparse(self.path).path
|
||||||
|
if path != "/v1/audio/speech":
|
||||||
|
self._drain()
|
||||||
|
self._json(404, _error("no such endpoint: %s" % path))
|
||||||
|
return
|
||||||
|
cfg = self.config()
|
||||||
|
try:
|
||||||
|
body = self._body()
|
||||||
|
except (ValueError, UnicodeDecodeError) as exc:
|
||||||
|
self._drain()
|
||||||
|
self._json(400, _error("malformed request body: %s" % exc))
|
||||||
|
return
|
||||||
|
|
||||||
|
text = body.get("input")
|
||||||
|
if not isinstance(text, str) or not text.strip():
|
||||||
|
self._json(400, _error("`input` must be a non-empty string"))
|
||||||
|
return
|
||||||
|
if len(text) > cfg.max_chars:
|
||||||
|
self._json(414, _error("input is %d characters, over the %d limit "
|
||||||
|
"(SHIM_MAX_CHARS); split it or raise the limit"
|
||||||
|
% (len(text), cfg.max_chars)))
|
||||||
|
return
|
||||||
|
|
||||||
|
fmt = str(body.get("response_format") or "wav").strip().lower()
|
||||||
|
if fmt not in ("wav", "mp3"):
|
||||||
|
self._json(400, _error("unsupported response_format %r; this "
|
||||||
|
"endpoint serves wav and mp3" % fmt))
|
||||||
|
return
|
||||||
|
|
||||||
|
# `voice_id` is the alias the omp harness forwards for `voice`.
|
||||||
|
voice = body.get("voice") or body.get("voice_id")
|
||||||
|
reference = resolve_voice(cfg.voices_dir, voice)
|
||||||
|
if reference is None:
|
||||||
|
available = list_voices(cfg.voices_dir)
|
||||||
|
listed = ", ".join(available) if available else "(none: no voice pack with a reference.wav in %s)" % cfg.voices_dir
|
||||||
|
self._json(400, _error("unknown voice %r; available voices: %s"
|
||||||
|
% (voice, listed), available))
|
||||||
|
return
|
||||||
|
|
||||||
|
model = str(body.get("model") or cfg.upstream_model)
|
||||||
|
try:
|
||||||
|
if cfg.backend == "omni-http":
|
||||||
|
audio = self._speech_http(cfg, model, text, str(voice), reference)
|
||||||
|
elif cfg.backend == "llama-tts-cli":
|
||||||
|
audio = self._speech_cli(cfg, text, reference)
|
||||||
|
else:
|
||||||
|
raise ShimError(500, "unknown TTS_BACKEND %r (expected omni-http or llama-tts-cli)" % cfg.backend)
|
||||||
|
if fmt == "mp3":
|
||||||
|
audio = to_mp3(audio, cfg.timeout)
|
||||||
|
except ShimError as exc:
|
||||||
|
self._json(exc.status, _error(exc.message))
|
||||||
|
return
|
||||||
|
|
||||||
|
content_type = MP3_CONTENT_TYPE if fmt == "mp3" else WAV_CONTENT_TYPE
|
||||||
|
self._bytes(200, audio, content_type)
|
||||||
|
self.log_message("speech voice=%s format=%s chars=%d -> %d bytes %s",
|
||||||
|
voice, fmt, len(text), len(audio), content_type)
|
||||||
|
|
||||||
|
# -- backends ----------------------------------------------------------
|
||||||
|
def _speech_http(self, cfg, model, text, voice, reference):
|
||||||
|
"""Forward to the upstream with the cloned voice riding along as
|
||||||
|
base64 ``reference_audio``. The upstream is asked for wav always; mp3
|
||||||
|
is our own re-encode on the way out."""
|
||||||
|
try:
|
||||||
|
payload = {
|
||||||
|
"model": model,
|
||||||
|
"input": text,
|
||||||
|
"voice": voice,
|
||||||
|
"response_format": "wav",
|
||||||
|
"reference_audio": base64.b64encode(reference.read_bytes()).decode("ascii"),
|
||||||
|
}
|
||||||
|
except OSError as exc:
|
||||||
|
raise ShimError(500, "cannot read %s: %s" % (reference, exc))
|
||||||
|
|
||||||
|
url = cfg.upstream + "/v1/audio/speech"
|
||||||
|
request = urllib.request.Request(
|
||||||
|
url, data=json.dumps(payload).encode("utf-8"),
|
||||||
|
headers={"Content-Type": "application/json"}, method="POST")
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=cfg.timeout) as response:
|
||||||
|
audio = response.read()
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
detail = exc.read(512).decode("utf-8", "replace").strip()
|
||||||
|
raise ShimError(502, "upstream %s returned HTTP %d: %s"
|
||||||
|
% (url, exc.code, detail or "no detail"))
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
raise ShimError(502, "upstream %s unreachable: %s" % (url, exc.reason))
|
||||||
|
except (TimeoutError, OSError) as exc:
|
||||||
|
raise ShimError(502, "upstream %s failed: %s" % (url, exc))
|
||||||
|
if not audio:
|
||||||
|
raise ShimError(502, "upstream %s returned no audio" % url)
|
||||||
|
return audio
|
||||||
|
|
||||||
|
def _speech_cli(self, cfg, text, reference):
|
||||||
|
"""Run the local TTS binary once, in a temp dir that is always removed
|
||||||
|
— also on failure, or a crash-looping binary fills /tmp."""
|
||||||
|
if not cfg.llama_bin or not cfg.llama_model:
|
||||||
|
raise ShimError(500, "TTS_BACKEND=llama-tts-cli needs LLAMA_TTS_BIN and LLAMA_TTS_MODEL")
|
||||||
|
work = tempfile.TemporaryDirectory(prefix="frank-tts-")
|
||||||
|
try:
|
||||||
|
out_path = Path(work.name) / "speech.wav"
|
||||||
|
argv = ["nice", "-n", "19", "env", "OMP_NUM_THREADS=%d" % cfg.llama_threads,
|
||||||
|
cfg.llama_bin, "-m", cfg.llama_model]
|
||||||
|
if cfg.llama_mmproj:
|
||||||
|
argv += ["-mm", cfg.llama_mmproj]
|
||||||
|
argv += ["--tts-lang", cfg.llama_lang,
|
||||||
|
"--tts-speaker-file", str(reference),
|
||||||
|
"-p", text,
|
||||||
|
"--output", str(out_path)]
|
||||||
|
try:
|
||||||
|
proc = subprocess.run(argv, cwd=work.name, capture_output=True,
|
||||||
|
timeout=cfg.timeout)
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise ShimError(500, "cannot run the TTS command: %s" % exc)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
raise ShimError(502, "TTS binary timed out after %gs" % cfg.timeout)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise ShimError(502, "TTS binary failed (exit %s): %s"
|
||||||
|
% (proc.returncode, _tail(proc.stderr)))
|
||||||
|
if not out_path.is_file():
|
||||||
|
raise ShimError(502, "TTS binary exited 0 but wrote no %s" % out_path.name)
|
||||||
|
return out_path.read_bytes()
|
||||||
|
finally:
|
||||||
|
work.cleanup()
|
||||||
|
|
||||||
|
|
||||||
|
class SpeechServer(ThreadingHTTPServer):
|
||||||
|
"""Threads so a health probe answers while a long synthesis runs; daemon
|
||||||
|
threads so SIGTERM is never held hostage by a stuck request."""
|
||||||
|
|
||||||
|
daemon_threads = True
|
||||||
|
allow_reuse_address = True
|
||||||
|
config = None
|
||||||
|
|
||||||
|
|
||||||
|
def create_server(config=None, host="0.0.0.0", port=None):
|
||||||
|
"""Bind the shim. ``config``/``port`` are the injectable seams: tests pass
|
||||||
|
a Config with a temp VOICES_DIR and ``port=0`` for an ephemeral port."""
|
||||||
|
cfg = Config.from_env() if config is None else config
|
||||||
|
server = SpeechServer((host, cfg.port if port is None else port), SpeechHandler)
|
||||||
|
server.config = cfg
|
||||||
|
return server
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv=None):
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
prog="frank-tts", description="OpenAI-compatible speech endpoint for cloned voice packs")
|
||||||
|
parser.add_argument("--host", default="0.0.0.0",
|
||||||
|
help="bind address (default: all interfaces)")
|
||||||
|
parser.add_argument("--port", type=int, default=None,
|
||||||
|
help="bind port (default: $SHIM_PORT, 0 = ephemeral)")
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
server = create_server(host=args.host, port=args.port)
|
||||||
|
cfg = server.config
|
||||||
|
bound = server.server_address[1]
|
||||||
|
print("frank-tts: backend=%s voices_dir=%s listening on http://%s:%d"
|
||||||
|
% (cfg.backend, cfg.voices_dir, args.host, bound), flush=True)
|
||||||
|
try:
|
||||||
|
server.serve_forever()
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
pass
|
||||||
|
finally:
|
||||||
|
server.server_close()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# Quadlet unit: frank-tts (rootless, boot-persistent).
|
||||||
|
#
|
||||||
|
# Install path on dankclaw: ~/.config/containers/systemd/frank-tts.container
|
||||||
|
# Unit name: frank-tts.service (container name: frank-tts)
|
||||||
|
#
|
||||||
|
# WHAT IT IS. The fleet's one OpenAI-compatible speech endpoint: consumers
|
||||||
|
# (omp, OpenWebUI, open-notebook) POST /v1/audio/speech with voice=<name> and
|
||||||
|
# get audio in that cloned voice. A voice is a directory under
|
||||||
|
# /home/dank/voices holding reference.wav; the shim refuses any name without
|
||||||
|
# one (400 + the list that does exist) rather than narrating in a fallback
|
||||||
|
# voice nobody chose. The engine behind it is a separate decision —
|
||||||
|
# TTS_BACKEND in scripts/frank-tts.env picks between forwarding to an
|
||||||
|
# OpenAI-shaped upstream on host :8880 and running a local CLI binary per
|
||||||
|
# request.
|
||||||
|
#
|
||||||
|
# IMAGE. Built locally from dankclaw/voiceservice/Containerfile by
|
||||||
|
# deploy-remote.sh (build_compose_image), like loki/alloy: python 3.12-alpine
|
||||||
|
# pinned by digest + ffmpeg, app.py copied in, running as uid 10000.
|
||||||
|
# Pull=never and the localhost/ prefix are required — this host's quadlet
|
||||||
|
# path cannot resolve a locally built tag from local storage otherwise.
|
||||||
|
#
|
||||||
|
# NETWORK. Bridge (dankclaw_default), NOT host: nothing here needs a shared
|
||||||
|
# loopback namespace, and the comfyui/loki/authentik units are bridge too.
|
||||||
|
# Consequences both ways: consumers reach it on the published 7772, and the
|
||||||
|
# shim reaches the host-side TTS engine through host.containers.internal
|
||||||
|
# (never the LAN IP — a bridge netns cannot route to the host's own address
|
||||||
|
# for every bind shape, and the fleet's law is the gateway name). NOTE for
|
||||||
|
# whoever starts the :8880 engine: host.containers.internal reaches the host
|
||||||
|
# network namespace, so the engine must bind an address that namespace
|
||||||
|
# accepts; a 127.0.0.1-only bind is not it (the 2026-09-17 signal-relay hop
|
||||||
|
# needed the host LAN bind for the same reason).
|
||||||
|
#
|
||||||
|
# VOICES. /home/dank/voices is host state, never repo state, and is mounted
|
||||||
|
# READ-ONLY: this endpoint has no business writing voice packs. Because the
|
||||||
|
# container runs as uid 10000, the packs must be world-readable (644 files,
|
||||||
|
# 755 directories), which is how the pack builder writes them.
|
||||||
|
#
|
||||||
|
# NO GPU. The shim is a façade: audio conversion via ffmpeg on the CPU and,
|
||||||
|
# for the CLI backend, whatever CPU threads LLAMA_TTS_THREADS allows, at nice
|
||||||
|
# 19. No /dev/kfd, no /dev/dri, no keep-groups — unlike comfyui, whose unit
|
||||||
|
# carries them deliberately.
|
||||||
|
[Unit]
|
||||||
|
Description=frank-tts (cloned-voice OpenAI-compatible TTS facade)
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Container]
|
||||||
|
ContainerName=frank-tts
|
||||||
|
Image=localhost/franksdank/frank-tts:latest
|
||||||
|
Pull=never
|
||||||
|
Network=dankclaw_default
|
||||||
|
PublishPort=7772:7772
|
||||||
|
EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts.env
|
||||||
|
Volume=/home/dank/voices:/voices:ro
|
||||||
|
# The probe is a probe, not a voice test: /health answers 200 with the backend
|
||||||
|
# and the resolved voice list, so a mounted-but-empty /voices still reads
|
||||||
|
# healthy while a dead app does not. busybox wget ships in the alpine base;
|
||||||
|
# keep the command argument-only — the podman 5.x generator splits HealthCmd
|
||||||
|
# on spaces (see the comfyui unit for the quoting trap).
|
||||||
|
HealthCmd=wget --spider -q http://127.0.0.1:7772/health
|
||||||
|
HealthInterval=30s
|
||||||
|
HealthTimeout=10s
|
||||||
|
HealthRetries=3
|
||||||
|
HealthStartPeriod=30s
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Restart=always
|
||||||
|
RestartSec=10
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
@@ -26,6 +26,7 @@ SERVICE_ORDER = (
|
|||||||
"marreta",
|
"marreta",
|
||||||
"pihole",
|
"pihole",
|
||||||
"open-webui",
|
"open-webui",
|
||||||
|
"open-notebook",
|
||||||
"gitea-postgres",
|
"gitea-postgres",
|
||||||
"gitea-server",
|
"gitea-server",
|
||||||
"comfyui",
|
"comfyui",
|
||||||
@@ -74,6 +75,7 @@ DANKCLAW_SERVICES = {
|
|||||||
"marreta",
|
"marreta",
|
||||||
"pihole",
|
"pihole",
|
||||||
"open-webui",
|
"open-webui",
|
||||||
|
"open-notebook",
|
||||||
"gitea-postgres",
|
"gitea-postgres",
|
||||||
"gitea-server",
|
"gitea-server",
|
||||||
"comfyui",
|
"comfyui",
|
||||||
@@ -316,6 +318,14 @@ def services_for_path(path: str, target: str) -> set[str]:
|
|||||||
if len(parts) >= 2 and parts[1] == "open-webui":
|
if len(parts) >= 2 and parts[1] == "open-webui":
|
||||||
_add(services, "open-webui")
|
_add(services, "open-webui")
|
||||||
return services
|
return services
|
||||||
|
if len(parts) >= 2 and parts[1] == "open-notebook":
|
||||||
|
# One deployable owns the app AND its SurrealDB, like the marreta
|
||||||
|
# app/companion pair: the app alone cannot write a note and the
|
||||||
|
# database alone is a volume nothing reads. Any change under the
|
||||||
|
# directory (unit, network, or the interim episode mount) therefore
|
||||||
|
# plans the pair, which deploy_open_notebook starts database-first.
|
||||||
|
_add(services, "open-notebook")
|
||||||
|
return services
|
||||||
# frank-dragoon left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
|
# frank-dragoon left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
|
||||||
# build, root unit): its files live in scripts/ and map there.
|
# build, root unit): its files live in scripts/ and map there.
|
||||||
if len(parts) >= 2 and parts[1] == "observability":
|
if len(parts) >= 2 and parts[1] == "observability":
|
||||||
@@ -426,6 +436,12 @@ def services_for_path(path: str, target: str) -> set[str]:
|
|||||||
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches
|
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches
|
||||||
# the payload, which is host-side state, so it selects nothing.
|
# the payload, which is host-side state, so it selects nothing.
|
||||||
_add(services, "frank-archon")
|
_add(services, "frank-archon")
|
||||||
|
if name == "open-notebook.env":
|
||||||
|
# open-notebook's tunables live here — the only repo-tracked part of
|
||||||
|
# either container's environment (the credentials are seeded on the
|
||||||
|
# host, so they are never a deploy trigger). Changing it is a
|
||||||
|
# restart of the pair, which the single deployable performs.
|
||||||
|
_add(services, "open-notebook")
|
||||||
# Deployment/test/benchmark scripts and other repo tooling do not
|
# Deployment/test/benchmark scripts and other repo tooling do not
|
||||||
# restart a runtime service.
|
# restart a runtime service.
|
||||||
return services
|
return services
|
||||||
|
|||||||
@@ -90,6 +90,7 @@ service_is_allowed() {
|
|||||||
dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
||||||
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\
|
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\
|
||||||
dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
|
dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
|
||||||
|
dankclaw:open-notebook|\
|
||||||
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
|
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
|
||||||
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
||||||
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
||||||
@@ -646,6 +647,103 @@ deploy_open_webui() {
|
|||||||
systemctl --user restart open-webui.service
|
systemctl --user restart open-webui.service
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# open-notebook is the fleet's podcast generator: one app container (API,
|
||||||
|
# worker and web UI under supervisord) in front of a SurrealDB container, with
|
||||||
|
# a private bridge between them. One handler owns the pair, because restarting
|
||||||
|
# the app without its database yields a UI that answers /health and fails every
|
||||||
|
# write; scripts/deploy-plan.py therefore maps every path of the pair here.
|
||||||
|
#
|
||||||
|
# Credentials follow the marreta convention (per-service file under the deploy
|
||||||
|
# user's home, mode 600, seeded from dankclaw/.env) with one difference: they
|
||||||
|
# are not optional. An empty OPEN_NOTEBOOK_PASSWORD is "authentication OFF",
|
||||||
|
# not "unconfigured" — the API, episode audio included, then answers anyone on
|
||||||
|
# the LAN — and an empty SURREAL_PASS lets the database come up with different
|
||||||
|
# root credentials than the app will present. So this handler refuses to start
|
||||||
|
# either unit until every key carries a value. An existing file is NEVER
|
||||||
|
# rewritten: regenerating an operator-set password would desync it from the
|
||||||
|
# credentials already stored in the database.
|
||||||
|
seed_open_notebook_env() {
|
||||||
|
local dest=/home/dank/Containers/open-notebook/env
|
||||||
|
local var missing=0
|
||||||
|
for var in OPEN_NOTEBOOK_ENCRYPTION_KEY OPEN_NOTEBOOK_PASSWORD SURREAL_PASSWORD SURREAL_PASS; do
|
||||||
|
grep -q "^${var}=." "$dest" 2>/dev/null || missing=1
|
||||||
|
done
|
||||||
|
if [[ "$missing" == 0 ]]; then
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
if [[ ! -e "$dest" ]]; then
|
||||||
|
if [[ ! -f dankclaw/.env ]]; then
|
||||||
|
echo "open-notebook: neither $dest nor dankclaw/.env exists; put OPEN_NOTEBOOK_ENCRYPTION_KEY, OPEN_NOTEBOOK_PASSWORD and SURREAL_PASSWORD in dankclaw/.env (see dankclaw/.env.example)" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
mkdir -p /home/dank/Containers/open-notebook
|
||||||
|
{
|
||||||
|
awk -F= '/^OPEN_NOTEBOOK_ENCRYPTION_KEY=/ || /^OPEN_NOTEBOOK_PASSWORD=/ || /^SURREAL_PASSWORD=/ { print }' \
|
||||||
|
dankclaw/.env
|
||||||
|
# The `surreal` CLI reads SURREAL_PASS from the environment while
|
||||||
|
# the app reads SURREAL_PASSWORD. One value, written twice, so the
|
||||||
|
# pair can never be seeded with two different root passwords.
|
||||||
|
sed -n 's/^SURREAL_PASSWORD=/SURREAL_PASS=/p' dankclaw/.env
|
||||||
|
} > "$dest"
|
||||||
|
chmod 600 "$dest"
|
||||||
|
fi
|
||||||
|
for var in OPEN_NOTEBOOK_ENCRYPTION_KEY OPEN_NOTEBOOK_PASSWORD SURREAL_PASSWORD SURREAL_PASS; do
|
||||||
|
grep -q "^${var}=." "$dest" \
|
||||||
|
|| { echo "open-notebook: $dest carries no value for ${var}; fill it in directly (or in dankclaw/.env before the first seed) — a blank value here means an unauthenticated service or a mismatched database password" >&2; return 1; }
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# Both units ship Pull=never: the digest pins in the .container files are the
|
||||||
|
# contract, and the generator must never re-resolve a moving tag at start. That
|
||||||
|
# makes fetching a pin a precondition the deployer satisfies once per pin
|
||||||
|
# change; the existence check keeps every later re-deploy offline-safe.
|
||||||
|
ensure_pinned_image() {
|
||||||
|
local image="$1"
|
||||||
|
if podman image exists "$image"; then
|
||||||
|
echo "open-notebook: $image is already in local storage"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
echo "open-notebook: pulling pinned image $image"
|
||||||
|
podman pull "$image"
|
||||||
|
}
|
||||||
|
|
||||||
|
deploy_open_notebook() {
|
||||||
|
seed_open_notebook_env || return 1
|
||||||
|
|
||||||
|
ensure_pinned_image 'docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723' || return 1
|
||||||
|
ensure_pinned_image 'docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212' || return 1
|
||||||
|
|
||||||
|
# install_quadlet copies the sibling open-notebook.network with the app
|
||||||
|
# unit. That cp is load-bearing, not tidiness: when a referenced Network=
|
||||||
|
# file is absent the v1 generator refuses to produce the container unit at
|
||||||
|
# all ("requested Quadlet unit open-notebook.network was not found").
|
||||||
|
install_quadlet open-notebook-surrealdb dankclaw/open-notebook/quadlet open-notebook-surrealdb
|
||||||
|
install_quadlet open-notebook dankclaw/open-notebook/quadlet open-notebook
|
||||||
|
systemctl --user start open-notebook-network.service
|
||||||
|
|
||||||
|
# Interim episode destination (Gitea issue #7, `human-only`): finished
|
||||||
|
# episode audio is bind-mounted out to this host directory. Created here as
|
||||||
|
# the deploy user so the container's mapped root writes files dank can
|
||||||
|
# read, and so the dependency is explicit rather than podman's default.
|
||||||
|
mkdir -p /home/dank/media/podcasts
|
||||||
|
|
||||||
|
# No `systemctl --user enable`: generated Quadlet units cannot be enabled
|
||||||
|
# on this host (see deploy_authentik); boot persistence is the [Install]
|
||||||
|
# section inside each .container file. Database first, and both on every
|
||||||
|
# run — a stale app left attached to a recreated database is exactly the
|
||||||
|
# half-deployed state this handler exists to prevent.
|
||||||
|
systemctl --user restart open-notebook-surrealdb.service
|
||||||
|
wait_container_healthy open-notebook-surrealdb 90 || return 1
|
||||||
|
systemctl --user restart open-notebook.service
|
||||||
|
# The gate is the API's own health route: upstream serves `GET /health` on
|
||||||
|
# 5055 with {"status":"healthy"} and exempts it from the password
|
||||||
|
# middleware, so it stays a real answer once OPEN_NOTEBOOK_PASSWORD is set.
|
||||||
|
# Host port 5056, because seerr already holds 127.0.0.1:5055. The UI on
|
||||||
|
# 8502 is deliberately not gated: supervisord starts it only after the API
|
||||||
|
# answers (the image's wait-for-api.sh), so this gate already implies it.
|
||||||
|
wait_for_health open-notebook http://127.0.0.1:5056/health 180
|
||||||
|
}
|
||||||
|
|
||||||
deploy_comfyui() {
|
deploy_comfyui() {
|
||||||
# Build first: the unit's Image= resolves against local storage, so the
|
# Build first: the unit's Image= resolves against local storage, so the
|
||||||
# image has to exist before the generator starts a container from it.
|
# image has to exist before the generator starts a container from it.
|
||||||
@@ -1573,6 +1671,7 @@ deploy_service() {
|
|||||||
gluetun) deploy_gluetun ;;
|
gluetun) deploy_gluetun ;;
|
||||||
searxng-valkey) deploy_searxng_valkey ;;
|
searxng-valkey) deploy_searxng_valkey ;;
|
||||||
open-webui) deploy_open_webui ;;
|
open-webui) deploy_open_webui ;;
|
||||||
|
open-notebook) deploy_open_notebook ;;
|
||||||
pihole) deploy_pihole ;;
|
pihole) deploy_pihole ;;
|
||||||
gitea-postgres) deploy_gitea_unit gitea-postgres gitea-postgres ;;
|
gitea-postgres) deploy_gitea_unit gitea-postgres gitea-postgres ;;
|
||||||
gitea-server) deploy_gitea_unit gitea-server gitea-server ;;
|
gitea-server) deploy_gitea_unit gitea-server gitea-server ;;
|
||||||
|
|||||||
@@ -141,6 +141,7 @@ valid_service() {
|
|||||||
dankclaw:alertmanager|\
|
dankclaw:alertmanager|\
|
||||||
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
||||||
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
|
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
|
||||||
|
dankclaw:open-notebook|\
|
||||||
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
|
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
|
||||||
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
||||||
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
||||||
|
|||||||
@@ -23,6 +23,15 @@
|
|||||||
# moving 17 % less weight per step can pay. That is an argument, not a
|
# moving 17 % less weight per step can pay. That is an argument, not a
|
||||||
# measurement: this script exists so the decision is made on OUR card, against
|
# measurement: this script exists so the decision is made on OUR card, against
|
||||||
# the PQ2_0 record (76.6 t/s single-stream decode, 1,258 t/s prefill at 35k).
|
# the PQ2_0 record (76.6 t/s single-stream decode, 1,258 t/s prefill at 35k).
|
||||||
|
# MEASURED ON OUR CARD (2026-10-18). Frank ordered this shape at 02:36 — Q8
|
||||||
|
# mmproj, batch 3072/ubatch 1536, two contexts — and the unit ran it as
|
||||||
|
# production 03:14:39→03:38:57 (pid 191358, journal `type ptq1_0: 402 tensors`).
|
||||||
|
# Its own print_timing lines: decode **48.0 / 47.6 t/s at 512 tokens** (47.9 at
|
||||||
|
# 229, 49.0 at 451), 25–27 t/s per stream with two streams live, prefill 1,295
|
||||||
|
# t/s at 30,637 → 943 at 97,856 → 801 at 141,365. The 1,201 MiB arrived and the
|
||||||
|
# decode tax came with it: −37 % against the PQ2_0 record, so the Ampere warning
|
||||||
|
# DOES generalize to GA102. Frank scratched the 27B plan at 03:33 for the 9B;
|
||||||
|
# this payload is the 27B rollback, not a candidate.
|
||||||
#
|
#
|
||||||
# THE RUNTIME LOCK CARRIES OVER. Both packings are custom GGML tensor types and
|
# THE RUNTIME LOCK CARRIES OVER. Both packings are custom GGML tensor types and
|
||||||
# stock llama.cpp rejects them ("It rejects PQ2_0 and PTQ1_0 as unknown types"
|
# stock llama.cpp rejects them ("It rejects PQ2_0 and PTQ1_0 as unknown types"
|
||||||
|
|||||||
+15
-12
@@ -68,10 +68,11 @@
|
|||||||
# ceiling does NOT move with it: HALOGEN_CTX stays 200,704, because the
|
# ceiling does NOT move with it: HALOGEN_CTX stays 200,704, because the
|
||||||
# checkpoint's native window is 262,144 and the engine refuses to go past it
|
# checkpoint's native window is 262,144 and the engine refuses to go past it
|
||||||
# without HALOGEN_ROPE_YARN (its own 400 text says so). ~28 KiB/position
|
# without HALOGEN_ROPE_YARN (its own 400 text says so). ~28 KiB/position
|
||||||
# (FLAGS.md) ⇒ ~10.8 GiB, where pool == ctx (200,704) cost 5.5, 262,144 cost
|
# (FLAGS.md) ⇒ the DEPLOYED 2026-10-18 boot RESERVES 11.0 GiB (`reserving the
|
||||||
# 7.2 and the 2026-10-06 shape's 524,288 measured 14.6. The engine's preflight
|
# KV pool, 401408 positions (about 11.0 GiB)`), where pool == ctx (200,704)
|
||||||
# formula bills a little more than the reservation (block scratch + ~115
|
# cost 5.5, 262,144 cost 7.2 and the 2026-10-06 shape's 524,288 measured 14.6.
|
||||||
# MiB/slot of O(1) state) — quote the RESERVATION. FLAGS.md law: "RAM the pool
|
# The preflight formula bills 11.3 (block scratch + ~115 MiB/slot of O(1)
|
||||||
|
# state) — quote the RESERVATION. FLAGS.md law: "RAM the pool
|
||||||
# takes is RAM that table loses" — the 47.68 GiB n-gram table surrenders this
|
# takes is RAM that table loses" — the 47.68 GiB n-gram table surrenders this
|
||||||
# much page cache, so the warm prompt-cache ratio is what to re-measure.
|
# much page cache, so the warm prompt-cache ratio is what to re-measure.
|
||||||
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
||||||
@@ -82,14 +83,16 @@
|
|||||||
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
||||||
# mmap'd page cache beside the checkpoint, not a pinned charge — the boot
|
# mmap'd page cache beside the checkpoint, not a pinned charge — the boot
|
||||||
# ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
|
# ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
|
||||||
# weights ~62.9 GiB locked (0.8 of them the vision tower) + KV ~10.8 +
|
# weights 62.9 GiB locked (0.8 of them the vision tower) + KV 11.0 +
|
||||||
# working ~5.7 (HALOGEN_MAX_TOK=8192) ≈ 79.4 GiB engine-side, with the
|
# working 5.7 (HALOGEN_MAX_TOK=8192) = 79.6 GiB engine-side (the boot
|
||||||
# prompt cache (~1.0 at this ctx; 1.3 was measured at 262144) booked
|
# ledger of the 401408-position deploy, 2026-10-18), with the 1.3 GiB
|
||||||
# separately by the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
# prompt cache (12 entries x 111 MiB at ctx 200704) booked separately by
|
||||||
# honest. Against the 121.2 GiB box that leaves ~22 GiB over the reserve for
|
# the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host honest. Against
|
||||||
# baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot
|
# the 121.2 GiB box that leaves ~20 GiB over the reserve (MemAvailable
|
||||||
# ledger is the referee — read it after deploy and re-size if the engine
|
# read 22 GiB right after the raise) for baseline co-tenancy
|
||||||
# prints a clamp.
|
# (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot ledger is the
|
||||||
|
# referee — read it after deploy and re-size if the engine prints a
|
||||||
|
# clamp.
|
||||||
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
||||||
# turns the MTP head off for whole requests after low acceptance and
|
# turns the MTP head off for whole requests after low acceptance and
|
||||||
# HALVES decode as a result. Frank accepted the head always-on trade
|
# HALVES decode as a result. Frank accepted the head always-on trade
|
||||||
|
|||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# frank-tts.env — tunables for **frank-tts**, the fleet's cloned-voice speech
|
||||||
|
# endpoint on Dankclaw, host port **7772** (the LLM port family: 7770
|
||||||
|
# frank-dragoon, 7771 frank-zealot, 7772 frank-tts). Read by the Quadlet unit
|
||||||
|
# (EnvironmentFile=), which is why the paths below are CONTAINER paths.
|
||||||
|
#
|
||||||
|
# WHAT THIS IS. One OpenAI-compatible /v1/audio/speech façade in front of
|
||||||
|
# whichever TTS engine wins the engine decision (dankclaw/voiceservice/app.py
|
||||||
|
# is a single stdlib-only file in a locally built image). Consumers — omp's
|
||||||
|
# TTS device, OpenWebUI, open-notebook — address a VOICE BY NAME
|
||||||
|
# (voice=narrator1, or the harness' voice_id=narrator1) and never see a path:
|
||||||
|
# a name is real exactly when /voices/<name>/reference.wav exists, and
|
||||||
|
# anything else is a 400 that lists the names that are.
|
||||||
|
#
|
||||||
|
# THE PACKS. /home/dank/voices on the host, mounted read-only at /voices. One
|
||||||
|
# directory per voice, and a directory is a PACK only when it carries BOTH
|
||||||
|
# required members:
|
||||||
|
# reference.wav 24 kHz mono PCM, 18-25 s of clean single-speaker audio,
|
||||||
|
# plus the 16 kHz variant some engines want;
|
||||||
|
# the engine artifact the pinned backend's own speaker file (embedding /
|
||||||
|
# speaker weights). REQUIRED, beside the WAV and never
|
||||||
|
# replacing it — a voice that cannot be reused by another
|
||||||
|
# runtime is not a saved voice.
|
||||||
|
# plus manifest.json for provenance: source file, offsets (always past the
|
||||||
|
# first 10 s of that file, the source-material law), engine, sha256. Packs are
|
||||||
|
# host state and stay out of this repo; they must be world-readable because the
|
||||||
|
# container runs as uid 10000. Who enforces what: the shim gates on
|
||||||
|
# reference.wav because that is the only engine-agnostic term, so the pack
|
||||||
|
# builder is what refuses to emit a pack without its engine artifact — until a
|
||||||
|
# winning engine pins the format and this gate can name the file.
|
||||||
|
#
|
||||||
|
# BACKEND SWITCH. TTS_BACKEND picks the shape and nothing else changes for the
|
||||||
|
# consumer:
|
||||||
|
# omni-http forward to an OpenAI-shaped upstream on the host (:8880),
|
||||||
|
# carrying the voice along as base64 reference_audio. The
|
||||||
|
# engine does the inference; the shim resolves the voice and
|
||||||
|
# re-encodes to mp3 when asked.
|
||||||
|
# llama-tts-cli run LLAMA_TTS_BIN once per request with
|
||||||
|
# --tts-speaker-file <reference.wav>, nice 19 so a narration
|
||||||
|
# never starves the text endpoints on this box. Needs
|
||||||
|
# LLAMA_TTS_BIN + LLAMA_TTS_MODEL filled in; LLAMA_TTS_MMPROJ
|
||||||
|
# only if that binary takes a projector.
|
||||||
|
# The upstream address is host.containers.internal — the gateway name a bridge
|
||||||
|
# container uses for the host. It is NOT the LAN IP and it is NOT
|
||||||
|
# 127.0.0.1: a bridge netns has neither.
|
||||||
|
VOICES_DIR=/voices
|
||||||
|
TTS_BACKEND=omni-http
|
||||||
|
TTS_UPSTREAM=http://host.containers.internal:8880
|
||||||
|
TTS_UPSTREAM_MODEL=voxcpm2
|
||||||
|
|
||||||
|
# CLI-backend knobs, inert while TTS_BACKEND=omni-http. Empty means "not
|
||||||
|
# configured": the endpoint answers a 500 that says so instead of running
|
||||||
|
# `nice env -m ...` with an empty binary name.
|
||||||
|
LLAMA_TTS_BIN=
|
||||||
|
LLAMA_TTS_MODEL=
|
||||||
|
LLAMA_TTS_MMPROJ=
|
||||||
|
LLAMA_TTS_LANG=english
|
||||||
|
LLAMA_TTS_THREADS=8
|
||||||
|
|
||||||
|
# The wire shape. 7772 is frank-tts's host port (the container's too), the next
|
||||||
|
# slot in the fleet's model-endpoint family; the TTS engine upstream of it
|
||||||
|
# keeps 8880, which is not a fleet endpoint.
|
||||||
|
SHIM_PORT=7772
|
||||||
|
# Longest single narration, in characters of `input`. Podcast-length text is
|
||||||
|
# the consumer's job to chunk; one request over this limit is refused with 414
|
||||||
|
# rather than silently truncated into a half-sentence of audio.
|
||||||
|
SHIM_MAX_CHARS=15000
|
||||||
|
# Seconds one request may take, covering both an upstream synthesis and a CLI
|
||||||
|
# run. Voice is slow compared to text — a long paragraph on a CPU engine is
|
||||||
|
# minutes, not milliseconds — and a client that gave up earlier would only
|
||||||
|
# retry into a second synthesis competing for the same CPU.
|
||||||
|
SHIM_TIMEOUT=600
|
||||||
+20
-13
@@ -65,9 +65,11 @@ UBATCH_SIZE=2048
|
|||||||
# the ledger said exactly that. HALF the 27B's 34,816 B/token, which is the
|
# the ledger said exactly that. HALF the 27B's 34,816 B/token, which is the
|
||||||
# whole reason three native windows fit. f16 would bill 26,112 MiB at this pool
|
# whole reason three native windows fit. f16 would bill 26,112 MiB at this pool
|
||||||
# (against a 24,123 MiB card) and dies at cudaMalloc; the 24 DeltaNet layers
|
# (against a 24,123 MiB card) and dies at cudaMalloc; the 24 DeltaNet layers
|
||||||
# bill a fixed recurrent state per SLOT instead of per token — the ledger's
|
# bill a fixed recurrent state per SLOT instead of per token — the PRODUCTION
|
||||||
# "RS buffer size = 452.25 MiB (3 cells, 32 layers, 3 seqs)" = 150.75 MiB a
|
# ledger prints "RS buffer size = 150.75 MiB (3 cells, 32 layers, 3 seqs)"
|
||||||
# slot — which is why pool arithmetic alone under-prices the shape.
|
# (~50 MiB a cell), while the SPEC-ARMED arms print 452.25 MiB, because MTP
|
||||||
|
# depth 2 bills three seqs per slot (9 cells) — which is why pool arithmetic
|
||||||
|
# alone under-prices the shape.
|
||||||
CACHE_TYPE_K=q8_0
|
CACHE_TYPE_K=q8_0
|
||||||
CACHE_TYPE_V=q8_0
|
CACHE_TYPE_V=q8_0
|
||||||
# SPECULATIVE DECODING: OFF, MEASURED. The head is in the file and the drafter
|
# SPECULATIVE DECODING: OFF, MEASURED. The head is in the file and the drafter
|
||||||
@@ -103,18 +105,23 @@ SPEC_DRAFT_P_MIN=0
|
|||||||
# PRESENCE_PENALTY="" is the documented opt-out.
|
# PRESENCE_PENALTY="" is the documented opt-out.
|
||||||
PRESENCE_PENALTY=1.0
|
PRESENCE_PENALTY=1.0
|
||||||
NGL=99
|
NGL=99
|
||||||
# Boot ledger (2026-10-18, this exact shape, nvidia-smi settled):
|
# Boot ledger (2026-10-18 PRODUCTION boot under the unit, verbosity 4):
|
||||||
# 17,876 MiB used / 6,249 MiB free of 24,123
|
# 17,922 MiB used / 6,203 MiB free of 24,123 (nvidia-smi settled)
|
||||||
|
# CUDA0 model 2,036.71 MiB + CPU_Mapped 257.66 MiB
|
||||||
# CUDA0 KV 13,056.00 MiB (q8_0 K + q8_0 V, 786,432 positions)
|
# CUDA0 KV 13,056.00 MiB (q8_0 K + q8_0 V, 786,432 positions)
|
||||||
# CUDA0 RS 452.25 MiB (3 recurrent-state cells)
|
# CUDA0 RS 150.75 MiB (3 cells, 32 layers, 3 seqs)
|
||||||
# compute 1,409 MiB CUDA + 1,201 MiB CUDA_Host, projector 595 + 248 MiB
|
# compute 1,409.02 MiB CUDA + 1,088.87 MiB CUDA_Host (+2.84 MiB output buf)
|
||||||
|
# mmproj worst-case estimate 868.32 MiB (248.10 CUDA + 24.93 CPU compute
|
||||||
|
# meta) — charged AFTER the engine's own fit check, as always
|
||||||
# no clamp line, no cudaMalloc error
|
# no clamp line, no cudaMalloc error
|
||||||
# Measured on the same boot: 167.2 t/s greedy decode (/completion, 512 tokens),
|
# Measured on this boot (/completion timings, greedy, ignore_eos, 512 tokens):
|
||||||
# 158.9 t/s single-stream chat, 280.2 t/s aggregate across THREE concurrent
|
# 168.7 t/s decode (three runs 168.4-168.7) and ~300 t/s aggregate across
|
||||||
# streams (93.8 each), 2,877 t/s prefill at 106,418 tokens. The 27B record it
|
# THREE concurrent streams (102.8 each). The trial arm's record (167.2 decode,
|
||||||
# replaces: 76.6 decode, 109.4 aggregate across two streams, 691.7 prefill at
|
# 280.2 aggregate, 2,877 t/s prefill at 106,418) stands; the 27B it replaces
|
||||||
# 166k — beaten on every axis. The FIRST request after a boot is a warmup
|
# gave 76.6 / 109.4 / 691.7 at 166k. The FIRST request after a boot is a
|
||||||
# artifact (37.7 t/s class); never report it.
|
# warmup artifact (37.7 t/s class) and a natural generation that stops early
|
||||||
|
# makes a wall-clock aggregate read 4x low - measure with ignore_eos and read
|
||||||
|
# the engine's own timings.
|
||||||
# Fallback rung if the margin is ever eaten (desktop, co-tenant): q4_0 KV on the
|
# Fallback rung if the margin is ever eaten (desktop, co-tenant): q4_0 KV on the
|
||||||
# 27B era measured 16,392 used / 7,733 free at 2 × 200,704; the same drop here
|
# 27B era measured 16,392 used / 7,733 free at 2 × 200,704; the same drop here
|
||||||
# frees ~6.5 GiB at three windows. UBATCH_SIZE=1024 is the cheaper lever
|
# frees ~6.5 GiB at three windows. UBATCH_SIZE=1024 is the cheaper lever
|
||||||
|
|||||||
@@ -0,0 +1,100 @@
|
|||||||
|
# open-notebook.env — configuration for **open-notebook**, the fleet's podcast
|
||||||
|
# generator on Dankclaw (UI host port **8502**, API host port **5056**). Read by
|
||||||
|
# BOTH Quadlet units (dankclaw/open-notebook/quadlet/) as the FIRST
|
||||||
|
# EnvironmentFile= line, which is why every path and host name below is the
|
||||||
|
# value the CONTAINER must see.
|
||||||
|
#
|
||||||
|
# TWO FILES, ONE ORDER. This file is tracked; it holds knobs, not credentials.
|
||||||
|
# The three secret keys below are declared EMPTY on purpose — the `scripts/*.env`
|
||||||
|
# gitignore exception exists for "paths and numbers, never credentials" — and the
|
||||||
|
# real values live in the host file the deployer seeds at
|
||||||
|
# /home/dank/Containers/open-notebook/env (mode 600, seeded from dankclaw/.env,
|
||||||
|
# the same convention grafana/marreta/pihole follow). Both files are handed to
|
||||||
|
# podman in EnvironmentFile= order and the LAST one wins (verified against the
|
||||||
|
# generated run line), so the host file overrides these blanks. Do not reorder
|
||||||
|
# the two lines.
|
||||||
|
#
|
||||||
|
# WHY THE DEPLOYER REFUSES TO START THESE UNITS WITHOUT THE SECRET FILE. An empty
|
||||||
|
# OPEN_NOTEBOOK_PASSWORD is not "unconfigured", it is **authentication off**, and
|
||||||
|
# the episode-audio download is one of the routes the API serves without a
|
||||||
|
# session; an empty SURREAL_PASS lets the database come up with different root
|
||||||
|
# credentials than the app will present; an empty OPEN_NOTEBOOK_ENCRYPTION_KEY
|
||||||
|
# means no provider credential can be saved at all. So the guard is in
|
||||||
|
# deploy-remote.sh (seed_open_notebook_env), not in the app's opinion.
|
||||||
|
#
|
||||||
|
# THE CPU-SHUTDOWN GUARD (the two knobs that matter most on this box). Defaults
|
||||||
|
# are TTS_BATCH_SIZE=5 and OPEN_NOTEBOOK_WORKER_MAX_TASKS=5 — five parallel
|
||||||
|
# synthesis requests from five parallel jobs. frank-tts answers each request by
|
||||||
|
# spawning its own engine process (~8 GiB resident, 8 threads, measured in the
|
||||||
|
# frank-tts CPU trial of 2026-10-18), so the defaults mean ~40 GiB and forty
|
||||||
|
# threads aimed at a machine that also serves frank-dragoon and frank-zealot and
|
||||||
|
# whose TTS engine runs on the SAME CPU. Both numbers are 1 here: one episode at
|
||||||
|
# a time, one voice line synthesized at a time. That is a throughput choice, not
|
||||||
|
# a correctness one — a podcast is a background job — and it is the difference
|
||||||
|
# between a slow render and a box that stops answering the text endpoints
|
||||||
|
# mid-sentence. Raise them only with a TTS engine that batches internally.
|
||||||
|
#
|
||||||
|
# THE TTS CLOCK. ESPERANTO_TTS_TIMEOUT is the worker's patience per DIALOGUE
|
||||||
|
# LINE, not per episode: one episode is dozens of POSTs to /v1/audio/speech
|
||||||
|
# (the app synthesises each speaker turn separately and concatenates the clips
|
||||||
|
# with ffmpeg afterwards), and a CPU engine takes single-digit seconds for a
|
||||||
|
# line, with the tail of a long turn well past the 300 s default. 660 sits one
|
||||||
|
# round-trip OUTSIDE SHIM_TIMEOUT=600 in scripts/frank-tts.env — the shim's own
|
||||||
|
# per-request ceiling — so a frank-tts give-up arrives as frank-tts's error and
|
||||||
|
# not as open-notebook abandoning a synthesis still in flight. An abandoned
|
||||||
|
# request does not cancel the engine process; it just leaves the episode stuck in
|
||||||
|
# "generating" while the audio arrives.
|
||||||
|
#
|
||||||
|
# THE SCRIPTWRITER. ESPERANTO_LLM_TIMEOUT is the app's own default (180 s), kept
|
||||||
|
# visible beside the TTS number because both belong to one podcast run: the
|
||||||
|
# outline call and the transcript call are the slow text work, and frank-zealot
|
||||||
|
# answers them well inside 180. Provider choice is made in the UI, not here —
|
||||||
|
# see [[Open Notebook]] for the one-time credential provisioning — and it is
|
||||||
|
# frank-zealot on purpose: script writing asks for a JSON-schema response, which
|
||||||
|
# the OpenAI-compatible llama.cpp endpoint is known to serve, while halogen's
|
||||||
|
# structured-output behaviour on frank-dragoon is UNVERIFIED. halogen remains
|
||||||
|
# the unproven alternative, not the fallback.
|
||||||
|
#
|
||||||
|
# THE DATABASE COORDINATES. SURREAL_URL addresses the sibling container BY NAME
|
||||||
|
# on systemd-open-notebook (container DNS is scoped to the owning network, so
|
||||||
|
# this is the only form that works). NAMESPACE/DATABASE/USER are plain names.
|
||||||
|
# SURREAL_PASSWORD and SURREAL_PASS are ONE value under two spellings: the app
|
||||||
|
# reads SURREAL_PASSWORD, the `surreal` CLI reads SURREAL_PASS. Keep them equal;
|
||||||
|
# the deployer writes both into the seeded file from a single SURREAL_PASSWORD in
|
||||||
|
# dankclaw/.env, so hand-editing one line is the only way to desync them.
|
||||||
|
|
||||||
|
SURREAL_URL=ws://open-notebook-surrealdb:8000/rpc
|
||||||
|
SURREAL_NAMESPACE=open_notebook
|
||||||
|
SURREAL_DATABASE=open_notebook
|
||||||
|
SURREAL_USER=open_notebook
|
||||||
|
# Secrets — filled on the host, never here (see the header).
|
||||||
|
SURREAL_PASSWORD=
|
||||||
|
SURREAL_PASS=
|
||||||
|
OPEN_NOTEBOOK_ENCRYPTION_KEY=
|
||||||
|
OPEN_NOTEBOOK_PASSWORD=
|
||||||
|
|
||||||
|
# WHERE THE BROWSER FINDS THE API. The UI auto-detects `<host-you-opened>:5055`,
|
||||||
|
# and 5055 on this host belongs to seerr, so the API leaves the container on 5056
|
||||||
|
# and this line has to say so. Both origins are listed in CORS_ORIGINS because
|
||||||
|
# the API is called cross-origin from the UI's own port: the LAN origin is the
|
||||||
|
# normal one, the localhost one covers a browser opened on Dankclaw itself. The
|
||||||
|
# app logs a warning at start while CORS_ORIGINS is unset and defaults to `*`,
|
||||||
|
# which is a wildcard in front of an endpoint that serves every episode's audio.
|
||||||
|
API_URL=http://192.168.0.143:5056
|
||||||
|
CORS_ORIGINS=http://192.168.0.143:8502,http://localhost:8502
|
||||||
|
|
||||||
|
# Concurrency: see "THE CPU-SHUTDOWN GUARD" above. Defaults are 5 and 5.
|
||||||
|
TTS_BATCH_SIZE=1
|
||||||
|
OPEN_NOTEBOOK_WORKER_MAX_TASKS=1
|
||||||
|
|
||||||
|
# Timeouts: see "THE TTS CLOCK" and "THE SCRIPTWRITER" above.
|
||||||
|
ESPERANTO_TTS_TIMEOUT=660
|
||||||
|
ESPERANTO_LLM_TIMEOUT=180
|
||||||
|
|
||||||
|
# Heavy extraction runtimes stay OFF, which is also what keeps boot fast: each
|
||||||
|
# one installs a multi-gigabyte ML/Chromium stack into the image's venv on first
|
||||||
|
# startup (cached on the /app/data volume after that). Podcast generation reads
|
||||||
|
# notes and sources that are already text, so neither is needed here; enable one
|
||||||
|
# deliberately, per the upstream docs, if source ingestion starts to matter.
|
||||||
|
# OPEN_NOTEBOOK_ENABLE_DOCLING=true
|
||||||
|
# OPEN_NOTEBOOK_ENABLE_CRAWL4AI=true
|
||||||
@@ -128,6 +128,16 @@ for path in "${CHANGED[@]}"; do
|
|||||||
add_test tests/docs_completeness.bats
|
add_test tests/docs_completeness.bats
|
||||||
add_test tests/deployment_scoping.bats
|
add_test tests/deployment_scoping.bats
|
||||||
;;
|
;;
|
||||||
|
# open-notebook is the fleet's podcast generator: an app quadlet, the
|
||||||
|
# SurrealDB quadlet behind it, the private network between them, and the
|
||||||
|
# one env file both units read. Like the caddy block above, this case
|
||||||
|
# has to precede the dankclaw/*.container glob, whose `*` crosses `/`
|
||||||
|
# and would otherwise swallow dankclaw/open-notebook/quadlet/*.
|
||||||
|
dankclaw/open-notebook/*|scripts/open-notebook.env)
|
||||||
|
add_test tests/open_notebook.bats
|
||||||
|
add_test tests/docs_completeness.bats
|
||||||
|
add_test tests/deployment_scoping.bats
|
||||||
|
;;
|
||||||
dankclaw/*.container|dankclaw/*.network)
|
dankclaw/*.container|dankclaw/*.network)
|
||||||
add_test tests/open_webui_quadlet.bats
|
add_test tests/open_webui_quadlet.bats
|
||||||
add_test tests/pihole.bats
|
add_test tests/pihole.bats
|
||||||
|
|||||||
@@ -0,0 +1,132 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
description: "open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out — script written by frank-zealot, voices rendered by frank-tts, no cloud provider anywhere in the path."
|
||||||
|
tags:
|
||||||
|
- dankclaw
|
||||||
|
- open-notebook
|
||||||
|
- podcasts
|
||||||
|
- quadlet
|
||||||
|
timestamp: 2026-10-18
|
||||||
|
---
|
||||||
|
|
||||||
|
# Open Notebook
|
||||||
|
|
||||||
|
open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook)) is the notebook layer that turns existing notes and sources into a **spoken episode**: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by **frank-zealot**, the voices are rendered by **frank-tts**, and nothing leaves the LAN.
|
||||||
|
|
||||||
|
**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Deploying it is a deliberate next step, not a pending one.
|
||||||
|
|
||||||
|
**Host:** Dankclaw (rootless Podman, user `dank`) — **Ports:** 8502 UI and 5056 API, published on the LAN, **no vhost and no Caddy route** — **Deployer:** `open-notebook` (one service, two units) — **Docs:** https://github.com/lfnovo/open-notebook
|
||||||
|
|
||||||
|
## Facts (verified 2026-10-18 against the pinned images, upstream source, and the live host)
|
||||||
|
|
||||||
|
| Item | Value |
|
||||||
|
|------|-------|
|
||||||
|
| App image | `docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212` (`open-notebook.container:79`) — the tag upstream's own `docker-compose.yml:27` pins for the multi-container shape. `v1-latest` is a moving tag, so the manifest-list digest (resolved from the registry 2026-10-18) is what makes this reproducible |
|
||||||
|
| Database image | `docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723` (`open-notebook-surrealdb.container:66`) — that digest is **SurrealDB 2.7.0** (image built 2026-09-23; measured by running the digest: `surreal version`) |
|
||||||
|
| Do NOT use | the `-single` tags (`v1-latest-single`). That variant bundles SurrealDB inside the app image and is deprecated upstream; here it would give the host a second, hidden database |
|
||||||
|
| Ports | container 8502 (Next.js UI) + 5055 (FastAPI API) → published `0.0.0.0:8502:8502` and `0.0.0.0:5056:5055` (`open-notebook.container:83-84`). **Host 5055 is taken**: seerr publishes `127.0.0.1:5055` (`dankclaw/seerr/quadlet/seerr.container`, and `ss -ltn` on Dankclaw shows `127.0.0.1:5055` live, 2026-10-18). The API therefore leaves on 5056, which is what forces `API_URL` (below) |
|
||||||
|
| Database port | 8000 inside its own namespace, **never published** (`open-notebook-surrealdb.container` has no `PublishPort`). Upstream publishes `127.0.0.1:8000` for debugging (`docker-compose.yml:18`); this unit does not, and the seam asserts that it stays that way |
|
||||||
|
| Data | `open-notebook-data:/app/data` (named volume: uploads, LangGraph SQLite checkpoints, extraction caches) **plus** the interim host bind `/home/dank/media/podcasts:/app/data/podcasts` (`open-notebook.container:87-88`); database in the `open-notebook-db` named volume |
|
||||||
|
| Networks | app: `dankclaw_default` first, then the private `systemd-open-notebook` (10.89.4.0/24, `open-notebook.network:33`); database: the private bridge only. Order is load-bearing — the generated `--network` flags keep file order, so the default route rides the network where `host.containers.internal` is proven |
|
||||||
|
| Units | `dankclaw/open-notebook/quadlet/open-notebook.container` → `open-notebook.service`; `open-notebook-surrealdb.container` → `open-notebook-surrealdb.service`; both `Restart=always`, `RestartSec=10` / `5`, `[Install] WantedBy=default.target` — boot-persistent per [[Quadlet Lifecycle]] (generated units cannot be `enable`d on this host; the `[Install]` section is what the v1 generator honors at user-manager start) |
|
||||||
|
| Health | app: `curl -fsS http://127.0.0.1:5055/health` (`open-notebook.container:89`); database: `/surreal is-ready` (`open-notebook-surrealdb.container:73`) |
|
||||||
|
| Env | `scripts/open-notebook.env` (repo, knobs) then `/home/dank/Containers/open-notebook/env` (host, secrets, mode 600) — two `EnvironmentFile=` lines per unit, repo file **first** because the last file wins |
|
||||||
|
| Deployer | `deploy_open_notebook` in `scripts/deploy-remote.sh`: seed secrets → ensure both digest pins are in local storage → install both units + the network → `mkdir -p /home/dank/media/podcasts` → restart database, wait for its container health → restart app → `wait_for_health open-notebook http://127.0.0.1:5056/health 180` |
|
||||||
|
|
||||||
|
## What the image actually runs
|
||||||
|
|
||||||
|
One container, three processes under supervisord (`supervisord.conf` upstream): `uvicorn api.main:app --port 5055` (`:12`), the job worker (`:25`), and the Next.js frontend (`:39`). Two consequences the config depends on:
|
||||||
|
|
||||||
|
- **One unit owns the whole app.** There is no separate worker or frontend container to scale or restart, and the frontend refuses to start until the API answers (`wait-for-api.sh && node server.js`, `supervisord.conf:39`) — so gating the API gates the UI.
|
||||||
|
- **The worker's concurrency comes from the environment**, not a flag baked into the image: `--max-tasks ${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5}` (`supervisord.conf:25`). Setting it in the env file is a real change, not decoration.
|
||||||
|
|
||||||
|
Also from the image itself: `WORKDIR /app` and `EXPOSE 8502 5055` (`Dockerfile:84`, `:126`), **no `USER` line** (the app runs as container root, which in this host's rootless map lands as `dank` — the [[Seerr]] UID note measured `0 → 1000` on this machine), and `curl` **and** `ffmpeg` installed in the runtime layer (`Dockerfile:74-76`) — curl makes the health probe possible without a shell wrapper, ffmpeg is how the per-line clips become one file.
|
||||||
|
|
||||||
|
`DATA_FOLDER` is hard-coded `"./data"` (`open_notebook/config.py:5`) and `PODCASTS_FOLDER` is `"./data/podcasts"` (`:19`); the database stores each episode's audio path **relative** to that folder (`episodes/<uuid>/audio/<uuid>.mp3`), and the download route is `GET /api/podcasts/episodes/{id}/audio`. That is why covering one subdirectory with a bind mount is sufficient — nothing in the database has to change if the mount moves.
|
||||||
|
|
||||||
|
## Provisioning the providers (after first boot; UI or REST, never a unit)
|
||||||
|
|
||||||
|
Credentials are stored in the database, encrypted, and are **not expressible in env or a Quadlet file**. The sequence is one-time, in **Manage → Models**, and both entries use provider type **OpenAI Compatible** (upstream: any server that implements `/v1/audio/speech` works this way — `docs/5-CONFIGURATION/local-tts.md:5`).
|
||||||
|
|
||||||
|
1. **TTS credential** — base URL **`http://frank-tts:7772/v1`**. The `/v1` is part of the URL: the client posts to `<base_url>/audio/speech`, so a base URL without `/v1` 404s on every dialogue line (`local-tts.md:72-80`, the base-URL table, always carries `/v1`). `frank-tts` resolves by container name because the app is attached to `dankclaw_default`, the network frank-tts's unit joins (`dankclaw/voiceservice/quadlet/frank-tts.container:51-52`). If frank-tts is ever moved off that network, the published-port form is `http://192.168.0.143:7772/v1` — a published port is dialable at the host's LAN address from any netns. The API-key field stays empty; frank-tts does not authenticate.
|
||||||
|
2. **LLM credential** — base URL **`http://host.containers.internal:7771/v1`**, model row `frank-zealot`. frank-zealot is a **host process** on `0.0.0.0:7771` (`ss -ltn` on Dankclaw shows the listener with no container owning it, 2026-10-18), and on this netavark host a bridge container reaches the host's LAN IP only at a *published* port — which is why Prometheus scrapes the same endpoint as `host.containers.internal:7771` (`shared/prometheus/prometheus.yml:89-111`, "not their own LAN addresses"). See [[Container Network Namespaces]]. `http://192.168.0.143:7771/v1` is the variant to try only if the gateway name ever stops resolving; it is expected to be refused from a bridge netns.
|
||||||
|
3. **Model rows are created by name.** Discovery is optional (see the next section): type the model id and confirm the add. One base URL per credential, so the TTS and LLM credentials are necessarily two separate configurations.
|
||||||
|
|
||||||
|
**Verify reachability before saving**, from inside the container, because both addresses are netns-dependent:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
podman exec open-notebook curl -sS http://frank-tts:7772/health
|
||||||
|
podman exec open-notebook curl -sS http://host.containers.internal:7771/v1/models
|
||||||
|
```
|
||||||
|
|
||||||
|
**The blocker to state plainly:** frank-tts is built, named and tested but **not deployed** — nothing answers on 7772 (`ss -ltn`, Dankclaw, 2026-10-18; [[Local TTS and Voice Cloning]] records the same census). open-notebook deployed today writes scripts and fails at the first voice line. frank-tts ships first, or podcast generation waits.
|
||||||
|
|
||||||
|
**Why frank-zealot and not frank-dragoon:** podcast scriptwriting asks the model for a JSON-schema response (with a plain-JSON fallback). llama.cpp's OpenAI-compatible endpoint is a known-good target for that; frank-dragoon's halogen runtime has **unverified** structured-output behaviour, so it is the unproven alternative, not the fallback. `ESPERANTO_LLM_TIMEOUT=180` (the app's own default) covers an outline or transcript call on zealot comfortably; upstream's note is to keep it under 600 so a slow call fails with a clear error instead of a UI timeout.
|
||||||
|
|
||||||
|
## Speakers and voices: 1–4, and the IDs are free text
|
||||||
|
|
||||||
|
A **speaker profile** carries one to four speakers (`docs/3-USER-GUIDE/creating-podcasts.md:27`), each with a personality and a **Voice ID that is typed exactly and passed through to the TTS request untouched** (`:29`) — there is no whitelist, no enum, no validation against a voice list. So the fleet's voice-pack names work verbatim:
|
||||||
|
|
||||||
|
| Speaker | Voice ID to type |
|
||||||
|
|---------|------------------|
|
||||||
|
| Speaker 1 | `narrator1` |
|
||||||
|
| Speaker 2 | `narrator2` |
|
||||||
|
| Speaker 3 | `narrator3` |
|
||||||
|
|
||||||
|
Those are the pack names frank-tts resolves from `/home/dank/voices` (`scripts/frank-tts.env` header: a voice is real exactly when `/voices/<name>/reference.wav` exists). Two traps, both from upstream's shipped defaults:
|
||||||
|
|
||||||
|
- The three seeded speaker profiles carry **OpenAI voice names** (`nova`, `alloy`, …) and **no voice model at all** — they show a "Setup required" badge, and generation fails with a voice error until they are edited (`creating-podcasts.md:13`, `:33`; `local-tts.md:89`). Add a `narrator1`/`narrator2` pair (or a third/fourth) rather than editing the seeded ones in place.
|
||||||
|
- Two speakers with the same voice are hard to follow (`creating-podcasts.md:89`). Distinct packs per speaker is the point of the 1–4 shape.
|
||||||
|
|
||||||
|
## Two buttons that fail against frank-tts, and why that is cosmetic
|
||||||
|
|
||||||
|
- **Test Connection** probes `GET <base_url>/models` (`open_notebook/ai/connection_tester.py:208-229`). frank-tts serves exactly two GET routes — `/health` and `/v1/audio/voices` — and 404s every other path (`dankclaw/voiceservice/app.py:228-238`). The button reports a failure against a healthy server.
|
||||||
|
- **Discover Models** does the same `GET <base>/models` (`open_notebook/ai/model_discovery.py:26-28`) and finds nothing to list.
|
||||||
|
- The TTS test path also synthesises with a **hard-coded voice `alloy`** for `openai_compatible` (`connection_tester.py:328-336`), which is not a fleet voice pack and would be refused even if the probe reached frank-tts.
|
||||||
|
|
||||||
|
None of it blocks anything: model rows are added **by name** ("If a model isn't listed, type its id and click Add", `local-tts.md:82`) and the Voice ID field is free text. The failure is a missing `/models` route on a deliberately minimal shim, not a broken credential. The honest end-to-end test is a real generation job, or the `podman exec … curl /v1/audio/speech` call frank-tts's own seam documents.
|
||||||
|
|
||||||
|
## The CPU guard, and why both numbers are 1
|
||||||
|
|
||||||
|
Defaults are `TTS_BATCH_SIZE=5` and `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` (`docs/5-CONFIGURATION/environment-reference.md:129-130`) — five parallel synthesis requests from five parallel jobs. frank-tts spawns an engine process per request (about 8 GiB resident, 8 threads, from the frank-tts CPU trial of 2026-10-18; `scripts/frank-tts.env` pins `LLAMA_TTS_THREADS=8`), on a machine that also serves frank-dragoon and frank-zealot. Five of those is ~40 GiB and forty threads competing with two LLM endpoints for the same cores. `scripts/open-notebook.env:87-88` sets both to 1: one episode at a time, one voice line at a time. Upstream's own guidance is the same direction ("Set 1 for a local model on a single GPU", "Lower it … for rate-limited or single-threaded TTS servers").
|
||||||
|
|
||||||
|
The clock next to it: `ESPERANTO_TTS_TIMEOUT` (`:91`) is patience **per dialogue line**, not per episode — a three-minute episode is dozens of separate POSTs, and the tail of a long turn on a CPU engine goes past upstream's 300 s default. It is set to **660, one round-trip outside `SHIM_TIMEOUT=600`** in `scripts/frank-tts.env` (frank-tts's own per-request ceiling): nested budgets go inside-out — the same rule that file states for `LLAMA_TIMEOUT` — so a frank-tts give-up arrives labelled as frank-tts's error instead of the worker's own timeout, and re-measuring frank-tts cannot silently strand this number below it. An abandoned request does not cancel the engine process; it only strands the episode in "generating".
|
||||||
|
|
||||||
|
## Secrets, and the difference between blank and unset
|
||||||
|
|
||||||
|
`OPEN_NOTEBOOK_PASSWORD` unset means **authentication off** (`environment-reference.md:46`) — every API route, including the episode-audio download, answers anyone who can reach the published port. That is a real exposure on a LAN-published service with no vhost and no guard chain in front of it, which is why `deploy_open_notebook` refuses to start either unit unless the seeded host file carries a value for all four secret keys (`OPEN_NOTEBOOK_ENCRYPTION_KEY`, `OPEN_NOTEBOOK_PASSWORD`, `SURREAL_PASSWORD`, `SURREAL_PASS`). `OPEN_NOTEBOOK_ENCRYPTION_KEY` is separately load-bearing: it is **required to save any provider credential**, and changing it later makes stored keys undecryptable (`:44`).
|
||||||
|
|
||||||
|
The tracked file declares the secrets **empty** and the host file supplies them; podman applies `--env-file` arguments in order and the last one wins (verified against the generated `ExecStart` line, 2026-10-18). Reordering the two `EnvironmentFile=` lines in a unit would blank every credential — that is the trap the seam's ordering assertion exists for.
|
||||||
|
|
||||||
|
The database has the same trap in a different spelling: the app reads `SURREAL_PASSWORD`, the `surreal` CLI reads `SURREAL_PASS` (its `start --help` prints `[env: SURREAL_PASS=]`; measured against the pinned digest). One value, written twice, by the seed function — so the pair cannot be given different root passwords. Both arrive through the environment, **not** through `Exec=`: `--pass` on the command line would put the database password in `/proc/<pid>/cmdline`, readable by any `ps` on the host. Measured with the pinned image: env-only credentials produce `Initialising credentials user=…`, the root user is created on first boot, a correct login authenticates and a wrong password is refused — no `--user`/`--pass` flags needed, and no `--bind` either, because the image's own `SURREAL_BIND=0.0.0.0:8000` is what makes the RPC reachable from another container at all.
|
||||||
|
|
||||||
|
## Episode destination: interim, and reversible (issue #7)
|
||||||
|
|
||||||
|
**Episode files currently exist in two places by design.** The app's own state stays in the `open-notebook-data` named volume; finished audio additionally lands on the host, because `/home/dank/media/podcasts` is bind-mounted over `/app/data/podcasts` (`open-notebook.container:88`). That is the **interim** answer to where episodes live: the destination is an open decision tracked as **Gitea issue #7, labelled `human-only` — agents must not work it.** Until Frank settles it, episode MP3s are ordinary files in his home directory, readable without `podman unshare`, which the database volume would require (measured with the pinned SurrealDB image).
|
||||||
|
|
||||||
|
Reversibility is one line: delete the `Volume=/home/dank/media/podcasts:/app/data/podcasts` line and the episodes go back to living solely inside the named volume. Nothing else in the config assumes the bind. See [[Storage Policy]] for the repo's storage conventions this interim state deliberately bends.
|
||||||
|
|
||||||
|
## Deploying, and the digest-pull step
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# on Dankclaw, as dank — the pins are Pull=never, so fetch them first
|
||||||
|
podman pull docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
|
||||||
|
podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
|
||||||
|
```
|
||||||
|
|
||||||
|
`deploy_open_notebook` runs the same two pulls when the image is absent from local storage, so the manual step is only needed to pre-seed an offline host or to bump a pin. Then `scripts/deploy.sh --target dankclaw --service open-notebook`. Both `Pull=never` and the pin comment live in the unit files; upgrading means **editing the digest**, not repulling a tag — see [[Deployment Flow]].
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
|
||||||
|
- **Nothing deployed (today's state):** revert the commits. There is no host state to undo — no unit, no container, no volume, no `/home/dank/media/podcasts` directory beyond what the deployer would create.
|
||||||
|
- **Deployed:** `systemctl --user stop open-notebook.service open-notebook-surrealdb.service`, then move the three files (`open-notebook.container`, `open-notebook-surrealdb.container`, `open-notebook.network`) out of `~/.config/containers/systemd/` into `~/.AGENTTRASH/` — never `rm` — and `systemctl --user daemon-reload`. The generated units disappear with their sources.
|
||||||
|
- **Kept on purpose:** the `open-notebook-data` and `open-notebook-db` named volumes and `/home/dank/media/podcasts`. Episode audio and the notebook database are user data; removing them is a separate, deliberate act, and episode files are host files precisely so they can outlive the container.
|
||||||
|
- **Homepage:** the card is `wip: "down"` + `showLink: false`, so retiring the service needs no card surgery beyond setting it back to that state.
|
||||||
|
|
||||||
|
## Citations
|
||||||
|
|
||||||
|
- Repo: `dankclaw/open-notebook/quadlet/{open-notebook.container,open-notebook-surrealdb.container,open-notebook.network}`, `scripts/open-notebook.env`, `dankclaw/.env.example`, `scripts/deploy-plan.py`, `scripts/deploy-remote.sh` (`seed_open_notebook_env`, `ensure_pinned_image`, `deploy_open_notebook`), `scripts/deploy.sh`, `scripts/test-changed.sh`, `tests/open_notebook.bats`, `shared/homepage/services.json`, `dankclaw/seerr/quadlet/seerr.container` (the 5055 collision), `dankclaw/voiceservice/{quadlet/frank-tts.container,app.py}`, `scripts/frank-tts.env`, `shared/prometheus/prometheus.yml`
|
||||||
|
- Upstream `lfnovo/open-notebook` @ main, read 2026-10-18: `docker-compose.yml:3,10,11,18,27,29-30,39-43`, `Dockerfile:69-76,84,126,140,158`, `supervisord.conf:12,23-25,39`, `api/main.py:73,227-230,239-241,414`, `api/credentials_service.py:418-420,552`, `api/routers/credentials.py:426-429,433-437`, `open_notebook/config.py:5,19`, `open_notebook/podcasts/audio_paths.py`, `open_notebook/ai/connection_tester.py:208-238,326-336`, `open_notebook/ai/model_discovery.py:26-28`, `docs/3-USER-GUIDE/creating-podcasts.md:13,27,29,33,89`, `docs/5-CONFIGURATION/environment-reference.md:44,46,106,110,129,130`, `docs/5-CONFIGURATION/local-tts.md:5,45,72-82,89`
|
||||||
|
- Registry (2026-10-18): `lfnovo/open_notebook` tags (`v1-latest`, `latest`, the `-single` set) and manifest-list digests; `surrealdb/surrealdb` `v2` → 2.7.0
|
||||||
|
- Local measurements (2026-10-18, workstation, pinned digests): `surreal start --help` (credential and bind options carry `[env: SURREAL_*]`); running the pinned SurrealDB digest with env-only credentials (credential init in the log, correct login authenticates, wrong password refused, `/surreal is-ready` → `OK` exit 0, named volume owned by the mapped uid); the v1-compatible Quadlet generator dry-run of all three unit files (two `--env-file` in order, two `--network` in order, named volume + host bind, both health probes, `--pull never`)
|
||||||
|
- Host evidence (Dankclaw, 2026-10-18, read-only): `ss -ltn` (`127.0.0.1:5055` held by seerr, `0.0.0.0:7771` a host listener, nothing on 7772/8502/5056/8000), `podman ps -a` (no open-notebook container, `dankclaw_default` present), subnet registry from `dankclaw/open-webui/quadlet/open-webui.network`
|
||||||
@@ -53,11 +53,18 @@ Policy while candidate arms held the port): the candidate — Ternary-Bonsai-2-*
|
|||||||
PQ2_0 with the MTP head, 3 × 262,144 — was **deployed as production**, so the
|
PQ2_0 with the MTP head, 3 × 262,144 — was **deployed as production**, so the
|
||||||
picker row below was re-keyed to it (see the next paragraph).
|
picker row below was re-keyed to it (see the next paragraph).
|
||||||
|
|
||||||
The `model` table carries exactly one decoration row, `frank-zealot`
|
The `model` table carries exactly two decoration rows, `frank-zealot` and
|
||||||
(context length 262,144 = the per-slot window since 2026-10-18; description
|
`frank-dragoon`. Both the context length and the description live **inside
|
||||||
re-keyed the same day to Ternary-Bonsai-2-9B PQ2_0 + MTP + Q8_0 vision mmproj —
|
JSON columns** — `params` holds `{"context": {"length": …}}` and `meta` holds
|
||||||
the row's `description` is live DB state, so a payload swap must re-key it in
|
`{"description": …}`; there is no `context_length` or `description` column, so
|
||||||
the database too, not only in the repo).
|
`sqlite3` one-liners against those names fail. Re-keyed 2026-10-18:
|
||||||
|
`frank-zealot` → 262,144 with the Bonsai-2-9B description, and `frank-dragoon`
|
||||||
|
→ 200,704 (its `params` had kept the pre-lowering 262,144 window from
|
||||||
|
2026-10-17 while its own description already said 200,704). These rows are
|
||||||
|
live DB state, so a payload swap must re-key the database too, not only the
|
||||||
|
repo. There is no `sqlite3` binary on Dankclaw — use `python3 -c` with the
|
||||||
|
stdlib `sqlite3` module, and take a `Connection.backup()` snapshot first (the
|
||||||
|
DB is in WAL and the app is live).
|
||||||
|
|
||||||
A third gate sits *after* the table, and it is the one that bites family
|
A third gate sits *after* the table, and it is the one that bites family
|
||||||
accounts (2026-10-07): a model row is only visible to a non-admin user when
|
accounts (2026-10-07): a model row is only visible to a non-admin user when
|
||||||
|
|||||||
@@ -202,10 +202,9 @@ boot ledger is unchanged to the MiB — 62.9 GiB weights (0.8 of them vision) +
|
|||||||
GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP
|
GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP
|
||||||
depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all
|
depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all
|
||||||
answered 200. (Later the same day the pool DOUBLING to 401,408 positions for
|
answered 200. (Later the same day the pool DOUBLING to 401,408 positions for
|
||||||
slot parity was COMMITTED — `2b12716` — but as of 2026-10-18 NOT yet
|
slot parity was COMMITTED (`2b12716`) and **deployed the same evening**: the
|
||||||
redeployed: the live boot still reserves ONE 200,704-position pool (5.5 GiB),
|
boot now reserves `401408 positions (about 11.0 GiB)` and the engine-side total
|
||||||
host `frank-dragoon.env` included. When it lands, the reservation is ~10.8
|
is `79.6 GiB in all` (62.9 weights + 11.0 KV + 5.7 working),
|
||||||
GiB and the engine-side total ~79.4 GiB;
|
|
||||||
see [[frank-dragoon-flash-server-v2-native]].)
|
see [[frank-dragoon-flash-server-v2-native]].)
|
||||||
|
|
||||||
**The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
|
**The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
|
||||||
|
|||||||
@@ -94,9 +94,9 @@ the pool: **two long conversations can no longer share the endpoint** (two of
|
|||||||
≤100k can). That is acceptable here because frank-dragoon carries the `slow`
|
≤100k can). That is acceptable here because frank-dragoon carries the `slow`
|
||||||
(medium-thinking) role and the `security-reviewer` subagent, not the
|
(medium-thinking) role and the `security-reviewer` subagent, not the
|
||||||
interactive workhorse (that is frank-archon).
|
interactive workhorse (that is frank-archon).
|
||||||
Frank lifted exactly that concurrency limit on 2026-10-18 — see *Pool
|
Frank lifted exactly that concurrency limit on 2026-10-18 and the lift is
|
||||||
doubled for slot parity* below; that lift is COMMITTED but not yet deployed,
|
**deployed** (section below): the pool is again `slots × ctx`, so two
|
||||||
so the sentences above remain the live truth until the next redeploy.
|
maximum-length conversations fit the endpoint at once.
|
||||||
|
|
||||||
**Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot
|
**Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot
|
||||||
printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
|
printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
|
||||||
@@ -115,7 +115,7 @@ that long prompts read.
|
|||||||
`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
|
`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
|
||||||
stays pinned. Disk-backing the weights is not a lever on this format.
|
stays pinned. Disk-backing the weights is not a lever on this format.
|
||||||
|
|
||||||
## Pool doubled for slot parity — committed 2026-10-18, awaiting redeploy
|
## Pool doubled for slot parity — committed AND deployed 2026-10-18
|
||||||
|
|
||||||
`HALOGEN_KV_POOL_POSITIONS` **200,704 → 401,408** (commit `2b12716`), Frank's
|
`HALOGEN_KV_POOL_POSITIONS` **200,704 → 401,408** (commit `2b12716`), Frank's
|
||||||
call: frank-dragoon's total in-flight context was **half** frank-zealot's —
|
call: frank-dragoon's total in-flight context was **half** frank-zealot's —
|
||||||
@@ -133,24 +133,34 @@ conversation had to stay under ~100k (commit message;
|
|||||||
(`scripts/serve-frank-dragoon.sh:85-95`); 401,408 is `slots × ctx`, the top
|
(`scripts/serve-frank-dragoon.sh:85-95`); 401,408 is `slots × ctx`, the top
|
||||||
of the legal band, and exactly frank-zealot's pool number — parity, not
|
of the legal band, and exactly frank-zealot's pool number — parity, not
|
||||||
approximation.
|
approximation.
|
||||||
- Expected reservation **~10.8 GiB** at ~28 KiB/position (preflight formula,
|
- Measured reservation **11.0 GiB** — the boot ledger prints `reserving the KV
|
||||||
`scripts/frank-dragoon.env:70-72`); the measured points beside it are 5.5
|
pool, 401408 positions (about 11.0 GiB)`. The preflight KV-budget line bills
|
||||||
GiB at 200,704 (boot ledger 2026-10-18), 7.2 GiB reserved / 7.4 GiB billed
|
**11.3 GiB** (`~28 KiB/position incl. block scratch + ~115 MiB/slot`) plus
|
||||||
at 262,144, and 14.6 GiB at the 2026-10-06 shape's 524,288
|
1.3 GiB of prompt cache. The measured points beside it: 5.5 GiB at 200,704,
|
||||||
|
7.2 GiB at 262,144, and 14.6 GiB at the 2026-10-06 shape's 524,288
|
||||||
(`scripts/frank-dragoon.env:49-52`, `:74-76`).
|
(`scripts/frank-dragoon.env:49-52`, `:74-76`).
|
||||||
- That RAM comes out of the n-gram table's page cache — FLAGS.md law: RAM the
|
- That RAM comes out of the n-gram table's page cache — FLAGS.md law: RAM the
|
||||||
pool takes is RAM that table loses.
|
pool takes is RAM that table loses.
|
||||||
|
|
||||||
**Deploy state as of 2026-10-18: NOT yet deployed.** The live container
|
**Deployed 2026-10-18** (`./scripts/deploy.sh dankclaw --service
|
||||||
(booted 2026-10-18 01:25) still runs `HALOGEN_KV_POOL_POSITIONS=200704` —
|
frank-dragoon`, Playwright gate 33/33). The boot ledger confirms the shape:
|
||||||
verified in its `flash_serve` `/proc/<pid>/environ` and in the host-deployed
|
`reserving the KV pool, 401408 positions (about 11.0 GiB)`,
|
||||||
`frank-dragoon.env` — and the 2026-10-18 boot ledger reserves `the KV pool,
|
`2 slots over ONE 401408-position KV pool, each request up to 200704 (it
|
||||||
200704 positions (about 5.5 GiB)` with `74.1 GiB in all`
|
reserves prompt + max_tokens)`, and the sum
|
||||||
(`/home/dank/logs/frank_dragoon.log`). Until the next
|
`62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB
|
||||||
`./scripts/deploy.sh dankclaw --service frank-dragoon`, the boot ledger keeps
|
of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`. Prompt cache ON (12
|
||||||
printing `2 slots over ONE 200704-position KV pool`; the first boot at the
|
entries × 111 MiB at ctx 200704 = the 1.3 GiB in the KV-budget line), n-gram
|
||||||
new pool must show `ONE 401408-position KV pool` and the ~10.8 GiB
|
47.68 GiB still disk-backed (`Pin::None`), no clamp line. Host `MemAvailable`
|
||||||
reservation — read that line before quoting 401,408 as live anywhere.
|
fell 26 → 22 GiB across the raise, which is the FLAGS.md collision being paid
|
||||||
|
in page cache. **Warm reuse survived it — measured after the deploy.** One
|
||||||
|
**101,987**-token wikitext prompt through `/v1/chat/completions` took **81.5 s**
|
||||||
|
cold (1,251 t/s of prefill) and **0.3 s** on each of two repeats:
|
||||||
|
`HALOGEN_CACHE_INPLACE=1` keeps the KV rows in the pool, so a warm turn prefills
|
||||||
|
nothing and the doubling evicted nothing the cache needed. (The pre-raise
|
||||||
|
136.7× figure was a `ttft` reading; this engine sends **no `Server-Timing`**
|
||||||
|
header and has **no `/completion` path** — `/ping`, `/props`, `/tokenize` are
|
||||||
|
404 too — so wall-clock around the POST plus `usage.prompt_tokens` is the only
|
||||||
|
instrument available here.)
|
||||||
|
|
||||||
## Measured A/B — same battery, both boots (2026-10-17)
|
## Measured A/B — same battery, both boots (2026-10-17)
|
||||||
|
|
||||||
@@ -231,7 +241,12 @@ is mid-thought); vision via OpenAI `image_url` content parts only, ≥256 px
|
|||||||
probes; `response_format` unsupported; the chat template rejects
|
probes; `response_format` unsupported; the chat template rejects
|
||||||
`reasoning_effort` outside `xhigh|medium|low` (the `models.yml` fold stays
|
`reasoning_effort` outside `xhigh|medium|low` (the `models.yml` fold stays
|
||||||
mandatory); unknown sampling params are a 400; the prompt cache is on by
|
mandatory); unknown sampling params are a 400; the prompt cache is on by
|
||||||
default, so a warm 100k follow-up costs seconds. See
|
default, so a warm 100k follow-up costs seconds (measured after the pool raise:
|
||||||
|
**81.5 s cold vs 0.3 s warm** at 101,987 prompt tokens). The HTTP surface is
|
||||||
|
thin — `/health`, `/metrics`, `/v1/models` and POST `/v1/chat/completions`
|
||||||
|
answer, while `/ping`, `/props`, `/completion` and `/tokenize` are 404 and no
|
||||||
|
response carries `Server-Timing`: llama.cpp's timing instruments do not exist
|
||||||
|
on this engine, so measure wall-clock plus `usage.prompt_tokens`. See
|
||||||
[[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that
|
[[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that
|
||||||
carries verbatim, and [[deployment]] for the deploy/health gate.
|
carries verbatim, and [[deployment]] for the deploy/health gate.
|
||||||
|
|
||||||
@@ -307,9 +322,17 @@ restart; `webui.db.bak-20261017b` holds the pre-cutover row).
|
|||||||
**What the cut cost:** with pool == ctx one maximum-length request fills the
|
**What the cut cost:** with pool == ctx one maximum-length request fills the
|
||||||
pool, so two 200k conversations never coexist (two of ≤100 k do), and the
|
pool, so two 200k conversations never coexist (two of ≤100 k do), and the
|
||||||
harness window is now 155,648 — the 180 k compaction target of the 262,144 era
|
harness window is now 155,648 — the 180 k compaction target of the 262,144 era
|
||||||
is unreachable because the window itself is smaller. (The pool == ctx
|
is unreachable because the window itself is smaller. (That pool == ctx
|
||||||
concurrency limit is superseded — on redeploy only — by *Pool doubled for slot
|
concurrency limit IS superseded: the doubling deployed the next day — see
|
||||||
parity* above; the window math stands, the pool never moves the ceiling.)
|
*Pool doubled for slot parity* above. The window math stands, though: the pool
|
||||||
|
never moves the per-request ceiling.)
|
||||||
**What it bought:** 1.7
|
**What it bought:** 1.7
|
||||||
GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the
|
GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the
|
||||||
2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry.
|
2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry.
|
||||||
|
|
||||||
|
**Third production boot — the `401408` pool, deployed 2026-10-18.** Same ctx,
|
||||||
|
same working set, same 111 MiB prompt-cache entry; the pool line reads
|
||||||
|
`reserving the KV pool, 401408 positions (about 11.0 GiB)` and the total
|
||||||
|
`79.6 GiB in all`. Ledger and the measured cost in
|
||||||
|
*Pool doubled for slot parity* above; host `MemAvailable` after that boot:
|
||||||
|
22 GiB.
|
||||||
@@ -76,8 +76,10 @@ axis improved — decode, prefill, three-stream aggregate, and free VRAM.
|
|||||||
**13,056.00 MiB**, ledger-exact. That is exactly HALF the 27B's 34,816 B/token.
|
**13,056.00 MiB**, ledger-exact. That is exactly HALF the 27B's 34,816 B/token.
|
||||||
**f16 KV would bill 26,112 MiB against a 24,123 MiB card** and dies at
|
**f16 KV would bill 26,112 MiB against a 24,123 MiB card** and dies at
|
||||||
`cudaMalloc`. The 24 DeltaNet layers bill a fixed recurrent state per SLOT
|
`cudaMalloc`. The 24 DeltaNet layers bill a fixed recurrent state per SLOT
|
||||||
instead — `RS buffer size = 452.25 MiB (3 cells, 32 layers, 3 seqs)` ≈ 150 MiB
|
instead: the PRODUCTION ledger prints `RS buffer size = 150.75 MiB (3 cells,
|
||||||
each — so pool arithmetic alone under-prices the shape.
|
32 layers, 3 seqs)` ≈ 50 MiB a cell, while the SPEC-ARMED arms printed
|
||||||
|
452.25 MiB — MTP depth 2 bills three seqs per slot, so the drafter triples
|
||||||
|
even the recurrent state. Pool arithmetic alone under-prices the shape.
|
||||||
|
|
||||||
## The shape (`scripts/frank-zealot.env`)
|
## The shape (`scripts/frank-zealot.env`)
|
||||||
|
|
||||||
@@ -123,30 +125,36 @@ inputs plus 2–3 concurrent subagents — so the shipped arm takes the +17 %
|
|||||||
prefill and +12 % three-stream aggregate over single-stream speed, and keeps
|
prefill and +12 % three-stream aggregate over single-stream speed, and keeps
|
||||||
6.2 GiB free. Frank confirmed this arm on 2026-10-18.
|
6.2 GiB free. Frank confirmed this arm on 2026-10-18.
|
||||||
|
|
||||||
## Boot ledger (deployed arm, verbosity 4)
|
## Boot ledger (production boot 2026-10-18 13:19, under the unit, verbosity 4)
|
||||||
|
|
||||||
```
|
```
|
||||||
|
CUDA0 model buffer size = 2036.71 MiB # + 257.66 MiB CPU_Mapped (mmap)
|
||||||
CUDA0 KV buffer size = 13056.00 MiB # q8_0 K + q8_0 V, 786,432 positions
|
CUDA0 KV buffer size = 13056.00 MiB # q8_0 K + q8_0 V, 786,432 positions
|
||||||
CUDA0 RS buffer size = 452.25 MiB # 3 recurrent-state cells, 32 layers, 3 seqs
|
CUDA0 RS buffer size = 150.75 MiB # 3 cells, 32 layers, 3 seqs
|
||||||
compute 1,409 MiB CUDA + 1,201 MiB CUDA_Host
|
CUDA0 compute buffer = 1409.02 MiB # + 1088.87 MiB CUDA_Host, 2.84 output
|
||||||
mmproj 595 MiB + 248 MiB # charged AFTER the fit check
|
mmproj worst-case (est.) = 868.32 MiB # 248.10 CUDA + 24.93 CPU, AFTER fit
|
||||||
nvidia-smi settled: 17,876 MiB used / 6,249 MiB free of 24,123
|
nvidia-smi settled: 17,922 MiB used / 6,203 MiB free of 24,123
|
||||||
|
n_slots = 3, n_ctx_slot = 262144, kv_unified = false
|
||||||
```
|
```
|
||||||
|
|
||||||
No clamp line, no `cudaMalloc` error. The engine's own projection (21,439-class
|
No clamp line, no `cudaMalloc` error. The engine's own projection (21,439-class
|
||||||
numbers from the 27B era) excludes the projector — **read settled `nvidia-smi`,
|
numbers from the 27B era) excludes the projector — **read settled `nvidia-smi`,
|
||||||
never the projection**.
|
never the projection**.
|
||||||
|
|
||||||
## Measured (deployed arm)
|
## Measured (production boot, `/completion` `timings`, greedy, `ignore_eos`)
|
||||||
|
|
||||||
- **167.2 t/s** greedy decode (512 tokens, `/completion` `timings`); 158.9 t/s
|
- **168.7 t/s** decode — three runs of 512 tokens: 168.4 / 168.7 / 168.7
|
||||||
single-stream chat with the card's sampler.
|
(the trial arm printed 167.2; same shape, same noise band).
|
||||||
- **280.2 t/s aggregate** across three concurrent streams (≈93.8 each).
|
- **~300 t/s aggregate** across three concurrent 512-token streams
|
||||||
- **2,877 t/s prefill** at 106,418 prompt tokens.
|
(102.8 t/s per stream, round 1 and 2 both) — the trial arm's 280.2.
|
||||||
|
- **2,877 t/s prefill at 106,418 tokens** — the trial arm; the production boot
|
||||||
|
has not re-run that depth.
|
||||||
- The 27B record it replaces: 76.6 decode, 109.4 aggregate across two streams,
|
- The 27B record it replaces: 76.6 decode, 109.4 aggregate across two streams,
|
||||||
691.7 prefill at 166k. Beaten on every axis.
|
691.7 prefill at 166k. Beaten on every axis.
|
||||||
- The **first request after a boot is a warmup artifact** (37.7 t/s class) —
|
- The **first request after a boot is a warmup artifact** (37.7 t/s class) —
|
||||||
never report it.
|
never report it. And never time a wall-clock aggregate on a generation that
|
||||||
|
stops on its own: a 24-token countdown measured 72 t/s across three streams
|
||||||
|
while the same shape with `ignore_eos` gave 300.
|
||||||
|
|
||||||
## Contract traps
|
## Contract traps
|
||||||
|
|
||||||
@@ -186,6 +194,21 @@ never the projection**.
|
|||||||
7771 — a live port with no unit behind it. `TimeoutStopSec=180` is now on the
|
7771 — a live port with no unit behind it. `TimeoutStopSec=180` is now on the
|
||||||
unit (llama.cpp needs the drain time to release slots and the card); the
|
unit (llama.cpp needs the drain time to release slots and the card); the
|
||||||
hand-booted arm must be killed by PID, never `pkill -f llama-server`.
|
hand-booted arm must be killed by PID, never `pkill -f llama-server`.
|
||||||
|
- **A leftover arm drop-in hijacks the production boot, and no health check
|
||||||
|
sees it.** The sanctioned bench method is a transient
|
||||||
|
`Environment=FRANK_ZEALOT_ENV_FILE=/tmp/<arm>.env` drop-in, and an arm env
|
||||||
|
`source`s the repo env and then overrides it. On 2026-10-18 a stale
|
||||||
|
`frank-zealot.service.d/ptq1.conf` — written 03:29 for the 27B **PTQ1_0** arm
|
||||||
|
Frank had ordered at 02:36 and measured from 03:14 (deploy ledger below) —
|
||||||
|
was still installed, so the cutover deploy restarted
|
||||||
|
the unit and booted THAT: `--model …/Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf
|
||||||
|
--ctx-size 401408 --parallel 3 --batch-size 3072 --ubatch-size 1536`, three
|
||||||
|
slots of 133,888, 21,188 MiB used. `/health` was 200, `/v1/models` said
|
||||||
|
`frank-zealot` (the alias is a flag, not a fact), and the Playwright gate
|
||||||
|
passed 33/33 — none of those read the payload. **After deploying an endpoint
|
||||||
|
that was recently benched: `systemctl show -p DropInPaths`, then
|
||||||
|
`tr '\0' ' ' </proc/$pid/cmdline`, then the ledger — in that order.** Retire
|
||||||
|
the drop-in to `~/.AGENTTRASH/<date>-<topic>/`, `daemon-reload`, restart.
|
||||||
|
|
||||||
## Compaction hub
|
## Compaction hub
|
||||||
|
|
||||||
@@ -197,6 +220,47 @@ the window ≈ 222,822 est, worst case ≈ 245,891 ≤ 262,144). With `--paralle
|
|||||||
a compaction request no longer serializes behind a live generation (the Gemma4
|
a compaction request no longer serializes behind a live generation (the Gemma4
|
||||||
era's `--parallel 1` law).
|
era's `--parallel 1` law).
|
||||||
|
|
||||||
|
## Deploy ledger (2026-10-18)
|
||||||
|
|
||||||
|
- `./scripts/deploy.sh dankclaw --service frank-zealot` at 13:16 PDT; the e2e
|
||||||
|
gate ran 33/33. The first boot was the hijacked arm above; the drop-in went
|
||||||
|
to `~/.AGENTTRASH/20261018-zealot-ptq1-arm/` and the 13:19 restart is the
|
||||||
|
ledger recorded here.
|
||||||
|
- Live probes on the shipped boot: chat `27*59` → **1593**; `where_am_i` tool
|
||||||
|
call fired with schema-correct empty arguments; 512 px solid-red PNG →
|
||||||
|
**Red** at 278 prompt tokens with `process_mtmd` chunk encoding in the
|
||||||
|
journal; `/props` reports `n_ctx 262144` and `presence_penalty 1.0`;
|
||||||
|
`/metrics` answers 200 with 15 `llamacpp:*` families and
|
||||||
|
`up{job="frank-zealot"}` is **1** in Prometheus for the first time in its
|
||||||
|
retained history.
|
||||||
|
- OpenWebUI picker rows re-keyed in the live DB (`Connection.backup()` snapshot
|
||||||
|
`webui.db.bak-20261018` taken first; `frank-dragoon`'s `context.length` also
|
||||||
|
corrected 262,144 → 200,704 — see [[OpenWebUI]]).
|
||||||
|
- **What the bench window left behind is not garbage — the PTQ1_0 arm was
|
||||||
|
ordered, measured, and then scratched by Frank himself.** The instruction at
|
||||||
|
**02:36** was: run frank-zealot on `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, the
|
||||||
|
**Q8** mmproj, batch/ubatch **3072/1536**, and as much context as fits in
|
||||||
|
**two** contexts. The unit booted it at **03:14** (journal:
|
||||||
|
`type ptq1_0: 402 tensors`, `file type = PTQ1_0 - 1.75 bpw ternary`, mmproj
|
||||||
|
loaded, 2 × 200,704) and served 24 minutes of traffic. Its own
|
||||||
|
`print_timing` lines measure **~48 t/s** single-stream decode
|
||||||
|
(48.0 / 47.9 / 47.6 / 49.0 t/s on 512 / 229 / 512 / 451-token generations),
|
||||||
|
25–27 t/s per stream with two streams live, and prefill **1,295 t/s at
|
||||||
|
30,637 → 943 at 97,856 → 801 t/s at 141,365** tokens. It bought the
|
||||||
|
1,201 MiB the fetch script promised and paid for it on decode, exactly as the
|
||||||
|
publisher's Ampere row warned (the PQ2_0 record for the same weights is
|
||||||
|
76.6 t/s). At **03:33** Frank scratched the whole 27B plan and ordered the
|
||||||
|
9B PQ2_0 + MTP shape deployed here; the arm process ended at 03:38.
|
||||||
|
- The actual failure was **documentation, not measurement.** An approved
|
||||||
|
payload that survives only as a systemd drop-in plus a journal window is,
|
||||||
|
a day later, indistinguishable from a rogue arm — which is why the 13:16 boot
|
||||||
|
read as a hijack. Payload decisions belong in this runbook the moment they
|
||||||
|
are made. The evidence is still perishable: `/tmp/zealot-arm-current.env` and
|
||||||
|
the 9B `/tmp/arm-*.jsonl` results die on reboot, and only the journal
|
||||||
|
(`journalctl -u frank-zealot --since "2026-10-18 03:10"`) holds the PTQ1_0
|
||||||
|
numbers. `/home/dank/models/Bonsai2-PTQ1_0/` (5,946,648,928 B + hardlinked
|
||||||
|
mmproj) stays on disk as the 27B rollback.
|
||||||
|
|
||||||
## Fallback rungs
|
## Fallback rungs
|
||||||
|
|
||||||
1. `UBATCH_SIZE=1024` — frees **1,306 MiB** measured on this shape (prefill
|
1. `UBATCH_SIZE=1024` — frees **1,306 MiB** measured on this shape (prefill
|
||||||
|
|||||||
@@ -22,14 +22,15 @@ and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
|||||||
digest serving the NATIVE v2 conversion of the abliterated
|
digest serving the NATIVE v2 conversion of the abliterated
|
||||||
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
|
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
|
||||||
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
|
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
|
||||||
200,704-position KV pool at ctx 200704 — the 2 × ctx = 401,408 raise is
|
401,408-position KV pool — 2 × ctx, raised and DEPLOYED 2026-10-18 — at ctx
|
||||||
committed 2026-10-18 but NOT yet deployed, vision ON)** and
|
200704, vision ON)** and
|
||||||
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
||||||
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
|
Ternary-Bonsai-2-9B at **PQ2_0** — 9B qwen35 hybrid (8 full-attention + 24
|
||||||
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
|
Gated DeltaNet layers + the blk.32 MTP block), THREE 262,144-token
|
||||||
|
conversations, MTP present in the file but OFF by measurement,
|
||||||
q8_0 K + q8_0 V (f16 KV does not fit the RTX 3090 at this window), vision via
|
q8_0 K + q8_0 V (f16 KV does not fit the RTX 3090 at this window), vision via
|
||||||
the publisher's own Q8_0 Qwen3-VL mmproj, on the RTX 3090 — and the custom
|
the publisher's own Q8_0 mmproj, on the RTX 3090 — and the custom
|
||||||
tensor type 142 + `prism.hadamard.*` metadata mean no other loader can open the
|
tensor type 141 + `prism.hadamard.*` metadata mean no other loader can open the
|
||||||
file)**.
|
file)**.
|
||||||
|
|
||||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||||
@@ -58,11 +59,11 @@ gfx1151 iGPU 114 GiB GTT]
|
|||||||
E[flash-server container gfx1151, digest-pinned image
|
E[flash-server container gfx1151, digest-pinned image
|
||||||
frank-dragoon, port 7770]
|
frank-dragoon, port 7770]
|
||||||
F[flash-server v2 native abliterated .hgn + ngram table + vision tower
|
F[flash-server v2 native abliterated .hgn + ngram table + vision tower
|
||||||
ONE 200704-position KV pool at ctx 200704 (401408 committed, awaiting deploy), 2 slots, MTP always-on, image input]
|
ONE 401408-position KV pool at ctx 200704 = 2 slots x ctx, MTP always-on, image input]
|
||||||
G[llama.cpp PrismML fork CUDA build
|
G[llama.cpp PrismML fork CUDA build
|
||||||
frank-zealot, port 7771]
|
frank-zealot, port 7771]
|
||||||
H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj
|
H[Ternary-Bonsai-2-9B PQ2_0 GGUF + Q8_0 mmproj
|
||||||
2 x 200704 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision]
|
3 x 262144 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision]
|
||||||
E2 --> E --> F
|
E2 --> E --> F
|
||||||
E2 --> G --> H
|
E2 --> G --> H
|
||||||
end
|
end
|
||||||
@@ -94,13 +95,10 @@ frank-zealot, port 7771]
|
|||||||
`HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=401408` (**2 × ctx**):
|
`HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=401408` (**2 × ctx**):
|
||||||
**one pool behind two slots**, sized so TWO maximum-length requests coexist —
|
**one pool behind two slots**, sized so TWO maximum-length requests coexist —
|
||||||
raised from 200,704 (5.5 GiB) on 2026-10-18 for frank-zealot parity on total
|
raised from 200,704 (5.5 GiB) on 2026-10-18 for frank-zealot parity on total
|
||||||
in-flight context, ~10.8 GiB reserved against the card's 262,144/262,144 =
|
in-flight context and **deployed the same evening**: the boot ledger reserves
|
||||||
7.2 GiB. **Deploy state 2026-10-18: the running container and the
|
`401408 positions (about 11.0 GiB)` with `79.6 GiB in all`
|
||||||
host-deployed env still carry the 200,704-position pool** (boot ledger
|
(`scripts/frank-dragoon.env:70-75`). It bought CONCURRENCY, not a longer
|
||||||
`reserving the KV pool, 200704 positions (about 5.5 GiB)`; host
|
window: the per-request ceiling stays 200,704, because 262,144 is the
|
||||||
`frank-dragoon.env` `HALOGEN_KV_POOL_POSITIONS=200704`) — the doubling lands
|
|
||||||
on the next redeploy (commit `2b12716`).
|
|
||||||
longer window: the per-request ceiling stays 200,704, because 262,144 is the
|
|
||||||
native wall and the engine refuses more without `HALOGEN_ROPE_YARN`.
|
native wall and the engine refuses more without `HALOGEN_ROPE_YARN`.
|
||||||
MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
|
MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
|
||||||
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
|
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
type: research
|
type: research
|
||||||
title: Local TTS and Voice Cloning
|
title: Local TTS and Voice Cloning
|
||||||
description: How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the full local AI audio stack, and a recommended homelab deployment.
|
description: How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the measured CPU engine A/B behind frank-tts and the arm A decision, the full local AI audio stack, and a recommended homelab deployment.
|
||||||
tags:
|
tags:
|
||||||
- research
|
- research
|
||||||
- tts
|
- tts
|
||||||
@@ -10,7 +10,7 @@ tags:
|
|||||||
- stt
|
- stt
|
||||||
- vllm-omni
|
- vllm-omni
|
||||||
- kokoro
|
- kokoro
|
||||||
timestamp: 2026-10-18T00:00:00Z
|
timestamp: 2026-10-18T16:05:00Z
|
||||||
---
|
---
|
||||||
|
|
||||||
# Local TTS and Voice Cloning
|
# Local TTS and Voice Cloning
|
||||||
@@ -47,13 +47,16 @@ greenfield: every fleet endpoint declares `input: [text, image]`
|
|||||||
local server needs a speech-role entry, not a patch.
|
local server needs a speech-role entry, not a patch.
|
||||||
- **Fleet plan (2026-10-18): CPU-only on Dankclaw, not the 3090.** Gitea issue
|
- **Fleet plan (2026-10-18): CPU-only on Dankclaw, not the 3090.** Gitea issue
|
||||||
#2 rules out every GPU option (the 3090 has carried frank-zealot since
|
#2 rules out every GPU option (the 3090 has carried frank-zealot since
|
||||||
2026-10-17; both R9700s are frank-archon's), so the shape is a VoxCPM2 GGUF
|
2026-10-17; both R9700s are frank-archon's). The shape is a TTS engine on
|
||||||
engine on **8880** behind a voice-registry shim on **8890**
|
**8880** behind **frank-tts**, the fleet's speech endpoint on **7772**
|
||||||
(`scripts/voiceshim.env`, `dankclaw/voiceservice/`) that speaks OpenAI
|
(`scripts/frank-tts.env`, `dankclaw/voiceservice/`): one OpenAI
|
||||||
`/v1/audio/speech` and resolves voices by name. Nothing is deployed yet — on
|
`/v1/audio/speech` façade that resolves a voice *by name* and never shows a
|
||||||
2026-10-18 `voiceshim` has no unit installed and nothing answers on 8880 or
|
path. Nothing is deployed — no `frank-tts` unit and no container exist on the
|
||||||
8890. STT (Parakeet TDT 1.1B, CC-BY-4.0) stays on CPU. **No RTF or quality
|
host and nothing listens on 7772 or 8880 (checked 2026-10-18). STT (Parakeet
|
||||||
number has been measured for this shape**, so this page states none.
|
TDT 1.1B, CC-BY-4.0) stays on CPU. **The engine is chosen: arm A**
|
||||||
|
`qwentts.cpp` **resident `tts-server`, 1.7 B talker** — measured in
|
||||||
|
*Measured CPU engine A/B* below and picked on the live-output criterion in
|
||||||
|
[Decision: arm A for live output](#decision-arm-a-for-live-output-2026-10-18).
|
||||||
|
|
||||||
## Can Kokoro learn a new voice?
|
## Can Kokoro learn a new voice?
|
||||||
|
|
||||||
@@ -133,11 +136,11 @@ accepts only its built-in voice list, so a custom pack needs a patch.
|
|||||||
|
|
||||||
Paths for a Frank-cloned voice (2026-10-18):
|
Paths for a Frank-cloned voice (2026-10-18):
|
||||||
|
|
||||||
1. **Local server + speech role, no patch.** Register the local engine (or the
|
1. **Local server + speech role, no patch.** Register frank-tts (port 7772) as
|
||||||
8890 voice shim) as a provider model on the `openai-speech` API and point
|
a provider model on the `openai-speech` API and point `modelRoles.speech` at
|
||||||
`modelRoles.speech` at it; the shim turns a voice *name* into a reference
|
it; the façade turns a voice *name* into that pack's reference clip and
|
||||||
clip and forwards it as base64 `reference_audio`
|
forwards it as base64 `reference_audio` (`scripts/frank-tts.env`,
|
||||||
(`scripts/voiceshim.env:19-35`). Not deployed as of 2026-10-18.
|
`TTS_BACKEND=omni-http`). Not deployed as of 2026-10-18.
|
||||||
2. **xAI console custom voice — works today, zero code.** Record 2–3 min clean
|
2. **xAI console custom voice — works today, zero code.** Record 2–3 min clean
|
||||||
speech (≤120 s, 24 kHz 16-bit mono WAV) → console → `voice_id` → speech
|
speech (≤120 s, 24 kHz 16-bit mono WAV) → console → `voice_id` → speech
|
||||||
tool. Free (≤30 voices), cloud, US-only.
|
tool. Free (≤30 voices), cloud, US-only.
|
||||||
@@ -206,29 +209,161 @@ The CPU shape does not need a Python server at all:
|
|||||||
`VoxCPM2-Acoustic-F16.gguf`, Apache-2.0 card) or from the fork's own
|
`VoxCPM2-Acoustic-F16.gguf`, Apache-2.0 card) or from the fork's own
|
||||||
`tools/omni/voxcpm2/convert_voxcpm2_to_gguf.py`.
|
`tools/omni/voxcpm2/convert_voxcpm2_to_gguf.py`.
|
||||||
|
|
||||||
Nothing has been measured on this fleet for any of these engines — no RTF, no
|
Arm B (`llama-tts`) and arm A (`qwentts.cpp`) have both been measured on this
|
||||||
latency, no MOS. That absence is the finding, not an omission.
|
box — RTF, first-audio latency and RSS are in the next section. The other
|
||||||
|
servers in the table above (vLLM-Omni, Kokoro-FastAPI, CosyVoice, F5-TTS,
|
||||||
|
GPT-SoVITS) remain unmeasured here; that absence is still the finding.
|
||||||
|
|
||||||
|
## Measured CPU engine A/B (2026-10-18)
|
||||||
|
|
||||||
|
Both candidate upstreams for frank-tts were measured on Dankclaw's CPU — 8
|
||||||
|
threads, no GPU touched, three runs per arm, seed 42, identical text, against
|
||||||
|
the fleet's own audition voice packs.
|
||||||
|
|
||||||
|
- **Arm A — [`ServeurpersoCom/qwentts.cpp`](https://github.com/ServeurpersoCom/qwentts.cpp)**
|
||||||
|
at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` (2026-10-16), weights
|
||||||
|
from [`Serveurperso/Qwen3-TTS-GGUF`](https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF):
|
||||||
|
`qwen-talker-1.7b-base-Q4_K_M.gguf` (1,219,245,248 B),
|
||||||
|
`qwen-talker-0.6b-base-Q4_K_M.gguf` (628,905,056 B),
|
||||||
|
`qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B) — byte counts verified
|
||||||
|
against the Hub API. Builds with zero compiler warnings; `GGML_BLAS=OFF` is
|
||||||
|
forced on this host (only `libopenblas.so.0` exists, no dev headers, so the
|
||||||
|
vendor `buildcpu.sh` cannot configure).
|
||||||
|
- **Arm B — mainline `llama-tts`** with `--tts-speaker-file`, i.e. the
|
||||||
|
`llama-tts-cli` backend `scripts/frank-tts.env` already implements.
|
||||||
|
|
||||||
|
RTF is **seconds of compute per second of audio** (lower is better; below 1 is
|
||||||
|
faster than real time).
|
||||||
|
|
||||||
|
| Arm | RTF | Run spread | Peak RSS |
|
||||||
|
|---|---|---|---|
|
||||||
|
| A · 0.6 B talker, CLI | 0.44 | ±0.002 | 3.4 GiB |
|
||||||
|
| A · 1.7 B, CLI, uncloned | 0.57–0.58 | ±0.006 | 4.7 GiB |
|
||||||
|
| A · 1.7 B, CLI, embedding-only clone | 0.63 | ±0.02 | n/m |
|
||||||
|
| A · 1.7 B, CLI, ICL clone | 0.92 | ±0.01 | n/m |
|
||||||
|
| A · 1.7 B, resident `tts-server`, warm | 0.55–0.57 | ±0.01 | 5.7 GiB |
|
||||||
|
| A · same, ICL voice warm / **cold (first request)** | 0.67 / **1.65** | one-shot | 5.7 GiB |
|
||||||
|
| B · mainline `llama-tts` | 0.93–0.99 | ±0.25 | n/m |
|
||||||
|
|
||||||
|
**Read RTF only within an arm.** Conditioning changes clip length — the same
|
||||||
|
seed yields 18.00 s of audio embedding-only and 15.12 s under ICL — so the
|
||||||
|
denominator is the arm's own audio duration.
|
||||||
|
|
||||||
|
Arm A's resident server is ~1.7× faster than arm B and about 40× tighter in
|
||||||
|
run-to-run spread, and it already serves the HTTP shape frank-tts's façade
|
||||||
|
speaks: `/health`, `/v1/models`, `/v1/audio/voices` (GET/POST/DELETE),
|
||||||
|
`/v1/audio/speech` (`wav`/`pcm`), chunked streaming with the first audio byte
|
||||||
|
at 86–451 ms and a server-side time-to-first-audio of 94–112 ms. Arm B has no
|
||||||
|
HTTP path at all — that is what the façade's CLI backend exists for.
|
||||||
|
|
||||||
|
Four laws came out of the trial, each worth a line of configuration:
|
||||||
|
|
||||||
|
- **Boot-warm every voice.** The first request for a registered voice decodes
|
||||||
|
the whole reference to snapshot codec state: 6.2–8.0 s, first audio byte at
|
||||||
|
5,530 ms. Later requests restore that state in 0.5 ms. An engine unit must
|
||||||
|
run one throwaway synthesis per voice at start, or the first request of the
|
||||||
|
day pays five seconds.
|
||||||
|
- **Speaker artifacts are reusable and exact.** `qwen-codec` turns a reference
|
||||||
|
WAV into `.spk` (8,192 B x-vector) + `.rvq` (5,456–6,864 B, 16 codebooks × T,
|
||||||
|
11-bit packed) in 1.8–2.1 s at 1.3–1.5 GiB peak; replaying them yields a
|
||||||
|
**byte-identical** WAV and saves 1.4 s per request. That pair is the
|
||||||
|
engine-artifact term the voice-pack law in `scripts/frank-tts.env` reserves a
|
||||||
|
slot for — arm A pins it as `.spk` + `.rvq` beside `reference.wav`.
|
||||||
|
- **Streaming is not bit-identical to buffered**, whatever the upstream README
|
||||||
|
claims: same text, seed and voice, 92.2 % of samples differ (peak delta
|
||||||
|
1139/32768, −41.6 dB RMS — inaudible). A fixed seed does not pin output
|
||||||
|
across `response_format`, so never mix the two paths in a cache or a golden
|
||||||
|
fixture.
|
||||||
|
- **A voice cannot be registered from an embedding alone**: `POST
|
||||||
|
/v1/audio/voices` accepts `wav_b64` or **both** `spk_b64` and `rvq_b64`
|
||||||
|
(`src/tts-server.h:480`), so an embedding-only voice has to be sent as the
|
||||||
|
WAV, or the lone-`spk` path has to be upstreamed.
|
||||||
|
|
||||||
|
**Likeness is not measurable at this resolution.** x-vector cosine against each
|
||||||
|
speaker's own reference: ICL 0.9886 / 0.9901, embedding-only 0.9844 / 0.9812,
|
||||||
|
uncloned 0.9228 — but **two different speakers score 0.9497** on the same
|
||||||
|
metric. The whole clone effect lives in a 0.04-wide band and the
|
||||||
|
ICL-over-embedding gap is inside noise, so the four audition clips
|
||||||
|
(`/home/blank/voice-audition/qwentts-*.mp3`, each against its `armB-*.mp3`
|
||||||
|
mainline partner, guide in `LISTEN-engine.md`) are the instrument. **The
|
||||||
|
operator heard them on 2026-10-18 and passed every clone**, which removed the
|
||||||
|
last axis measurement could not reach; the engine was then picked on
|
||||||
|
live-output grounds ([Decision](#decision-arm-a-for-live-output-2026-10-18)).
|
||||||
|
The cosine
|
||||||
|
extractor is still worth keeping: it is engine-neutral (it scores mainline
|
||||||
|
output through the same encoder) and it separates an uncloned voice cleanly.
|
||||||
|
|
||||||
|
Two operational details. The engine takes 8 threads and exposes **no `-t`
|
||||||
|
flag** — `src/backend.h` uses `hardware_concurrency()/2` and prints the choice
|
||||||
|
at load, so pinning a cpuset on the quadlet changes that number implicitly. And
|
||||||
|
ICL needs a reference transcript that did not exist: the fleet has no ASR
|
||||||
|
endpoint and no audio input, so the crops were transcribed with
|
||||||
|
`vosk-small-en-us` inside the work dir, and two of three transcripts came back
|
||||||
|
suspect ("demon souls" for *Dark Souls*) — a reason to curate `ref_text` into
|
||||||
|
the pack manifest instead of transcribing on the host.
|
||||||
|
|
||||||
|
## Decision: arm A for live output (2026-10-18)
|
||||||
|
|
||||||
|
Every audition pair passed, so likeness stopped being the tiebreak. The
|
||||||
|
question became **which engine is better for live output** — the harness
|
||||||
|
speaking a reply, an alert read aloud, a click-to-speak — and on that criterion
|
||||||
|
RTF only has to stay under 1.0 with margin: what separates the arms is **time to
|
||||||
|
first audio** and the **spread**, because an arm whose RTF can cross 1.0 stalls
|
||||||
|
a live stream mid-sentence.
|
||||||
|
|
||||||
|
| | arm A · `qwentts.cpp` `tts-server` | arm B · mainline `llama-tts` |
|
||||||
|
|---|---|---|
|
||||||
|
| first audio byte | 86–451 ms | model load, per request |
|
||||||
|
| warm RTF | 0.55–0.67 | 0.93–0.99 |
|
||||||
|
| run spread | ±0.01 | ±0.25 |
|
||||||
|
| HTTP | serves `/v1/audio/speech` | none — façade CLI only |
|
||||||
|
| RAM | 5.7 GiB resident | per-request, freed after |
|
||||||
|
|
||||||
|
**Chosen: arm A, 1.7 B talker, resident `tts-server` in a `frank-tts-engine`
|
||||||
|
container, with `TTS_BACKEND=llama-tts-cli` kept in the repo as the rollback.**
|
||||||
|
Three consequences are configuration, not opinion: the unit **boot-warms every
|
||||||
|
voice** (5.5 s cold first byte otherwise), it **does not pin a cpuset** casually
|
||||||
|
(threads are `hardware_concurrency()/2`, no flag), and every pack registers with
|
||||||
|
the **WAV** because the engine rejects a lone embedding.
|
||||||
|
|
||||||
|
Two follow-ups stay open on purpose, and neither blocks the deploy:
|
||||||
|
|
||||||
|
- **Talker size.** The 0.6 B arm measured 0.44 RTF at 3.4 GiB and has never
|
||||||
|
been auditioned — every clip above is 1.7 B. It is the lever for a CPU
|
||||||
|
co-tenancy problem, not a preemptive saving.
|
||||||
|
- **ICL vs embedding-only.** Both passed the ear test. Embedding-only is the
|
||||||
|
shipped default: 0.57–0.63 vs 0.67 warm, and it needs no `ref_text`, which
|
||||||
|
this fleet cannot produce honestly (the ASR crops came back suspect). Packs
|
||||||
|
still carry `.spk` + `.rvq`, so ICL remains a per-voice opt-in for batch work
|
||||||
|
like the podcast render, where total time matters more than first byte.
|
||||||
|
|
||||||
## Recommended deployment (Dankclaw)
|
## Recommended deployment (Dankclaw)
|
||||||
|
|
||||||
As decided in Gitea issue #2, this stays **CPU-only on Dankclaw**: the RTX 3090
|
As decided in Gitea issue #2, this stays **CPU-only on Dankclaw**: the RTX 3090
|
||||||
has carried frank-zealot since 2026-10-17 and both R9700s are frank-archon's,
|
has carried frank-zealot since 2026-10-17 and both R9700s are frank-archon's,
|
||||||
so no card may be asked to co-host TTS. The shape is one engine plus a thin
|
so no card may be asked to co-host TTS. The shape is one engine plus one
|
||||||
voice shim, and **nothing here is deployed as of 2026-10-18** — `systemctl
|
façade, and **nothing here is deployed as of 2026-10-18**: `frank-tts` has
|
||||||
--user is-enabled voiceshim` reports `not-found` and nothing answers on 8880 or
|
neither a unit nor a container on the host, and nothing listens on 7772 or
|
||||||
8890.
|
8880.
|
||||||
|
|
||||||
| Service | Image / install | Port | CPU | Notes |
|
| Service | Image / install | Port | CPU | Notes |
|
||||||
|---|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
| TTS engine | `llama-tts-server` (`tc-mb/llama.cpp-omni`) on a VoxCPM2 GGUF; `vllm/vllm-omni` + Qwen3-TTS only if a GPU is ever deliberately freed | 8880 | engine decision still open | OpenAI `/v1/audio/speech`; `n_gpu_layers` is there for a GPU boot |
|
| TTS engine | **arm A `qwentts.cpp` `tts-server`** on the Qwen3-TTS GGUFs (decided 2026-10-18, above); arm B mainline `llama-tts` stays in the repo as the `llama-tts-cli` rollback; `vllm/vllm-omni` + Qwen3-TTS only if a GPU is ever deliberately freed | 8880 (container-internal) | open, evidence above | arm A serves OpenAI `/v1/audio/speech` itself and boot-warms each voice; arm B has no HTTP path and runs through the façade's CLI backend |
|
||||||
| Voice shim | `dankclaw/voiceservice/` (stdlib container) + `scripts/voiceshim.env` + `dankclaw/voiceservice/quadlet/voiceshim.container` | 8890 | yes | voice *name* → `reference.wav`, `voice`/`voice_id` on the wire, mp3 re-encode; backends `omni-http` (forwards base64 `reference_audio`) and `llama-tts-cli` (`--tts-speaker-file`, `nice 19`) |
|
| frank-tts | `dankclaw/voiceservice/` (stdlib container) + `scripts/frank-tts.env` + its quadlet | 7772 | yes | voice *name* → `/voices/<name>/reference.wav`, `voice`/`voice_id` on the wire, mp3 re-encode; backends `omni-http` (forwards base64 `reference_audio`) and `llama-tts-cli` (`--tts-speaker-file`, `nice 19`) |
|
||||||
| STT (English) | Parakeet TDT 1.1B behind a FastAPI wrapper, or whisper.cpp server | 8882 | yes | CC-BY-4.0; LibriSpeech-clean WER 1.39 (model card) |
|
| STT (English) | Parakeet TDT 1.1B behind a FastAPI wrapper, or whisper.cpp server | 8882 | yes | CC-BY-4.0; LibriSpeech-clean WER 1.39 (model card) |
|
||||||
| STT (multilingual, opt.) | faster-whisper `large-v3` int8 | 8883 | yes | MIT |
|
| STT (multilingual, opt.) | faster-whisper `large-v3` int8 | 8883 | yes | MIT |
|
||||||
| VAD | Silero (embedded) | — | yes | MIT; model file 2 327 524 B |
|
| VAD | Silero (embedded) | — | yes | MIT; model file 2 327 524 B |
|
||||||
|
|
||||||
8880/8890 are what `scripts/voiceshim.env:35,49` carry (`TTS_UPSTREAM=
|
7772 is frank-tts's host port — the next slot in the fleet's model-endpoint
|
||||||
http://host.containers.internal:8880`, `SHIM_PORT=8890`); 8882-8883 for STT
|
family (7770 frank-dragoon, 7771 frank-zealot). The engine behind it keeps 8880,
|
||||||
keep clear of the fleet's LLM ports 7770/7771.
|
but **not as a host-loopback publish**: netavark DNATs published ports only, and
|
||||||
|
a bridge container cannot reach a `127.0.0.1` bind on the host — that is the
|
||||||
|
lesson that 502'd hermes (`AGENTS.md`). `host.containers.internal`
|
||||||
|
(169.254.1.2) is the gateway, so it reaches a host service only when that
|
||||||
|
service binds a routable address; `scripts/frank-tts.env` names it as the
|
||||||
|
engine's address, which holds for a `0.0.0.0` bind and not for a loopback one.
|
||||||
|
The deployed pair therefore shares one quadlet network and the façade addresses
|
||||||
|
the engine by container name.
|
||||||
|
|
||||||
## Voice-creation workflow (the "from sample files" answer)
|
## Voice-creation workflow (the "from sample files" answer)
|
||||||
|
|
||||||
@@ -284,6 +419,10 @@ keep clear of the fleet's LLM ports 7770/7771.
|
|||||||
https://huggingface.co/openbmb/VoxCPM2
|
https://huggingface.co/openbmb/VoxCPM2
|
||||||
- llama.cpp-omni TTS server: https://github.com/tc-mb/llama.cpp-omni ·
|
- llama.cpp-omni TTS server: https://github.com/tc-mb/llama.cpp-omni ·
|
||||||
https://raw.githubusercontent.com/tc-mb/llama.cpp-omni/master/tools/omni/voxcpm2/README.md
|
https://raw.githubusercontent.com/tc-mb/llama.cpp-omni/master/tools/omni/voxcpm2/README.md
|
||||||
|
- qwentts.cpp (arm A): https://github.com/ServeurpersoCom/qwentts.cpp at commit
|
||||||
|
`5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` · weights
|
||||||
|
https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF (byte counts against the
|
||||||
|
Hub API, 2026-10-18)
|
||||||
- mainline `llama-tts`:
|
- mainline `llama-tts`:
|
||||||
https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/tts/README.md ·
|
https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/tts/README.md ·
|
||||||
Pocket-TTS weights: https://huggingface.co/kyutai/pocket-tts (CC-BY-4.0)
|
Pocket-TTS weights: https://huggingface.co/kyutai/pocket-tts (CC-BY-4.0)
|
||||||
|
|||||||
@@ -79,6 +79,7 @@ macOS mini) as quadlets inside its podman machine — not on Dankserver.
|
|||||||
- [[Postgres]] — Gitea database backend
|
- [[Postgres]] — Gitea database backend
|
||||||
- [[Seerr]] — unified *arr request manager (guard-chained; fronts FrankBook-Pro's stack)
|
- [[Seerr]] — unified *arr request manager (guard-chained; fronts FrankBook-Pro's stack)
|
||||||
- [[Marreta]] — content-cleaning proxy (guard-chained; shares SearXNG's Gluetun egress)
|
- [[Marreta]] — content-cleaning proxy (guard-chained; shares SearXNG's Gluetun egress)
|
||||||
|
- [[Open Notebook]] — podcast generator (scripts by frank-zealot, voices by frank-tts); repo-owned config, not deployed
|
||||||
|
|
||||||
## Infrastructure
|
## Infrastructure
|
||||||
|
|
||||||
@@ -314,8 +315,9 @@ The LLM inference fleet on the two hosts. Full index:
|
|||||||
since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2
|
since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2
|
||||||
abliterated** conversion of orcarouter's pinned BF16
|
abliterated** conversion of orcarouter's pinned BF16
|
||||||
(`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) at ctx **200,704** over a
|
(`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) at ctx **200,704** over a
|
||||||
**401,408**-position pool (`HALOGEN_KV_POOL_POSITIONS=401408`, ~10.8 GiB — one
|
**401,408**-position pool (`HALOGEN_KV_POOL_POSITIONS=401408`, 11.0 GiB
|
||||||
pool behind both slots, raised to 2 × ctx on 2026-10-18 so TWO maximum-length
|
reserved — one pool behind both slots, raised to 2 × ctx and deployed
|
||||||
|
2026-10-18 so TWO maximum-length
|
||||||
requests coexist) and the stock peonist n-gram/vision companions **hardlinked**
|
requests coexist) and the stock peonist n-gram/vision companions **hardlinked**
|
||||||
in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP
|
in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP
|
||||||
always-on, vision ON, no server-side sampler change
|
always-on, vision ON, no server-side sampler change
|
||||||
|
|||||||
@@ -2156,4 +2156,288 @@ every restart since the cutover.
|
|||||||
picker row (DB `description` + context length 262,144) re-keyed. Rollback:
|
picker row (DB `description` + context length 262,144) re-keyed. Rollback:
|
||||||
revert the swap commit — the 27B payload and the build-10754 fork tree are
|
revert the swap commit — the 27B payload and the build-10754 fork tree are
|
||||||
both still on disk.
|
both still on disk.
|
||||||
|
- Deployed 13:16 PDT (`deploy.sh dankclaw --service frank-zealot`, e2e 33/33)
|
||||||
|
— and the first boot was **not the shipped payload**: a leftover
|
||||||
|
`frank-zealot.service.d/ptq1.conf` from the bench window (written 03:29 for
|
||||||
|
the 27B **PTQ1_0** arm Frank ordered at 02:36 and measured from 03:14 — see
|
||||||
|
the record below) pointed
|
||||||
|
`FRANK_ZEALOT_ENV_FILE` at `/tmp/zealot-arm-current.env`, so the unit booted
|
||||||
|
that arm: 3 slots of 133,888, 21,188 MiB used, batch 3072/ubatch 1536.
|
||||||
|
`/health` 200, `/v1/models` echoed the alias, Playwright 33/33 — nothing in
|
||||||
|
the gate reads the payload. Drop-in moved to
|
||||||
|
`~/.AGENTTRASH/20261018-zealot-ptq1-arm/`, `daemon-reload`, restart at 13:19:
|
||||||
|
`n_slots = 3, n_ctx_slot = 262144`, KV 13,056.00 MiB, settled 17,922/6,203.
|
||||||
|
- Production measures (engine `timings`, greedy, `ignore_eos`): **168.7 t/s**
|
||||||
|
decode (168.4/168.7/168.7) and **~300 t/s** aggregate across three 512-token
|
||||||
|
streams (102.8 each). My first battery reported 117 and 72 t/s: it timed
|
||||||
|
natural generations that stopped at 24 tokens — wall-clock aggregate on an
|
||||||
|
early EOS reads 4× low. `ignore_eos` + the engine's own numbers is the only
|
||||||
|
honest way to time this.
|
||||||
|
- Ledger correction: the shipped shape prints **RS 150.75 MiB** (3 cells), not
|
||||||
|
the 452.25 MiB my first-pass notes carried — that figure belongs to the
|
||||||
|
SPEC-ARMED arms, where MTP depth 2 bills three seqs per slot (9 cells). The
|
||||||
|
drafter triples the recurrent state as well as billing 3,072 MiB of f16 KV.
|
||||||
|
Corrected in AGENTS.md (topology row + architecture section), the env header
|
||||||
|
and the runbook, with the production boot ledger replacing the trial's.
|
||||||
|
- Verified live: chat `27*59` → 1593; `where_am_i` tool call fired; 512 px
|
||||||
|
solid-red PNG → **Red** (278 prompt tokens, `process_mtmd` in the journal);
|
||||||
|
`/props` `n_ctx 262144`, `presence_penalty 1.0`; `/metrics` 200 with 15
|
||||||
|
`llamacpp:*` families and `up{job="frank-zealot"} = 1` — the first non-zero
|
||||||
|
sample in that job's retained history.
|
||||||
|
- OpenWebUI `model` rows re-keyed in the live DB (backup
|
||||||
|
`webui.db.bak-20261018`): `frank-zealot` → 262,144 + the 9B description, and
|
||||||
|
`frank-dragoon`'s `context.length` corrected 262,144 → **200,704** (it had
|
||||||
|
kept the pre-lowering window from 2026-10-17 while its own description said
|
||||||
|
200,704). Note the columns are `params`/`meta` JSON — the table has no
|
||||||
|
`context_length`/`description` columns.
|
||||||
|
|
||||||
|
## 2026-10-18 — frank-zealot PTQ1_0 arm: measured, approved, and never written down
|
||||||
|
|
||||||
|
- **Correction to the entry above and to my own report.** The
|
||||||
|
`frank-zealot.service.d/ptq1.conf` drop-in was NOT a half-finished experiment.
|
||||||
|
The session archive (`01a14c93`) carries Frank's own instruction at **02:36**:
|
||||||
|
replace frank-zealot with `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, use the **Q8**
|
||||||
|
mmproj, batch/ubatch **3072/1536**, "fit as much context as possible in 2
|
||||||
|
contexts".
|
||||||
|
- It was the production payload **03:14:39 → 03:38:57** (pid 191358; journal
|
||||||
|
`type ptq1_0: 402 tensors`, `file type = PTQ1_0 — 1.75 bpw ternary`, mmproj
|
||||||
|
loaded, 2 × 200,704) and served real traffic. Recovered from that process's
|
||||||
|
own `print_timing` lines (17 timed tasks): single-stream greedy decode
|
||||||
|
**48.0 and 47.6 t/s at 512 tokens, 47.9 at 229, 49.0 at 451**; 25–27 t/s per
|
||||||
|
stream with two streams live; prefill **1,295 t/s @ 30,637 → 943 @ 97,856 →
|
||||||
|
801 t/s @ 141,365** (one 176 t/s row at 97,024 is a concurrent-slot request,
|
||||||
|
not a clean probe).
|
||||||
|
- Against the PQ2_0 27B record (76.6 t/s decode, 691.7 t/s prefill at 166k):
|
||||||
|
PTQ1_0 freed the **1,201 MiB** its fetch script promised and paid **−37 %
|
||||||
|
decode** — the publisher's "dense ternary unpacking loses on Ampere" warning,
|
||||||
|
now confirmed on GA102 — while prefilling deep prompts faster than that
|
||||||
|
record ever managed. Frank scratched the whole 27B plan at **03:33** ("scratch
|
||||||
|
the current frank-zealot plan completely… 9B… enable mtp… if you can possibly
|
||||||
|
fit a third context, do it"), which is what shipped at 13:16.
|
||||||
|
- The durable failure is mine: those numbers lived only in `/tmp` and the
|
||||||
|
journal, never in the repo, so the next agent on the host could not tell an
|
||||||
|
approved payload from a rogue drop-in and called it a hijack. The runbook
|
||||||
|
deploy ledger now carries the measurement; `/home/dank/models/Bonsai2-PTQ1_0/`
|
||||||
|
stays on disk as the 27B rollback and the drop-in stays trashed.
|
||||||
|
|
||||||
|
|
||||||
|
## 2026-10-18 — frank-dragoon: the 401,408-position pool goes live; docs re-keyed to the measured boot
|
||||||
|
|
||||||
|
- `./scripts/deploy.sh dankclaw --service frank-dragoon` — e2e 33/33. The
|
||||||
|
drop-in check the zealot incident bought ran FIRST: `DropInPaths` empty, so
|
||||||
|
the repo env governed. Boot ledger: `reserving the KV pool, 401408 positions
|
||||||
|
(about 11.0 GiB)`, `2 slots over ONE 401408-position KV pool, each request up
|
||||||
|
to 200704`, and `62.9 GiB of weights locked … 11.0 GiB of KV pool, 5.7 GiB of
|
||||||
|
working memory, 79.6 GiB in all`; prompt cache ON (12 entries × 111 MiB at
|
||||||
|
ctx 200704 = 1.3 GiB), n-gram 47.68 GiB still `Pin::None`, no clamp line.
|
||||||
|
- Warm reuse survived the raise, measured right after the deploy: one
|
||||||
|
**101,987**-token wikitext prompt through `/v1/chat/completions` took
|
||||||
|
**81.5 s** cold (1,251 t/s of prefill) and **0.3 s** on each of two repeats —
|
||||||
|
`HALOGEN_CACHE_INPLACE=1` keeps the KV rows in the pool, so a warm turn
|
||||||
|
prefills nothing and the doubling evicted nothing the cache needed. The RAM
|
||||||
|
price is still real: host `MemAvailable` fell **26 → 22 GiB**, because every
|
||||||
|
pool position is RAM the n-gram table's page cache loses.
|
||||||
|
- Every document quoted **~10.8 GiB**, the ~28 KiB/position arithmetic; the
|
||||||
|
engine reserves **11.0 GiB** and its preflight bills **11.3**. Re-keyed in
|
||||||
|
AGENTS.md (topology row, KV-pool law, memory ledger), the `frank-dragoon.env`
|
||||||
|
lore block, both index pages, the iommu page and the runbook, and every
|
||||||
|
"committed but not yet deployed" clause was retired with the measurement.
|
||||||
|
- The same sweep caught the LLM index still describing frank-zealot as
|
||||||
|
**Bonsai-2-27B at 2 × 200,704** in its intro paragraph and topology diagram —
|
||||||
|
the 9B swap of the same day had not reached those two spots. Re-keyed: 8
|
||||||
|
full-attention + 24 Gated DeltaNet + the `blk.32` MTP block, 3 × 262,144,
|
||||||
|
tensor type **141**.
|
||||||
|
- PTQ1_0 provenance corrected (see the dedicated section above): the arm that
|
||||||
|
hijacked the zealot deploy was a **repo-tracked, Frank-ordered, measured**
|
||||||
|
candidate — `scripts/download-frank-zealot-bonsai2-ptq1.sh` is committed, and
|
||||||
|
its header now carries the recovered numbers (≈48 t/s decode, 801 t/s prefill
|
||||||
|
at 141k) and the verdict: −37 % decode against PQ2_0 for 1,201 MiB freed, then
|
||||||
|
scratched by Frank's 03:33 order in favour of the 9B. Nothing is owed on it.
|
||||||
|
- OpenWebUI: both decoration rows re-keyed in the live DB. The values live
|
||||||
|
inside JSON columns — `params.context.length` and `meta.description` — not in
|
||||||
|
fields named for them, and there is no `sqlite3` binary on the host, so
|
||||||
|
`python3` stdlib + `Connection.backup()` (`webui.db.bak-20261018`). Zealot
|
||||||
|
200,704 → **262,144** with the 9B description; dragoon 262,144 → **200,704**,
|
||||||
|
which had kept the pre-lowering window while its own description said
|
||||||
|
200,704.
|
||||||
|
- Census of the halogen HTTP surface while doing it, because the fleet bench
|
||||||
|
law quoted the wrong engine: frank-dragoon answers **`/health`, `/metrics`,
|
||||||
|
`/v1/models`, `/v1/chat/completions`(POST)** and 404s `/ping`, `/props`,
|
||||||
|
`/completion`, `/tokenize`; the chat response carries **no `Server-Timing`**
|
||||||
|
header (`server: uvicorn`, only `date`/`content-*`). So the `/completion`
|
||||||
|
`timings` object and the `ttft;dur=` header are **llama.cpp** instruments —
|
||||||
|
frank-zealot's, not dragoon's. AGENTS.md's Benchmarking Policy re-keyed to
|
||||||
|
say which engine owns which instrument.
|
||||||
|
|
||||||
|
## 2026-10-18 — frank-tts phase 1: the cloned-voice endpoint, named and tested (not deployed)
|
||||||
|
|
||||||
|
- Voice-over goal, phase 1 (Gitea issue #2). New service **frank-tts** on
|
||||||
|
Dankclaw: `dankclaw/voiceservice/` (stdlib-only `app.py` + `Containerfile` +
|
||||||
|
rootless quadlet), `scripts/frank-tts.env`, `tests/frank_tts_unit.bats`
|
||||||
|
(12 tests). One OpenAI-shaped `POST /v1/audio/speech` surface; `voice=<name>`
|
||||||
|
is honoured only when `/voices/<name>/reference.wav` exists, otherwise a 400
|
||||||
|
listing the voices that are — no silent fallback voice. `voice_id` is the
|
||||||
|
harness alias; `response_format=mp3` really is mp3; oversized input is a 414,
|
||||||
|
not a half-narrated sentence; `/health` reports backend + voice list; no audio
|
||||||
|
ever reaches the log. Two backends behind it (`TTS_BACKEND=omni-http` forwards
|
||||||
|
to the host's `:8880` engine with the voice as base64 `reference_audio`;
|
||||||
|
`llama-tts-cli` runs a binary per request at `nice -n 19`), both exercised
|
||||||
|
through stubs — no GPU, no containers in the seam.
|
||||||
|
- **Naming law applied (Frank's call):** treat it as a first-class model
|
||||||
|
endpoint → alias-named `frank-tts` (TTS endpoint, so not `frank-asr`),
|
||||||
|
**port 7772**, joining `7770 frank-dragoon` / `7771 frank-zealot`; the engine
|
||||||
|
upstream keeps 8880 and is deliberately not a fleet endpoint. The internal
|
||||||
|
knob names stay `SHIM_*` (they name the façade component, not the service).
|
||||||
|
- **Voice-pack law recorded** in the env header: a pack needs `reference.wav`
|
||||||
|
(24 kHz mono, 18–25 s) **and** the pinned engine's own speaker artifact —
|
||||||
|
required, beside the WAV, never replacing it — plus `manifest.json` provenance
|
||||||
|
whose offsets are always past the first 10 s of the source (the
|
||||||
|
source-material law). Enforcement is split honestly: the shim gates on
|
||||||
|
`reference.wav` (the only engine-agnostic term); the pack builder is what will
|
||||||
|
refuse to emit a pack without its artifact.
|
||||||
|
- Suite green: **384/384** (`./tests/run.sh`, wall 66 s). Landed as **PR #4**
|
||||||
|
(branch `wt/20261017-181502` → `main`, merged as `49c4e19`) — the first PR on
|
||||||
|
this repo through the API, which cost three 422s to get right: the merge
|
||||||
|
endpoint takes **`{"Do":"merge"}`**, `Do` being a *string* naming the strategy.
|
||||||
|
`DoMerge`/`merge_method` do not exist on this Gitea; swagger JSON is disabled
|
||||||
|
on the instance, so the 422 messages are the schema. Law saved to the
|
||||||
|
`gitea-api-pr-operations` skill.
|
||||||
|
- Deliberately NOT in this phase: the `:8880` engine decision and its
|
||||||
|
speaker-artifact format, the pack builder, deployer/homepage/Prometheus
|
||||||
|
registration, the `models.yml` row that lets omp call it, and open-notebook.
|
||||||
|
Harness fact established while writing the PR body: the **`tts` tool** will
|
||||||
|
dispatch an `api: openai-speech` provider row to `POST <baseUrl>/audio/speech`
|
||||||
|
(`{model, input, response_format, voice}`), but the live `speech.*` vocalizer is
|
||||||
|
Kokoro-enum-only — so "the agent talks in my voice" needs that provider row or
|
||||||
|
a `pi.registerProvider` extension, decided in phase 2.
|
||||||
|
|
||||||
|
## 2026-10-18 — TTS engine picked: arm A `qwentts.cpp`, on the live-output criterion
|
||||||
|
|
||||||
|
- Frank listened to the four audition pairs (`qwentts-*.mp3` against their
|
||||||
|
`armB-*.mp3` mainline partners) and **passed every clone**. That deleted the
|
||||||
|
only axis the A/B could not measure, and the question was reframed: **which
|
||||||
|
engine is better for live output**. On that criterion RTF only has to stay
|
||||||
|
under 1.0 with margin — the discriminators are time-to-first-audio and
|
||||||
|
run-to-run spread, because an arm whose RTF can cross 1.0 stalls a live
|
||||||
|
stream mid-sentence.
|
||||||
|
- **Chosen: arm A `qwentts.cpp` @ `5916bf7`, the resident `tts-server`, 1.7 B
|
||||||
|
talker Q4_K_M + Q8_0 tokenizer, in a `frank-tts-engine` container on 8880.**
|
||||||
|
The `TTS_BACKEND=llama-tts-cli` path in `scripts/frank-tts.env` stays in the
|
||||||
|
repo as the rollback. The numbers that carried it: first audio byte 86–451 ms
|
||||||
|
against a per-request model load, warm RTF 0.55–0.67 against 0.93–0.99, spread
|
||||||
|
±0.01 against ±0.25, and a native `/v1/audio/speech` against no HTTP path at
|
||||||
|
all. Three consequences are configuration, not preference: boot-warm every
|
||||||
|
voice, do not pin a cpuset (threads are `hardware_concurrency()/2`), register
|
||||||
|
every pack with its WAV (a lone embedding is rejected).
|
||||||
|
- Two follow-ups deliberately **not** decided: the **0.6 B talker** (0.44 RTF,
|
||||||
|
3.4 GiB, never auditioned — every clip is 1.7 B) and **ICL vs
|
||||||
|
embedding-only** conditioning. Embedding-only is the shipped default because
|
||||||
|
it is cheaper and needs no `ref_text`, which this fleet cannot transcribe
|
||||||
|
honestly; `.spk` + `.rvq` still ride in every pack, so ICL remains a per-voice
|
||||||
|
opt-in for batch work where total time beats first byte.
|
||||||
|
- **Addressing law corrected** in
|
||||||
|
[[Local TTS and Voice Cloning]]: the pair may not meet at a host-loopback
|
||||||
|
publish. netavark DNATs published ports only and a bridge container cannot
|
||||||
|
reach a host `127.0.0.1` bind — the lesson that 502'd hermes — so
|
||||||
|
`host.containers.internal:8880`, which `scripts/frank-tts.env` documents, is
|
||||||
|
correct only for a routable bind. The deployed pair shares one quadlet network
|
||||||
|
and the façade addresses the engine by container name.
|
||||||
|
- **PR #12** (`feat/open-notebook-podcast` @ `114dd7a`, 8 commits, 403 tests) is
|
||||||
|
**merged** as `1eb8b21` (Frank's OK, given in session): open-notebook +
|
||||||
|
SurrealDB quadlets on the private bridge `10.89.4.0/24`, the API published on
|
||||||
|
**5056** because seerr holds `127.0.0.1:5055`, and `ESPERANTO_TTS_TIMEOUT`
|
||||||
|
moved 600 → **660** to sit one round-trip outside frank-tts's own
|
||||||
|
`SHIM_TIMEOUT`. Merged is not runnable: nothing listens on 7772 until the
|
||||||
|
voice stack lands, and the episode destination (`/home/dank/media/podcasts`)
|
||||||
|
stays interim pending Gitea #7.
|
||||||
|
- Still in flight, not in this branch: `feat/frank-tts-engine` — the arm A image
|
||||||
|
build, the quadlet pair on one network, the boot-warm step, deployer /
|
||||||
|
planner / `test-changed.sh` routing, homepage and docs. This branch is also
|
||||||
|
where the two-branch overlap got resolved: merging `main` here conflicted on
|
||||||
|
`log.md` alone (both branches appended an entry) and took `index.md`,
|
||||||
|
`deploy-remote.sh`, `deploy-plan.py` and `test-changed.sh` clean.
|
||||||
|
|
||||||
|
## 2026-10-18 — open-notebook: the podcast generator's config lands (nothing deployed)
|
||||||
|
|
||||||
|
- Goal: **open-notebook** as the fleet's podcast generator on Dankclaw — notes
|
||||||
|
and sources in, a multi-speaker episode out, script written by
|
||||||
|
**frank-zealot**, voices rendered by **frank-tts**, no cloud provider
|
||||||
|
anywhere in the path. Repo-side only: no host touched, no container pulled or
|
||||||
|
started. Runbook: [[Open Notebook]].
|
||||||
|
- **Two containers, not one.** `lfnovo/open_notebook:v1-latest` plus
|
||||||
|
`surrealdb/surrealdb:v2`, both **digest-pinned** with `Pull=never` and a
|
||||||
|
documented one-time `podman pull` — frank-dragoon's pinning style, because
|
||||||
|
`:latest` moving underneath a rootless service is how you get an unbootable
|
||||||
|
pod after an unrelated `systemctl-daemon-reload`. The registry round-trip is
|
||||||
|
what proved `v2` is SurrealDB **2.7.0** and that the app image is Debian
|
||||||
|
trixie with **curl and ffmpeg already inside** — which is how both health
|
||||||
|
probes can be argument-only: `curl -fsS …/health` (route is upstream's,
|
||||||
|
`api/main.py:239`, and it sits on the auth exclude list) and
|
||||||
|
`/surreal is-ready`, which I ran against the digest here: green in 2 s, and
|
||||||
|
it refuses to pass without `SURREAL_PASS` in the environment — a credential
|
||||||
|
that must never ride the command line, so `Exec=` carries flags only.
|
||||||
|
- **Host 5055 was already spoken for.** seerr has held `127.0.0.1:5055` since
|
||||||
|
2026-10-14 (live `ss -ltn` agrees today), so the API publishes
|
||||||
|
`0.0.0.0:5056:5055`. open-notebook's frontend derives its API URL from the
|
||||||
|
window origin when `API_URL` is unset — the upstream env reference spells the
|
||||||
|
reverse-proxy symptom — so a silent remap would hang the UI on a port nobody
|
||||||
|
listens on, and `API_URL=http://192.168.0.143:5056` is set accordingly. 8502
|
||||||
|
publishes on `0.0.0.0` because this service carries its own password (the
|
||||||
|
episode-audio route is unauthenticated), unlike the loopback-and-proxied
|
||||||
|
observability stack; **8000 is published nowhere**, where upstream exposes it
|
||||||
|
on loopback with root credentials, and the seam forbids it.
|
||||||
|
- **The CPU guard, pre-armed.** `frank-tts` spawns one ~8 GiB, 8-thread engine
|
||||||
|
per request on a box that also serves two LLMs, while open-notebook defaults
|
||||||
|
to `TTS_BATCH_SIZE=5` × `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` — a five-way
|
||||||
|
self-DoS. Both are pinned to `1` in `scripts/open-notebook.env`, and the
|
||||||
|
worker knob is only real because the image's own supervisord expands
|
||||||
|
`${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5}` (`supervisord.conf:25`). The TTS clock
|
||||||
|
goes to **660 — one round-trip outside** frank-tts's own `SHIM_TIMEOUT=600`, so
|
||||||
|
a shim give-up reads as the shim's error rather than the worker's; the seam
|
||||||
|
asserts the direction (`>`) against the frank-tts env instead of restating a
|
||||||
|
number. `ESPERANTO_LLM_TIMEOUT` stays at the app's 180 s default: frank-zealot
|
||||||
|
answers an outline or a transcript well inside it, and upstream says to keep it
|
||||||
|
under 600 so a slow call fails with a name on it.
|
||||||
|
- **What a credential needs in order to exist:** `OPEN_NOTEBOOK_ENCRYPTION_KEY`
|
||||||
|
must be set before the UI can save *any* provider credential, and
|
||||||
|
`OPEN_NOTEBOOK_PASSWORD` must stay **absent** to remain auth-off — the app
|
||||||
|
reads `os.environ.get(…) is not None`, so `PASSWORD=` boots a login form
|
||||||
|
asking for the empty string. The repo env file carries both keys empty; the
|
||||||
|
deployer seeds `/home/dank/Containers/open-notebook/env` (600) from the host
|
||||||
|
vault and **refuses to start the pair while either secret is blank**, because
|
||||||
|
a half-configured open-notebook stores no credentials and fails quietly.
|
||||||
|
- **Episode destination is interim, by decision.** The destination is **Gitea
|
||||||
|
issue #7, `human-only`** — agents must not work it. Until Frank settles it,
|
||||||
|
the app keeps its own named volume and `/home/dank/media/podcasts` is
|
||||||
|
bind-mounted over `/app/data/podcasts` (`DATA_FOLDER` is hard-coded `./data`),
|
||||||
|
so finished MP3s are ordinary files in his home directory instead of volume
|
||||||
|
contents needing `podman unshare` — the database volume demonstrably lands on
|
||||||
|
a mapped subuid, measured with the pinned image. One mount line; one deletion
|
||||||
|
undoes it.
|
||||||
|
- **Two UI buttons will fail, and that is cosmetic.** Test Connection does
|
||||||
|
`GET <base>/models` and the TTS probe synthesises voice `alloy`; frank-tts
|
||||||
|
serves `/health` and `/v1/audio/speech` only. Model rows are created *by
|
||||||
|
name* and the voice field is free text, so `narrator1/2/3` is typed in and
|
||||||
|
never discovered. Documented, not worked around.
|
||||||
|
- **Blocked on frank-tts, stated plainly:** phase 1 landed without deployer or
|
||||||
|
homepage registration and nothing listens on 7772 today, so a generation run
|
||||||
|
writes its script and then fails on the first voice. That is upstream's
|
||||||
|
creation-podcasts warning in fleet clothing.
|
||||||
|
- Suite green: **403/403** (`./tests/run.sh`, wall 67 s) — 19 tests in the new
|
||||||
|
`tests/open_notebook.bats`, structure only: pins, boot persistence, the
|
||||||
|
unpublishable database port, the 5055→5056 remap plus the `API_URL` that has
|
||||||
|
to match it, the env-file read order, the guard knobs, the health gate naming
|
||||||
|
`/health`, planner routing, the homepage card hidden while undeployed.
|
||||||
|
- Three deployer seams cost time and are now law-shaped: a service must be
|
||||||
|
registered in **five** places — `SERVICE_ORDER`, `DANKCLAW_SERVICES`, the
|
||||||
|
`deploy-remote.sh` dispatch case, its pre-deploy requirement, and
|
||||||
|
`deploy.sh`'s dispatch regex, where an omission silently plans an empty set
|
||||||
|
for `--only`; a `dankclaw/<svc>/*` case must precede the `dankclaw/*.container`
|
||||||
|
glob because that `*` crosses `/`; and the changed-path routing lives *inside*
|
||||||
|
the per-path loop, after the `scripts/*.env)` case, not at the top of the file.
|
||||||
|
- Deliberately NOT done: the deploy (pulling 1.27 GiB to Dankclaw was not
|
||||||
|
asked for), any Caddy vhost or public exposure, audiobook/EPUB (issue #5), any
|
||||||
|
`frank-tts` code change, and the harbor templates on Dankserver —
|
||||||
|
`~/.harbor/compose.opennotebook.yml` pins the deprecated `-single` image,
|
||||||
|
carries no encryption key and injects the whole harbor env — left untouched,
|
||||||
|
read-only, as the record of what not to copy.
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#101a12" stroke="#7ee787" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#7ee787" text-anchor="middle">ON</text></svg>
|
||||||
|
After Width: | Height: | Size: 293 B |
@@ -28,6 +28,19 @@
|
|||||||
"art_bg": "#f5f5f5",
|
"art_bg": "#f5f5f5",
|
||||||
"warn": "Chats stay on Frank's server, but OpenWebUI's web search rides the same SearXNG path as the search app: obfuscated, not anonymous. The upstream search API keys are tied to Frank’s identity."
|
"warn": "Chats stay on Frank's server, but OpenWebUI's web search rides the same SearXNG path as the search app: obfuscated, not anonymous. The upstream search API keys are tied to Frank’s identity."
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
"name": "open-notebook",
|
||||||
|
"category": "user",
|
||||||
|
"section": "AI",
|
||||||
|
"description": "Podcast generator — notes in, spoken episode out: script written by frank-zealot, voices rendered by frank-tts. No public route; LAN UI only once it is deployed.",
|
||||||
|
"url": "",
|
||||||
|
"docs": "",
|
||||||
|
"probe": "127.0.0.1:8502",
|
||||||
|
"method": "http",
|
||||||
|
"wip": "down",
|
||||||
|
"showLink": false,
|
||||||
|
"art": "/img/apps/mono-on.svg"
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"name": "Gitea",
|
"name": "Gitea",
|
||||||
"category": "user",
|
"category": "user",
|
||||||
|
|||||||
Executable
+343
@@ -0,0 +1,343 @@
|
|||||||
|
#!/usr/bin/env bats
|
||||||
|
# Seam: frank-tts, the cloned-voice speech façade on Dankclaw :7772.
|
||||||
|
#
|
||||||
|
# The contract this pins is the one every consumer (omp, OpenWebUI,
|
||||||
|
# open-notebook) codes against: POST /v1/audio/speech with voice=<name> is
|
||||||
|
# answered with audio in THAT voice pack, where "that voice pack" is exactly
|
||||||
|
# $VOICES_DIR/<name>/reference.wav — the file that makes a name real. A name
|
||||||
|
# without a reference must come back as a 400 naming the voices that do exist
|
||||||
|
# (a silent fallback voice is how a wrong-voice endpoint stays wrong for
|
||||||
|
# weeks), the harness' voice_id must work as an alias for voice, oversized text
|
||||||
|
# must be refused rather than half-narrated, and the mp3 response_format must
|
||||||
|
# really be mp3. Whatever engine sits behind it is a separate decision — so
|
||||||
|
# both backends are driven here through stubs, never against a GPU.
|
||||||
|
#
|
||||||
|
# Everything runs against the real app.py over 127.0.0.1 ephemeral ports with
|
||||||
|
# a temp VOICES_DIR: no Dankclaw, no network, no containers, no pkill — the
|
||||||
|
# processes this file starts are recorded by pid and killed in teardown.
|
||||||
|
|
||||||
|
load helpers
|
||||||
|
|
||||||
|
setup() {
|
||||||
|
APP="$REPO_ROOT/dankclaw/voiceservice/app.py"
|
||||||
|
UNIT="$REPO_ROOT/dankclaw/voiceservice/quadlet/frank-tts.container"
|
||||||
|
ENV_FILE="$REPO_ROOT/scripts/frank-tts.env"
|
||||||
|
TEST_TMP="$(mktemp -d)"
|
||||||
|
PIDS="$TEST_TMP/pids"
|
||||||
|
: > "$PIDS"
|
||||||
|
|
||||||
|
# One real voice pack and one directory that LOOKS like a pack but has no
|
||||||
|
# reference.wav — it must never be offered as a voice.
|
||||||
|
VOICES="$TEST_TMP/voices"
|
||||||
|
mkdir -p "$VOICES/narrator1" "$VOICES/nopack"
|
||||||
|
python3 -c 'import sys; sys.stdout.buffer.write(b"RIFF" + bytes(range(256)) * 4)' \
|
||||||
|
> "$VOICES/narrator1/reference.wav"
|
||||||
|
printf 'a manifest is not a voice\n' > "$VOICES/nopack/manifest.json"
|
||||||
|
|
||||||
|
# The audio the stub upstream "synthesises": distinct bytes, so a response
|
||||||
|
# can be proven to be the engine's output and not something the shim made
|
||||||
|
# up on its way past.
|
||||||
|
python3 -c 'import sys; sys.stdout.buffer.write(b"RIFF" + bytes(reversed(range(128))) * 3)' \
|
||||||
|
> "$TEST_TMP/upstream.wav"
|
||||||
|
|
||||||
|
write_stub
|
||||||
|
}
|
||||||
|
|
||||||
|
teardown() {
|
||||||
|
# Kill only what this file started, by recorded pid.
|
||||||
|
while IFS= read -r pid; do
|
||||||
|
[ -n "$pid" ] && kill "$pid" 2>/dev/null || true
|
||||||
|
done < "$PIDS"
|
||||||
|
wait 2>/dev/null || true
|
||||||
|
rm -rf "$TEST_TMP"
|
||||||
|
}
|
||||||
|
|
||||||
|
# write_stub — the upstream TTS server arm A forwards to: records the JSON body
|
||||||
|
# it was handed (so "did the reference voice actually travel?" is observable)
|
||||||
|
# and answers with $STUB_WAV. Prints its bound port, chosen by the kernel.
|
||||||
|
write_stub() {
|
||||||
|
cat > "$TEST_TMP/stub_upstream.py" <<'PY'
|
||||||
|
import json, os
|
||||||
|
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||||
|
|
||||||
|
class Stub(BaseHTTPRequestHandler):
|
||||||
|
def do_POST(self):
|
||||||
|
length = int(self.headers.get('content-length') or 0)
|
||||||
|
with open(os.environ['STUB_CAPTURE'], 'w') as fh:
|
||||||
|
json.dump(json.loads(self.rfile.read(length) or b'{}'), fh)
|
||||||
|
with open(os.environ['STUB_WAV'], 'rb') as fh:
|
||||||
|
audio = fh.read()
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header('content-type', 'audio/wav')
|
||||||
|
self.send_header('content-length', str(len(audio)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(audio)
|
||||||
|
|
||||||
|
def log_message(self, *args):
|
||||||
|
pass
|
||||||
|
|
||||||
|
server = HTTPServer(('127.0.0.1', 0), Stub)
|
||||||
|
print(server.server_address[1], flush=True)
|
||||||
|
server.serve_forever()
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
start_stub() {
|
||||||
|
: > "$TEST_TMP/stub.log"
|
||||||
|
STUB_WAV="$TEST_TMP/upstream.wav" STUB_CAPTURE="$TEST_TMP/capture.json" \
|
||||||
|
python3 "$TEST_TMP/stub_upstream.py" > "$TEST_TMP/stub.log" 2>&1 &
|
||||||
|
echo "$!" >> "$PIDS"
|
||||||
|
local waited=0
|
||||||
|
while [ "$waited" -lt 100 ]; do
|
||||||
|
STUB_PORT="$(head -n1 "$TEST_TMP/stub.log")"
|
||||||
|
[ -n "$STUB_PORT" ] && return 0
|
||||||
|
sleep 0.05
|
||||||
|
waited=$((waited + 1))
|
||||||
|
done
|
||||||
|
echo "stub upstream never bound a port:"
|
||||||
|
cat "$TEST_TMP/stub.log"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# start_app — boot app.py on 127.0.0.1 with an ephemeral port and read that
|
||||||
|
# port back off its own startup line: no port guessing, no race. The caller
|
||||||
|
# exports the env the run needs first; app.py reads its knobs at request time.
|
||||||
|
start_app() {
|
||||||
|
: > "$TEST_TMP/app.log"
|
||||||
|
python3 "$APP" --host 127.0.0.1 --port 0 > "$TEST_TMP/app.log" 2>&1 &
|
||||||
|
echo "$!" >> "$PIDS"
|
||||||
|
local waited=0
|
||||||
|
while [ "$waited" -lt 200 ]; do
|
||||||
|
APP_PORT="$(sed -nE 's|.*listening on http://127\.0\.0\.1:([0-9]+)|\1|p' "$TEST_TMP/app.log" | head -n1)"
|
||||||
|
[ -n "$APP_PORT" ] && return 0
|
||||||
|
sleep 0.05
|
||||||
|
waited=$((waited + 1))
|
||||||
|
done
|
||||||
|
echo "frank-tts never came up:"
|
||||||
|
cat "$TEST_TMP/app.log"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# arm_omni — the shim forwarding to an OpenAI-shaped upstream (arm A).
|
||||||
|
arm_omni() {
|
||||||
|
start_stub
|
||||||
|
export VOICES_DIR="$VOICES"
|
||||||
|
export TTS_BACKEND=omni-http
|
||||||
|
export TTS_UPSTREAM="http://127.0.0.1:$STUB_PORT"
|
||||||
|
}
|
||||||
|
|
||||||
|
speech() {
|
||||||
|
# speech <json-payload-inline> — the HTTP code lands in $output, the body
|
||||||
|
# in $TEST_TMP/out.bin and the response headers in head.txt.
|
||||||
|
local data="$1"
|
||||||
|
run curl -sS -o "$TEST_TMP/out.bin" -D "$TEST_TMP/head.txt" -w '%{http_code}' \
|
||||||
|
-H 'Content-Type: application/json' --data-binary "$data" \
|
||||||
|
"http://127.0.0.1:$APP_PORT/v1/audio/speech"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "app.py parses as python the runtime can import" {
|
||||||
|
run env PYTHONPYCACHEPREFIX="$TEST_TMP/pycache" python3 -m py_compile "$APP"
|
||||||
|
[ "$status" -eq 0 ] || { echo "$output"; false; }
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "voice=narrator1 returns the engine's audio with the pack's reference attached" {
|
||||||
|
arm_omni
|
||||||
|
start_app
|
||||||
|
speech '{"model":"tts-1","input":"hello world","voice":"narrator1"}'
|
||||||
|
[ "$status" -eq 0 ] || { cat "$TEST_TMP/app.log"; false; }
|
||||||
|
[ "$output" = "200" ] || { cat "$TEST_TMP/app.log"; false; }
|
||||||
|
grep -qi '^content-type: audio/wav' "$TEST_TMP/head.txt"
|
||||||
|
cmp "$TEST_TMP/out.bin" "$TEST_TMP/upstream.wav"
|
||||||
|
# The voice only arrives at the engine as base64 reference_audio — prove it
|
||||||
|
# is the pack's bytes, not a path and not the engine's own default voice.
|
||||||
|
python3 - "$TEST_TMP/capture.json" "$VOICES/narrator1/reference.wav" <<'PY'
|
||||||
|
import base64, json, sys
|
||||||
|
body = json.load(open(sys.argv[1]))
|
||||||
|
reference = open(sys.argv[2], 'rb').read()
|
||||||
|
assert body['voice'] == 'narrator1', body
|
||||||
|
assert body['input'] == 'hello world', body
|
||||||
|
assert body['response_format'] == 'wav', body
|
||||||
|
assert base64.b64decode(body['reference_audio']) == reference, 'reference_audio is not the pack wav'
|
||||||
|
PY
|
||||||
|
# What a consumer discovers before it speaks: /health and the voice list.
|
||||||
|
# A directory with a manifest but no reference.wav is not a voice.
|
||||||
|
run curl -sS "http://127.0.0.1:$APP_PORT/v1/audio/voices"
|
||||||
|
[ "$output" = '["narrator1"]' ]
|
||||||
|
run curl -sS "http://127.0.0.1:$APP_PORT/health"
|
||||||
|
[[ "$output" == *'"status": "ok"'* ]]
|
||||||
|
[[ "$output" == *'"backend": "omni-http"'* ]]
|
||||||
|
[[ "$output" == *'"voices": ["narrator1"]'* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "voice_id works as the alias the omp harness sends" {
|
||||||
|
arm_omni
|
||||||
|
start_app
|
||||||
|
speech '{"model":"tts-1","input":"hello","voice_id":"narrator1"}'
|
||||||
|
[ "$output" = "200" ] || { cat "$TEST_TMP/app.log"; false; }
|
||||||
|
cmp "$TEST_TMP/out.bin" "$TEST_TMP/upstream.wav"
|
||||||
|
python3 - "$TEST_TMP/capture.json" <<'PY'
|
||||||
|
import json, sys
|
||||||
|
body = json.load(open(sys.argv[1]))
|
||||||
|
assert body['voice'] == 'narrator1', 'voice_id never became the upstream voice'
|
||||||
|
assert 'voice_id' not in body, body
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "an unknown voice is a 400 naming the voices that exist" {
|
||||||
|
arm_omni
|
||||||
|
start_app
|
||||||
|
speech '{"model":"tts-1","input":"hello","voice":"nosuch"}'
|
||||||
|
[ "$output" = "400" ]
|
||||||
|
grep -q 'narrator1' "$TEST_TMP/out.bin"
|
||||||
|
# Refused before the engine saw anything: no audio, no upstream call.
|
||||||
|
[ ! -s "$TEST_TMP/capture.json" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "oversized input is refused outright, not narrated half-way" {
|
||||||
|
arm_omni
|
||||||
|
# The limit is a knob the operator sets, not a constant in the seam: the
|
||||||
|
# app reads its knobs at request time, so exporting it before boot works.
|
||||||
|
export SHIM_MAX_CHARS=10
|
||||||
|
start_app
|
||||||
|
python3 - "$TEST_TMP/payload.json" <<'PY'
|
||||||
|
import json, sys
|
||||||
|
json.dump({'model': 'tts-1', 'input': 'A' * 11, 'voice': 'narrator1'},
|
||||||
|
open(sys.argv[1], 'w'))
|
||||||
|
PY
|
||||||
|
run curl -sS -o "$TEST_TMP/out.bin" -w '%{http_code}' \
|
||||||
|
-H 'Content-Type: application/json' --data-binary "@$TEST_TMP/payload.json" \
|
||||||
|
"http://127.0.0.1:$APP_PORT/v1/audio/speech"
|
||||||
|
[ "$output" = "414" ]
|
||||||
|
[ ! -s "$TEST_TMP/capture.json" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "every other route is a 404, GET or POST" {
|
||||||
|
arm_omni
|
||||||
|
start_app
|
||||||
|
run curl -sS -o /dev/null -w '%{http_code}' "http://127.0.0.1:$APP_PORT/v1/audio/speeck"
|
||||||
|
[ "$output" = "404" ]
|
||||||
|
run curl -sS -o /dev/null -w '%{http_code}' -X POST \
|
||||||
|
-H 'Content-Type: application/json' -d '{"input":"hi","voice":"narrator1"}' \
|
||||||
|
"http://127.0.0.1:$APP_PORT/v1/speech"
|
||||||
|
[ "$output" = "404" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "an unrouted POST does not poison a reused connection" {
|
||||||
|
# The shim speaks HTTP/1.1, so a client may keep one TCP connection alive
|
||||||
|
# across requests. A body left unread on the way to a 404 is parsed as the
|
||||||
|
# NEXT request line — the consumer would blame the following, perfectly
|
||||||
|
# valid call. So: 404 first, then /health on the same connection.
|
||||||
|
arm_omni
|
||||||
|
start_app
|
||||||
|
python3 - "$APP_PORT" <<'PY'
|
||||||
|
import http.client, json, sys
|
||||||
|
conn = http.client.HTTPConnection("127.0.0.1", int(sys.argv[1]), timeout=15)
|
||||||
|
conn.request("POST", "/v1/audio/speeck",
|
||||||
|
body=json.dumps({"input": "hi", "voice": "narrator1"}),
|
||||||
|
headers={"Content-Type": "application/json"})
|
||||||
|
first = conn.getresponse()
|
||||||
|
assert first.status == 404, first.status
|
||||||
|
first.read()
|
||||||
|
conn.request("GET", "/health")
|
||||||
|
second = conn.getresponse()
|
||||||
|
assert second.status == 200, second.status
|
||||||
|
assert json.loads(second.read())["status"] == "ok"
|
||||||
|
conn.close()
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the CLI backend gets the resolved reference as --tts-speaker-file" {
|
||||||
|
# A fake binary standing in for llama-tts: it records its argv, writes a
|
||||||
|
# WAV where --output points, and reports the temp dir it was asked to work
|
||||||
|
# in so the cleanup promise is observable too.
|
||||||
|
cat > "$TEST_TMP/fake-tts" <<'FAKE'
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
printf '%s\n' "$@" > "$ARGS_FILE"
|
||||||
|
prev=""
|
||||||
|
for arg in "$@"; do
|
||||||
|
if [ "$prev" = "--output" ]; then
|
||||||
|
printf 'CLI-AUDIO' > "$arg"
|
||||||
|
dirname "$arg" > "$WORKDIR_FILE"
|
||||||
|
fi
|
||||||
|
prev="$arg"
|
||||||
|
done
|
||||||
|
exit 0
|
||||||
|
FAKE
|
||||||
|
chmod +x "$TEST_TMP/fake-tts"
|
||||||
|
printf 'CLI-AUDIO' > "$TEST_TMP/expected.bin"
|
||||||
|
|
||||||
|
export VOICES_DIR="$VOICES"
|
||||||
|
export TTS_BACKEND=llama-tts-cli
|
||||||
|
export LLAMA_TTS_BIN="$TEST_TMP/fake-tts"
|
||||||
|
export LLAMA_TTS_MODEL="$TEST_TMP/model.bin"
|
||||||
|
export ARGS_FILE="$TEST_TMP/argv.txt"
|
||||||
|
export WORKDIR_FILE="$TEST_TMP/workdir.txt"
|
||||||
|
start_app
|
||||||
|
speech '{"model":"tts-1","input":"narrate this","voice":"narrator1"}'
|
||||||
|
[ "$output" = "200" ] || { cat "$TEST_TMP/app.log"; false; }
|
||||||
|
grep -qi '^content-type: audio/wav' "$TEST_TMP/head.txt"
|
||||||
|
cmp "$TEST_TMP/out.bin" "$TEST_TMP/expected.bin"
|
||||||
|
# The speaker file must be the pack's reference under VOICES_DIR — that is
|
||||||
|
# the whole product; anything else and the clone never happens.
|
||||||
|
speaker="$(awk 'p { print; exit } $0 == "--tts-speaker-file" { p = 1 }' "$TEST_TMP/argv.txt")"
|
||||||
|
[ "$speaker" = "$VOICES/narrator1/reference.wav" ]
|
||||||
|
prompt="$(awk 'p { print; exit } $0 == "-p" { p = 1 }' "$TEST_TMP/argv.txt")"
|
||||||
|
[ "$prompt" = "narrate this" ]
|
||||||
|
grep -qx -- '--tts-lang' "$TEST_TMP/argv.txt"
|
||||||
|
# A per-request temp dir that survives is how a TTS box fills its /tmp.
|
||||||
|
workdir="$(cat "$TEST_TMP/workdir.txt")"
|
||||||
|
[ -n "$workdir" ]
|
||||||
|
[ ! -d "$workdir" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "response_format=mp3 ships mp3, converted from the engine's wav" {
|
||||||
|
arm_omni
|
||||||
|
# A fake ffmpeg on PATH: whatever wav arrives on stdin is tapped (so the
|
||||||
|
# conversion provably ran on the engine's audio) and mp3 bytes come out.
|
||||||
|
mkdir -p "$TEST_TMP/bin"
|
||||||
|
cat > "$TEST_TMP/bin/ffmpeg" <<'FF'
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
cat > "$STDIN_TAP"
|
||||||
|
printf 'MP3-BYTES'
|
||||||
|
FF
|
||||||
|
chmod +x "$TEST_TMP/bin/ffmpeg"
|
||||||
|
export STDIN_TAP="$TEST_TMP/ffmpeg-in.wav"
|
||||||
|
# The fake must reach the APP's PATH: the shim runs `ffmpeg` by name.
|
||||||
|
export PATH="$TEST_TMP/bin:$PATH"
|
||||||
|
start_app
|
||||||
|
speech '{"model":"tts-1","input":"hello","voice":"narrator1","response_format":"mp3"}'
|
||||||
|
[ "$output" = "200" ] || { cat "$TEST_TMP/app.log"; false; }
|
||||||
|
grep -qi '^content-type: audio/mpeg' "$TEST_TMP/head.txt"
|
||||||
|
printf 'MP3-BYTES' > "$TEST_TMP/expected-mp3.bin"
|
||||||
|
cmp "$TEST_TMP/out.bin" "$TEST_TMP/expected-mp3.bin"
|
||||||
|
cmp "$TEST_TMP/ffmpeg-in.wav" "$TEST_TMP/upstream.wav"
|
||||||
|
# The upstream is still asked for wav: mp3 is this shim's re-encode.
|
||||||
|
python3 - "$TEST_TMP/capture.json" <<'PY'
|
||||||
|
import json, sys
|
||||||
|
assert json.load(open(sys.argv[1]))['response_format'] == 'wav'
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the shipped env file boots the shim with its declared backend" {
|
||||||
|
# Not a grep of the env file: source it the way systemd's EnvironmentFile
|
||||||
|
# would, point the two machine-local values at the sandbox, and require the
|
||||||
|
# endpoint to actually speak. A typo, a shell-ism or an expansion systemd
|
||||||
|
# cannot parse shows up here as a dead process, not a green grep.
|
||||||
|
set -a
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
source "$ENV_FILE"
|
||||||
|
set +a
|
||||||
|
arm_omni
|
||||||
|
start_app
|
||||||
|
speech '{"model":"tts-1","input":"hello","voice":"narrator1"}'
|
||||||
|
[ "$output" = "200" ] || { cat "$TEST_TMP/app.log"; false; }
|
||||||
|
cmp "$TEST_TMP/out.bin" "$TEST_TMP/upstream.wav"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the quadlet unit keeps boot persistence and a read-only voice mount" {
|
||||||
|
grep -qE '^Restart=always$' "$UNIT"
|
||||||
|
grep -qE '^RestartSec=[0-9]+$' "$UNIT"
|
||||||
|
grep -qE '^WantedBy=default.target$' "$UNIT"
|
||||||
|
grep -qE '^PublishPort=([0-9.]+:)?7772:7772$' "$UNIT"
|
||||||
|
grep -qE '^Volume=/home/dank/voices:/voices:ro$' "$UNIT"
|
||||||
|
}
|
||||||
Executable
+274
@@ -0,0 +1,274 @@
|
|||||||
|
#!/usr/bin/env bats
|
||||||
|
# Seam: open-notebook, the fleet's podcast generator on Dankclaw — the Quadlet
|
||||||
|
# pair (app + SurrealDB), the private network between them, the env file both
|
||||||
|
# units read, the deployer routing, and the homepage card. Runbook:
|
||||||
|
# serverAppConfigs-docs/Dankclaw/Open Notebook.md.
|
||||||
|
#
|
||||||
|
# Structure only: what the files declare and what the deployer can see. Nothing
|
||||||
|
# here starts a container, opens a socket, or touches the network — the app's
|
||||||
|
# behaviour inside the image is upstream's contract, cited in the docs page.
|
||||||
|
|
||||||
|
load helpers
|
||||||
|
|
||||||
|
UNIT_DIR="dankclaw/open-notebook/quadlet"
|
||||||
|
APP="$REPO_ROOT/$UNIT_DIR/open-notebook.container"
|
||||||
|
DB="$REPO_ROOT/$UNIT_DIR/open-notebook-surrealdb.container"
|
||||||
|
NET="$REPO_ROOT/$UNIT_DIR/open-notebook.network"
|
||||||
|
ENVF="$REPO_ROOT/scripts/open-notebook.env"
|
||||||
|
PLANNER="$REPO_ROOT/scripts/deploy-plan.py"
|
||||||
|
REMOTE="$REPO_ROOT/scripts/deploy-remote.sh"
|
||||||
|
# The host-side path the units read (they run on Dankclaw, not here), spelled
|
||||||
|
# out once so the ordering test below compares against a real constant.
|
||||||
|
HOST_TUNABLES="EnvironmentFile=/home/dank/serverAppConfigs/scripts/open-notebook.env"
|
||||||
|
HOST_SECRETS="EnvironmentFile=/home/dank/Containers/open-notebook/env"
|
||||||
|
|
||||||
|
# The pins are part of the contract, so a silent re-resolution to a different
|
||||||
|
# build has to fail here rather than at the next deploy. Both digests were
|
||||||
|
# resolved from the registry on 2026-10-18; the SurrealDB one is 2.7.0.
|
||||||
|
APP_PIN="sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212"
|
||||||
|
DB_PIN="sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723"
|
||||||
|
|
||||||
|
@test "open-notebook ships a network, a database unit and an app unit" {
|
||||||
|
[ -f "$NET" ]
|
||||||
|
[ -f "$DB" ]
|
||||||
|
[ -f "$APP" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "both units are boot-persistent and self-healing" {
|
||||||
|
for unit in "$APP" "$DB"; do
|
||||||
|
grep -qE '^Restart=(always|on-failure)$' "$unit" || {
|
||||||
|
echo "$(basename "$unit") has no Restart="; false; }
|
||||||
|
# "short RestartSec": a podcast service has no reason to wait minutes.
|
||||||
|
restart_sec="$(sed -n 's/^RestartSec=\([0-9]*\)s\?$/\1/p' "$unit")"
|
||||||
|
[ -n "$restart_sec" ] || {
|
||||||
|
echo "$(basename "$unit") has no RestartSec"; false; }
|
||||||
|
[ "$restart_sec" -le 30 ] || {
|
||||||
|
echo "$(basename "$unit") RestartSec=${restart_sec}s is not short"; false; }
|
||||||
|
grep -qE '^\[Install\]' "$unit"
|
||||||
|
grep -qE '^WantedBy=default\.target$' "$unit"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "both images are digest-pinned and never re-resolved at start" {
|
||||||
|
grep -qxF "Image=docker.io/lfnovo/open_notebook:v1-latest@$APP_PIN" "$APP"
|
||||||
|
grep -qxF "Image=docker.io/surrealdb/surrealdb:v2@$DB_PIN" "$DB"
|
||||||
|
grep -qxF 'Pull=never' "$APP"
|
||||||
|
grep -qxF 'Pull=never' "$DB"
|
||||||
|
# Pull=never is only honest if the fetch it depends on is written down where
|
||||||
|
# the operator reads the unit, and wired into the deployer's precondition.
|
||||||
|
grep -qF 'podman pull' "$APP"
|
||||||
|
grep -qF 'podman pull' "$DB"
|
||||||
|
grep -qF 'ensure_pinned_image' "$REMOTE"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the database port is never published" {
|
||||||
|
# 8000 belongs to the private bridge. A published 8000 would put the RPC,
|
||||||
|
# root credentials included, on every interface of the host.
|
||||||
|
! grep -qE '^PublishPort=' "$DB"
|
||||||
|
! grep -qE '^PublishPort=.*:8000(:|$)' "$APP"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the app publishes its UI and moves the API off the host port seerr holds" {
|
||||||
|
grep -qxF 'PublishPort=0.0.0.0:8502:8502' "$APP"
|
||||||
|
# seerr already binds 127.0.0.1:5055 on this host, so the API cannot leave
|
||||||
|
# the container on the host port upstream uses.
|
||||||
|
publish="$(grep -E '^PublishPort=[^:]+:[0-9]+:5055$' "$APP")"
|
||||||
|
[ -n "$publish" ] || { echo "no PublishPort mapping to the API's 5055"; false; }
|
||||||
|
mapping="${publish#PublishPort=}" # e.g. 0.0.0.0:5056:5055
|
||||||
|
host_port="${mapping#*:}"
|
||||||
|
host_port="${host_port%%:*}"
|
||||||
|
[ "$host_port" != "5055" ] || {
|
||||||
|
echo "the API is published on host 5055, which seerr already holds"; false; }
|
||||||
|
# The frontend auto-detects the API as <host it was opened on>:5055, so a
|
||||||
|
# remap is only correct if API_URL names the published port.
|
||||||
|
api_url="$(sed -n 's/^API_URL=//p' "$ENVF")"
|
||||||
|
[ "$api_url" = "http://192.168.0.143:${host_port}" ] || {
|
||||||
|
echo "API_URL ($api_url) does not name the published API port ($host_port)"; false; }
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the pair shares one network and the app finds the database by container name" {
|
||||||
|
grep -qxF 'Network=open-notebook.network' "$DB"
|
||||||
|
grep -qxF 'Network=open-notebook.network' "$APP"
|
||||||
|
[ "$(grep -cE '^Network=' "$DB")" = "1" ]
|
||||||
|
# dankclaw_default is the app's second network and must come FIRST: the
|
||||||
|
# generated --network flags keep file order, so the default route (and the
|
||||||
|
# gateway name proven on that network) rides it.
|
||||||
|
[ "$(grep -cE '^Network=' "$APP")" = "2" ]
|
||||||
|
[ "$(grep -E '^Network=' "$APP" | head -1)" = "Network=dankclaw_default" ]
|
||||||
|
db_name="$(sed -n 's/^ContainerName=//p' "$DB")"
|
||||||
|
[ "$db_name" = "open-notebook-surrealdb" ]
|
||||||
|
grep -qxF "SURREAL_URL=ws://${db_name}:8000/rpc" "$ENVF"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "each unit reads the repo tunables first and the host secrets second" {
|
||||||
|
# podman applies --env-file in argument order and the LAST file wins, so
|
||||||
|
# swapping these two lines would blank every credential the app needs.
|
||||||
|
for unit in "$APP" "$DB"; do
|
||||||
|
files="$(grep -E '^EnvironmentFile=' "$unit")"
|
||||||
|
[ "$(echo "$files" | wc -l)" = "2" ] || {
|
||||||
|
echo "$(basename "$unit") must read exactly two env files"; false; }
|
||||||
|
[ "$(echo "$files" | head -1)" = "$HOST_TUNABLES" ] || {
|
||||||
|
echo "$(basename "$unit") must read the tracked env file first"; false; }
|
||||||
|
[ "$(echo "$files" | tail -1)" = "$HOST_SECRETS" ] || {
|
||||||
|
echo "$(basename "$unit") must read the host secret file last"; false; }
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the interim episode destination is mounted exactly once" {
|
||||||
|
# The bind that answers Gitea issue #7 provisionally: two of them is a
|
||||||
|
# shadowing bug, none is episode audio locked inside a named volume.
|
||||||
|
[ "$(grep -cE '^Volume=/home/dank/media/podcasts:/app/data/podcasts$' "$APP")" = "1" ]
|
||||||
|
# The app's own state stays in a named volume; the database gets its own.
|
||||||
|
grep -qE '^Volume=open-notebook-data:/app/data$' "$APP"
|
||||||
|
grep -qE '^Volume=open-notebook-db:/mydata$' "$DB"
|
||||||
|
# The deployer creates the host directory as the deploy user.
|
||||||
|
grep -qF 'mkdir -p /home/dank/media/podcasts' "$REMOTE"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the env file declares the CPU-shutdown guard and both timeouts" {
|
||||||
|
# Upstream defaults are 5 and 5; frank-tts spawns ~8 GiB / 8 threads per
|
||||||
|
# request on a box that also serves two LLM endpoints.
|
||||||
|
grep -qxF 'TTS_BATCH_SIZE=1' "$ENVF"
|
||||||
|
grep -qxF 'OPEN_NOTEBOOK_WORKER_MAX_TASKS=1' "$ENVF"
|
||||||
|
# Per dialogue line, and strictly beyond frank-tts's own per-request
|
||||||
|
# ceiling: nested budgets go inside-out (the rule scripts/frank-tts.env
|
||||||
|
# states for LLAMA_TIMEOUT), so a shim give-up surfaces as frank-tts's
|
||||||
|
# error instead of the worker's own timeout. The seam reads the ceiling
|
||||||
|
# rather than restating it, so re-measuring frank-tts cannot leave this
|
||||||
|
# file silently behind.
|
||||||
|
tts="$(sed -n 's/^ESPERANTO_TTS_TIMEOUT=//p' "$ENVF")"
|
||||||
|
shim="$(sed -n 's/^SHIM_TIMEOUT=//p' "$REPO_ROOT/scripts/frank-tts.env")"
|
||||||
|
[ -n "$tts" ] || { echo "no ESPERANTO_TTS_TIMEOUT in the env file"; false; }
|
||||||
|
[ -n "$shim" ] || { echo "no SHIM_TIMEOUT in scripts/frank-tts.env"; false; }
|
||||||
|
[ "$tts" -gt "$shim" ] || {
|
||||||
|
echo "ESPERANTO_TTS_TIMEOUT=$tts is not beyond frank-tts SHIM_TIMEOUT=$shim"; false; }
|
||||||
|
# Upstream: keep the scriptwriting call under the web UI's ~600 s patience,
|
||||||
|
# or the failure arrives as a UI timeout with no name on it.
|
||||||
|
llm="$(sed -n 's/^ESPERANTO_LLM_TIMEOUT=//p' "$ENVF")"
|
||||||
|
[[ "$llm" =~ ^[0-9]+$ ]] && [ "$llm" -lt 600 ] || {
|
||||||
|
echo "ESPERANTO_LLM_TIMEOUT=$llm is not a number under 600"; false; }
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "secrets stay declared-but-empty in the tracked env file" {
|
||||||
|
# scripts/*.env is un-ignored for "paths and numbers, never credentials":
|
||||||
|
# the values belong to the host file the deployer seeds.
|
||||||
|
for key in OPEN_NOTEBOOK_ENCRYPTION_KEY OPEN_NOTEBOOK_PASSWORD SURREAL_PASSWORD SURREAL_PASS; do
|
||||||
|
grep -qxF "${key}=" "$ENVF" || {
|
||||||
|
echo "$key is not declared, empty, in $ENVF"; false; }
|
||||||
|
done
|
||||||
|
has_no_leaked_secret "$ENVF"
|
||||||
|
has_no_leaked_sudo_pw "$ENVF"
|
||||||
|
# And the deployer treats blank as fatal, because blank means "auth off",
|
||||||
|
# not "unconfigured".
|
||||||
|
grep -qF 'seed_open_notebook_env' "$REMOTE"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the database password never rides in a command line" {
|
||||||
|
# `surreal start` reads SURREAL_USER / SURREAL_PASS from the environment;
|
||||||
|
# --pass would put it in /proc/<pid>/cmdline for any ps on the host.
|
||||||
|
grep -qE '^Exec=start' "$DB"
|
||||||
|
! grep -qE '^Exec=.*--(pass|user)' "$DB"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "both health probes are argument-only and name real paths" {
|
||||||
|
for unit in "$APP" "$DB"; do
|
||||||
|
cmd="$(sed -n 's/^HealthCmd=//p' "$unit")"
|
||||||
|
[ -n "$cmd" ] || { echo "$(basename "$unit") has no HealthCmd"; false; }
|
||||||
|
# The quadlet generator splits HealthCmd on spaces, so anything needing
|
||||||
|
# a shell (quotes, pipes, &&) would be silently mis-sent.
|
||||||
|
! printf '%s' "$cmd" | grep -qE '["&;|`]'
|
||||||
|
done
|
||||||
|
# The API's own route, exempt from its password middleware upstream.
|
||||||
|
grep -qxF 'HealthCmd=curl -fsS http://127.0.0.1:5055/health' "$APP"
|
||||||
|
# The distroless database image has no shell and no curl; it ships a probe.
|
||||||
|
grep -qxF 'HealthCmd=/surreal is-ready' "$DB"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the deployer gates open-notebook on the health path that actually answers" {
|
||||||
|
grep -qF 'open-notebook) deploy_open_notebook ;;' "$REMOTE"
|
||||||
|
gate="$(grep -E '^[[:space:]]*wait_for_health open-notebook ' "$REMOTE" | head -1)"
|
||||||
|
[ -n "$gate" ] || { echo "deploy_open_notebook has no wait_for_health gate"; false; }
|
||||||
|
# /health is the route upstream serves; the suite must not bless a guess.
|
||||||
|
printf '%s' "$gate" | grep -qE '/health( |$)'
|
||||||
|
# The gate is a host-loopback URL (the publish is wildcard, so 127.0.0.1
|
||||||
|
# answers) on the PUBLISHED port, never the container's 5055.
|
||||||
|
printf '%s' "$gate" | grep -qE 'http://127\.0\.0\.1:[0-9]+/health'
|
||||||
|
! printf '%s' "$gate" | grep -qE ':5055/'
|
||||||
|
# The database is gated by its container health before the app is started.
|
||||||
|
grep -qE 'wait_container_healthy open-notebook-surrealdb' "$REMOTE"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the deployer refuses to start the pair on blank secrets and never rewrites a seeded file" {
|
||||||
|
# Read the seed function itself: the refusal, and the no-clobber rule that
|
||||||
|
# protects a password an operator set by hand.
|
||||||
|
seed="$(sed -n '/^seed_open_notebook_env()/,/^}/p' "$REMOTE")"
|
||||||
|
[ -n "$seed" ]
|
||||||
|
printf '%s' "$seed" | grep -qF 'return 1'
|
||||||
|
printf '%s' "$seed" | grep -qF 'if [[ ! -e "$dest" ]]'
|
||||||
|
printf '%s' "$seed" | grep -qF 'chmod 600'
|
||||||
|
# One value, two spellings: the app reads SURREAL_PASSWORD, the CLI SURREAL_PASS.
|
||||||
|
printf '%s' "$seed" | grep -qF 'SURREAL_PASS='
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the planner maps every open-notebook path to the open-notebook service" {
|
||||||
|
run python3 "$PLANNER" --target dankclaw --paths \
|
||||||
|
dankclaw/open-notebook/quadlet/open-notebook.container \
|
||||||
|
dankclaw/open-notebook/quadlet/open-notebook-surrealdb.container \
|
||||||
|
dankclaw/open-notebook/quadlet/open-notebook.network \
|
||||||
|
scripts/open-notebook.env
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ "$output" = "open-notebook" ]
|
||||||
|
# One deployable owns the pair: the unit files never plan separately.
|
||||||
|
run python3 "$PLANNER" --target dankclaw --paths \
|
||||||
|
dankclaw/open-notebook/quadlet/open-notebook-surrealdb.container
|
||||||
|
[ "$output" = "open-notebook" ]
|
||||||
|
# And the service belongs to dankclaw only.
|
||||||
|
run python3 "$PLANNER" --target dankserver --paths scripts/open-notebook.env
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ -z "$output" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "changed-path routing selects this seam" {
|
||||||
|
run "$REPO_ROOT/scripts/test-changed.sh" --list --paths \
|
||||||
|
dankclaw/open-notebook/quadlet/open-notebook.container
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
printf '%s\n' "$output" | grep -qx "tests/open_notebook.bats"
|
||||||
|
|
||||||
|
run "$REPO_ROOT/scripts/test-changed.sh" --list --paths scripts/open-notebook.env
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
printf '%s\n' "$output" | grep -qx "tests/open_notebook.bats"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "both deployer entry points know the service" {
|
||||||
|
grep -qF 'dankclaw:open-notebook|' "$REMOTE"
|
||||||
|
grep -qF 'dankclaw:open-notebook|' "$REPO_ROOT/scripts/deploy.sh"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the homepage card exists and stays hidden until the service is deployed" {
|
||||||
|
python3 - "$REPO_ROOT/shared/homepage/services.json" <<'PY'
|
||||||
|
import json, os, sys
|
||||||
|
data = json.load(open(sys.argv[1]))
|
||||||
|
cards = [s for s in data if s["name"] == "open-notebook"]
|
||||||
|
assert len(cards) == 1, f"expected exactly one open-notebook card, got {len(cards)}"
|
||||||
|
card = cards[0]
|
||||||
|
assert card["category"] == "user" and card["section"] == "AI", card
|
||||||
|
assert card["probe"] == "127.0.0.1:8502", card["probe"]
|
||||||
|
assert card["method"] == "http"
|
||||||
|
# Not deployed yet: the card is reported wip-down and carries no link, so the
|
||||||
|
# page shows no dead button (the Seerr precedent).
|
||||||
|
assert card.get("wip") == "down", card.get("wip")
|
||||||
|
assert card.get("showLink") is False, card.get("showLink")
|
||||||
|
assert not card.get("url"), "a hidden card must not carry a url"
|
||||||
|
art = card["art"].lstrip("/")
|
||||||
|
assert os.path.exists(os.path.join(os.path.dirname(sys.argv[1]), art)), art
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the runbook page exists and names the interim mount's owning issue" {
|
||||||
|
page="$REPO_ROOT/serverAppConfigs-docs/Dankclaw/Open Notebook.md"
|
||||||
|
[ -f "$page" ]
|
||||||
|
grep -qF 'issue #7' "$page"
|
||||||
|
grep -qF 'human-only' "$page"
|
||||||
|
# The interim decision is one mount line; the page has to say how to undo it.
|
||||||
|
grep -qF '## Rollback' "$page"
|
||||||
|
}
|
||||||
Reference in new issue
Block a user