Merge remote-tracking branch 'origin/main' into wt/20261017-181502
# Conflicts: # serverAppConfigs-docs/log.md
This commit is contained in:
Frank
@@ -0,0 +1,133 @@
|
||||
---
|
||||
type: reference
|
||||
description: "frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker, on a private podman bridge behind frank-tts — deployed 2026-10-18, and it publishes no port on purpose."
|
||||
tags:
|
||||
- dankclaw
|
||||
- tts
|
||||
- voice
|
||||
- quadlet
|
||||
- cpu
|
||||
timestamp: 2026-10-18
|
||||
---
|
||||
|
||||
# Frank TTS Engine
|
||||
|
||||
`frank-tts-engine` is the **arm A text-to-speech engine** on Dankclaw: [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp)'s resident `tts-server` HTTP process, holding the Qwen3-TTS **1.7 B base talker (Q4_K_M)** and the **12 Hz tokenizer (Q8_0)** in RAM once and synthesizing from there. It listens on container port **8880** and is **deliberately not a fleet endpoint** — the endpoint consumers address is `frank-tts` on **7772**, which resolves voice names, re-encodes to mp3 and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page.
|
||||
|
||||
**Deployed 2026-10-18.** The chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified, 18 s), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit `active` with no drop-ins, container healthy 31 s after the build, **4.0 GiB** resident on a host that kept 17 GiB available, `3 voice(s) registered and warm` in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a **file**, not an argument (Troubleshooting), and `STEP 3/13 RUN useradd` is the first honest proof of the image's non-root user.
|
||||
|
||||
**Host:** Dankclaw (`192.168.0.143`), rootless Podman, user `dank` — **Port:** 8880, **never published** — **Network:** the private `systemd-frank-tts-engine` bridge (10.89.5.0/24) and no other — **Deployer:** `frank-tts-engine` — **Consumers:** `frank-tts` (7772) and, through it, every voice request in the fleet including [[Open Notebook]]'s podcast renders.
|
||||
|
||||
## Facts (verified 2026-10-18 against the pinned upstream commit, the Hub, and the CPU trial on this host)
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Engine | `tts-server` from qwentts.cpp, pinned at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` (`dankclaw/voiceservice/Containerfile.engine`). The source is **not vendored**: the image clones that commit and its recorded `ggml` submodule, so the build needs outbound HTTPS and the `cfhash` label is what keeps an unchanged re-deploy offline |
|
||||
| Build line | `cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF && cmake --build build -j 8` — the exact line the 2026-10-18 trial ran on this box. `GGML_BLAS=OFF` is **forced**: Dankclaw has `libopenblas.so.0` and no BLAS dev headers, so upstream's own `Dockerfile` and `buildcpu.sh` (both of which pass BLAS on) cannot be copied. No `-DGGML_NATIVE` is passed, and ggml defaults it on, so the binary is `-march=native` for the machine that built it: **build this image on the host that runs it** |
|
||||
| Image | built locally as `franksdank/frank-tts-engine:latest`, two stages from one digest-pinned base `docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac` (index digest resolved 2026-10-18). Runtime stage: `libgomp1` (OpenMP is on), `curl` (the unit's `HealthCmd` runs as `/bin/sh -c`, and the warmer speaks HTTP), uid **10000**, `LD_LIBRARY_PATH=/app` because ggml builds shared by default and the link-time RPATH names the build tree |
|
||||
| Talker | `qwen-talker-1.7b-base-Q4_K_M.gguf`, **1 219 245 248 B**, sha256 `ea393ebaf2167ea23ce9fc18b093822851358a950d7075cd47ab4f6ce23e887d` (the Hub's LFS pointer oid) |
|
||||
| Tokenizer | `qwen-tokenizer-12hz-Q8_0.gguf`, **291 150 624 B**, sha256 `1883beeed99348fc35e23dd225e9082f93f6f8c109330a33d935baa8acdbfd94` |
|
||||
| Source of both | `https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF`, fetched by `scripts/download-frank-tts-engine-qwentts.sh` into `/home/dank/models/Qwen3-TTS-GGUF`, mounted read-only at `/models`. It has to be the **base** variant: the engine refuses every voice registration unless the model reports type `base`, so on `customvoice`/`voicedesign` the cloned-voice product does not exist. A 0.6 B talker (628 905 056 B) exists and is **not** the decided artifact |
|
||||
| Memory | **5.7 GiB peak RSS** with both files resident (2026-10-18 trial). No `LimitMEMORY`/`MemoryMax`: a limit would turn a busy moment into an OOM kill in the middle of an utterance |
|
||||
| Concurrency | `--max-batch 1`: a serialised FIFO. Warm RTF on this box is 0.55–0.67 against real time 1.0, and that margin is the entire headroom a live stream has; two concurrent syntheses split the same cores and the second caller's first byte additionally waits for the first utterance to finish |
|
||||
| Network | `systemd-frank-tts-engine` (10.89.5.0/24, `dankclaw/voiceservice/quadlet/frank-tts-engine.network`) and **nothing else**, with **no `PublishPort` at all**. `frank-tts` joins the same bridge and dials `http://frank-tts-engine:8880` |
|
||||
| Voices | `/home/dank/voices` mounted read-only at `/voices`, and it is the **warmer** that reads it, not the server: `tts-server` receives reference audio as base64 in a registration body and keeps its registry in memory only |
|
||||
| Health | `GET /health` → `{"status":"ok"}`, served by the HTTP thread rather than the synthesis queue, so a busy engine still reads healthy. Probed by the unit (`HealthCmd`, 30 s interval, 120 s start period) **inside the container's namespace**, because there is no published port to probe from outside |
|
||||
| Routes | `GET /health`, `GET /v1/models`, `GET|POST /v1/audio/voices`, `DELETE /v1/audio/voices/{name}`, `POST /v1/audio/speech` (`response_format` = `wav` or `pcm`; **default `pcm`**). Everything else is a bare 404. `POST /v1/audio/speech` requires `input`; unknown `voice`, missing `input` and a bad `speed` are 400s, an empty generation is a 502 |
|
||||
|
||||
## Why this engine, and not the alternatives
|
||||
|
||||
The criterion Frank set on 2026-10-18 was **"whatever is better for live output"**, and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. RTF only has to stay under 1.0 with margin; what separates the arms is first-byte latency and **spread**. Measured on this host the same day (CPU; the full A/B is the section *"Decision: arm A for live output (2026-10-18)"* in [[Local TTS and Voice Cloning]], which is the source of these numbers):
|
||||
|
||||
| Measure | Resident `tts-server` (arm A, chosen) | Mainline per-request CLI path (arm B) |
|
||||
|---|---|---|
|
||||
| first audio byte | **86–451 ms** end to end; **94–112 ms** server-side | model load, per request |
|
||||
| warm real-time factor while streaming | **0.55–0.67** | 0.93–0.99 |
|
||||
| run-to-run spread | **±0.01** | ±0.25 |
|
||||
| HTTP surface | serves `/v1/audio/speech` itself | none — façade CLI only |
|
||||
|
||||
The spread is the whole argument: at ±0.01 the engine finishes synthesizing 1.5–1.8× faster than the audio plays, every run; at ±0.25 some runs fall behind real time and a live listener hears the stream stall mid-sentence. The rollback path stays in the repo untouched — `TTS_BACKEND=llama-tts-cli` in `scripts/frank-tts.env` still selects the per-request CLI engine, and that file's header documents what it costs.
|
||||
|
||||
## Three laws this service lives under
|
||||
|
||||
**1. The registry law → the unit warms at boot.** The voice registry is **memory only**: a restart forgets every voice, and re-registering one does **not** warm it — only synthesizing does. A boot that registers nothing leaves the façade answering 400 for every name, which is why `frank-tts-engine.container` carries
|
||||
|
||||
```
|
||||
ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh
|
||||
TimeoutStartSec=900
|
||||
```
|
||||
|
||||
and the warm pass runs on every start including boot. What it buys is registration (mandatory) plus a measurably smaller first request — and the size of that saving is **not** the number the trial published. **Re-measured on the deployed build, 2026-10-18,** against an empty registry (register, then time the synthesis): the first request for `narrator1` cost **1 883 ms**, the second **1 283 ms**, the third **1 226 ms** — about **0.6 s** of saved silence. A ~110-word request cost **9.4 s** and then **9.8 s**, so request time tracks the audio generated, not snapshot state. The trial's **6.2–8.0 s / first audio byte 5 530 ms / later requests 0.5 ms** ladder does **not** reproduce here; the 0.5 ms figure is an internal state-restore cost that was never an end-to-end request time, and the honest claim is the ~0.6 s above. The `-` prefix is load-bearing: with `Restart=always`, a warmer that exits non-zero would restart the container and crash-loop the endpoint forever. The hook is `podman exec` rather than a host script **because this container publishes nothing** — there is no address the host could dial — and a consequence worth knowing: an exec inherits the unit's `EnvironmentFile`, so the warmer's knobs (`VOICES_DIR`, `HEALTH_WAIT`, `WARM_TEXT`, `REQUEST_TIMEOUT`) live in `scripts/frank-tts-engine.env` beside the engine's own. The deployer runs the same hook a second time so a failed warm becomes a deploy result rather than only a journal line, and so the per-voice numbers land on the console:
|
||||
|
||||
```
|
||||
frank-tts-engine-warm: voice narrator1 cold=1402ms warm=1812ms
|
||||
frank-tts-engine-warm: 3 voice(s) registered and warm
|
||||
```
|
||||
|
||||
**2. The thread law → nothing is ever pinned.** `tts-server` in this build has **no `-t`/`--threads` flag**: `src/backend.h` computes the CPU thread count as `std::thread::hardware_concurrency() / 2` (its comment: GEMM shares SIMD units across hyperthreads). Thread count therefore follows what the kernel *reports*, not what a cgroup *allows* — so `CPUQuota=`, `CPUAffinity=`, `taskset` or a slice silently changes the engine's GEMM tiling and its timings, with no log line. Nothing in this fleet pins this container. If a neighbour starves it, fix the neighbour.
|
||||
|
||||
**3. Streaming is not buffered.** At the same seed, `response_format=pcm` (chunked, codec chunks pushed as they decode) and `response_format=wav` (buffered) differ in **92.2 % of samples**, −41.6 dB RMS. Nothing may compare the two paths, cache one against the other, or assume a seed pins output across `response_format`. `frank-tts` always asks for `wav`; the warmer asks for `wav`; the test seam is forbidden from pinning either.
|
||||
|
||||
## The voice registry contract
|
||||
|
||||
A voice becomes known to the engine through `POST /v1/audio/voices`, and only these shapes are accepted:
|
||||
|
||||
- `{"name": …, "wav_b64": …}` — the reference clip travels as base64 WAV (the engine resamples to 24 kHz internally). **This is what the boot warmer sends.**
|
||||
- `{"name": …, "spk_b64": …, "rvq_b64": …}` — the extracted embedding plus the reference codec codes. **`spk_b64` alone is rejected** (`src/tts-server.h:480`): it is `wav_b64` or *both* of the pair.
|
||||
|
||||
Whether a registered voice is *embedding-only* or *in-context-learning* is then decided by `ref_text`: `tools/tts-server.cpp:258` selects ICL only when **both** `ref_text` and the reference codes are present. Registering `{name, wav_b64}` with no `ref_text` therefore yields the **x-vector voice** — the shipped default (0.57–0.63 warm RTF against ICL's 0.67) and the only one this fleet can produce honestly, because the audition crops have no trustworthy transcript to put in `ref_text`. The ICL door stays open for batch work like the podcast render: a pack that wants it needs its text in the pack plus one key in the payload built by `dankclaw/voiceservice/tts-engine-warm.sh`.
|
||||
|
||||
The packs themselves are host state at `/home/dank/voices/<name>/reference.wav` (24 kHz mono PCM), alongside the `reference.spk` / `reference.rvq` extraction and a manifest. Both halves of the voice stack read that directory — `frank-tts` for its name list, the warmer for its registrations — and `tts-server` never opens it. Two naming facts, both from the source: pack names must be **lowercase**, because `tts_voice_name` lowercases every name engine-side while the façade matches the directory name exactly (the warmer refuses anything outside `[a-z0-9._-]`); and `GET /v1/audio/voices` at this commit returns an empty JSON object rather than the voice list (the handler builds the array and never attaches it to the document), so **use the façade's `/v1/voices` for "which voices exist"** and treat the engine's listing route as broken, not empty.
|
||||
|
||||
## Status visibility
|
||||
|
||||
The engine is scraped by **nothing in Prometheus** and has **no homepage card**, and both are shape decisions rather than gaps. It exposes no metrics endpoint; and since it publishes no port, there is no address the host's status checker (`scripts/homepage-status.py`, which dials `host:port`) or the `dankclaw_default` Prometheus container could reach. That absence is the point of the network shape, not a cost of it (see [[Container Network Namespaces]] for the netavark rule that makes a host-loopback publish useless to a peer container anyway). The pair's public signal is the endpoint in front of it: [[Frank TTS]] carries the card and is what a consumer can dial. Its up-state is visible in four places, and the runbook uses all of them:
|
||||
|
||||
- `systemctl --user status frank-tts-engine` — the unit, including whatever the warm hook logged on the way to `active`.
|
||||
- `podman healthcheck run frank-tts-engine` — runs the unit's own `HealthCmd` against `/health` on demand and exits non-zero when the engine does not answer. This is also what `scripts/deploy-remote.sh` gates the deploy on (`wait_container_healthy`), precisely because there is no URL to gate on.
|
||||
- `podman exec frank-tts-engine curl -fsS http://127.0.0.1:8880/health` — loopback is valid **inside** the container's namespace, which is where the address lives.
|
||||
- The journal of the warm pass: `journalctl --user -u frank-tts-engine`, whose per-voice `cold=` / `warm=` lines are the only signal that voices are actually registered — a live HTTP listener with an empty registry still looks healthy.
|
||||
|
||||
The **façade** on 7772 is the pair's public surface and carries the probe. Note the honest limit there: a façade that answers `/health` does **not** prove the engine behind it is alive — it reports its backend and its voice list, not an upstream ping. That is exactly why the warm pass is part of the deploy and not only of the boot.
|
||||
|
||||
## Files this service owns
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `dankclaw/voiceservice/Containerfile.engine` | two-stage build of the pinned commit, proven cmake line, digest-pinned base, ships both scripts |
|
||||
| `dankclaw/voiceservice/tts-engine-start.sh` | image entrypoint (`/app/start.sh`): env → flags, refuses to boot without weights |
|
||||
| `dankclaw/voiceservice/tts-engine-warm.sh` | boot warmer (`/app/warm-voices.sh`): register from `reference.wav` (body via a temp file), speak a fixed sentence twice, report cold/warm |
|
||||
| `dankclaw/voiceservice/quadlet/frank-tts-engine.container` | the unit: bridge only, no publish, env file, two read-only mounts, health probe, warm hook, `Restart=always` |
|
||||
| `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the private 10.89.5.0/24 bridge, with the subnet registry |
|
||||
| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law and the registry law in its header |
|
||||
| `scripts/download-frank-tts-engine-qwentts.sh` | weight fetcher: no resume, byte count + LFS oid check, `--verify-only` |
|
||||
| `scripts/deploy-remote.sh` → `deploy_frank_tts_engine` | weights gate → build → quadlet → restart → container health → warm |
|
||||
| `tests/frank_tts_engine.bats` | the seam: generator output, entrypoint argv, warmer against a stub engine, planner and routing maps |
|
||||
|
||||
## Deploy record (host steps, in the order they ran)
|
||||
|
||||
Run 2026-10-18 as `./scripts/deploy.sh dankclaw --service frank-tts-engine`, then
|
||||
`--service frank-tts`. Steps 1–6 are this service and the façade; only step 7 is
|
||||
still open.
|
||||
|
||||
1. **Fetch the weights** — `scripts/download-frank-tts-engine-qwentts.sh` (1.4 GiB, byte + sha256 verified, idempotent; `--verify-only` is what the deployer calls).
|
||||
2. **Deploy the engine** — `./scripts/deploy.sh dankclaw --service frank-tts-engine` (`--service` is the only way to name one service; `deploy-remote.sh` is the remote half and takes `<target> <git-revision> [service]...`, so never call it by hand with a bare service name). It builds the image (needs github.com once), installs the unit and the network file, gates on the container's health status, then warms.
|
||||
3. **Confirm the endpoint is alive** — `podman exec frank-tts-engine curl -fsS http://127.0.0.1:8880/health` → `{"status":"ok"}`, and `systemctl --user status frank-tts-engine` showing the unit `active` with the warm pass behind it.
|
||||
4. **Confirm the voices are known** — the warmer's journal lines name every pack, and once the façade is up `curl http://127.0.0.1:7772/v1/audio/voices` is the authoritative list (the façade has exactly three routes: `/health`, `/v1/audio/voices`, `/v1/audio/speech`).
|
||||
5. **Measure the ladder instead of trusting the trial** — two identical requests for one voice back to back. The design expected 6.2-8.0 s then 0.5 ms; the deployed container gave **1,883 ms then 1,226-1,283 ms**, so the hook buys ~0.6 s and stays only for the registry reason (law 1). The retired numbers must not be quoted.
|
||||
6. **Then the façade on 7772** (its own page), after the engine answered — that order is the whole point of step 6: publishing the façade in front of a dead engine would make every consumer fail at the first voice line.
|
||||
7. **Open:** [[Open Notebook]] is still not installed, so nothing renders a podcast yet; the voice leg it waits on now exists.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Meaning | Move |
|
||||
|---|---|---|
|
||||
| Unit restarts every 10 s, journal says `cannot read /models/…` | weights missing or the mount is empty | run the fetcher; re-deploy |
|
||||
| Unit `active`, but every voice name returns 400 from the façade | the engine restarted and lost its in-memory registry, or the warm pass failed | `journalctl --user -u frank-tts-engine` for the warmer's lines; `podman exec frank-tts-engine /app/warm-voices.sh` |
|
||||
| Every voice logs `registration failed (HTTP no response)`, with `/usr/bin/curl: Argument list too long` on the line above it | the hook handed the registration body to curl as one argument: the kernel caps a single argument at 131 072 bytes (`MAX_ARG_STRLEN`), and a shipped `reference.wav` base64's to about 1.27 MB | the body goes through a temp file and `--data-binary @file` (2026-10-18). Any pack over ~96 KB trips this, so no small fixture can see it — the seam ships a 300 KB pack for exactly that reason |
|
||||
| Everything is ~2× slower after a systemd tweak | a cpuset/cap pin changed `hardware_concurrency()` and the engine resized its thread pools | remove the pin (law 2) |
|
||||
| `502` from the façade with `frank-tts upstream request failed` | the façade cannot resolve or reach the engine | `podman network inspect systemd-frank-tts-engine` and `podman inspect frank-tts --format '{{json .NetworkSettings.Networks}}'` — the façade must be attached to the bridge; a `127.0.0.1:8880` or `host.containers.internal:8880` upstream in `scripts/frank-tts.env` is the old shape and cannot work |
|
||||
| A voice is missing from the list | its directory has no `reference.wav`, or the name is not lowercase | fix the pack; both are refused loudly by design |
|
||||
| `podman exec` in the deploy says `no such file or directory` for `/app/warm-voices.sh` | the image predates the warmer | re-deploy: since 2026-10-18 `build_compose_image` hashes every file the image `COPY`s, not only `Containerfile.engine`, so editing the warmer rebuilds it — see [[deployment]] |
|
||||
| The warm pass logs `registered, but the warm synthesis failed` for EVERY voice, while a hand-run `/app/warm-voices.sh` succeeds | `WARM_TEXT` arrived **with its quotes**. podman's `--env-file` does no shell parsing — it takes the literal remainder of the line — so the env-file reader got `"Warm up, one two three."` where `source` gives `Warm up, one two three.`; the quotes plus a backslash then make the synthesis body invalid JSON | the hook strips one matched pair and escapes what is left; prove the value with `podman exec frank-tts-engine env \| grep WARM_TEXT`. The seam pins both readers, because a test that only sources the env file cannot see this |
|
||||
| A `502` for `/v1/voices` where the engine answers `200` on `/health` from the same boot | the engine restarted and its in-memory registry is empty again — `/health` is model-loaded, NOT voices-registered | re-run `podman exec frank-tts-engine /app/warm-voices.sh`; that is the same distinction the `frank-tts` healthcheck enforces |
|
||||
@@ -0,0 +1,203 @@
|
||||
---
|
||||
type: reference
|
||||
description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; deployed 2026-10-18 and wired as the harness speech role."
|
||||
tags:
|
||||
- dankclaw
|
||||
- tts
|
||||
- voice
|
||||
- quadlet
|
||||
---
|
||||
|
||||
# Frank TTS
|
||||
|
||||
`frank-tts` is the fleet's only speech endpoint: an OpenAI-shaped
|
||||
`POST /v1/audio/speech` on **Dankclaw :7772** that answers `voice=<name>` with
|
||||
audio in that cloned voice. Everything else in the voice stack sits behind it —
|
||||
the engine ([[Frank TTS Engine]]) publishes no port on purpose, and consumers
|
||||
never learn a path.
|
||||
|
||||
**Deployed 2026-10-18** — `./scripts/deploy.sh dankclaw --service frank-tts`,
|
||||
both units `active`, no drop-ins, and the homepage card's `wip` gate came off in
|
||||
the same change. Probes on the deployed endpoint: `/health` answers
|
||||
`{"status":"ok","backend":"omni-http","voices":["narrator1","narrator2","narrator3"]}`,
|
||||
and two different voices rendered the same sentence into **different** bytes
|
||||
(24 kHz mono PCM16, 4.24 s, healthy RMS) — so name resolution is real, not a
|
||||
pass-through of one default. The operator harness routes its `speech` role here;
|
||||
see [[#The harness speech role]]. What still waits is [[Open Notebook]]: its
|
||||
render needs the app installed, not this endpoint.
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| Host | Dankclaw (`192.168.0.143`), rootless podman, user unit `frank-tts.service` |
|
||||
| Port | **7772**, published (`PublishPort=7772:7772`); the family is 7770 dragoon, 7771 zealot, 7772 tts |
|
||||
| Routes | `GET /health`, `GET /v1/audio/voices`, `POST /v1/audio/speech`; everything else 404, and speech is POST-only |
|
||||
| Image | `localhost/franksdank/frank-tts:latest`, built from `dankclaw/voiceservice/Containerfile`, `Pull=never` |
|
||||
| Networks | `dankclaw_default` first (it owns the default route and the consumers), then `frank-tts-engine.network` (10.89.5.0/24, the bridge to the engine) |
|
||||
| Voice packs | `/home/dank/voices` mounted read-only at `/voices` — host state, never repo state |
|
||||
| Upstream | `http://frank-tts-engine:8880` (`scripts/frank-tts.env`, `TTS_UPSTREAM`) |
|
||||
| Default voice | `DEFAULT_VOICE` (`scripts/frank-tts.env`) answers a request that names none; the journal marks that line `(default)`, and an **unknown** name is still a 400 |
|
||||
| Rollback | `TTS_BACKEND=llama-tts-cli` runs a CLI binary per request and needs no upstream at all |
|
||||
|
||||
## Naming a voice
|
||||
|
||||
A voice **is** a directory name. `voice=narrator1` resolves to
|
||||
`/voices/narrator1/reference.wav`, and that file — not a manifest, not a config
|
||||
entry — is what makes the name real. omp's TTS device may send `voice_id`; the
|
||||
shim accepts either. A call that sends **neither** — the harness's normal shape,
|
||||
because no harness setting carries a voice for this transport — speaks
|
||||
`$DEFAULT_VOICE`; empty default makes it a 400 again.
|
||||
|
||||
The usable shape is `^[a-z0-9][a-z0-9._-]*$`: lowercase, starting with a letter
|
||||
or digit, then letters, digits, `.`, `_`, `-`, at most 64 characters. No slash,
|
||||
no `..`, no space, no uppercase. The lowercase requirement is where two rules
|
||||
meet: `app.py` would accept `[A-Za-z0-9…]`, but `tts-server` **lowercases
|
||||
names**, so a pack called `Narrator1` resolves through the façade and not
|
||||
through the engine. The boot warmer therefore refuses a mixed-case directory
|
||||
loudly instead of letting it half-work.
|
||||
|
||||
Names are user-visible: an unknown voice comes back as
|
||||
`400 unknown voice 'x'; available voices: …`, and that list is the honest
|
||||
census. The podcast speakers are `narrator1`, `narrator2`, `narrator3` —
|
||||
[[Open Notebook]] is configured with exactly those three, so renaming them
|
||||
breaks episode rendering, not just speech. A personal voice gets the person's
|
||||
name, lowercase.
|
||||
|
||||
## Voice packs
|
||||
|
||||
One directory per voice under `/home/dank/voices`:
|
||||
|
||||
```
|
||||
reference.wav 24 kHz mono PCM, 18-25 s of clean single-speaker audio
|
||||
reference.spk engine speaker embedding (8 192 B)
|
||||
reference.rvq reference codec codes (22 B per 12.5 Hz frame)
|
||||
manifest.json source file, offsets, engine, sha256
|
||||
```
|
||||
|
||||
`reference.wav` is the engine-agnostic term the façade gates on; the other two
|
||||
are what makes a voice reusable by this engine, and they come from the pack
|
||||
builder ([[Voice Packs]]) — never by hand, never fabricated from the WAV. Two
|
||||
laws apply to the
|
||||
source material: offsets always start **past the first 10 seconds** of the
|
||||
sample, and packs are host state that stays out of this repo. Because the
|
||||
container runs as uid 10000, files must be world-readable (`644`/`755`), and the
|
||||
directory must exist before the first deploy: rootless podman auto-creates a
|
||||
missing bind source **root-owned**, which reads as "no voices" rather than as an
|
||||
error.
|
||||
|
||||
Adding a voice to a running stack is a two-step thing — drop the pack, then
|
||||
register **and warm** it with
|
||||
`podman exec frank-tts-engine /app/warm-voices.sh` — because registering alone
|
||||
does not warm it. The reason is the cold-voice law in [[Frank TTS Engine]], and
|
||||
the procedure with its proof (the second request under a second) is the
|
||||
`frank-tts-add-cloned-voice` skill.
|
||||
|
||||
## Why the upstream is a container name
|
||||
|
||||
`scripts/frank-tts.env` used to point at `http://host.containers.internal:8880`.
|
||||
That cannot work, and it is the trap this shape exists to avoid: a bridge
|
||||
container reaches the host's network namespace, netavark DNATs **published**
|
||||
ports only, and an engine on the host's loopback is published to nobody — the
|
||||
same shape that 502'd hermes ([[Container Network Namespaces]]). So both units
|
||||
attach to one committed Quadlet network and the façade dials the engine's
|
||||
container DNS name.
|
||||
|
||||
The consequence worth stating before an operator meets it: **the engine is not
|
||||
reachable from the host at all** — not over the LAN, not over loopback. Its
|
||||
health, voice registry and logs are observable only through
|
||||
`podman exec frank-tts-engine …` and `systemctl --user status`. And the façade's
|
||||
own `/health` answers 200 with the backend name and the voice list it resolved
|
||||
without dialing the engine, so "façade healthy, engine down" is a real state: it
|
||||
shows up as a failed synthesis, never as a down probe.
|
||||
|
||||
## Files
|
||||
|
||||
| Path | What |
|
||||
| --- | --- |
|
||||
| `dankclaw/voiceservice/app.py` | the shim: stdlib only — resolves the voice, forwards, re-encodes mp3 |
|
||||
| `dankclaw/voiceservice/Containerfile` | python 3.12-alpine pinned by digest + ffmpeg, uid 10000 |
|
||||
| `dankclaw/voiceservice/quadlet/frank-tts.container` | the unit: two networks, `Restart=always`, `HealthCmd`, `[Install]` |
|
||||
| `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the shared bridge, named after the engine unit and attached by both |
|
||||
| `scripts/frank-tts.env` | backend, upstream, port, limits, and the default voice — paths and numbers, no credentials |
|
||||
| `tests/frank_tts_unit.bats` | the shim against stub engines over real sockets |
|
||||
| `tests/frank_tts_engine.bats` | the pair's wiring: generator output, planner, homepage, deployer |
|
||||
|
||||
## Deploying it
|
||||
|
||||
`./scripts/deploy.sh dankclaw --service frank-tts` (or `--service frank-tts-engine --service frank-tts` for the pair):
|
||||
builds the façade image, installs **both** Quadlet units — including the
|
||||
engine's `.network` file, because a referenced `Network=` file the generator
|
||||
cannot find means the façade unit is not produced at all — creates
|
||||
`/home/dank/voices` as the deploy user, restarts the unit (whose `Wants=` and
|
||||
`After=` pull the engine and its warm pass ahead of it) and gates on podman's
|
||||
recorded health. It never runs `systemctl --user enable`; boot persistence is
|
||||
the `[Install]` section in each unit. Order matters in a full deploy:
|
||||
[[Frank TTS Engine]] is planned before this, and both before [[Open Notebook]].
|
||||
|
||||
## Not scraped, and why
|
||||
|
||||
`shared/prometheus/` gets **no** frank-tts job, and that is a considered
|
||||
absence. The façade exports no `/metrics` — `app.py` serves `/health` plus the
|
||||
speech route only — and this Prometheus 3 install refuses non-metrics content
|
||||
types: [[Prometheus]] records that the existing HTML probes (`comfyui`, the two
|
||||
`services` targets) sit **down** with
|
||||
`received unsupported Content-Type … and no fallback_scrape_protocol specified`.
|
||||
A `/health` probe job would therefore be structurally never-green, and because
|
||||
`LlmEndpointDown` is scoped `up{job=~"frank-.*"} == 0`, a permanently red
|
||||
`frank-tts` target would page on Signal every few minutes for an endpoint that
|
||||
is fine — the frank-archon lesson pointed the other way. The engine cannot be
|
||||
scraped at all: it publishes nothing. Status lives in `systemctl --user status`,
|
||||
`podman healthcheck run`, the two unit journals, and the homepage card, which
|
||||
probes `127.0.0.1:7772` for real now — the `wip: down` gate came off with the
|
||||
2026-10-18 deploy.
|
||||
|
||||
## The harness speech role
|
||||
|
||||
The operator harness reaches this endpoint through
|
||||
`~/.omp/agent/models.yml` — provider `frank-tts (dankclaw)`,
|
||||
`baseUrl: http://192.168.0.143:7772/v1`, `auth: none`, and the runner api
|
||||
`openai-speech` on the **model** row (naming a runner api at *provider* level
|
||||
fails the whole file's validation and disables every custom provider, which is
|
||||
how the fleet's chat endpoints disappear) — plus `config.yml`
|
||||
`modelRoles.speech: frank-tts (dankclaw)/frank-tts`. Four laws, read out of the
|
||||
harness bundle and measured the day of the flip:
|
||||
|
||||
- **The only voice the harness can send is the `voice_id` argument of the call.**
|
||||
The transport builds `{model, input, response_format, voice?}` and adds `voice`
|
||||
**only** when the caller passed `voice_id`; no setting feeds it.
|
||||
`tts.localVoice` is the *local Kokoro backend's* voice (it is what answered
|
||||
`voice=kokoro/bm_george, backend=local` in the control test), and
|
||||
`speech.voice` is the TUI's **speak-aloud** voice — its enum is Kokoro's list,
|
||||
and it never reaches an HTTP request. So a harness call arrives voice-less by
|
||||
default, which is exactly why the endpoint grew `DEFAULT_VOICE`: the declared
|
||||
pack answers a voice-less request and the journal marks that line `(default)`,
|
||||
while an **unknown** name still 400s with the list that exists.
|
||||
- **`speech.voice` must stay a Kokoro name.** Setting it to a pack name does not
|
||||
select a pack; it hands the vocalizer a value outside its enum. (The first
|
||||
draft of this section said the opposite and was reverted the same day, after a
|
||||
fresh session's call came back `unknown voice None`.)
|
||||
- **There is no silent fallback.** An explicitly configured `modelRoles.speech`
|
||||
yields exactly one candidate, so a dead 7772 fails the speech device with
|
||||
*"No available speech model matches the speech role"* instead of quietly
|
||||
degrading to Kokoro. That is the price of `auth: none` (which makes a dead
|
||||
endpoint read as available) and it is the wanted shape: a narration job should
|
||||
fail loudly rather than ship a voice Frank never asked for.
|
||||
- **A session older than the models.yml row cannot see it.** `models.yml` loads
|
||||
at process start while `config.yml` is read live, so a long-lived session
|
||||
carrying the flipped role fails exactly like a dead endpoint while a fresh
|
||||
process succeeds in the same minute. Measured 2026-10-18: the open session
|
||||
failed twice (provider/id form, then the bare id), `local/kokoro` succeeded in
|
||||
that same session, and a fresh `omp -p --tools tts` returned `Saved 180524
|
||||
bytes (model=frank-tts (dankclaw)/frank-tts, voice=narrator1, codec=wav)`
|
||||
while the façade logged the matching `speech voice=narrator1 format=wav
|
||||
chars=51 -> 180524 bytes` from the podman bridge. Rule the stale registry out
|
||||
before blaming the service.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause |
|
||||
| --- | --- |
|
||||
| `400 unknown voice 'x'; available voices: …` | the name is not a pack: case, spelling, missing `reference.wav`, or an empty/root-owned `/home/dank/voices` |
|
||||
| `(none: no voice pack with a reference.wav in /voices)` | the mount, not the voices: `podman inspect frank-tts --format '{{json .Mounts}}'`, then the directory owner |
|
||||
| Endpoint answers, synthesis fails | the engine: `podman healthcheck run frank-tts-engine`, then `systemctl --user status frank-tts-engine.service` |
|
||||
| First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
|
||||
| The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
|
||||
@@ -0,0 +1,132 @@
|
||||
---
|
||||
type: reference
|
||||
description: "open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out — script written by frank-zealot, voices rendered by frank-tts, no cloud provider anywhere in the path."
|
||||
tags:
|
||||
- dankclaw
|
||||
- open-notebook
|
||||
- podcasts
|
||||
- quadlet
|
||||
timestamp: 2026-10-18
|
||||
---
|
||||
|
||||
# Open Notebook
|
||||
|
||||
open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook)) is the notebook layer that turns existing notes and sources into a **spoken episode**: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by **frank-zealot**, the voices are rendered by **frank-tts**, and nothing leaves the LAN.
|
||||
|
||||
**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Deploying it is a deliberate next step, not a pending one.
|
||||
|
||||
**Host:** Dankclaw (rootless Podman, user `dank`) — **Ports:** 8502 UI and 5056 API, published on the LAN, **no vhost and no Caddy route** — **Deployer:** `open-notebook` (one service, two units) — **Docs:** https://github.com/lfnovo/open-notebook
|
||||
|
||||
## Facts (verified 2026-10-18 against the pinned images, upstream source, and the live host)
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| App image | `docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212` (`open-notebook.container:79`) — the tag upstream's own `docker-compose.yml:27` pins for the multi-container shape. `v1-latest` is a moving tag, so the manifest-list digest (resolved from the registry 2026-10-18) is what makes this reproducible |
|
||||
| Database image | `docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723` (`open-notebook-surrealdb.container:66`) — that digest is **SurrealDB 2.7.0** (image built 2026-09-23; measured by running the digest: `surreal version`) |
|
||||
| Do NOT use | the `-single` tags (`v1-latest-single`). That variant bundles SurrealDB inside the app image and is deprecated upstream; here it would give the host a second, hidden database |
|
||||
| Ports | container 8502 (Next.js UI) + 5055 (FastAPI API) → published `0.0.0.0:8502:8502` and `0.0.0.0:5056:5055` (`open-notebook.container:83-84`). **Host 5055 is taken**: seerr publishes `127.0.0.1:5055` (`dankclaw/seerr/quadlet/seerr.container`, and `ss -ltn` on Dankclaw shows `127.0.0.1:5055` live, 2026-10-18). The API therefore leaves on 5056, which is what forces `API_URL` (below) |
|
||||
| Database port | 8000 inside its own namespace, **never published** (`open-notebook-surrealdb.container` has no `PublishPort`). Upstream publishes `127.0.0.1:8000` for debugging (`docker-compose.yml:18`); this unit does not, and the seam asserts that it stays that way |
|
||||
| Data | `open-notebook-data:/app/data` (named volume: uploads, LangGraph SQLite checkpoints, extraction caches) **plus** the interim host bind `/home/dank/media/podcasts:/app/data/podcasts` (`open-notebook.container:87-88`); database in the `open-notebook-db` named volume |
|
||||
| Networks | app: `dankclaw_default` first, then the private `systemd-open-notebook` (10.89.4.0/24, `open-notebook.network:33`); database: the private bridge only. Order is load-bearing — the generated `--network` flags keep file order, so the default route rides the network where `host.containers.internal` is proven |
|
||||
| Units | `dankclaw/open-notebook/quadlet/open-notebook.container` → `open-notebook.service`; `open-notebook-surrealdb.container` → `open-notebook-surrealdb.service`; both `Restart=always`, `RestartSec=10` / `5`, `[Install] WantedBy=default.target` — boot-persistent per [[Quadlet Lifecycle]] (generated units cannot be `enable`d on this host; the `[Install]` section is what the v1 generator honors at user-manager start) |
|
||||
| Health | app: `curl -fsS http://127.0.0.1:5055/health` (`open-notebook.container:89`); database: `/surreal is-ready` (`open-notebook-surrealdb.container:73`) |
|
||||
| Env | `scripts/open-notebook.env` (repo, knobs) then `/home/dank/Containers/open-notebook/env` (host, secrets, mode 600) — two `EnvironmentFile=` lines per unit, repo file **first** because the last file wins |
|
||||
| Deployer | `deploy_open_notebook` in `scripts/deploy-remote.sh`: seed secrets → ensure both digest pins are in local storage → install both units + the network → `mkdir -p /home/dank/media/podcasts` → restart database, wait for its container health → restart app → `wait_for_health open-notebook http://127.0.0.1:5056/health 180` |
|
||||
|
||||
## What the image actually runs
|
||||
|
||||
One container, three processes under supervisord (`supervisord.conf` upstream): `uvicorn api.main:app --port 5055` (`:12`), the job worker (`:25`), and the Next.js frontend (`:39`). Two consequences the config depends on:
|
||||
|
||||
- **One unit owns the whole app.** There is no separate worker or frontend container to scale or restart, and the frontend refuses to start until the API answers (`wait-for-api.sh && node server.js`, `supervisord.conf:39`) — so gating the API gates the UI.
|
||||
- **The worker's concurrency comes from the environment**, not a flag baked into the image: `--max-tasks ${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5}` (`supervisord.conf:25`). Setting it in the env file is a real change, not decoration.
|
||||
|
||||
Also from the image itself: `WORKDIR /app` and `EXPOSE 8502 5055` (`Dockerfile:84`, `:126`), **no `USER` line** (the app runs as container root, which in this host's rootless map lands as `dank` — the [[Seerr]] UID note measured `0 → 1000` on this machine), and `curl` **and** `ffmpeg` installed in the runtime layer (`Dockerfile:74-76`) — curl makes the health probe possible without a shell wrapper, ffmpeg is how the per-line clips become one file.
|
||||
|
||||
`DATA_FOLDER` is hard-coded `"./data"` (`open_notebook/config.py:5`) and `PODCASTS_FOLDER` is `"./data/podcasts"` (`:19`); the database stores each episode's audio path **relative** to that folder (`episodes/<uuid>/audio/<uuid>.mp3`), and the download route is `GET /api/podcasts/episodes/{id}/audio`. That is why covering one subdirectory with a bind mount is sufficient — nothing in the database has to change if the mount moves.
|
||||
|
||||
## Provisioning the providers (after first boot; UI or REST, never a unit)
|
||||
|
||||
Credentials are stored in the database, encrypted, and are **not expressible in env or a Quadlet file**. The sequence is one-time, in **Manage → Models**, and both entries use provider type **OpenAI Compatible** (upstream: any server that implements `/v1/audio/speech` works this way — `docs/5-CONFIGURATION/local-tts.md:5`).
|
||||
|
||||
1. **TTS credential** — base URL **`http://frank-tts:7772/v1`**. The `/v1` is part of the URL: the client posts to `<base_url>/audio/speech`, so a base URL without `/v1` 404s on every dialogue line (`local-tts.md:72-80`, the base-URL table, always carries `/v1`). `frank-tts` resolves by container name because the app is attached to `dankclaw_default`, the network frank-tts's unit joins (`dankclaw/voiceservice/quadlet/frank-tts.container:51-52`). If frank-tts is ever moved off that network, the published-port form is `http://192.168.0.143:7772/v1` — a published port is dialable at the host's LAN address from any netns. The API-key field stays empty; frank-tts does not authenticate.
|
||||
2. **LLM credential** — base URL **`http://host.containers.internal:7771/v1`**, model row `frank-zealot`. frank-zealot is a **host process** on `0.0.0.0:7771` (`ss -ltn` on Dankclaw shows the listener with no container owning it, 2026-10-18), and on this netavark host a bridge container reaches the host's LAN IP only at a *published* port — which is why Prometheus scrapes the same endpoint as `host.containers.internal:7771` (`shared/prometheus/prometheus.yml:89-111`, "not their own LAN addresses"). See [[Container Network Namespaces]]. `http://192.168.0.143:7771/v1` is the variant to try only if the gateway name ever stops resolving; it is expected to be refused from a bridge netns.
|
||||
3. **Model rows are created by name.** Discovery is optional (see the next section): type the model id and confirm the add. One base URL per credential, so the TTS and LLM credentials are necessarily two separate configurations.
|
||||
|
||||
**Verify reachability before saving**, from inside the container, because both addresses are netns-dependent:
|
||||
|
||||
```bash
|
||||
podman exec open-notebook curl -sS http://frank-tts:7772/health
|
||||
podman exec open-notebook curl -sS http://host.containers.internal:7771/v1/models
|
||||
```
|
||||
|
||||
**The blocker moved; it did not vanish.** frank-tts and frank-tts-engine are **deployed** (2026-10-18) and answer on 7772, so the voice leg works end to end — `podman exec open-notebook curl -sS http://frank-tts:7772/health` is the check to run the day the app exists. What still blocks a render is open-notebook itself: not installed on this host yet. When it is, the two provider rows above are the only wiring, and the seeded speaker profiles carrying OpenAI voice names are the first trap.
|
||||
|
||||
**Why frank-zealot and not frank-dragoon:** podcast scriptwriting asks the model for a JSON-schema response (with a plain-JSON fallback). llama.cpp's OpenAI-compatible endpoint is a known-good target for that; frank-dragoon's halogen runtime has **unverified** structured-output behaviour, so it is the unproven alternative, not the fallback. `ESPERANTO_LLM_TIMEOUT=180` (the app's own default) covers an outline or transcript call on zealot comfortably; upstream's note is to keep it under 600 so a slow call fails with a clear error instead of a UI timeout.
|
||||
|
||||
## Speakers and voices: 1–4, and the IDs are free text
|
||||
|
||||
A **speaker profile** carries one to four speakers (`docs/3-USER-GUIDE/creating-podcasts.md:27`), each with a personality and a **Voice ID that is typed exactly and passed through to the TTS request untouched** (`:29`) — there is no whitelist, no enum, no validation against a voice list. So the fleet's voice-pack names work verbatim:
|
||||
|
||||
| Speaker | Voice ID to type |
|
||||
|---------|------------------|
|
||||
| Speaker 1 | `narrator1` |
|
||||
| Speaker 2 | `narrator2` |
|
||||
| Speaker 3 | `narrator3` |
|
||||
|
||||
Those are the pack names frank-tts resolves from `/home/dank/voices` (`scripts/frank-tts.env` header: a voice is real exactly when `/voices/<name>/reference.wav` exists). Two traps, both from upstream's shipped defaults:
|
||||
|
||||
- The three seeded speaker profiles carry **OpenAI voice names** (`nova`, `alloy`, …) and **no voice model at all** — they show a "Setup required" badge, and generation fails with a voice error until they are edited (`creating-podcasts.md:13`, `:33`; `local-tts.md:89`). Add a `narrator1`/`narrator2` pair (or a third/fourth) rather than editing the seeded ones in place.
|
||||
- Two speakers with the same voice are hard to follow (`creating-podcasts.md:89`). Distinct packs per speaker is the point of the 1–4 shape.
|
||||
|
||||
## Two buttons that fail against frank-tts, and why that is cosmetic
|
||||
|
||||
- **Test Connection** probes `GET <base_url>/models` (`open_notebook/ai/connection_tester.py:208-229`). frank-tts serves exactly two GET routes — `/health` and `/v1/audio/voices` — and 404s every other path (`dankclaw/voiceservice/app.py:228-238`). The button reports a failure against a healthy server.
|
||||
- **Discover Models** does the same `GET <base>/models` (`open_notebook/ai/model_discovery.py:26-28`) and finds nothing to list.
|
||||
- The TTS test path also synthesises with a **hard-coded voice `alloy`** for `openai_compatible` (`connection_tester.py:328-336`), which is not a fleet voice pack and would be refused even if the probe reached frank-tts.
|
||||
|
||||
None of it blocks anything: model rows are added **by name** ("If a model isn't listed, type its id and click Add", `local-tts.md:82`) and the Voice ID field is free text. The failure is a missing `/models` route on a deliberately minimal shim, not a broken credential. The honest end-to-end test is a real generation job, or the `podman exec … curl /v1/audio/speech` call frank-tts's own seam documents.
|
||||
|
||||
## The CPU guard, and why both numbers are 1
|
||||
|
||||
Defaults are `TTS_BATCH_SIZE=5` and `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` (`docs/5-CONFIGURATION/environment-reference.md:129-130`) — five parallel synthesis requests from five parallel jobs. frank-tts spawns an engine process per request (about 8 GiB resident, 8 threads, from the frank-tts CPU trial of 2026-10-18; `scripts/frank-tts.env` pins `LLAMA_TTS_THREADS=8`), on a machine that also serves frank-dragoon and frank-zealot. Five of those is ~40 GiB and forty threads competing with two LLM endpoints for the same cores. `scripts/open-notebook.env:87-88` sets both to 1: one episode at a time, one voice line at a time. Upstream's own guidance is the same direction ("Set 1 for a local model on a single GPU", "Lower it … for rate-limited or single-threaded TTS servers").
|
||||
|
||||
The clock next to it: `ESPERANTO_TTS_TIMEOUT` (`:91`) is patience **per dialogue line**, not per episode — a three-minute episode is dozens of separate POSTs, and the tail of a long turn on a CPU engine goes past upstream's 300 s default. It is set to **660, one round-trip outside `SHIM_TIMEOUT=600`** in `scripts/frank-tts.env` (frank-tts's own per-request ceiling): nested budgets go inside-out — the same rule that file states for `LLAMA_TIMEOUT` — so a frank-tts give-up arrives labelled as frank-tts's error instead of the worker's own timeout, and re-measuring frank-tts cannot silently strand this number below it. An abandoned request does not cancel the engine process; it only strands the episode in "generating".
|
||||
|
||||
## Secrets, and the difference between blank and unset
|
||||
|
||||
`OPEN_NOTEBOOK_PASSWORD` unset means **authentication off** (`environment-reference.md:46`) — every API route, including the episode-audio download, answers anyone who can reach the published port. That is a real exposure on a LAN-published service with no vhost and no guard chain in front of it, which is why `deploy_open_notebook` refuses to start either unit unless the seeded host file carries a value for all four secret keys (`OPEN_NOTEBOOK_ENCRYPTION_KEY`, `OPEN_NOTEBOOK_PASSWORD`, `SURREAL_PASSWORD`, `SURREAL_PASS`). `OPEN_NOTEBOOK_ENCRYPTION_KEY` is separately load-bearing: it is **required to save any provider credential**, and changing it later makes stored keys undecryptable (`:44`).
|
||||
|
||||
The tracked file declares the secrets **empty** and the host file supplies them; podman applies `--env-file` arguments in order and the last one wins (verified against the generated `ExecStart` line, 2026-10-18). Reordering the two `EnvironmentFile=` lines in a unit would blank every credential — that is the trap the seam's ordering assertion exists for.
|
||||
|
||||
The database has the same trap in a different spelling: the app reads `SURREAL_PASSWORD`, the `surreal` CLI reads `SURREAL_PASS` (its `start --help` prints `[env: SURREAL_PASS=]`; measured against the pinned digest). One value, written twice, by the seed function — so the pair cannot be given different root passwords. Both arrive through the environment, **not** through `Exec=`: `--pass` on the command line would put the database password in `/proc/<pid>/cmdline`, readable by any `ps` on the host. Measured with the pinned image: env-only credentials produce `Initialising credentials user=…`, the root user is created on first boot, a correct login authenticates and a wrong password is refused — no `--user`/`--pass` flags needed, and no `--bind` either, because the image's own `SURREAL_BIND=0.0.0.0:8000` is what makes the RPC reachable from another container at all.
|
||||
|
||||
## Episode destination: interim, and reversible (issue #7)
|
||||
|
||||
**Episode files currently exist in two places by design.** The app's own state stays in the `open-notebook-data` named volume; finished audio additionally lands on the host, because `/home/dank/media/podcasts` is bind-mounted over `/app/data/podcasts` (`open-notebook.container:88`). That is the **interim** answer to where episodes live: the destination is an open decision tracked as **Gitea issue #7, labelled `human-only` — agents must not work it.** Until Frank settles it, episode MP3s are ordinary files in his home directory, readable without `podman unshare`, which the database volume would require (measured with the pinned SurrealDB image).
|
||||
|
||||
Reversibility is one line: delete the `Volume=/home/dank/media/podcasts:/app/data/podcasts` line and the episodes go back to living solely inside the named volume. Nothing else in the config assumes the bind. See [[Storage Policy]] for the repo's storage conventions this interim state deliberately bends.
|
||||
|
||||
## Deploying, and the digest-pull step
|
||||
|
||||
```bash
|
||||
# on Dankclaw, as dank — the pins are Pull=never, so fetch them first
|
||||
podman pull docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
|
||||
podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
|
||||
```
|
||||
|
||||
`deploy_open_notebook` runs the same two pulls when the image is absent from local storage, so the manual step is only needed to pre-seed an offline host or to bump a pin. Then `scripts/deploy.sh --target dankclaw --service open-notebook`. Both `Pull=never` and the pin comment live in the unit files; upgrading means **editing the digest**, not repulling a tag — see [[Deployment Flow]].
|
||||
|
||||
## Rollback
|
||||
|
||||
- **Nothing deployed (today's state):** revert the commits. There is no host state to undo — no unit, no container, no volume, no `/home/dank/media/podcasts` directory beyond what the deployer would create.
|
||||
- **Deployed:** `systemctl --user stop open-notebook.service open-notebook-surrealdb.service`, then move the three files (`open-notebook.container`, `open-notebook-surrealdb.container`, `open-notebook.network`) out of `~/.config/containers/systemd/` into `~/.AGENTTRASH/` — never `rm` — and `systemctl --user daemon-reload`. The generated units disappear with their sources.
|
||||
- **Kept on purpose:** the `open-notebook-data` and `open-notebook-db` named volumes and `/home/dank/media/podcasts`. Episode audio and the notebook database are user data; removing them is a separate, deliberate act, and episode files are host files precisely so they can outlive the container.
|
||||
- **Homepage:** the card is `wip: "down"` + `showLink: false`, so retiring the service needs no card surgery beyond setting it back to that state.
|
||||
|
||||
## Citations
|
||||
|
||||
- Repo: `dankclaw/open-notebook/quadlet/{open-notebook.container,open-notebook-surrealdb.container,open-notebook.network}`, `scripts/open-notebook.env`, `dankclaw/.env.example`, `scripts/deploy-plan.py`, `scripts/deploy-remote.sh` (`seed_open_notebook_env`, `ensure_pinned_image`, `deploy_open_notebook`), `scripts/deploy.sh`, `scripts/test-changed.sh`, `tests/open_notebook.bats`, `shared/homepage/services.json`, `dankclaw/seerr/quadlet/seerr.container` (the 5055 collision), `dankclaw/voiceservice/{quadlet/frank-tts.container,app.py}`, `scripts/frank-tts.env`, `shared/prometheus/prometheus.yml`
|
||||
- Upstream `lfnovo/open-notebook` @ main, read 2026-10-18: `docker-compose.yml:3,10,11,18,27,29-30,39-43`, `Dockerfile:69-76,84,126,140,158`, `supervisord.conf:12,23-25,39`, `api/main.py:73,227-230,239-241,414`, `api/credentials_service.py:418-420,552`, `api/routers/credentials.py:426-429,433-437`, `open_notebook/config.py:5,19`, `open_notebook/podcasts/audio_paths.py`, `open_notebook/ai/connection_tester.py:208-238,326-336`, `open_notebook/ai/model_discovery.py:26-28`, `docs/3-USER-GUIDE/creating-podcasts.md:13,27,29,33,89`, `docs/5-CONFIGURATION/environment-reference.md:44,46,106,110,129,130`, `docs/5-CONFIGURATION/local-tts.md:5,45,72-82,89`
|
||||
- Registry (2026-10-18): `lfnovo/open_notebook` tags (`v1-latest`, `latest`, the `-single` set) and manifest-list digests; `surrealdb/surrealdb` `v2` → 2.7.0
|
||||
- Local measurements (2026-10-18, workstation, pinned digests): `surreal start --help` (credential and bind options carry `[env: SURREAL_*]`); running the pinned SurrealDB digest with env-only credentials (credential init in the log, correct login authenticates, wrong password refused, `/surreal is-ready` → `OK` exit 0, named volume owned by the mapped uid); the v1-compatible Quadlet generator dry-run of all three unit files (two `--env-file` in order, two `--network` in order, named volume + host bind, both health probes, `--pull never`)
|
||||
- Host evidence (Dankclaw, 2026-10-18, read-only): `ss -ltn` (`127.0.0.1:5055` held by seerr, `0.0.0.0:7771` a host listener, nothing on 7772/8502/5056/8000), `podman ps -a` (no open-notebook container, `dankclaw_default` present), subnet registry from `dankclaw/open-webui/quadlet/open-webui.network`
|
||||
@@ -0,0 +1,261 @@
|
||||
---
|
||||
type: reference
|
||||
description: "How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material gate scripts/build-voice-pack.sh enforces, how to add a fourth voice, and what to do when a pinned engine bump changes the extractor's output sizes."
|
||||
tags:
|
||||
- dankclaw
|
||||
- frank-tts
|
||||
- voices
|
||||
- tts
|
||||
timestamp: 2026-10-18
|
||||
---
|
||||
|
||||
# Voice Packs
|
||||
|
||||
A **voice pack** is the unit of identity for **frank-tts**: one directory under
|
||||
`/home/dank/voices`, named by the voice, carrying the reference audio that makes
|
||||
the name real plus the pinned engine's own speaker artifacts and a provenance
|
||||
record. `voice=narrator1` on `POST /v1/audio/speech` resolves to
|
||||
`/home/dank/voices/narrator1/reference.wav` and to nothing else — a name without
|
||||
that file is a 400 that lists the names that do exist, because a silent fallback
|
||||
voice is how a wrong-voice endpoint stays wrong for weeks.
|
||||
|
||||
**Host:** Dankclaw (192.168.0.143) — **Pack root:** `/home/dank/voices`, host
|
||||
state that stays out of the repo, mounted read-only at `/voices` by the façade
|
||||
unit — **Builder:** `scripts/build-voice-pack.sh` with tunables in
|
||||
`scripts/voice-packs.env` — **Seam:** `tests/voice_pack_unit.bats` — **Engine:**
|
||||
arm A `qwentts.cpp`, decided in [[Local TTS and Voice Cloning]].
|
||||
|
||||
Packs are **world-readable on purpose** (`drwxr-xr-x`, owner `dank`): the façade
|
||||
container runs as uid 10000 and reads the mount from outside its own uid map.
|
||||
|
||||
## The required files
|
||||
|
||||
| File | What it is | Who reads it |
|
||||
|------|------------|--------------|
|
||||
| `reference.wav` | the crop: 24 kHz mono `pcm_s16le`, **18–25 s** of clean single-speaker audio | the façade (it is the existence gate for the name) and the engine's boot warmer, which base64s it and registers the voice with no transcript |
|
||||
| `reference.spk` | the engine's x-vector: **8,192 B** = `enc_dim` 2048 × f32 | the engine, for the ICL/`--ref-spk` path |
|
||||
| `reference.rvq` | the engine's RVQ code stream: **5,456–6,864 B**, 16 codebooks × T frames packed at 11 bits = 22 B per 12.5 Hz frame | the engine, for the ICL/`--ref-rvq` path |
|
||||
| `manifest.json` | provenance: source path and sha256, crop start/end offsets, sample rate, duration, the tool and engine commit that made the artifacts, and the `voice` name | humans, and the builder itself on its second gate assertion |
|
||||
|
||||
`qwen-codec.log` rides along with the extractor's own record of the run; the
|
||||
façade ignores anything it does not name and never deletes anything.
|
||||
|
||||
Division of enforcement, which is deliberate and not a gap: the **façade** gates
|
||||
on `reference.wav` because that is the only engine-agnostic term (see the
|
||||
voice-pack law in `scripts/frank-tts.env`); the **builder** is what refuses to
|
||||
emit a pack without its engine artifacts; the **engine** registers a voice from
|
||||
the WAV alone because its own `POST /v1/audio/voices` refuses a lone embedding —
|
||||
it wants `wav_b64` or **both** `spk_b64` and `rvq_b64`. So the artifacts are in
|
||||
the pack to make ICL a per-voice opt-in, not to boot it.
|
||||
|
||||
## How a pack is built
|
||||
|
||||
```bash
|
||||
# everything except the extractor: gate, crop, probe, plan, the manifest it would write
|
||||
scripts/build-voice-pack.sh --dry-run narrator1 /home/dank/voices
|
||||
|
||||
# the real thing, on the host that holds the engine build
|
||||
scripts/build-voice-pack.sh narrator1 /home/dank/voices
|
||||
```
|
||||
|
||||
Crops come from `scripts/voice-packs.env` — per voice: source path, offset,
|
||||
duration, and the source's pinned sha256 — and any of them can be overridden on
|
||||
the command line (`--source`, `--offset`, `--duration`, `--sha256`, `--tool`).
|
||||
`--all` walks `VOICE_PACKS`. The builder assumes no GPU, no container, no fleet
|
||||
endpoint and no network: `ffmpeg` cuts the crop, the pinned extractor makes the
|
||||
artifacts, and both run `nice -n 19` so a build never competes with the text
|
||||
endpoints on that box.
|
||||
|
||||
The extractor is `qwen-codec` in the `qwentts.cpp` tree at
|
||||
`VOICE_PACK_ENGINE_COMMIT` (`5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`). Its
|
||||
usage text, read out of that tree and not guessed:
|
||||
|
||||
```
|
||||
Usage: qwen-codec --model <gguf> [-i <input>] [--talker <gguf>] [--format <fmt>]
|
||||
|
||||
Required:
|
||||
--model <gguf> Codec GGUF (qwen-tokenizer-12hz-*.gguf)
|
||||
|
||||
Optional:
|
||||
--device <name> Compute device: CUDA0, Vulkan0, MTL0, CPU...
|
||||
(default: best available)
|
||||
-i <path> Input. WAV -> encode, .rvq -> decode
|
||||
--talker <gguf> Talker GGUF (Base only). Encode also extracts the speaker
|
||||
embedding and writes it next to the .rvq as a .spk file
|
||||
--format <fmt> WAV output format: wav16, wav24, wav32 (default: wav16)
|
||||
|
||||
Output is auto-named next to input : clip.wav -> clip.rvq, clip.rvq -> clip.wav.
|
||||
```
|
||||
|
||||
Three consequences shape the builder. **There is no `-o`**, so the pack's WAV is
|
||||
staged under its final stem and the artifacts land beside it with nothing to
|
||||
rename. **`--talker` must be a _Base_ talker** — the customvoice and voicedesign
|
||||
GGUFs carry no speaker encoder and the tool says `has no speaker encoder (Base
|
||||
only)`. **Encode truncates to the hop boundary**, so the `.rvq` size is arithmetic,
|
||||
not chance: `22 × floor(samples / 1920)` bytes, which is where the recorded
|
||||
5,456–6,864 B window comes from and why the **effective crop window is
|
||||
19.84–25 s** even though the WAV law says 18–25 s. A crop shorter than that is
|
||||
refused before the extractor runs, with the number in the message.
|
||||
|
||||
The build is staged and published, never assembled in place: `reference.wav` is
|
||||
the file the façade reads, so it moves into position **last**, after the
|
||||
artifacts and the manifest. A failed extractor therefore leaves a pack directory
|
||||
that is not a voice. Nothing is ever deleted — an existing pack is refused unless
|
||||
`--force`, and `--force` moves the whole old directory to
|
||||
`/home/dank/voices/.AGENTTRASH/<voice>-<UTC>/` and builds fresh beside it.
|
||||
|
||||
Measured on the shipped `narrator1` crop (2026-10-18, `--dry-run` against the
|
||||
real source): 98.739–118.636 s → `955,378 B` of 24 kHz mono `pcm_s16le`, 19.897
|
||||
s, planned `.rvq` 5,456 B — the same byte counts the trial produced by hand, and
|
||||
the extractor itself costs 1.8–2.1 s and 1.3–1.5 GiB peak per voice.
|
||||
|
||||
## The 10-second gate
|
||||
|
||||
Project-wide source-material invariant: **the first 10 seconds of any sample
|
||||
file used to generate voices is forbidden material.** An offset that starts
|
||||
inside it is refused with exit 1 and a message that names the invariant, before
|
||||
a directory is created:
|
||||
|
||||
```
|
||||
ERROR: source-material invariant violated: the crop must start past the first
|
||||
10 s of the source file, and 'narrator1' asks for 4.5 s into … — nothing was
|
||||
written.
|
||||
```
|
||||
|
||||
The same gate is then asserted **from the manifest** that was just written —
|
||||
`.crop.offset_seconds >= .invariant.forbidden_source_seconds`, plus the recorded
|
||||
sha256 against the source it claims — so a pack cannot reach disk carrying a
|
||||
forbidden crop even if the offsets in the env file are edited into nonsense.
|
||||
`10` is not a knob: `scripts/voice-packs.env` carries the offsets, the sample
|
||||
rate, the windows and the engine facts, but the forbidden region is a constant in
|
||||
the builder. `tests/voice_pack_unit.bats` pins both halves (a refusal that
|
||||
creates nothing, and one legal crop that builds the whole pack) and applies the
|
||||
invariant to every crop in the tracked pack list.
|
||||
|
||||
## Adding a fourth voice
|
||||
|
||||
1. Pick the source file and hash it: `sha256sum /path/to/sample.wav`. A crop
|
||||
offset is a claim about one exact file, so the digest goes in the env with it.
|
||||
2. Find a speech-dense stretch **past 10 s**. The trial's recipe, applied once:
|
||||
`ffmpeg -ss 60 -t 600 -i SRC -af silencedetect=noise=-35dB:d=0.6 -f null -`,
|
||||
build the runs of non-silence, keep runs ≥ 18 s, take the longest (earliest
|
||||
on ties), and cap at 25 s. Aim for 20–25 s: shorter than 19.84 s and the
|
||||
`.rvq` falls under the floor.
|
||||
3. Add the four lines to `scripts/voice-packs.env` and the name to
|
||||
`VOICE_PACKS`:
|
||||
|
||||
```
|
||||
VOICE_PACKS="narrator1 narrator2 narrator3 narrator4"
|
||||
|
||||
VOICE_PACK_NARRATOR4_SOURCE=/home/dank/voice-sources/audiosample4.m4b
|
||||
VOICE_PACK_NARRATOR4_OFFSET=142.31
|
||||
VOICE_PACK_NARRATOR4_DURATION=24
|
||||
VOICE_PACK_NARRATOR4_SHA256=<64 hex>
|
||||
```
|
||||
|
||||
The name must be lowercase `[a-z0-9._-]`: the façade matches the directory
|
||||
name verbatim while `tts-server` lowercases names internally, so an uppercase
|
||||
name would be reported one way by the façade and another by the engine. The
|
||||
builder refuses one anyway.
|
||||
4. `scripts/build-voice-pack.sh --dry-run narrator4 /home/dank/voices`, read the
|
||||
plan, then drop `--dry-run`.
|
||||
5. `./tests/run.sh` (the pack-list invariant test covers the new row for free).
|
||||
6. Register it with the running engine. The engine's voice registry is
|
||||
memory-only and warm-on-boot, but a restart re-decodes **every** voice (6–8 s
|
||||
each), so adding one pack uses the hook instead:
|
||||
|
||||
```bash
|
||||
podman exec frank-tts-engine /app/warm-voices.sh
|
||||
curl -s localhost:7772/v1/audio/voices # the façade's list, not the engine's
|
||||
```
|
||||
|
||||
The hook walks `/voices` and registers **every** pack that has a
|
||||
`reference.wav`, on every run — the engine replaces a name it already holds
|
||||
(`tools/tts-server.cpp:148` at the pinned commit: *"Re-registering a name
|
||||
replaces the previous entry."*) — then synthesizes twice per voice and logs
|
||||
`voice <name> cold=…ms warm=…ms` to the unit's journal. The two numbers should
|
||||
be close; see law 1 of [[Frank TTS Engine]] for what they measure. The rootless
|
||||
unit restart — `systemctl --user restart frank-tts-engine.service` — is for when
|
||||
the unit or its env changed, not for a new pack.
|
||||
|
||||
## When the extractor's output size changes on an engine bump
|
||||
|
||||
The builder treats the two artifact sizes as properties of the pinned engine, so
|
||||
an engine bump that changes them **fails the build**: `reference.spk is N B,
|
||||
expected 8192 B` or `reference.rvq is N B, expected …`. That is the failure being
|
||||
loud, not a bug. Work through it in order:
|
||||
|
||||
1. Re-read the tool's own numbers rather than trusting this page: `qwen-codec`
|
||||
prints `SpeakerEncoder] Loaded: enc_dim=…` and `Pipeline] Ready: hop … , …
|
||||
codebooks @ … Hz` on every run, and the pinned tree's
|
||||
`tools/qwen-codec.cpp` documents the packing. `enc_dim × 4` is the `.spk`;
|
||||
`codebooks × code_bits / 8` bytes per frame × frames is the `.rvq`.
|
||||
2. Decide whether the new number is intended. A changed `enc_dim` means the old
|
||||
`.spk` files are unreadable by the new engine, so **every** pack must be
|
||||
rebuilt, not just the failing one.
|
||||
3. Update the four arithmetic lines in `scripts/voice-packs.env`
|
||||
(`VOICE_PACK_SPK_BYTES`, `VOICE_PACK_RVQ_MIN_BYTES`, `VOICE_PACK_RVQ_MAX_BYTES`,
|
||||
and `VOICE_PACK_RVQ_BYTES_PER_FRAME` if the codebook packing changed) **and**
|
||||
`VOICE_PACK_ENGINE_COMMIT`, `VOICE_PACK_TOOL`, `VOICE_PACK_CODEC_MODEL`,
|
||||
`VOICE_PACK_TALKER_MODEL` to the new build. An env diff that moves the sizes
|
||||
without moving the commit is the mistake to avoid.
|
||||
4. Rebuild all three packs with `--force` (the retired directories keep the old
|
||||
bytes readable beside their replacements until a human clears them), then
|
||||
re-run the warm hook: `podman exec frank-tts-engine /app/warm-voices.sh`. That
|
||||
hook registers **every** pack it finds on every run, and the engine replaces a
|
||||
name it already holds — `tools/tts-server.cpp:148` at the pinned commit says so
|
||||
in its own words: *"Re-registering a name replaces the previous entry."* So the
|
||||
rebuilt `reference.wav` is what the running engine speaks afterwards, and the
|
||||
hook's cold/warm pass is what re-snapshots it. A restart
|
||||
(`systemctl --user restart frank-tts-engine.service`) is what an ENGINE bump
|
||||
needs — the registry is memory-only and lives in that process — and the unit
|
||||
runs the same hook on every start anyway, so a restart is never worse, only
|
||||
slower.
|
||||
5. `./tests/voice_pack_unit.bats` — the stub in that seam reproduces the packing
|
||||
law, so it needs the same arithmetic update as the env file. The seam's
|
||||
engine-bump test is exactly this scenario with a wrong `.spk` size, and its
|
||||
assertion — no `reference.wav`, no manifest — is the property that matters.
|
||||
|
||||
## Host state (census 2026-10-18, read-only)
|
||||
|
||||
- `/home/dank/voices/` holds `narrator1`, `narrator2`, `narrator3`, each with
|
||||
`reference.wav`, a `reference16k.wav`, and a `manifest.json` **in the trial's
|
||||
own schema** (`kind: voice-reference-pack`, a silence-scan record). They carry
|
||||
**no** `reference.spk` / `reference.rvq`: the trial's artifacts live in its own
|
||||
work directory (`/home/dank/voice-audition/qwentts/voices/`, and only for
|
||||
narrator1 and narrator2). The builder's first real run is therefore a
|
||||
migration, and it should be a `--force` run per voice so the old directories
|
||||
are retired rather than left half-updated.
|
||||
- The three source files (`/home/blank/audiosample.mp3`,
|
||||
`audiosample2.m4b`, `audiosample3.m4b`, digests pinned in
|
||||
`scripts/voice-packs.env`) live on the workstation, not on Dankclaw; the
|
||||
builder reads them from `$VOICE_PACK_SOURCE_DIR=/home/dank/voice-sources`, so
|
||||
they have to be copied there once before a build.
|
||||
- `VOICE_PACK_TOOL` points at the trial's build tree
|
||||
(`/home/dank/voice-audition/qwentts/qwentts.cpp/build/qwen-codec`, checked out
|
||||
at exactly `5916bf7c`), and the two GGUFs at
|
||||
`/home/dank/models/Qwen3-TTS-GGUF/`, which is where the engine slice's
|
||||
download script puts them — that directory does not exist yet, so until it
|
||||
does the models have to come from the trial's `models/` directory.
|
||||
- Nothing is deployed: nothing listens on 7772 or 8880 today, so building packs
|
||||
is possible on that host while serving them is not.
|
||||
|
||||
## Known gaps, on purpose
|
||||
|
||||
- **No transcript in the manifest.** ICL wants a `ref_text` that matches the
|
||||
crop, and the trial's machine transcripts came back wrong ("Demon Souls" for
|
||||
*Dark Souls*). Transcribing on the host is not a fix, so the pack carries none
|
||||
and the engine registers the WAV-only voice, which is what its boot warmer
|
||||
does anyway. When ICL becomes a per-voice opt-in, the curated transcript
|
||||
belongs in this manifest — a human-supplied field, not something the builder
|
||||
can derive.
|
||||
- **No 16 kHz variant.** The packs the trial left behind carry
|
||||
`reference16k.wav` for the `llama-tts-cli` rollback engine; arm A is native
|
||||
24 kHz, so the builder emits one reference. `--force` retires the whole
|
||||
directory including that variant, so if the rollback engine is ever revived:
|
||||
`ffmpeg -i reference.wav -ar 16000 -c:a pcm_s16le reference16k.wav`.
|
||||
- **No loudness pass.** The trial's pipeline also measured and normalised
|
||||
(two-pass `loudnorm`, `astats`, `ebur128`, spectrograms) to a `-16 LUFS` spec.
|
||||
None of that is in the pack law, which asks for 24 kHz mono inside the
|
||||
duration window, so the builder does only that; a future engine that is
|
||||
loudness-sensitive should get the pass as an explicit, documented step.
|
||||
@@ -116,6 +116,34 @@ On a later ordinary deployment, a service remains pending only while its
|
||||
recorded revision differs from the checkout and the planner still finds a
|
||||
change in that service's inputs.
|
||||
|
||||
## Locally built images
|
||||
|
||||
`build_compose_image` in `scripts/deploy-remote.sh` builds the images that have
|
||||
no registry digest — `comfyui-rocm`, `franksdank/loki`, `franksdank/alloy` and
|
||||
the voice pair `franksdank/frank-tts` / `franksdank/frank-tts-engine` — and
|
||||
**skips** the build when a `cfhash` label on the installed image matches a hash
|
||||
it computes from the checkout now. That skip is what keeps the engine's
|
||||
ten-minute `git clone --recursive` off every re-deploy, so the rule has to be
|
||||
honest about what the image bakes:
|
||||
|
||||
- The hash covers the `Containerfile` **plus the files its `COPY` lines pull
|
||||
in**, named as `extra=(…)` in each case arm. Hashing the `Containerfile`
|
||||
alone was wrong in practice: editing `dankclaw/voiceservice/app.py` or
|
||||
`tts-engine-warm.sh` left the label matching, `podman build` never ran, and
|
||||
the redeployed container still carried the previous code while `/health`
|
||||
answered — invisible to every gate. Corrected 2026-10-18;
|
||||
`tests/frank_tts_engine.bats` drives the function against a stub `podman`
|
||||
(first deploy builds, unchanged tree does not, a `COPY`'d file rebuilds).
|
||||
- A base-image assumption is part of the image's identity too. The engine's
|
||||
runtime stage creates its unprivileged user with `useradd`, not `adduser`,
|
||||
because the pinned `ubuntu:24.04` base ships no `adduser` — checked by
|
||||
running the command against the digest on 2026-10-18. That failure mode is a
|
||||
build that dies at the `RUN` line, so it is loud, unlike the stale-hash one.
|
||||
|
||||
Both engines keep serving during a rebuild: `build_compose_image` runs after
|
||||
`install_quadlet` only for the LLM-style handlers, and the voice deploy gates on
|
||||
recorded health rather than a fixed sleep — see [[Frank TTS Engine]].
|
||||
|
||||
## Manual recovery
|
||||
|
||||
If the LLM handler fails after the target pull:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: research
|
||||
title: Local TTS and Voice Cloning
|
||||
description: How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the full local AI audio stack, and a recommended homelab deployment.
|
||||
description: How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the measured CPU engine A/B behind frank-tts and the arm A decision, the full local AI audio stack, and a recommended homelab deployment.
|
||||
tags:
|
||||
- research
|
||||
- tts
|
||||
@@ -10,7 +10,7 @@ tags:
|
||||
- stt
|
||||
- vllm-omni
|
||||
- kokoro
|
||||
timestamp: 2026-10-18T00:00:00Z
|
||||
timestamp: 2026-10-18T16:05:00Z
|
||||
---
|
||||
|
||||
# Local TTS and Voice Cloning
|
||||
@@ -47,13 +47,16 @@ greenfield: every fleet endpoint declares `input: [text, image]`
|
||||
local server needs a speech-role entry, not a patch.
|
||||
- **Fleet plan (2026-10-18): CPU-only on Dankclaw, not the 3090.** Gitea issue
|
||||
#2 rules out every GPU option (the 3090 has carried frank-zealot since
|
||||
2026-10-17; both R9700s are frank-archon's), so the shape is a VoxCPM2 GGUF
|
||||
engine on **8880** behind a voice-registry shim on **8890**
|
||||
(`scripts/voiceshim.env`, `dankclaw/voiceservice/`) that speaks OpenAI
|
||||
`/v1/audio/speech` and resolves voices by name. Nothing is deployed yet — on
|
||||
2026-10-18 `voiceshim` has no unit installed and nothing answers on 8880 or
|
||||
8890. STT (Parakeet TDT 1.1B, CC-BY-4.0) stays on CPU. **No RTF or quality
|
||||
number has been measured for this shape**, so this page states none.
|
||||
2026-10-17; both R9700s are frank-archon's). The shape is a TTS engine on
|
||||
**8880** behind **frank-tts**, the fleet's speech endpoint on **7772**
|
||||
(`scripts/frank-tts.env`, `dankclaw/voiceservice/`): one OpenAI
|
||||
`/v1/audio/speech` façade that resolves a voice *by name* and never shows a
|
||||
path. Nothing is deployed — no `frank-tts` unit and no container exist on the
|
||||
host and nothing listens on 7772 or 8880 (checked 2026-10-18). STT (Parakeet
|
||||
TDT 1.1B, CC-BY-4.0) stays on CPU. **The engine is chosen: arm A**
|
||||
`qwentts.cpp` **resident `tts-server`, 1.7 B talker** — measured in
|
||||
*Measured CPU engine A/B* below and picked on the live-output criterion in
|
||||
[Decision: arm A for live output](#decision-arm-a-for-live-output-2026-10-18).
|
||||
|
||||
## Can Kokoro learn a new voice?
|
||||
|
||||
@@ -133,11 +136,11 @@ accepts only its built-in voice list, so a custom pack needs a patch.
|
||||
|
||||
Paths for a Frank-cloned voice (2026-10-18):
|
||||
|
||||
1. **Local server + speech role, no patch.** Register the local engine (or the
|
||||
8890 voice shim) as a provider model on the `openai-speech` API and point
|
||||
`modelRoles.speech` at it; the shim turns a voice *name* into a reference
|
||||
clip and forwards it as base64 `reference_audio`
|
||||
(`scripts/voiceshim.env:19-35`). Not deployed as of 2026-10-18.
|
||||
1. **Local server + speech role, no patch.** Register frank-tts (port 7772) as
|
||||
a provider model on the `openai-speech` API and point `modelRoles.speech` at
|
||||
it; the façade turns a voice *name* into that pack's reference clip and
|
||||
forwards it as base64 `reference_audio` (`scripts/frank-tts.env`,
|
||||
`TTS_BACKEND=omni-http`). Not deployed as of 2026-10-18.
|
||||
2. **xAI console custom voice — works today, zero code.** Record 2–3 min clean
|
||||
speech (≤120 s, 24 kHz 16-bit mono WAV) → console → `voice_id` → speech
|
||||
tool. Free (≤30 voices), cloud, US-only.
|
||||
@@ -206,29 +209,169 @@ The CPU shape does not need a Python server at all:
|
||||
`VoxCPM2-Acoustic-F16.gguf`, Apache-2.0 card) or from the fork's own
|
||||
`tools/omni/voxcpm2/convert_voxcpm2_to_gguf.py`.
|
||||
|
||||
Nothing has been measured on this fleet for any of these engines — no RTF, no
|
||||
latency, no MOS. That absence is the finding, not an omission.
|
||||
Arm B (`llama-tts`) and arm A (`qwentts.cpp`) have both been measured on this
|
||||
box — RTF, first-audio latency and RSS are in the next section. The other
|
||||
servers in the table above (vLLM-Omni, Kokoro-FastAPI, CosyVoice, F5-TTS,
|
||||
GPT-SoVITS) remain unmeasured here; that absence is still the finding.
|
||||
|
||||
## Measured CPU engine A/B (2026-10-18)
|
||||
|
||||
Both candidate upstreams for frank-tts were measured on Dankclaw's CPU — 8
|
||||
threads, no GPU touched, three runs per arm, seed 42, identical text, against
|
||||
the fleet's own audition voice packs.
|
||||
|
||||
- **Arm A — [`ServeurpersoCom/qwentts.cpp`](https://github.com/ServeurpersoCom/qwentts.cpp)**
|
||||
at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` (2026-10-16), weights
|
||||
from [`Serveurperso/Qwen3-TTS-GGUF`](https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF):
|
||||
`qwen-talker-1.7b-base-Q4_K_M.gguf` (1,219,245,248 B),
|
||||
`qwen-talker-0.6b-base-Q4_K_M.gguf` (628,905,056 B),
|
||||
`qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B) — byte counts verified
|
||||
against the Hub API. Builds with zero compiler warnings; `GGML_BLAS=OFF` is
|
||||
forced on this host (only `libopenblas.so.0` exists, no dev headers, so the
|
||||
vendor `buildcpu.sh` cannot configure).
|
||||
- **Arm B — mainline `llama-tts`** with `--tts-speaker-file`, i.e. the
|
||||
`llama-tts-cli` backend `scripts/frank-tts.env` already implements.
|
||||
|
||||
RTF is **seconds of compute per second of audio** (lower is better; below 1 is
|
||||
faster than real time).
|
||||
|
||||
| Arm | RTF | Run spread | Peak RSS |
|
||||
|---|---|---|---|
|
||||
| A · 0.6 B talker, CLI | 0.44 | ±0.002 | 3.4 GiB |
|
||||
| A · 1.7 B, CLI, uncloned | 0.57–0.58 | ±0.006 | 4.7 GiB |
|
||||
| A · 1.7 B, CLI, embedding-only clone | 0.63 | ±0.02 | n/m |
|
||||
| A · 1.7 B, CLI, ICL clone | 0.92 | ±0.01 | n/m |
|
||||
| A · 1.7 B, resident `tts-server`, warm | 0.55–0.57 | ±0.01 | 5.7 GiB |
|
||||
| A · same, ICL voice warm / **cold (first request)** | 0.67 / **1.65** | one-shot | 5.7 GiB |
|
||||
| B · mainline `llama-tts` | 0.93–0.99 | ±0.25 | n/m |
|
||||
|
||||
**Read RTF only within an arm.** Conditioning changes clip length — the same
|
||||
seed yields 18.00 s of audio embedding-only and 15.12 s under ICL — so the
|
||||
denominator is the arm's own audio duration.
|
||||
|
||||
Arm A's resident server is ~1.7× faster than arm B and about 40× tighter in
|
||||
run-to-run spread, and it already serves the HTTP shape frank-tts's façade
|
||||
speaks: `/health`, `/v1/models`, `/v1/audio/voices` (GET/POST/DELETE),
|
||||
`/v1/audio/speech` (`wav`/`pcm`), chunked streaming with the first audio byte
|
||||
at 86–451 ms and a server-side time-to-first-audio of 94–112 ms. Arm B has no
|
||||
HTTP path at all — that is what the façade's CLI backend exists for.
|
||||
|
||||
Four laws came out of the trial, each worth a line of configuration:
|
||||
|
||||
- **Boot-warm every voice.** The trial measured the first request for a
|
||||
registered voice at 6.2–8.0 s (first audio byte 5,530 ms) and later requests
|
||||
restoring the snapshot in 0.5 ms. **That ladder did not survive deployment:**
|
||||
measured 2026-10-18 on the built container against an empty registry — register,
|
||||
then time — the first synthesis of a freshly registered voice cost **1,883 ms**
|
||||
against **1,226–1,283 ms** for the requests after it, and a ~110-word request
|
||||
cost **9.4 s** then **9.8 s**, so request time tracks the audio generated rather
|
||||
than snapshot state. What still holds is the reason to warm at all: the registry
|
||||
is memory-only and re-registering does not warm, so an engine unit must run one
|
||||
throwaway synthesis per voice at start. It buys registration and about 0.6 s,
|
||||
not five seconds.
|
||||
- **Speaker artifacts are reusable and exact.** `qwen-codec` turns a reference
|
||||
WAV into `.spk` (8,192 B x-vector) + `.rvq` (5,456–6,864 B, 16 codebooks × T,
|
||||
11-bit packed) in 1.8–2.1 s at 1.3–1.5 GiB peak; replaying them yields a
|
||||
**byte-identical** WAV and saves 1.4 s per request. That pair is the
|
||||
engine-artifact term the voice-pack law in `scripts/frank-tts.env` reserves a
|
||||
slot for — arm A pins it as `.spk` + `.rvq` beside `reference.wav`.
|
||||
- **Streaming is not bit-identical to buffered**, whatever the upstream README
|
||||
claims: same text, seed and voice, 92.2 % of samples differ (peak delta
|
||||
1139/32768, −41.6 dB RMS — inaudible). A fixed seed does not pin output
|
||||
across `response_format`, so never mix the two paths in a cache or a golden
|
||||
fixture.
|
||||
- **A voice cannot be registered from an embedding alone**: `POST
|
||||
/v1/audio/voices` accepts `wav_b64` or **both** `spk_b64` and `rvq_b64`
|
||||
(`src/tts-server.h:480`), so an embedding-only voice has to be sent as the
|
||||
WAV, or the lone-`spk` path has to be upstreamed.
|
||||
|
||||
**Likeness is not measurable at this resolution.** x-vector cosine against each
|
||||
speaker's own reference: ICL 0.9886 / 0.9901, embedding-only 0.9844 / 0.9812,
|
||||
uncloned 0.9228 — but **two different speakers score 0.9497** on the same
|
||||
metric. The whole clone effect lives in a 0.04-wide band and the
|
||||
ICL-over-embedding gap is inside noise, so the four audition clips
|
||||
(`/home/blank/voice-audition/qwentts-*.mp3`, each against its `armB-*.mp3`
|
||||
mainline partner, guide in `LISTEN-engine.md`) are the instrument. **The
|
||||
operator heard them on 2026-10-18 and passed every clone**, which removed the
|
||||
last axis measurement could not reach; the engine was then picked on
|
||||
live-output grounds ([Decision](#decision-arm-a-for-live-output-2026-10-18)).
|
||||
The cosine
|
||||
extractor is still worth keeping: it is engine-neutral (it scores mainline
|
||||
output through the same encoder) and it separates an uncloned voice cleanly.
|
||||
|
||||
Two operational details. The engine takes 8 threads and exposes **no `-t`
|
||||
flag** — `src/backend.h` uses `hardware_concurrency()/2` and prints the choice
|
||||
at load, so pinning a cpuset on the quadlet changes that number implicitly. And
|
||||
ICL needs a reference transcript that did not exist: the fleet has no ASR
|
||||
endpoint and no audio input, so the crops were transcribed with
|
||||
`vosk-small-en-us` inside the work dir, and two of three transcripts came back
|
||||
suspect ("demon souls" for *Dark Souls*) — a reason to curate `ref_text` into
|
||||
the pack manifest instead of transcribing on the host.
|
||||
|
||||
## Decision: arm A for live output (2026-10-18)
|
||||
|
||||
Every audition pair passed, so likeness stopped being the tiebreak. The
|
||||
question became **which engine is better for live output** — the harness
|
||||
speaking a reply, an alert read aloud, a click-to-speak — and on that criterion
|
||||
RTF only has to stay under 1.0 with margin: what separates the arms is **time to
|
||||
first audio** and the **spread**, because an arm whose RTF can cross 1.0 stalls
|
||||
a live stream mid-sentence.
|
||||
|
||||
| | arm A · `qwentts.cpp` `tts-server` | arm B · mainline `llama-tts` |
|
||||
|---|---|---|
|
||||
| first audio byte | 86–451 ms | model load, per request |
|
||||
| warm RTF | 0.55–0.67 | 0.93–0.99 |
|
||||
| run spread | ±0.01 | ±0.25 |
|
||||
| HTTP | serves `/v1/audio/speech` | none — façade CLI only |
|
||||
| RAM | 5.7 GiB resident | per-request, freed after |
|
||||
|
||||
**Chosen: arm A, 1.7 B talker, resident `tts-server` in a `frank-tts-engine`
|
||||
container, with `TTS_BACKEND=llama-tts-cli` kept in the repo as the rollback.**
|
||||
Three consequences are configuration, not opinion: the unit **boot-warms every
|
||||
voice** — the registry is memory-only, so an unwarm'd name is a 400 and the first
|
||||
synthesis otherwise costs about 0.6 s more than the ones after it, as corrected
|
||||
above —, it **does not pin a cpuset** casually (threads are
|
||||
`hardware_concurrency()/2`, no flag), and every pack registers with the **WAV**
|
||||
because the engine rejects a lone embedding.
|
||||
|
||||
Two follow-ups stay open on purpose, and neither blocks the deploy:
|
||||
|
||||
- **Talker size.** The 0.6 B arm measured 0.44 RTF at 3.4 GiB and has never
|
||||
been auditioned — every clip above is 1.7 B. It is the lever for a CPU
|
||||
co-tenancy problem, not a preemptive saving.
|
||||
- **ICL vs embedding-only.** Both passed the ear test. Embedding-only is the
|
||||
shipped default: 0.57–0.63 vs 0.67 warm, and it needs no `ref_text`, which
|
||||
this fleet cannot produce honestly (the ASR crops came back suspect). Packs
|
||||
still carry `.spk` + `.rvq`, so ICL remains a per-voice opt-in for batch work
|
||||
like the podcast render, where total time matters more than first byte.
|
||||
|
||||
## Recommended deployment (Dankclaw)
|
||||
|
||||
As decided in Gitea issue #2, this stays **CPU-only on Dankclaw**: the RTX 3090
|
||||
has carried frank-zealot since 2026-10-17 and both R9700s are frank-archon's,
|
||||
so no card may be asked to co-host TTS. The shape is one engine plus a thin
|
||||
voice shim, and **nothing here is deployed as of 2026-10-18** — `systemctl
|
||||
--user is-enabled voiceshim` reports `not-found` and nothing answers on 8880 or
|
||||
8890.
|
||||
so no card may be asked to co-host TTS. The shape is one engine plus one
|
||||
façade, and **nothing here is deployed as of 2026-10-18**: `frank-tts` has
|
||||
neither a unit nor a container on the host, and nothing listens on 7772 or
|
||||
8880.
|
||||
|
||||
| Service | Image / install | Port | CPU | Notes |
|
||||
|---|---|---|---|---|
|
||||
| TTS engine | `llama-tts-server` (`tc-mb/llama.cpp-omni`) on a VoxCPM2 GGUF; `vllm/vllm-omni` + Qwen3-TTS only if a GPU is ever deliberately freed | 8880 | engine decision still open | OpenAI `/v1/audio/speech`; `n_gpu_layers` is there for a GPU boot |
|
||||
| Voice shim | `dankclaw/voiceservice/` (stdlib container) + `scripts/voiceshim.env` + `dankclaw/voiceservice/quadlet/voiceshim.container` | 8890 | yes | voice *name* → `reference.wav`, `voice`/`voice_id` on the wire, mp3 re-encode; backends `omni-http` (forwards base64 `reference_audio`) and `llama-tts-cli` (`--tts-speaker-file`, `nice 19`) |
|
||||
| TTS engine | **arm A `qwentts.cpp` `tts-server`** on the Qwen3-TTS GGUFs (decided 2026-10-18, above); arm B mainline `llama-tts` stays in the repo as the `llama-tts-cli` rollback; `vllm/vllm-omni` + Qwen3-TTS only if a GPU is ever deliberately freed | 8880 (container-internal) | open, evidence above | arm A serves OpenAI `/v1/audio/speech` itself and boot-warms each voice; arm B has no HTTP path and runs through the façade's CLI backend |
|
||||
| frank-tts | `dankclaw/voiceservice/` (stdlib container) + `scripts/frank-tts.env` + its quadlet | 7772 | yes | voice *name* → `/voices/<name>/reference.wav`, `voice`/`voice_id` on the wire, mp3 re-encode; backends `omni-http` (forwards base64 `reference_audio`) and `llama-tts-cli` (`--tts-speaker-file`, `nice 19`) |
|
||||
| STT (English) | Parakeet TDT 1.1B behind a FastAPI wrapper, or whisper.cpp server | 8882 | yes | CC-BY-4.0; LibriSpeech-clean WER 1.39 (model card) |
|
||||
| STT (multilingual, opt.) | faster-whisper `large-v3` int8 | 8883 | yes | MIT |
|
||||
| VAD | Silero (embedded) | — | yes | MIT; model file 2 327 524 B |
|
||||
|
||||
8880/8890 are what `scripts/voiceshim.env:35,49` carry (`TTS_UPSTREAM=
|
||||
http://host.containers.internal:8880`, `SHIM_PORT=8890`); 8882-8883 for STT
|
||||
keep clear of the fleet's LLM ports 7770/7771.
|
||||
7772 is frank-tts's host port — the next slot in the fleet's model-endpoint
|
||||
family (7770 frank-dragoon, 7771 frank-zealot). The engine behind it keeps 8880,
|
||||
but **not as a host-loopback publish**: netavark DNATs published ports only, and
|
||||
a bridge container cannot reach a `127.0.0.1` bind on the host — that is the
|
||||
lesson that 502'd hermes (`AGENTS.md`). `host.containers.internal`
|
||||
(169.254.1.2) is the gateway, so it reaches a host service only when that
|
||||
service binds a routable address; `scripts/frank-tts.env` names it as the
|
||||
engine's address, which holds for a `0.0.0.0` bind and not for a loopback one.
|
||||
The deployed pair therefore shares one quadlet network and the façade addresses
|
||||
the engine by container name.
|
||||
|
||||
## Voice-creation workflow (the "from sample files" answer)
|
||||
|
||||
@@ -284,6 +427,10 @@ keep clear of the fleet's LLM ports 7770/7771.
|
||||
https://huggingface.co/openbmb/VoxCPM2
|
||||
- llama.cpp-omni TTS server: https://github.com/tc-mb/llama.cpp-omni ·
|
||||
https://raw.githubusercontent.com/tc-mb/llama.cpp-omni/master/tools/omni/voxcpm2/README.md
|
||||
- qwentts.cpp (arm A): https://github.com/ServeurpersoCom/qwentts.cpp at commit
|
||||
`5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` · weights
|
||||
https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF (byte counts against the
|
||||
Hub API, 2026-10-18)
|
||||
- mainline `llama-tts`:
|
||||
https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/tts/README.md ·
|
||||
Pocket-TTS weights: https://huggingface.co/kyutai/pocket-tts (CC-BY-4.0)
|
||||
|
||||
@@ -79,6 +79,10 @@ macOS mini) as quadlets inside its podman machine — not on Dankserver.
|
||||
- [[Postgres]] — Gitea database backend
|
||||
- [[Seerr]] — unified *arr request manager (guard-chained; fronts FrankBook-Pro's stack)
|
||||
- [[Marreta]] — content-cleaning proxy (guard-chained; shares SearXNG's Gluetun egress)
|
||||
- [[Open Notebook]] — podcast generator (scripts by frank-zealot, voices by frank-tts); repo-owned config, not deployed
|
||||
- [[Frank TTS Engine]] — the arm A TTS engine (`qwentts.cpp` resident `tts-server`, 1.7 B talker) behind frank-tts on a private bridge; deployed on Dankclaw, CPU-only, publishes no port by design
|
||||
- [[Frank TTS]] — the 7772 cloned-voice endpoint consumers address: voice naming, pack layout, why the upstream is a container name; deployed, and it is the harness `speech` role
|
||||
- [[Voice Packs]] — how a frank-tts voice is built: reference.wav + the pinned engine's speaker artifacts, cut past the 10-second source-material gate
|
||||
|
||||
## Infrastructure
|
||||
|
||||
|
||||
@@ -2311,6 +2311,449 @@ every restart since the cutover.
|
||||
Kokoro-enum-only — so "the agent talks in my voice" needs that provider row or
|
||||
a `pi.registerProvider` extension, decided in phase 2.
|
||||
|
||||
## 2026-10-18 — TTS engine picked: arm A `qwentts.cpp`, on the live-output criterion
|
||||
|
||||
- Frank listened to the four audition pairs (`qwentts-*.mp3` against their
|
||||
`armB-*.mp3` mainline partners) and **passed every clone**. That deleted the
|
||||
only axis the A/B could not measure, and the question was reframed: **which
|
||||
engine is better for live output**. On that criterion RTF only has to stay
|
||||
under 1.0 with margin — the discriminators are time-to-first-audio and
|
||||
run-to-run spread, because an arm whose RTF can cross 1.0 stalls a live
|
||||
stream mid-sentence.
|
||||
- **Chosen: arm A `qwentts.cpp` @ `5916bf7`, the resident `tts-server`, 1.7 B
|
||||
talker Q4_K_M + Q8_0 tokenizer, in a `frank-tts-engine` container on 8880.**
|
||||
The `TTS_BACKEND=llama-tts-cli` path in `scripts/frank-tts.env` stays in the
|
||||
repo as the rollback. The numbers that carried it: first audio byte 86–451 ms
|
||||
against a per-request model load, warm RTF 0.55–0.67 against 0.93–0.99, spread
|
||||
±0.01 against ±0.25, and a native `/v1/audio/speech` against no HTTP path at
|
||||
all. Three consequences are configuration, not preference: boot-warm every
|
||||
voice, do not pin a cpuset (threads are `hardware_concurrency()/2`), register
|
||||
every pack with its WAV (a lone embedding is rejected).
|
||||
- Two follow-ups deliberately **not** decided: the **0.6 B talker** (0.44 RTF,
|
||||
3.4 GiB, never auditioned — every clip is 1.7 B) and **ICL vs
|
||||
embedding-only** conditioning. Embedding-only is the shipped default because
|
||||
it is cheaper and needs no `ref_text`, which this fleet cannot transcribe
|
||||
honestly; `.spk` + `.rvq` still ride in every pack, so ICL remains a per-voice
|
||||
opt-in for batch work where total time beats first byte.
|
||||
- **Addressing law corrected** in
|
||||
[[Local TTS and Voice Cloning]]: the pair may not meet at a host-loopback
|
||||
publish. netavark DNATs published ports only and a bridge container cannot
|
||||
reach a host `127.0.0.1` bind — the lesson that 502'd hermes — so
|
||||
`host.containers.internal:8880`, which `scripts/frank-tts.env` documents, is
|
||||
correct only for a routable bind. The deployed pair shares one quadlet network
|
||||
and the façade addresses the engine by container name.
|
||||
- **PR #12** (`feat/open-notebook-podcast` @ `114dd7a`, 8 commits, 403 tests) is
|
||||
**merged** as `1eb8b21` (Frank's OK, given in session): open-notebook +
|
||||
SurrealDB quadlets on the private bridge `10.89.4.0/24`, the API published on
|
||||
**5056** because seerr holds `127.0.0.1:5055`, and `ESPERANTO_TTS_TIMEOUT`
|
||||
moved 600 → **660** to sit one round-trip outside frank-tts's own
|
||||
`SHIM_TIMEOUT`. Merged is not runnable: nothing listens on 7772 until the
|
||||
voice stack lands, and the episode destination (`/home/dank/media/podcasts`)
|
||||
stays interim pending Gitea #7.
|
||||
- Still in flight, not in this branch: `feat/frank-tts-engine` — the arm A image
|
||||
build, the quadlet pair on one network, the boot-warm step, deployer /
|
||||
planner / `test-changed.sh` routing, homepage and docs. This branch is also
|
||||
where the two-branch overlap got resolved: merging `main` here conflicted on
|
||||
`log.md` alone (both branches appended an entry) and took `index.md`,
|
||||
`deploy-remote.sh`, `deploy-plan.py` and `test-changed.sh` clean.
|
||||
|
||||
## 2026-10-18 — open-notebook: the podcast generator's config lands (nothing deployed)
|
||||
|
||||
- Goal: **open-notebook** as the fleet's podcast generator on Dankclaw — notes
|
||||
and sources in, a multi-speaker episode out, script written by
|
||||
**frank-zealot**, voices rendered by **frank-tts**, no cloud provider
|
||||
anywhere in the path. Repo-side only: no host touched, no container pulled or
|
||||
started. Runbook: [[Open Notebook]].
|
||||
- **Two containers, not one.** `lfnovo/open_notebook:v1-latest` plus
|
||||
`surrealdb/surrealdb:v2`, both **digest-pinned** with `Pull=never` and a
|
||||
documented one-time `podman pull` — frank-dragoon's pinning style, because
|
||||
`:latest` moving underneath a rootless service is how you get an unbootable
|
||||
pod after an unrelated `systemctl-daemon-reload`. The registry round-trip is
|
||||
what proved `v2` is SurrealDB **2.7.0** and that the app image is Debian
|
||||
trixie with **curl and ffmpeg already inside** — which is how both health
|
||||
probes can be argument-only: `curl -fsS …/health` (route is upstream's,
|
||||
`api/main.py:239`, and it sits on the auth exclude list) and
|
||||
`/surreal is-ready`, which I ran against the digest here: green in 2 s, and
|
||||
it refuses to pass without `SURREAL_PASS` in the environment — a credential
|
||||
that must never ride the command line, so `Exec=` carries flags only.
|
||||
- **Host 5055 was already spoken for.** seerr has held `127.0.0.1:5055` since
|
||||
2026-10-14 (live `ss -ltn` agrees today), so the API publishes
|
||||
`0.0.0.0:5056:5055`. open-notebook's frontend derives its API URL from the
|
||||
window origin when `API_URL` is unset — the upstream env reference spells the
|
||||
reverse-proxy symptom — so a silent remap would hang the UI on a port nobody
|
||||
listens on, and `API_URL=http://192.168.0.143:5056` is set accordingly. 8502
|
||||
publishes on `0.0.0.0` because this service carries its own password (the
|
||||
episode-audio route is unauthenticated), unlike the loopback-and-proxied
|
||||
observability stack; **8000 is published nowhere**, where upstream exposes it
|
||||
on loopback with root credentials, and the seam forbids it.
|
||||
- **The CPU guard, pre-armed.** `frank-tts` spawns one ~8 GiB, 8-thread engine
|
||||
per request on a box that also serves two LLMs, while open-notebook defaults
|
||||
to `TTS_BATCH_SIZE=5` × `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` — a five-way
|
||||
self-DoS. Both are pinned to `1` in `scripts/open-notebook.env`, and the
|
||||
worker knob is only real because the image's own supervisord expands
|
||||
`${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5}` (`supervisord.conf:25`). The TTS clock
|
||||
goes to **660 — one round-trip outside** frank-tts's own `SHIM_TIMEOUT=600`, so
|
||||
a shim give-up reads as the shim's error rather than the worker's; the seam
|
||||
asserts the direction (`>`) against the frank-tts env instead of restating a
|
||||
number. `ESPERANTO_LLM_TIMEOUT` stays at the app's 180 s default: frank-zealot
|
||||
answers an outline or a transcript well inside it, and upstream says to keep it
|
||||
under 600 so a slow call fails with a name on it.
|
||||
- **What a credential needs in order to exist:** `OPEN_NOTEBOOK_ENCRYPTION_KEY`
|
||||
must be set before the UI can save *any* provider credential, and
|
||||
`OPEN_NOTEBOOK_PASSWORD` must stay **absent** to remain auth-off — the app
|
||||
reads `os.environ.get(…) is not None`, so `PASSWORD=` boots a login form
|
||||
asking for the empty string. The repo env file carries both keys empty; the
|
||||
deployer seeds `/home/dank/Containers/open-notebook/env` (600) from the host
|
||||
vault and **refuses to start the pair while either secret is blank**, because
|
||||
a half-configured open-notebook stores no credentials and fails quietly.
|
||||
- **Episode destination is interim, by decision.** The destination is **Gitea
|
||||
issue #7, `human-only`** — agents must not work it. Until Frank settles it,
|
||||
the app keeps its own named volume and `/home/dank/media/podcasts` is
|
||||
bind-mounted over `/app/data/podcasts` (`DATA_FOLDER` is hard-coded `./data`),
|
||||
so finished MP3s are ordinary files in his home directory instead of volume
|
||||
contents needing `podman unshare` — the database volume demonstrably lands on
|
||||
a mapped subuid, measured with the pinned image. One mount line; one deletion
|
||||
undoes it.
|
||||
- **Two UI buttons will fail, and that is cosmetic.** Test Connection does
|
||||
`GET <base>/models` and the TTS probe synthesises voice `alloy`; frank-tts
|
||||
serves `/health` and `/v1/audio/speech` only. Model rows are created *by
|
||||
name* and the voice field is free text, so `narrator1/2/3` is typed in and
|
||||
never discovered. Documented, not worked around.
|
||||
- **Blocked on frank-tts, stated plainly:** phase 1 landed without deployer or
|
||||
homepage registration and nothing listens on 7772 today, so a generation run
|
||||
writes its script and then fails on the first voice. That is upstream's
|
||||
creation-podcasts warning in fleet clothing.
|
||||
- Suite green: **403/403** (`./tests/run.sh`, wall 67 s) — 19 tests in the new
|
||||
`tests/open_notebook.bats`, structure only: pins, boot persistence, the
|
||||
unpublishable database port, the 5055→5056 remap plus the `API_URL` that has
|
||||
to match it, the env-file read order, the guard knobs, the health gate naming
|
||||
`/health`, planner routing, the homepage card hidden while undeployed.
|
||||
- Three deployer seams cost time and are now law-shaped: a service must be
|
||||
registered in **five** places — `SERVICE_ORDER`, `DANKCLAW_SERVICES`, the
|
||||
`deploy-remote.sh` dispatch case, its pre-deploy requirement, and
|
||||
`deploy.sh`'s dispatch regex, where an omission silently plans an empty set
|
||||
for `--only`; a `dankclaw/<svc>/*` case must precede the `dankclaw/*.container`
|
||||
glob because that `*` crosses `/`; and the changed-path routing lives *inside*
|
||||
the per-path loop, after the `scripts/*.env)` case, not at the top of the file.
|
||||
- Deliberately NOT done: the deploy (pulling 1.27 GiB to Dankclaw was not
|
||||
asked for), any Caddy vhost or public exposure, audiobook/EPUB (issue #5), any
|
||||
`frank-tts` code change, and the harbor templates on Dankserver —
|
||||
`~/.harbor/compose.opennotebook.yml` pins the deprecated `-single` image,
|
||||
carries no encryption key and injects the whole harbor env — left untouched,
|
||||
read-only, as the record of what not to copy.
|
||||
|
||||
## 2026-10-18 — frank-tts-engine: the arm A engine becomes deployable (nothing deployed)
|
||||
|
||||
- Goal: turn the engine the day's A/B picked into a **repo-owned, boot-persistent,
|
||||
self-healing container** — `frank-tts-engine`, `qwentts.cpp` commit
|
||||
`5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`, resident `tts-server`, 1.7 B base
|
||||
talker Q4_K_M + Q8_0 tokenizer, container port 8880, and **not** a fleet
|
||||
endpoint: the endpoint stays `frank-tts` on 7772. Runbook: [[Frank TTS Engine]].
|
||||
- **No `PublishPort` at all, which is the shape the merged A/B states as law.**
|
||||
netavark DNATs published ports only, so a `127.0.0.1:8880` publish is not a path
|
||||
to a peer container — the façade could never have used it (the trap that 502'd
|
||||
hermes) — while being the one LAN-reachable surface of an unauthenticated
|
||||
synthesiser. Both voice containers therefore sit on one committed quadlet
|
||||
network, `frank-tts-engine.network` at **10.89.5.0/24** (the next free /24 after
|
||||
open-notebook's 10.89.4.0/24), and the façade addresses the engine by container
|
||||
name. Diagnostics moved to `podman exec`, including the deployer's health gate
|
||||
(`wait_container_healthy`, not `wait_for_health`: there is no URL to dial).
|
||||
- **The boot warmer is a unit hook, not an operator habit.** Cold-voice law: the
|
||||
first request for a registered voice decodes the whole reference to snapshot
|
||||
codec state (6.2–8.0 s, first byte 5 530 ms), later ones restore in 0.5 ms; the
|
||||
registry is memory-only and re-registering does not warm. So
|
||||
`ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh` with
|
||||
`TimeoutStartSec=900`, and the `-` prefix is what stops a failed warm from
|
||||
crash-looping the endpoint under `Restart=always`. It logs `cold=` and `warm=`
|
||||
per voice, so the under-a-second second request is proven on every boot.
|
||||
- **Thread law recorded before anyone tunes it away:** no `--threads` exists;
|
||||
`src/backend.h` uses `hardware_concurrency()/2`, so a cpuset pin silently
|
||||
changes the thread count. Nothing pins this container.
|
||||
- Image builds from the pinned commit with the **proven** line, verbatim —
|
||||
`GGML_BLAS=OFF` is forced (this host has `libopenblas.so.0`, no dev headers),
|
||||
every GPU backend off, `-j 8`, and no `-DGGML_NATIVE` because ggml's default is
|
||||
`-march=native` for the building host: the image is built where it runs. Base
|
||||
`ubuntu:24.04` pinned by index digest; two stages, because ggml builds shared
|
||||
and the RPATH names the build tree.
|
||||
- Weights fetched by `scripts/download-frank-tts-engine-qwentts.sh` with byte and
|
||||
LFS-oid checks (talker 1 219 245 248 B, tokenizer 291 150 624 B); the deployer
|
||||
gates on `--verify-only` rather than pulling 1.4 GiB inside a deploy.
|
||||
- Registered in all five places the open-notebook entry just named as law, plus
|
||||
the routing case above the `dankclaw/*.container` glob. New seam
|
||||
`tests/frank_tts_engine.bats` (18 tests): the Quadlet **generator's own output**
|
||||
for the network/publish/boot/warm shape, the entrypoint's argv driven from the
|
||||
tracked env file, the warmer against a stub `tts-server` over a real socket
|
||||
(registration keys, `ref_text` absent, late engine, hung synthesis, name
|
||||
refusal), the fetcher's verifier, and both routing tables.
|
||||
- Deliberately NOT done: the deploy (no host touched, no container started, no
|
||||
image built), any homepage card or Prometheus job for the engine — it publishes
|
||||
nothing and exposes no metrics, so there is nothing to probe, which is said out
|
||||
loud in the page rather than papered over. No ICL/`ref_text` work: the packs
|
||||
keep `reference.spk`/`reference.rvq` so it stays a per-voice opt-in.
|
||||
|
||||
## 2026-10-18 — frank-tts: the 7772 endpoint gets its deploy wiring, and voice naming is written down
|
||||
|
||||
- Stage 2 of the voice stack: the **façade** in front of the engine is now a
|
||||
deployable service too. Nothing deployed, no host touched. Runbook:
|
||||
[[Frank TTS]].
|
||||
- **Two attached networks, in a meaningful order.** `dankclaw_default` first — it
|
||||
owns the default route and is where consumers live (open-notebook dials
|
||||
`http://frank-tts:7772/v1` by container name) — then the voice pair's private
|
||||
bridge. `scripts/frank-tts.env`'s `TTS_UPSTREAM` moves from
|
||||
`host.containers.internal:8880` to `http://frank-tts-engine:8880`, which is the
|
||||
actual fix for a shape that could never have worked: a bridge container cannot
|
||||
reach a host-loopback bind, and the engine publishes nothing.
|
||||
- **`Wants=` + `After=` on the engine, never `Requires=`:** the façade still
|
||||
carries the `llama-tts-cli` rollback backend, which needs no upstream, so an
|
||||
engine that will not boot must delay this endpoint, not take it down.
|
||||
- The façade handler installs the **engine's** quadlet pair as well. Not tidiness:
|
||||
with a referenced `Network=` file absent, the v1 Quadlet generator produces no
|
||||
container unit at all, and `--only frank-tts` has no other way to obtain it. It
|
||||
also `mkdir -p /home/dank/voices` as the deploy user, because rootless podman
|
||||
auto-creates a missing bind source **root-owned** and the endpoint then reports
|
||||
"no voices" instead of an error.
|
||||
- **A voice is a lowercase directory name.** `^[a-z0-9][a-z0-9._-]*$`, ≤ 64 chars,
|
||||
no slash/`..`/space. The shape is the intersection of two rules that do not
|
||||
agree: `app.py` would accept mixed case, `tts-server` lowercases names — so a
|
||||
pack named `Narrator1` resolves through the façade and not through the engine,
|
||||
and the boot warmer refuses such a directory loudly. `narrator1/2/3` are
|
||||
load-bearing for open-notebook, and `frank-tts-add-cloned-voice` is the skill
|
||||
that walks the pack → register → warm → prove-a-second-request flow.
|
||||
- Registered everywhere the open-notebook entry named as law (planner order puts
|
||||
the engine before the façade, so a warm registry precedes the first synthesis),
|
||||
plus the homepage card — `wip: down` and hidden, because nothing listens on
|
||||
7772 yet.
|
||||
- **No Prometheus job, on purpose.** The façade exports no `/metrics`, Prometheus 3
|
||||
refuses HTML/JSON content types (the `comfyui` and `services` probes are
|
||||
documented down for exactly that), and `LlmEndpointDown` is scoped
|
||||
`up{job=~"frank-.*"}` — so a `/health` scrape would page on Signal every few
|
||||
minutes for a healthy endpoint. That is the frank-archon never-green lesson
|
||||
pointed the other way, and it is written in the page instead of being quietly
|
||||
ignored.
|
||||
- Tests added to the voice seam: the generator's own rendering of the pair (both
|
||||
attachments, first-network order, one publish, Wants-not-Requires), the
|
||||
env-file upstream checked as a **network join** rather than a string
|
||||
comparison, planner separation and order, the façade handler's steps, the
|
||||
homepage card's gating, and routing that selects both seams for either
|
||||
deployable's paths.
|
||||
|
||||
## 2026-10-18 — voice-pack builder: the 10-second source-material gate becomes code
|
||||
|
||||
- Third slice of the voice stack (after the façade and the engine decision):
|
||||
`scripts/build-voice-pack.sh` turns a source recording plus a crop offset into
|
||||
a deployable pack — `reference.wav` (24 kHz mono `pcm_s16le`, 18–25 s), the
|
||||
engine's own `reference.spk` (8,192 B) + `reference.rvq` (5,456–6,864 B) made
|
||||
from it, and `manifest.json` provenance. Tracked tunables in
|
||||
`scripts/voice-packs.env`: pack list, per-voice source/offset/duration/pinned
|
||||
source sha256, extractor path and engine commit, artifact arithmetic. New runbook
|
||||
[[Voice Packs]].
|
||||
- **The gate is the product.** The project-wide invariant — no voice material
|
||||
from the first 10 s of a sample file — had only ever been a sentence in
|
||||
`scripts/frank-tts.env`. An offset inside it now exits 1 before a directory is
|
||||
created, and the same assertion runs again against the manifest that was just
|
||||
written, next to the recorded sha256, so a pack cannot carry a forbidden crop
|
||||
even if the env file is edited into nonsense. `10` stays a constant in the
|
||||
builder; the env file carries offsets, never the law.
|
||||
- **The extractor's contract was read, not guessed:** `qwen-codec` in qwentts.cpp
|
||||
`5916bf7c` takes `--model <codec gguf> --talker <base talker gguf> -i <wav>`
|
||||
with an optional `--device`, has **no `-o`** and names both artifacts next to
|
||||
the input with the extension swapped. So the WAV is staged under its final stem
|
||||
and `reference.wav` — the file the façade gates on — is moved into place
|
||||
**last**, after the artifacts and the manifest: a failed extractor leaves a
|
||||
directory that is not a voice. `--force` retires an existing pack whole to
|
||||
`.AGENTTRASH/` instead of touching bytes; nothing in this script deletes.
|
||||
- Sizes are arithmetic, so they are checked: 16 codebooks × 11 bits = 22 B per
|
||||
12.5 Hz frame (`22 × floor(samples/1920)`) and `enc_dim 2048 × f32` = 8,192 B.
|
||||
That makes the **effective crop window 19.84–25 s**, narrower than the WAV
|
||||
law's 18–25 s, and a too-short crop is refused before the extractor runs. An
|
||||
engine bump that changes either number fails the build by design; the bump
|
||||
procedure is in [[Voice Packs]].
|
||||
- Verified against reality, not against the model: the pinned `narrator1` crop
|
||||
(98.739–118.636 s of `audiosample.mp3`, sha256 `2299510…`) comes out of a
|
||||
`--dry-run` at 955,378 B / 19.897 s with 5,456 B of `.rvq` planned — byte-for-byte
|
||||
the trial's hand-cut numbers — and all three pinned source digests match the
|
||||
files on disk.
|
||||
- Seam: `tests/voice_pack_unit.bats` (13 tests) — refusal inside the forbidden
|
||||
region creates nothing, a legal crop builds the whole pack, `ffprobe` on the
|
||||
generated reference, manifest round-trip through `jq` with a sha that matches
|
||||
the source, re-run-without-`--force` leaves every byte alone, `--force` retires
|
||||
rather than deletes, `--dry-run` needs no extractor, a wrong-sized artifact
|
||||
from a bumped engine cannot publish a voice, and every crop in the tracked pack
|
||||
list clears the invariant. `qwen-codec` is stood in by a stub with the real
|
||||
tool's I/O shape, so no engine, no GPU, no network, no fleet host. Suite green:
|
||||
**416/416** (`./tests/run.sh`, wall 77 s) — 403 before this branch.
|
||||
- Deliberately NOT done: no transcript in the manifest (ICL wants a curated
|
||||
`ref_text`; the trial's machine transcripts were wrong, and the engine
|
||||
registers WAV-only voices anyway), no 16 kHz variant (arm A is native 24 kHz),
|
||||
no loudness pass (not in the pack law), and no pack built on the host — the
|
||||
three sources still live on the workstation and the engine GGUFs have not been
|
||||
fetched to `/home/dank/models/Qwen3-TTS-GGUF` yet, so the first real run is a
|
||||
host step.
|
||||
|
||||
## 2026-10-18 — voice stack audit: three faults that no gate could see
|
||||
|
||||
- The voice stack had never been run end to end — no image built, nothing on
|
||||
7772 — so the whole path was audited as static code against the tools it
|
||||
calls, and three faults came out of it. All three are the same species: a
|
||||
step that **reports success while doing nothing**.
|
||||
- **The build would not have happened.** `Containerfile.engine` created its
|
||||
unprivileged user with `adduser`; the pinned ubuntu:24.04 base has no
|
||||
`adduser` at all. Proven in the base on Dankclaw rather than argued from
|
||||
memory: `command -v adduser` prints nothing, and the replacement line
|
||||
`useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin tts`
|
||||
answers `uid=10000(tts) gid=999(tts)`. Its `SYS_UID_MAX` warning is cosmetic
|
||||
and is the price of matching the façade's uid, so the comment says that
|
||||
before someone "fixes" it by dropping `--system`.
|
||||
- **The warm pass would not have warmed.** The unit hands its env file to podman
|
||||
via `--env-file`, which performs no shell parsing and takes the literal
|
||||
remainder of the line, while `source` strips one quoting layer. `WARM_TEXT`
|
||||
therefore reached the container with its quotes, the hook interpolated them
|
||||
into the synthesis body, the JSON was invalid, and every voice would log
|
||||
"registered, but the warm synthesis failed" beside an `active` unit. Fixed in
|
||||
the hook — strip exactly one matched pair, only when the inner content also
|
||||
carries a quote or backslash, then JSON-escape — and the seam now reads
|
||||
`scripts/frank-tts-engine.env` **both** ways and asserts the engine gets the
|
||||
same sentence in both shapes. The bug survived the first deploy precisely
|
||||
because every existing test used `source`.
|
||||
- **A rebuilt image would not have been rebuilt.** `build_compose_image` skips
|
||||
`podman build` on a matching `cfhash` label, and that hash covered the
|
||||
`Containerfile` alone — so editing `app.py` or `tts-engine-warm.sh` (both
|
||||
`COPY`'d) skipped the build, the deploy reported success, `/health` answered,
|
||||
and the container served old code. Each case arm now names the files its
|
||||
`Containerfile` copies and hashes them with it; comfyui/loki/alloy have no
|
||||
extras, so their labels are byte-unchanged and they do not rebuild here.
|
||||
- Tests as behaviour, not as greps: the new seam extracts the real
|
||||
`build_compose_image` from the deployer and drives it against a stub `podman`
|
||||
that replays per-image label state, asserting the observable question — did
|
||||
the deploy ask podman to build, or did it skip? — through build, skip,
|
||||
engine-hook-edit-rebuilds-engine-only, and `app.py`-rebuilds-the-façade.
|
||||
Reverting `scripts/deploy-remote.sh` turns it red at the warm-hook step, which
|
||||
is the fault being prevented. Suite green: **443/443** (`./tests/run.sh`,
|
||||
wall 110 s); each fix commit was left green on its own so the branch bisects.
|
||||
- Docs carry the lessons where an operator will hit them: a **Locally built
|
||||
images** section in [[deployment]], two new troubleshooting lines in
|
||||
[[Frank TTS Engine]] (all-voices-cold = quoted `WARM_TEXT`; façade 502 beside
|
||||
200 = model loaded ≠ voices registered), and `Voice Packs.md` step 4
|
||||
corrected — the warm hook re-registers **every** pack on every run, because
|
||||
the engine replaces a name it already holds (`tools/tts-server.cpp:148` at
|
||||
the pinned commit: *"Re-registering a name replaces the previous entry."*),
|
||||
so a restart is what an engine bump needs and not what a rebuilt pack needs.
|
||||
- Opened as PR **#14** against `main`. Deliberately NOT done: no deploy (the
|
||||
first `--service frank-tts-engine` run is a ~10-minute recursive source clone
|
||||
and a new unit on a host that serves people — Frank's call), no voice pack
|
||||
built on the host, no Playwright run (this branch publishes no host port and
|
||||
touches no vhost), and Gitea **#2** phase 1 stays open until 7772 answers.
|
||||
|
||||
## 2026-10-18 — frank-tts-engine deployed: it builds, it serves, and the warm hook died on the kernel's argument cap
|
||||
|
||||
- Merged as **#14** (`747c665`) and deployed `--service frank-tts-engine`. The
|
||||
payload took 18 s to fetch for the 1.4 GiB pair (byte count + Hub LFS sha256,
|
||||
`--verify-only` OK), and the image built clean — the first real proof of the
|
||||
`useradd` fix: `STEP 3/13: RUN useradd --system --uid 10000 …` passed with only
|
||||
the documented cosmetic `SYS_UID_MAX` warning, and the unit was `healthy` after
|
||||
31 s.
|
||||
- Then the deploy reported `0 voice(s) warm`:
|
||||
`/app/warm-voices.sh: line 152: /usr/bin/curl: Argument list too long`. The hook
|
||||
handed the registration body to curl as ONE argument; the kernel caps a single
|
||||
argument at 131,072 bytes (`MAX_ARG_STRLEN`) and a shipped 955 KB
|
||||
`reference.wav` base64's to about 1.27 MB, so curl never executed and the hook
|
||||
logged that as "HTTP no response". Fourth silent fault of the voice stack, same
|
||||
species as the other three: `/health` stayed green throughout.
|
||||
- Fix: the body goes to a `mktemp` file and curl gets `@file`. The regression test
|
||||
is the honest shape — the seam's stub engine is a real listener, `curl` is the
|
||||
real binary, the fixture pack is 300 KB — so it fails with the same kernel error
|
||||
the host produced (proved red first, green after). Every shipped pack sits well
|
||||
over the ~96 KB cliff, so no small fixture could ever have seen this.
|
||||
- Live proof on the deployed container with the three real packs:
|
||||
`narrator1 cold=1175ms warm=1874ms`, `narrator2 1373/1892`, `narrator3
|
||||
1816/1487`, exit 0, 15.2 s for the whole pass.
|
||||
- **A trial law did not survive deployment.** Against an empty registry —
|
||||
register only, then time — the FIRST synthesis of a freshly registered voice
|
||||
cost **1 883 ms**, the next two **1 283** and **1 226 ms**; a ~110-word request
|
||||
cost **9.4 s** then **9.8 s**. The trial's 6.2–8.0 s cold / 0.5 ms restore
|
||||
ladder, which the warmer's own header repeated, is retired: warming buys
|
||||
registration — mandatory, because the registry is memory-only — plus about
|
||||
**0.6 s**. Corrected in [[Frank TTS Engine]] law 1 and its troubleshooting
|
||||
table, in [[Local TTS and Voice Cloning]], and in [[Voice Packs]].
|
||||
- Host state, honestly: the engine is `active` with three voices warm, but the hook
|
||||
fix is not in the image yet, so the next restart re-breaks it until this branch
|
||||
is merged and re-deployed. `frank-tts` (the 7772 façade) is still undeployed.
|
||||
- Deliberately NOT done in this window: the `frank-tts` deploy, any Prometheus
|
||||
probe for the voice pair (neither container exposes `/metrics`; the blackbox
|
||||
question is still open), and the podcast render — open-notebook is on `main` but
|
||||
not installed.
|
||||
|
||||
- Sweep afterwards: the retired ladder still lived in six places the first pass
|
||||
missed — both quadlet files, `scripts/frank-tts-engine.env`,
|
||||
`scripts/frank-tts.env`, the [[Frank TTS]] troubleshooting row, the research
|
||||
brief's consequences paragraph, and one test comment. All re-keyed to the
|
||||
measured claim: about **0.6 s**, and the 400-for-every-name reason the pages
|
||||
should have led with. The trial entry above is left as written — a dated worklog
|
||||
entry is a record — and this line is the correction that points at it.
|
||||
|
||||
## 2026-10-18 — 7772 is live and the harness speaks through it
|
||||
|
||||
- Merged **#15** (`0315a62`, the warmer's argument-cap fix) and **#16**
|
||||
(`d2ce669`, the comment sweep), then deployed both voice services in order:
|
||||
`--service frank-tts-engine` (image rebuilt from the fixed warmer, unit
|
||||
`active`, zero drop-ins, warm pass in the unit's own start — `narrator1
|
||||
cold=1542ms warm=1709ms`, `narrator2 1329/1405`, `narrator3 1451/1394`, `3
|
||||
voice(s) registered and warm`) and `--service frank-tts`. The Playwright gate
|
||||
ran at the end of each deploy: 33/33 twice.
|
||||
- Deployed-state proof from a LAN client: `/health` → `omni-http` with all three
|
||||
voices; the same sentence in `narrator1` and `narrator2` came back with
|
||||
**different sha256** and different RMS (4.24 s, 24 kHz mono PCM16), so the name
|
||||
actually selects a voice. Engine-side ledger for the harness request:
|
||||
`TTFA 67.5 ms`, `Total 2158.8 ms`, `audio 3.76 s, RTF 0.574`.
|
||||
- **The harness `speech` role moved off Kokoro**: `config.yml`
|
||||
`modelRoles.speech: frank-tts (dankclaw)/frank-tts`. No voice key moves with it.
|
||||
The bundle is unambiguous: the speech device forwards the call's `voice_id` and
|
||||
nothing else, `tts.localVoice` belongs to the local backend (it is what answered
|
||||
in the control test), and `speech.voice` is the TUI's speak-aloud enum.
|
||||
- Two harness laws learned by breaking it first (both now on [[Frank TTS]]): an
|
||||
explicit speech role has **no fallback candidate**, so a dead endpoint errors
|
||||
instead of degrading; and `models.yml` loads at process start while
|
||||
`config.yml` is read live, so a session older than the `frank-tts` row fails
|
||||
exactly like a dead endpoint. Fresh-process proof in the same minute:
|
||||
`Saved 180524 bytes (model=frank-tts (dankclaw)/frank-tts, voice=narrator1)`
|
||||
with the matching façade log line `speech voice=narrator1 format=wav chars=51
|
||||
-> 180524 bytes`. This window's own session is that stale case and knows it.
|
||||
- Status sweep, because six pages still said the endpoint did not exist: both
|
||||
vault descriptions, the two `index.md` bullets, the Open Notebook blocker (the
|
||||
voice leg works; open-notebook itself is what is missing), the engine page's
|
||||
"Deploy order — nothing below has been run" is now a **Deploy record**, and the
|
||||
homepage card lost `wip: down` so the admin panel probes 7772 for real.
|
||||
- Corrected the same evening, by breaking it in a restarted session: a harness
|
||||
call that names no voice reaches 7772 with **no `voice` key at all** — `400
|
||||
unknown voice None; available voices: narrator1, narrator2, narrator3` —
|
||||
because no setting feeds one. The earlier `Saved 180524 bytes
|
||||
(voice=narrator1)` succeeded only because that run was *told* to pass
|
||||
`voice_id`. The endpoint therefore grew `DEFAULT_VOICE` (its own PR): a
|
||||
voice-less request speaks the declared pack and the journal line carries
|
||||
`(default)`, while an unknown name still 400s with the list. Reverting the role
|
||||
to Kokoro is one key, not two, and `speech.voice: narrator1` — briefly recorded
|
||||
above as wiring — went back to `am_fenrir`: that key is the vocalizer's Kokoro
|
||||
enum, so a pack name there breaks speak-aloud and does nothing for synthesis.
|
||||
|
||||
## 2026-10-18 — the default voice shipped: a harness voiceover needs no voice
|
||||
|
||||
- Merged **#17** (`b590695`) and **#18** (`7889a68`), then deployed both affected
|
||||
services in one dispatch — `--service frank-tts --service homepage`, 159 s,
|
||||
Playwright gate 33/33. The shim image rebuilt with the new handler; the
|
||||
homepage checker took the card on its next timer tick: `frank-tts → up`,
|
||||
probed for real rather than gated.
|
||||
- The call that had failed is the one that proves the fix. Same device, same
|
||||
restarted session, **no** `voice_id`: `Saved 430124 bytes (model=frank-tts
|
||||
(dankclaw)/frank-tts, voice=default, codec=wav)`. The façade journal says what
|
||||
actually spoke — `speech voice=narrator1 (default) format=wav chars=148 ->
|
||||
430124 bytes` — with the engine's ledger behind it (`audio 8.96 s, RTF 0.551`).
|
||||
The harness prints `voice=default` because it never had a name to print; the
|
||||
voice that answered is the operator's line in `scripts/frank-tts.env`, and the
|
||||
journal keeps them distinguishable forever after.
|
||||
- The page gained a **Default voice** row in its contract table and
|
||||
[[#Naming a voice]] now states that a call naming nothing is the harness's
|
||||
normal shape, not a client bug — the misconception that produced the 400 in
|
||||
the first place.
|
||||
|
||||
## 2026-10-18 — OpenWebUI "voice narrator1 not found": the rejector is in the browser
|
||||
|
||||
|
||||
Reference in New Issue
Block a user