live: heartbeat sync 2026-10-18T17:49:00-07:00 (Refs #11)
This commit is contained in:
Frank committed
1 parent
b3d3df4bdb
commit
57e255f930
15 files changed
+352
-19
No files matched your search
@@ -0,0 +1,125 @@
|
||||
---
|
||||
name: frank-tts-add-cloned-voice
|
||||
description: "Add or repair a cloned voice on the dank fleet: /home/dank/voices/<name> pack, name rules, boot-warm registration, and the under-a-second proof"
|
||||
---
|
||||
|
||||
Use for "give frank-tts a new voice", "voice X is missing/unknown", or "the first
|
||||
request of the day takes 5 s". Runbook of record: vault pages [[Frank TTS]] (voice
|
||||
packs, naming) and [[Frank TTS Engine]] (the warm registry). Everything below runs
|
||||
as the deploy user on **Dankclaw** (`ssh -o BatchMode=yes dankclaw '…'`); the
|
||||
containers are **rootless** (`systemctl --user …`, `podman …` without sudo).
|
||||
|
||||
## Naming (the question that gets asked first)
|
||||
|
||||
A voice is addressed **by its directory name** — `voice=narrator1` on
|
||||
`POST /v1/audio/speech` (omp's TTS device may send `voice_id`, same value). Use:
|
||||
|
||||
- `^[a-z0-9][a-z0-9._-]*$`, **lowercase**, ≤ 64 chars. No slash, no `..`, no
|
||||
space, no uppercase; must not start with `.`/`_`/`-`.
|
||||
- Why lowercase rather than just "alphanumeric": the shim accepts
|
||||
`[A-Za-z0-9][A-Za-z0-9._-]{0,63}` (`app.py`, `VOICE_NAME_RE`) but the **engine
|
||||
lowercases names**, so `Narrator1` resolves on one side and not the other. The
|
||||
boot warmer refuses a mixed-case pack loudly instead of half-working.
|
||||
- Names are user-visible: a 400 carries them in `available_voices`. Fleet
|
||||
convention is `narrator1`, `narrator2`, … for the open-notebook podcast
|
||||
speakers — open-notebook is configured with exactly `narrator1/2/3`, so renaming
|
||||
those breaks podcast rendering, not just TTS — and a lowercase personal name for
|
||||
a person's own voice.
|
||||
|
||||
## Pack layout (host state — never commit it)
|
||||
|
||||
`/home/dank/voices/<name>/` is a voice **iff** it holds `reference.wav`:
|
||||
|
||||
```
|
||||
reference.wav 24 kHz mono pcm_s16le, 18-25 s of clean single-speaker audio
|
||||
reference.spk engine x-vector: enc_dim 2048 f32 = 8192 B
|
||||
reference.rvq RVQ codes: 22 B per 12.5 Hz frame (5456-6864 B)
|
||||
manifest.json source path + sha256, crop start/end, engine commit, name
|
||||
```
|
||||
|
||||
Build it; do not hand-assemble and never fabricate `.spk`/`.rvq` from a WAV:
|
||||
|
||||
```bash
|
||||
scripts/build-voice-pack.sh --all /home/dank/voices # everything in VOICE_PACKS
|
||||
scripts/build-voice-pack.sh --source SRC --offset SEC --duration SEC --sha256 HEX NAME /home/dank/voices
|
||||
scripts/build-voice-pack.sh --dry-run ... # validate + crop, no extractor
|
||||
```
|
||||
|
||||
Knobs live in `scripts/voice-packs.env` (one `_SOURCE`/`_OFFSET`/`_DURATION`/
|
||||
`_SHA256` set per voice). Three things it enforces that an operator gets wrong:
|
||||
|
||||
1. **The 10-second gate.** A crop starting inside the first 10 s of the source is
|
||||
refused before anything is written (project source-material law), and the
|
||||
manifest is re-checked against the source's sha256 afterwards.
|
||||
2. **The effective crop window is 19.84-25 s, not 18-25 s.** An 18.0 s crop is a
|
||||
legal `reference.wav` but yields only 22 × 225 = 4 950 B of `.rvq`, under the
|
||||
5 456 B floor the codec config implies — so the builder says so up front.
|
||||
3. **It never deletes.** An existing pack is refused unless `--force`, and
|
||||
`--force` moves the old directory to `$PACKS_DIR/.AGENTTRASH/<voice>-<UTC>/`.
|
||||
|
||||
Files must be world-readable (the container runs as uid 10000): `644` files,
|
||||
`755` dirs. `mkdir -p /home/dank/voices` as the deploy user if it is absent —
|
||||
rootless podman auto-creates a missing bind source **root-owned**, and the
|
||||
endpoint then sees no voices at all.
|
||||
|
||||
## Register and warm (the part people forget)
|
||||
|
||||
The engine keeps its voice registry **in memory**, and the **first synthesis of a
|
||||
registered voice decodes the whole reference clip** to snapshot codec state:
|
||||
6.2-8.0 s, first audio byte at ~5 530 ms. Later requests restore in 0.5 ms.
|
||||
Re-registering does **not** warm — only synthesizing does. So:
|
||||
|
||||
```bash
|
||||
# registers every pack that has a reference.wav, then speaks the warm sentence once
|
||||
podman exec frank-tts-engine /app/warm-voices.sh
|
||||
systemctl --user --no-pager --full status frank-tts-engine.service | tail -20
|
||||
```
|
||||
|
||||
The unit's
|
||||
`ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh` does
|
||||
the same at every boot, so a fresh boot is already warm; the explicit call above
|
||||
is for **adding one voice to a running engine** and needs no restart — a restart
|
||||
drops every registration and re-pays the cold cost for the whole library. The
|
||||
`podman exec` form is not cosmetic: the engine **publishes no port**, so there is
|
||||
no host address to curl.
|
||||
|
||||
## Proof (do not report "added" without it)
|
||||
|
||||
```bash
|
||||
curl -s localhost:7772/v1/audio/voices # name is listed
|
||||
time curl -s -o /tmp/a.wav localhost:7772/v1/audio/speech \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"tts-1","voice":"NAME","input":"Checking the cloned voice.","response_format":"wav"}'
|
||||
time curl -s -o /tmp/b.wav localhost:7772/v1/audio/speech \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"tts-1","voice":"NAME","input":"And the second request.","response_format":"wav"}'
|
||||
file /tmp/a.wav /tmp/b.wav # really audio
|
||||
```
|
||||
|
||||
The warmer's journal lines carry `voice NAME cold=…ms warm=…ms` — quote that pair.
|
||||
Note that streaming and buffered output differ at the same seed (92.2 % of samples,
|
||||
-41.6 dB RMS): never compare two paths or two "identical" renders, only that each
|
||||
is audio and that the second request is fast.
|
||||
|
||||
## Failure modes worth naming
|
||||
|
||||
- `400 unknown voice …; available voices: …` → the name is not a pack: wrong
|
||||
spelling/case, missing `reference.wav`, or `/home/dank/voices` root-owned/empty.
|
||||
Trust the 400's list over any manifest.
|
||||
- 400 with `(none: no voice pack with a reference.wav in /voices)` → mount problem,
|
||||
not a voice problem.
|
||||
- First request slow **after** a boot → the warm pass failed; read the unit's
|
||||
journal (the warmer exits 0 and logs when there is no voice directory, so
|
||||
silence there is not success either).
|
||||
- Endpoint answers but synthesis fails 502 → the façade must dial
|
||||
`http://frank-tts-engine:8880` over the shared bridge
|
||||
(`frank-tts-engine.network`); a `host.containers.internal` / `127.0.0.1:8880`
|
||||
upstream can never work between containers, because netavark DNATs published
|
||||
ports only.
|
||||
|
||||
## Never
|
||||
|
||||
Never write under `/voices` from the endpoint (mounted `:ro` by design), never
|
||||
`git add` a voice pack, never `rm` a pack (repo policy: move to `.AGENTTRASH/`),
|
||||
and never benchmark against this endpoint while it serves — stop and mask the
|
||||
units first.
|
||||
Reference in new issue
Block a user