126 lines
6.3 KiB
Markdown
126 lines
6.3 KiB
Markdown
---
|
||
name: frank-tts-add-cloned-voice
|
||
description: "Add or repair a cloned voice on the dank fleet: /home/dank/voices/<name> pack, name rules, boot-warm registration, and the under-a-second proof"
|
||
---
|
||
|
||
Use for "give frank-tts a new voice", "voice X is missing/unknown", or "the first
|
||
request of the day takes 5 s". Runbook of record: vault pages [[Frank TTS]] (voice
|
||
packs, naming) and [[Frank TTS Engine]] (the warm registry). Everything below runs
|
||
as the deploy user on **Dankclaw** (`ssh -o BatchMode=yes dankclaw '…'`); the
|
||
containers are **rootless** (`systemctl --user …`, `podman …` without sudo).
|
||
|
||
## Naming (the question that gets asked first)
|
||
|
||
A voice is addressed **by its directory name** — `voice=narrator1` on
|
||
`POST /v1/audio/speech` (omp's TTS device may send `voice_id`, same value). Use:
|
||
|
||
- `^[a-z0-9][a-z0-9._-]*$`, **lowercase**, ≤ 64 chars. No slash, no `..`, no
|
||
space, no uppercase; must not start with `.`/`_`/`-`.
|
||
- Why lowercase rather than just "alphanumeric": the shim accepts
|
||
`[A-Za-z0-9][A-Za-z0-9._-]{0,63}` (`app.py`, `VOICE_NAME_RE`) but the **engine
|
||
lowercases names**, so `Narrator1` resolves on one side and not the other. The
|
||
boot warmer refuses a mixed-case pack loudly instead of half-working.
|
||
- Names are user-visible: a 400 carries them in `available_voices`. Fleet
|
||
convention is `narrator1`, `narrator2`, … for the open-notebook podcast
|
||
speakers — open-notebook is configured with exactly `narrator1/2/3`, so renaming
|
||
those breaks podcast rendering, not just TTS — and a lowercase personal name for
|
||
a person's own voice.
|
||
|
||
## Pack layout (host state — never commit it)
|
||
|
||
`/home/dank/voices/<name>/` is a voice **iff** it holds `reference.wav`:
|
||
|
||
```
|
||
reference.wav 24 kHz mono pcm_s16le, 18-25 s of clean single-speaker audio
|
||
reference.spk engine x-vector: enc_dim 2048 f32 = 8192 B
|
||
reference.rvq RVQ codes: 22 B per 12.5 Hz frame (5456-6864 B)
|
||
manifest.json source path + sha256, crop start/end, engine commit, name
|
||
```
|
||
|
||
Build it; do not hand-assemble and never fabricate `.spk`/`.rvq` from a WAV:
|
||
|
||
```bash
|
||
scripts/build-voice-pack.sh --all /home/dank/voices # everything in VOICE_PACKS
|
||
scripts/build-voice-pack.sh --source SRC --offset SEC --duration SEC --sha256 HEX NAME /home/dank/voices
|
||
scripts/build-voice-pack.sh --dry-run ... # validate + crop, no extractor
|
||
```
|
||
|
||
Knobs live in `scripts/voice-packs.env` (one `_SOURCE`/`_OFFSET`/`_DURATION`/
|
||
`_SHA256` set per voice). Three things it enforces that an operator gets wrong:
|
||
|
||
1. **The 10-second gate.** A crop starting inside the first 10 s of the source is
|
||
refused before anything is written (project source-material law), and the
|
||
manifest is re-checked against the source's sha256 afterwards.
|
||
2. **The effective crop window is 19.84-25 s, not 18-25 s.** An 18.0 s crop is a
|
||
legal `reference.wav` but yields only 22 × 225 = 4 950 B of `.rvq`, under the
|
||
5 456 B floor the codec config implies — so the builder says so up front.
|
||
3. **It never deletes.** An existing pack is refused unless `--force`, and
|
||
`--force` moves the old directory to `$PACKS_DIR/.AGENTTRASH/<voice>-<UTC>/`.
|
||
|
||
Files must be world-readable (the container runs as uid 10000): `644` files,
|
||
`755` dirs. `mkdir -p /home/dank/voices` as the deploy user if it is absent —
|
||
rootless podman auto-creates a missing bind source **root-owned**, and the
|
||
endpoint then sees no voices at all.
|
||
|
||
## Register and warm (the part people forget)
|
||
|
||
The engine keeps its voice registry **in memory**, and the **first synthesis of a
|
||
registered voice decodes the whole reference clip** to snapshot codec state:
|
||
6.2-8.0 s, first audio byte at ~5 530 ms. Later requests restore in 0.5 ms.
|
||
Re-registering does **not** warm — only synthesizing does. So:
|
||
|
||
```bash
|
||
# registers every pack that has a reference.wav, then speaks the warm sentence once
|
||
podman exec frank-tts-engine /app/warm-voices.sh
|
||
systemctl --user --no-pager --full status frank-tts-engine.service | tail -20
|
||
```
|
||
|
||
The unit's
|
||
`ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh` does
|
||
the same at every boot, so a fresh boot is already warm; the explicit call above
|
||
is for **adding one voice to a running engine** and needs no restart — a restart
|
||
drops every registration and re-pays the cold cost for the whole library. The
|
||
`podman exec` form is not cosmetic: the engine **publishes no port**, so there is
|
||
no host address to curl.
|
||
|
||
## Proof (do not report "added" without it)
|
||
|
||
```bash
|
||
curl -s localhost:7772/v1/audio/voices # name is listed
|
||
time curl -s -o /tmp/a.wav localhost:7772/v1/audio/speech \
|
||
-H 'content-type: application/json' \
|
||
-d '{"model":"tts-1","voice":"NAME","input":"Checking the cloned voice.","response_format":"wav"}'
|
||
time curl -s -o /tmp/b.wav localhost:7772/v1/audio/speech \
|
||
-H 'content-type: application/json' \
|
||
-d '{"model":"tts-1","voice":"NAME","input":"And the second request.","response_format":"wav"}'
|
||
file /tmp/a.wav /tmp/b.wav # really audio
|
||
```
|
||
|
||
The warmer's journal lines carry `voice NAME cold=…ms warm=…ms` — quote that pair.
|
||
Note that streaming and buffered output differ at the same seed (92.2 % of samples,
|
||
-41.6 dB RMS): never compare two paths or two "identical" renders, only that each
|
||
is audio and that the second request is fast.
|
||
|
||
## Failure modes worth naming
|
||
|
||
- `400 unknown voice …; available voices: …` → the name is not a pack: wrong
|
||
spelling/case, missing `reference.wav`, or `/home/dank/voices` root-owned/empty.
|
||
Trust the 400's list over any manifest.
|
||
- 400 with `(none: no voice pack with a reference.wav in /voices)` → mount problem,
|
||
not a voice problem.
|
||
- First request slow **after** a boot → the warm pass failed; read the unit's
|
||
journal (the warmer exits 0 and logs when there is no voice directory, so
|
||
silence there is not success either).
|
||
- Endpoint answers but synthesis fails 502 → the façade must dial
|
||
`http://frank-tts-engine:8880` over the shared bridge
|
||
(`frank-tts-engine.network`); a `host.containers.internal` / `127.0.0.1:8880`
|
||
upstream can never work between containers, because netavark DNATs published
|
||
ports only.
|
||
|
||
## Never
|
||
|
||
Never write under `/voices` from the endpoint (mounted `:ro` by design), never
|
||
`git add` a voice pack, never `rm` a pack (repo policy: move to `.AGENTTRASH/`),
|
||
and never benchmark against this endpoint while it serves — stop and mask the
|
||
units first.
|