Frank
9a17b93e8a
A consumer that lists the 29-name af_*/am_*/bf_*/bm_* pack and refuses narrator1 without a request reaching the façade is synthesizing in the browser. Frank TTS's troubleshooting table gains that symptom; OpenWebUI's See Also gains the service page.
205 lines
13 KiB
Markdown
205 lines
13 KiB
Markdown
---
|
|
type: reference
|
|
description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; deployed 2026-10-18 and wired as the harness speech role."
|
|
tags:
|
|
- dankclaw
|
|
- tts
|
|
- voice
|
|
- quadlet
|
|
---
|
|
|
|
# Frank TTS
|
|
|
|
`frank-tts` is the fleet's only speech endpoint: an OpenAI-shaped
|
|
`POST /v1/audio/speech` on **Dankclaw :7772** that answers `voice=<name>` with
|
|
audio in that cloned voice. Everything else in the voice stack sits behind it —
|
|
the engine ([[Frank TTS Engine]]) publishes no port on purpose, and consumers
|
|
never learn a path.
|
|
|
|
**Deployed 2026-10-18** — `./scripts/deploy.sh dankclaw --service frank-tts`,
|
|
both units `active`, no drop-ins, and the homepage card's `wip` gate came off in
|
|
the same change. Probes on the deployed endpoint: `/health` answers
|
|
`{"status":"ok","backend":"omni-http","voices":["narrator1","narrator2","narrator3"]}`,
|
|
and two different voices rendered the same sentence into **different** bytes
|
|
(24 kHz mono PCM16, 4.24 s, healthy RMS) — so name resolution is real, not a
|
|
pass-through of one default. The operator harness routes its `speech` role here;
|
|
see [[#The harness speech role]]. What still waits is [[Open Notebook]]: its
|
|
render needs the app installed, not this endpoint.
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| Host | Dankclaw (`192.168.0.143`), rootless podman, user unit `frank-tts.service` |
|
|
| Port | **7772**, published (`PublishPort=7772:7772`); the family is 7770 dragoon, 7771 zealot, 7772 tts |
|
|
| Routes | `GET /health`, `GET /v1/audio/voices`, `POST /v1/audio/speech`; everything else 404, and speech is POST-only |
|
|
| Image | `localhost/franksdank/frank-tts:latest`, built from `dankclaw/voiceservice/Containerfile`, `Pull=never` |
|
|
| Networks | `dankclaw_default` first (it owns the default route and the consumers), then `frank-tts-engine.network` (10.89.5.0/24, the bridge to the engine) |
|
|
| Voice packs | `/home/dank/voices` mounted read-only at `/voices` — host state, never repo state |
|
|
| Upstream | `http://frank-tts-engine:8880` (`scripts/frank-tts.env`, `TTS_UPSTREAM`) |
|
|
| Default voice | `DEFAULT_VOICE` (`scripts/frank-tts.env`) answers a request that names none; the journal marks that line `(default)`, and an **unknown** name is still a 400 |
|
|
| Rollback | `TTS_BACKEND=llama-tts-cli` runs a CLI binary per request and needs no upstream at all |
|
|
|
|
## Naming a voice
|
|
|
|
A voice **is** a directory name. `voice=narrator1` resolves to
|
|
`/voices/narrator1/reference.wav`, and that file — not a manifest, not a config
|
|
entry — is what makes the name real. omp's TTS device may send `voice_id`; the
|
|
shim accepts either. A call that sends **neither** — the harness's normal shape,
|
|
because no harness setting carries a voice for this transport — speaks
|
|
`$DEFAULT_VOICE`; empty default makes it a 400 again.
|
|
|
|
The usable shape is `^[a-z0-9][a-z0-9._-]*$`: lowercase, starting with a letter
|
|
or digit, then letters, digits, `.`, `_`, `-`, at most 64 characters. No slash,
|
|
no `..`, no space, no uppercase. The lowercase requirement is where two rules
|
|
meet: `app.py` would accept `[A-Za-z0-9…]`, but `tts-server` **lowercases
|
|
names**, so a pack called `Narrator1` resolves through the façade and not
|
|
through the engine. The boot warmer therefore refuses a mixed-case directory
|
|
loudly instead of letting it half-work.
|
|
|
|
Names are user-visible: an unknown voice comes back as
|
|
`400 unknown voice 'x'; available voices: …`, and that list is the honest
|
|
census. The podcast speakers are `narrator1`, `narrator2`, `narrator3` —
|
|
[[Open Notebook]] is configured with exactly those three, so renaming them
|
|
breaks episode rendering, not just speech. A personal voice gets the person's
|
|
name, lowercase.
|
|
|
|
## Voice packs
|
|
|
|
One directory per voice under `/home/dank/voices`:
|
|
|
|
```
|
|
reference.wav 24 kHz mono PCM, 18-25 s of clean single-speaker audio
|
|
reference.spk engine speaker embedding (8 192 B)
|
|
reference.rvq reference codec codes (22 B per 12.5 Hz frame)
|
|
manifest.json source file, offsets, engine, sha256
|
|
```
|
|
|
|
`reference.wav` is the engine-agnostic term the façade gates on; the other two
|
|
are what makes a voice reusable by this engine, and they come from the pack
|
|
builder ([[Voice Packs]]) — never by hand, never fabricated from the WAV. Two
|
|
laws apply to the
|
|
source material: offsets always start **past the first 10 seconds** of the
|
|
sample, and packs are host state that stays out of this repo. Because the
|
|
container runs as uid 10000, files must be world-readable (`644`/`755`), and the
|
|
directory must exist before the first deploy: rootless podman auto-creates a
|
|
missing bind source **root-owned**, which reads as "no voices" rather than as an
|
|
error.
|
|
|
|
Adding a voice to a running stack is a two-step thing — drop the pack, then
|
|
register **and warm** it with
|
|
`podman exec frank-tts-engine /app/warm-voices.sh` — because registering alone
|
|
does not warm it. The reason is the cold-voice law in [[Frank TTS Engine]], and
|
|
the procedure with its proof (the second request under a second) is the
|
|
`frank-tts-add-cloned-voice` skill.
|
|
|
|
## Why the upstream is a container name
|
|
|
|
`scripts/frank-tts.env` used to point at `http://host.containers.internal:8880`.
|
|
That cannot work, and it is the trap this shape exists to avoid: a bridge
|
|
container reaches the host's network namespace, netavark DNATs **published**
|
|
ports only, and an engine on the host's loopback is published to nobody — the
|
|
same shape that 502'd hermes ([[Container Network Namespaces]]). So both units
|
|
attach to one committed Quadlet network and the façade dials the engine's
|
|
container DNS name.
|
|
|
|
The consequence worth stating before an operator meets it: **the engine is not
|
|
reachable from the host at all** — not over the LAN, not over loopback. Its
|
|
health, voice registry and logs are observable only through
|
|
`podman exec frank-tts-engine …` and `systemctl --user status`. And the façade's
|
|
own `/health` answers 200 with the backend name and the voice list it resolved
|
|
without dialing the engine, so "façade healthy, engine down" is a real state: it
|
|
shows up as a failed synthesis, never as a down probe.
|
|
|
|
## Files
|
|
|
|
| Path | What |
|
|
| --- | --- |
|
|
| `dankclaw/voiceservice/app.py` | the shim: stdlib only — resolves the voice, forwards, re-encodes mp3 |
|
|
| `dankclaw/voiceservice/Containerfile` | python 3.12-alpine pinned by digest + ffmpeg, uid 10000 |
|
|
| `dankclaw/voiceservice/quadlet/frank-tts.container` | the unit: two networks, `Restart=always`, `HealthCmd`, `[Install]` |
|
|
| `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the shared bridge, named after the engine unit and attached by both |
|
|
| `scripts/frank-tts.env` | backend, upstream, port, limits, and the default voice — paths and numbers, no credentials |
|
|
| `tests/frank_tts_unit.bats` | the shim against stub engines over real sockets |
|
|
| `tests/frank_tts_engine.bats` | the pair's wiring: generator output, planner, homepage, deployer |
|
|
|
|
## Deploying it
|
|
|
|
`./scripts/deploy.sh dankclaw --service frank-tts` (or `--service frank-tts-engine --service frank-tts` for the pair):
|
|
builds the façade image, installs **both** Quadlet units — including the
|
|
engine's `.network` file, because a referenced `Network=` file the generator
|
|
cannot find means the façade unit is not produced at all — creates
|
|
`/home/dank/voices` as the deploy user, restarts the unit (whose `Wants=` and
|
|
`After=` pull the engine and its warm pass ahead of it) and gates on podman's
|
|
recorded health. It never runs `systemctl --user enable`; boot persistence is
|
|
the `[Install]` section in each unit. Order matters in a full deploy:
|
|
[[Frank TTS Engine]] is planned before this, and both before [[Open Notebook]].
|
|
|
|
## Not scraped, and why
|
|
|
|
`shared/prometheus/` gets **no** frank-tts job, and that is a considered
|
|
absence. The façade exports no `/metrics` — `app.py` serves `/health` plus the
|
|
speech route only — and this Prometheus 3 install refuses non-metrics content
|
|
types: [[Prometheus]] records that the existing HTML probes (`comfyui`, the two
|
|
`services` targets) sit **down** with
|
|
`received unsupported Content-Type … and no fallback_scrape_protocol specified`.
|
|
A `/health` probe job would therefore be structurally never-green, and because
|
|
`LlmEndpointDown` is scoped `up{job=~"frank-.*"} == 0`, a permanently red
|
|
`frank-tts` target would page on Signal every few minutes for an endpoint that
|
|
is fine — the frank-archon lesson pointed the other way. The engine cannot be
|
|
scraped at all: it publishes nothing. Status lives in `systemctl --user status`,
|
|
`podman healthcheck run`, the two unit journals, and the homepage card, which
|
|
probes `127.0.0.1:7772` for real now — the `wip: down` gate came off with the
|
|
2026-10-18 deploy.
|
|
|
|
## The harness speech role
|
|
|
|
The operator harness reaches this endpoint through
|
|
`~/.omp/agent/models.yml` — provider `frank-tts (dankclaw)`,
|
|
`baseUrl: http://192.168.0.143:7772/v1`, `auth: none`, and the runner api
|
|
`openai-speech` on the **model** row (naming a runner api at *provider* level
|
|
fails the whole file's validation and disables every custom provider, which is
|
|
how the fleet's chat endpoints disappear) — plus `config.yml`
|
|
`modelRoles.speech: frank-tts (dankclaw)/frank-tts`. Four laws, read out of the
|
|
harness bundle and measured the day of the flip:
|
|
|
|
- **The only voice the harness can send is the `voice_id` argument of the call.**
|
|
The transport builds `{model, input, response_format, voice?}` and adds `voice`
|
|
**only** when the caller passed `voice_id`; no setting feeds it.
|
|
`tts.localVoice` is the *local Kokoro backend's* voice (it is what answered
|
|
`voice=kokoro/bm_george, backend=local` in the control test), and
|
|
`speech.voice` is the TUI's **speak-aloud** voice — its enum is Kokoro's list,
|
|
and it never reaches an HTTP request. So a harness call arrives voice-less by
|
|
default, which is exactly why the endpoint grew `DEFAULT_VOICE`: the declared
|
|
pack answers a voice-less request and the journal marks that line `(default)`,
|
|
while an **unknown** name still 400s with the list that exists.
|
|
- **`speech.voice` must stay a Kokoro name.** Setting it to a pack name does not
|
|
select a pack; it hands the vocalizer a value outside its enum. (The first
|
|
draft of this section said the opposite and was reverted the same day, after a
|
|
fresh session's call came back `unknown voice None`.)
|
|
- **There is no silent fallback.** An explicitly configured `modelRoles.speech`
|
|
yields exactly one candidate, so a dead 7772 fails the speech device with
|
|
*"No available speech model matches the speech role"* instead of quietly
|
|
degrading to Kokoro. That is the price of `auth: none` (which makes a dead
|
|
endpoint read as available) and it is the wanted shape: a narration job should
|
|
fail loudly rather than ship a voice Frank never asked for.
|
|
- **A session older than the models.yml row cannot see it.** `models.yml` loads
|
|
at process start while `config.yml` is read live, so a long-lived session
|
|
carrying the flipped role fails exactly like a dead endpoint while a fresh
|
|
process succeeds in the same minute. Measured 2026-10-18: the open session
|
|
failed twice (provider/id form, then the bare id), `local/kokoro` succeeded in
|
|
that same session, and a fresh `omp -p --tools tts` returned `Saved 180524
|
|
bytes (model=frank-tts (dankclaw)/frank-tts, voice=narrator1, codec=wav)`
|
|
while the façade logged the matching `speech voice=narrator1 format=wav
|
|
chars=51 -> 180524 bytes` from the podman bridge. Rule the stale registry out
|
|
before blaming the service.
|
|
|
|
## Troubleshooting
|
|
|
|
| Symptom | Cause |
|
|
| --- | --- |
|
|
| `400 unknown voice 'x'; available voices: …` | the name is not a pack: case, spelling, missing `reference.wav`, or an empty/root-owned `/home/dank/voices` |
|
|
| `(none: no voice pack with a reference.wav in /voices)` | the mount, not the voices: `podman inspect frank-tts --format '{{json .Mounts}}'`, then the directory owner |
|
|
| Endpoint answers, synthesis fails | the engine: `podman healthcheck run frank-tts-engine`, then `systemctl --user status frank-tts-engine.service` |
|
|
| First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
|
|
| The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
|
|
| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
|