Frank
dd8f3974c1
Bare /say and /say menu open a ctx.ui.select settings surface: auto-speak toggle, voice from the frank-tts roster, PipeWire output device (pw-play --target / paplay --device=), player binary, spoken cap, and a test phrase. Choices persist to ~/.omp/cache/dankvoice.json (env wins over file); detectPlayer becomes availablePlayers + playerFor. Fixes the silent-route failure mode where synthesized audio played into an offline Bluetooth sink. 15/15 dankvoice bats, 51/51 suite, live pty TUI smoke armed the footer and wrote the settings file.
152 lines
8.0 KiB
Markdown
152 lines
8.0 KiB
Markdown
---
|
|
type: reference
|
|
title: dankvoice
|
|
description: omp extension that speaks assistant output with the fleet's cloned voices through frank-tts.
|
|
tags:
|
|
- omp
|
|
- extension
|
|
- tts
|
|
- voice
|
|
- frank-tts
|
|
- dankclaw
|
|
verified: 2026-10-18
|
|
timestamp: 2026-10-18
|
|
---
|
|
|
|
# dankvoice
|
|
|
|
`agent/extensions/dankvoice.ts` adds cloned-voice speech to the interactive
|
|
harness: `/say` opens the options menu, `/say <text>` speaks through the
|
|
speakers and `/say on` narrates the assistant's final answer after every
|
|
turn, all served by **frank-tts** (Dankclaw :7772, the `speech` model role
|
|
in [agent/models.yml](/core/index.md)).
|
|
|
|
## Why it exists
|
|
|
|
Nothing built into the harness can pronounce Frank's cloned voices.
|
|
|
|
| Surface | Engine | Verdict |
|
|
|---|---|---|
|
|
| TUI speak-aloud (`speech.enabled`, `speech.voice`) | bundled Kokoro ONNX worker, `synthesizeStream(voice, speed, signal)` | Kokoro voice enum only; never consults `models.yml` or the speech role |
|
|
| `omp say [TEXT]` | same local backend (`--model` = *local* model key) | `omp say --voice narrator1 "test"` → `error: Expected --voice to be one of: af_heart, af_bella, af_nicole, af_kore, af_sarah, am_michael, am_fenrir, am_puck, bf_emma, bm_george, bm_fable` (verified 2026-10-18) |
|
|
| `tts` tool / `xd://tts` device (`modelRoles.speech`) | frank-tts over OpenAI `/v1/audio/speech` | cloned voices work, but only as a **file** — nothing plays them |
|
|
|
|
`dankvoice` is not a patch to that vocalizer (harness code, not ours); it is a
|
|
second transport that happens to speak, so Kokoro stays available for quick
|
|
local speech and the cloned voices carry the real narration.
|
|
|
|
## Commands
|
|
|
|
| Command | Effect |
|
|
|---|---|
|
|
| `/say` \| `/say menu` | options menu (`ctx.ui.select`): auto-speak toggle, voice, output device, player, spoken cap, test phrase |
|
|
| `/say <text>` | synthesize once and play; interrupts whatever is playing |
|
|
| `/say on` \| `/say off` | arm/disarm auto-speak of the assistant's final text at `agent_end`; the footer shows `🔊 <voice>` while armed |
|
|
| `/say voice <name>` | pick a voice; an empty name defers to the server's own default |
|
|
| `/say voices` | list what frank-tts reports |
|
|
| `/say stop` | cut the current utterance |
|
|
| `/say status` \| `/say help` | endpoint, voice, player, device, cap |
|
|
|
|
Menu choices persist to `~/.omp/cache/dankvoice.json` (runtime state, never
|
|
tracked); the environment overrides the file, the file overrides defaults.
|
|
"Speak a test phrase" synthesizes a short line through the current voice,
|
|
player and device — the audible proof whenever a session "has say on" but
|
|
stays silent.
|
|
|
|
Auto-speak is skipped in `print`/`json` mode so benches and pipelines stay
|
|
silent. Markdown is stripped before synthesis — fenced code is dropped, links
|
|
collapse to their label, bullets and table pipes disappear — and the utterance
|
|
is truncated at a sentence boundary inside the cap.
|
|
|
|
## Endpoint resolution
|
|
|
|
The base URL is **read from `agent/models.yml`**, from the provider block whose
|
|
model row carries `api: openai-speech` — the same row the `speech` role
|
|
resolves, so no host address and no credential appears in this file. From
|
|
`baseUrl: http://192.168.0.143:7772/v1` it derives
|
|
`POST …/v1/audio/speech`, `GET …/v1/audio/voices` and `GET …/health`.
|
|
|
|
| Environment | Default | Meaning |
|
|
|---|---|---|
|
|
| `DANKVOICE_URL` | models.yml | speech base URL (`…/v1`), bypasses the YAML read |
|
|
| `DANKVOICE_MODEL` | the id in that block (`frank-tts`) | model named in the request body |
|
|
| `DANKVOICE_CONFIG` | `~/.omp/cache/dankvoice.json` | persisted menu choices (voice, player, sink, cap, auto) |
|
|
| `DANKVOICE_VOICE` | empty (server default) | default voice name |
|
|
| `DANKVOICE_AUTO=1` | off | start with auto-speak armed |
|
|
| `DANKVOICE_MAX` | `700` | spoken characters per utterance |
|
|
| `DANKVOICE_PLAYER` | detect | player binary, skipping detection |
|
|
| `DANKVOICE_MODELS` | `~/.omp/agent/models.yml` | YAML to read the endpoint from |
|
|
|
|
Player detection order is `pw-play` → `paplay` → `ffplay` (PipeWire first:
|
|
the workstations run WirePlumber). A saved output device routes the player
|
|
explicitly — `pw-play --target <sink>` or `paplay --device=<sink>`; ffplay
|
|
has no device concept and ignores it — while an unset device leaves the
|
|
player's own default. Sinks are enumerated with `wpctl list audio sinks`;
|
|
if wireplumber is unreachable the menu offers only "system default". Each
|
|
utterance is written to `$TMPDIR/dankvoice-*/say-<ms>.wav` and the file plus
|
|
its directory are removed when the player process exits.
|
|
|
|
## Verified facts (2026-10-18)
|
|
|
|
* **Extension discovery** — any `.ts` file in `~/.omp/agent/extensions/` loads
|
|
at startup: a probe file there printed `loaded 2026-10-19T02:36:55Z` from a
|
|
fresh `omp -p` run. `omp --help` documents `--no-extensions` (discovery off,
|
|
explicit `-e/--extension` and `--hook` still load) and `plugin install|link`.
|
|
* **Endpoint** — resolving the live `agent/models.yml` yields
|
|
`speechUrl http://192.168.0.143:7772/v1/audio/speech`,
|
|
`voicesUrl …/v1/audio/voices`, `healthUrl …/health`, `model frank-tts`.
|
|
* **Roster** — `GET /v1/audio/voices` answers the bare JSON array
|
|
`["narrator1","narrator2","narrator3"]`; `/health` answers
|
|
`{"status":"ok","backend":"omni-http","voices":[…]}`. `GET /v1/voices` and
|
|
`GET /v1/models` are **404** on this shim, and an unknown voice answers
|
|
`400 unknown voice 'zzz'; available voices: …` with `available_voices` in the
|
|
body — which is the fallback the extension uses if the roster route ever
|
|
disappears.
|
|
* **Omitting `voice` is legal** — frank-tts fills its `DEFAULT_VOICE` since the
|
|
2026-10-18 voice-pack work, so an unset voice synthesizes instead of failing.
|
|
* **Speech** — `/say "dankvoice live smoke test"` produced a 99,884 B / 2.08 s
|
|
WAV and `/say voice narrator2` + `/say "cloned voice check"` a 69,164 B /
|
|
1.44 s WAV (durations from `ffprobe`), each handed to the player; temp
|
|
directories were gone afterwards.
|
|
* **Auto-speak** — verified in a real TUI session: `/say on` armed the footer
|
|
(`🔊 narrator1`), the next finished turn synthesized and played the answer
|
|
(1.20 s WAV), and an unknown voice name answers with the roster instead of
|
|
speaking. An `agent_end` carrying `willContinue` (the harness scheduled an
|
|
auto-retry) is never narrated — only the terminal settle speaks.
|
|
* **Menu** — verified via driver scenarios 2026-10-18 (Refs #23): the menu
|
|
lists auto-speak/voice/device/player/cap/test-phrase; a voice chosen in the
|
|
menu is used by later utterances and survives a fresh load via the settings
|
|
file; a device chosen in the menu appears in the player invocation as
|
|
`--target <sink>`.
|
|
|
|
## Limits
|
|
|
|
* The TUI's own speak-aloud feature still narrates in Kokoro: its voice list is
|
|
a compile-time enum in the harness bundle, and no setting routes it to
|
|
frank-tts. Cloned voices live only on this file-producing/speaking path and
|
|
on the `tts` tool. See [Handy](/tools/handy.md) for the STT half.
|
|
* `agent_end` narrates the final message text only — mid-turn tool chatter is
|
|
never spoken.
|
|
* frank-tts is LAN-only: no Caddy vhost fronts :7772, so this extension is
|
|
usable on the home network.
|
|
* Deployment, packs and the voice registry are owned by the
|
|
`serverAppConfigs` repo (`serverAppConfigs-docs/Dankclaw/Frank TTS.md`), not
|
|
by this one.
|
|
|
|
## Citations
|
|
|
|
* Live probes on the workstation, 2026-10-18: `omp say --voice narrator1 …`,
|
|
`curl http://192.168.0.143:7772/health`, `/v1/audio/voices`,
|
|
`/v1/audio/…/speech` (invalid voice), `ffprobe` on the produced WAVs.
|
|
* `omp --help` (extension discovery flags, `say`, `plugin`) and
|
|
`@earendil-works/pi-coding-agent` `dist/core/extensions/types.d.ts`
|
|
(`ExtensionAPI.on/registerCommand`, `ExtensionMode = "tui" | "rpc" | "json" | "print"`).
|
|
* Gitea issue #21 on `dank/command-center` (the work item this page ships with).
|
|
* Gitea issue #23 on `dank/command-center` (the options-menu work item).
|
|
|
|
## See Also
|
|
|
|
* [Core Documentation](/core/index.md) — the live harness configuration.
|
|
* [Handy](/tools/handy.md) — offline speech-to-text, the input half of voice.
|
|
* `serverAppConfigs-docs/Dankclaw/Frank TTS.md` — the frank-tts service runbook.
|