feat(dankvoice): speak the cloned voices in the TUI (Refs #21) #22

Merged
dank merged 1 commits from feat/21-dankvoice into main 2026-10-18 20:06:31 -07:00
Owner

What

/say speaks in the TUI with the frank-tts cloned voices. Every speak-aloud path the harness ships (TUI vocalizer, omp say) gates --voice against the bundled Kokoro enum — omp say --voice narrator1 "test" → error: Expected --voice to be one of: af_heart, … — so narrator1/2/3 were reachable only as files.

agent/extensions/ auto-loads, so one tracked extension carries the missing transport, plus its black-box seam and a docs page.

Commands

Command Effect
/say <text> synthesize and play now
/say on | /say off narrate the final answer at agent_end; footer shows 🔊 <voice> while armed
/say voice <name> | /say voices pick a voice / list the endpoint roster
/say stop | /say status kill playback / print endpoint and state

Endpoint and served id come from the api: openai-speech row of agent/models.yml — the extension file carries no host and no secret. Overrides: DANKVOICE_URL, DANKVOICE_MODEL, DANKVOICE_VOICE, DANKVOICE_PLAYER, DANKVOICE_MAX, DANKVOICE_AUTO, DANKVOICE_MODELS. Utterances are markdown-stripped and bounded (DANKVOICE_MAX 700 chars, cut at a sentence); playback order pw-play → paplay → ffplay -nodisp -autoexit; the temp WAV directory is removed in a finally.

Verification

  • Live TUI (omp --no-extensions -e <ext>, stub player standing in for the speakers): roster GET /v1/audio/voices → narrator1, narrator2, narrator3; /say produced 99,884 B / 2.08 s and 69,164 B / 1.44 s WAVs handed to the player, temp dirs gone afterwards.
  • Auto-speak: /say on armed the footer (🔊 narrator1) and every finished turn spoke its answer (1.20 s WAV for a five-word answer); /say off, print mode, and an agent_end carrying willContinue (the harness scheduled an auto-retry — not a terminal settle) all stay silent.
  • Failure surface: unknown voice → dankvoice: unknown voice 'x'; available voices: narrator1, narrator2, narrator3; missing player → NOTIFY error dankvoice: no audio player found with zero plays attempted.
  • Suite: bats tests/ → 48/48 (12 in tests/dankvoice.bats — the extension is loaded against a fixture speech endpoint and a stub player, assertions on observable requests/playback only).

Notes

  • The TUI's own speak-aloud feature still narrates in Kokoro: its voice list is a compile-time enum in the harness bundle and no setting routes it to frank-tts. Cloned voices live on this path and the tts tool.
  • frank-tts is LAN-only (192.168.0.143:7772, no Caddy vhost); the engine, voice packs and registry are owned by dank/serverAppConfigs.

cwd: /home/blank/.omp/wt/feat-21-dankvoice
session: wt-20261017-181502-6d1fc1f

## What `/say` speaks in the TUI with the **frank-tts cloned voices**. Every speak-aloud path the harness ships (TUI vocalizer, `omp say`) gates `--voice` against the bundled Kokoro enum — `omp say --voice narrator1 "test"` → `error: Expected --voice to be one of: af_heart, …` — so `narrator1/2/3` were reachable only as files. `agent/extensions/` auto-loads, so one tracked extension carries the missing transport, plus its black-box seam and a docs page. ## Commands | Command | Effect | |---|---| | `/say <text>` | synthesize and play now | | `/say on` \| `/say off` | narrate the final answer at `agent_end`; footer shows `🔊 <voice>` while armed | | `/say voice <name>` \| `/say voices` | pick a voice / list the endpoint roster | | `/say stop` \| `/say status` | kill playback / print endpoint and state | Endpoint and served id come from the `api: openai-speech` row of `agent/models.yml` — the extension file carries **no host and no secret**. Overrides: `DANKVOICE_URL`, `DANKVOICE_MODEL`, `DANKVOICE_VOICE`, `DANKVOICE_PLAYER`, `DANKVOICE_MAX`, `DANKVOICE_AUTO`, `DANKVOICE_MODELS`. Utterances are markdown-stripped and bounded (`DANKVOICE_MAX` 700 chars, cut at a sentence); playback order `pw-play` → `paplay` → `ffplay -nodisp -autoexit`; the temp WAV directory is removed in a `finally`. ## Verification * **Live TUI** (`omp --no-extensions -e <ext>`, stub player standing in for the speakers): roster `GET /v1/audio/voices` → `narrator1, narrator2, narrator3`; `/say` produced 99,884 B / 2.08 s and 69,164 B / 1.44 s WAVs handed to the player, temp dirs gone afterwards. * **Auto-speak**: `/say on` armed the footer (`🔊 narrator1`) and every finished turn spoke its answer (1.20 s WAV for a five-word answer); `/say off`, print mode, and an `agent_end` carrying `willContinue` (the harness scheduled an auto-retry — not a terminal settle) all stay silent. * **Failure surface**: unknown voice → `dankvoice: unknown voice 'x'; available voices: narrator1, narrator2, narrator3`; missing player → `NOTIFY error dankvoice: no audio player found` with zero plays attempted. * **Suite**: `bats tests/` → **48/48** (12 in `tests/dankvoice.bats` — the extension is loaded against a fixture speech endpoint and a stub player, assertions on observable requests/playback only). ## Notes * The TUI's own speak-aloud feature still narrates in Kokoro: its voice list is a compile-time enum in the harness bundle and no setting routes it to frank-tts. Cloned voices live on this path and the `tts` tool. * frank-tts is LAN-only (`192.168.0.143:7772`, no Caddy vhost); the engine, voice packs and registry are owned by `dank/serverAppConfigs`. cwd: /home/blank/.omp/wt/feat-21-dankvoice session: wt-20261017-181502-6d1fc1f
dank added 1 commit 2026-10-18 20:01:26 -07:00
Every speak-aloud path in the harness gates `--voice` against the bundled
Kokoro enum, so the frank-tts cloned voices could only ever produce files.
`agent/extensions/` auto-loads, so one tracked extension adds the missing
transport: `/say <text>`, `/say on|off` (narrate the final answer at
`agent_end`, silent when the harness schedules a continuation),
`/say voice|voices|stop|status` — OpenAI speech POST plus playback through
pw-play/paplay/ffplay. The endpoint and served id come from the
`api: openai-speech` row of `agent/models.yml`, so the file carries no host
and no secret; markdown is stripped, over-long utterances are cut at a
sentence bound, and the temp WAV dir is always removed.

Verified live in a real TUI session on the workstation (roster
narrator1/narrator2/narrator3; 2.08 s and 1.44 s WAVs played; auto-speak
spoke each finished answer, 1.20 s WAV; unknown voice answers with the
roster). Black-box seam `tests/dankvoice.bats` (12 tests) drives the
extension against a fixture speech endpoint and a stub player.

cwd: /home/blank/.omp/wt/feat-21-dankvoice
session: wt-20261017-181502-6d1fc1f
dank merged commit 0c7fe8b4bf into main 2026-10-18 20:06:31 -07:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dank/command-center#22