dankvoice: speak long answers as sequential sentence-sized requests

agent_end awaited one whole-answer synthesis; long answers hit the
harness 30s extension-handler timeout (handler timed out after 30000ms)
and the single frank-tts request was the stall. Port the built-in
speak-aloud vocalizer's utterance segmenter (fHe in dist/cli.js) so
segmentation matches the known-working stock path: [.!?…]-run
boundaries, abbreviation/decimal/inline-code guards, fenced code and
tables swallowed, markup cleaned. Play segment N to player exit before
synthesizing N+1 (audio never stacks); fire-and-forget the chain so
utterances outlive the handler timeout; a run token retires superseded
chains. DANKVOICE_MAX becomes a per-answer budget of whole segments;
stripForSpeech is gone. Refs #23 follow-up.
This commit is contained in:
Frank
2026-10-19 01:21:34 -07:00
parent edd7e0cf68
commit bcd52bb0f4
4 changed files with 457 additions and 52 deletions
+48 -7
View File
@@ -9,8 +9,8 @@ tags:
- voice
- frank-tts
- dankclaw
verified: 2026-10-18
timestamp: 2026-10-18
verified: 2026-10-19
timestamp: 2026-10-19
---
# dankvoice
@@ -54,9 +54,38 @@ player and device — the audible proof whenever a session "has say on" but
stays silent.
Auto-speak is skipped in `print`/`json` mode so benches and pipelines stay
silent. Markdown is stripped before synthesis — fenced code is dropped, links
collapse to their label, bullets and table pipes disappear — and the utterance
is truncated at a sentence boundary inside the cap.
silent. A long answer is never one monolithic synthesis request; see
[Segmentation](#segmentation) for how utterances are cut and played.
## Segmentation
Utterances are cut by a state machine ported from the harness's own
speak-aloud vocalizer (the minified `fHe` class in the agent bundle's
`dist/cli.js`), kept deliberately in lockstep with the stock path:
* A strong boundary is a run of `[.!?…]` followed by whitespace — so
`!!!!!` and `...` are one boundary, never a pile of cuts.
* Boundaries after `e.g.`, `etc.`, `Mr.` and friends, or inside inline
code, don't count; decimals (`3.5`, `1,5`) survive because a boundary
requires trailing whitespace.
* Line starts are classified: table rows and fenced code are swallowed
whole (their content is never read aloud), heading and bullet markers
vanish, ordered-list markers speak their number ("1, …"), quote markers
strip.
* Per segment, markdown is cleaned: links collapse to their label, URLs
to their host, deep paths to their last segment; emphasis markup dies.
* Segments aim for one sentence; run-ons break at commas or word wraps
before 280 characters.
Playback is strictly sequential: segment N+1 is synthesized only after
segment N's player process exited, so audio never stacks. The chain runs
detached from `agent_end` (fire-and-forget) and may legitimately outlive
the harness's 30 s handler timeout — the old awaiting design produced a
"handler timed out after 30000ms" toast on long answers. A new utterance,
`/say stop`, or disarming auto-speak retires the live chain through a run
token: the current player is killed and pending segments are skipped.
`DANKVOICE_MAX` is a per-answer character budget: the leading whole
segments that fit are spoken, the rest are dropped.
## Endpoint resolution
@@ -73,7 +102,7 @@ resolves, so no host address and no credential appears in this file. From
| `DANKVOICE_CONFIG` | `~/.omp/cache/dankvoice.json` | persisted menu choices (voice, player, sink, cap, auto) |
| `DANKVOICE_VOICE` | empty (server default) | default voice name |
| `DANKVOICE_AUTO=1` | off | start with auto-speak armed |
| `DANKVOICE_MAX` | `700` | spoken characters per utterance |
| `DANKVOICE_MAX` | `700` | spoken character budget per answer (whole leading segments) |
| `DANKVOICE_PLAYER` | detect | player binary, skipping detection |
| `DANKVOICE_MODELS` | `~/.omp/agent/models.yml` | YAML to read the endpoint from |
@@ -86,7 +115,7 @@ if wireplumber is unreachable the menu offers only "system default". Each
utterance is written to `$TMPDIR/dankvoice-*/say-<ms>.wav` and the file plus
its directory are removed when the player process exits.
## Verified facts (2026-10-18)
## Verified facts (2026-10-18, 2026-10-19)
* **Extension discovery** — any `.ts` file in `~/.omp/agent/extensions/` loads
at startup: a probe file there printed `loaded 2026-10-19T02:36:55Z` from a
@@ -118,6 +147,12 @@ its directory are removed when the player process exits.
menu is used by later utterances and survives a fresh load via the settings
file; a device chosen in the menu appears in the player invocation as
`--target <sink>`.
* **Segmented playback** — driver scenarios 2026-10-19: "First stop here.
Second one goes!!! Really? Third ... done" produced exactly 3 sequential
requests and 3 plays; "Model 3.5 ships today. Version 1,5 too." produced
exactly 2 (decimals never split); the markdown fixture's fenced code and
URL never reached the fixture endpoint; with `DANKVOICE_MAX=60` only the
leading segment was spoken. `bats tests/dankvoice.bats` 17/17.
## Limits
@@ -141,6 +176,12 @@ its directory are removed when the player process exits.
* `omp --help` (extension discovery flags, `say`, `plugin`) and
`@earendil-works/pi-coding-agent` `dist/core/extensions/types.d.ts`
(`ExtensionAPI.on/registerCommand`, `ExtensionMode = "tui" | "rpc" | "json" | "print"`).
* The stock speak-aloud implementation — vocalizer pipeline `I6o` and
utterance segmenter `fHe` — in `@oh-my-pi/pi-coding-agent`'s
`dist/cli.js`, read off the minified bundle 2026-10-19; the enhanced
rewrite ("with the tiny/smol model before synthesis") is that bundle's
own `speech.enhanced` description. dankvoice ports the mechanical half
only: no LLM rewrite happens before frank-tts.
* Gitea issue #21 on `dank/command-center` (the work item this page ships with).
* Gitea issue #23 on `dank/command-center` (the options-menu work item).
+23
View File
@@ -12,6 +12,29 @@ timestamp: 2026-08-28
# Bundle Change Log
## 2026-10-19 — dankvoice: segmented sequential speech, copied from the bundle's vocalizer
Silent-answer incident follow-up: `agent_end` awaiting one whole-answer
synthesis hit the harness's 30 s extension-handler timeout
(`handler timed out after 30000ms`) on long answers, and the single frank-tts
request for the entire answer was the stall. Instead of inventing a heuristic,
ported the utterance-splitting state machine out of the built-in speak-aloud
vocalizer (minified class `fHe` in `dist/cli.js`): strong boundaries are
`[.!?…]+` runs followed by whitespace — `!!!!!`/`...` stay one cut — with
abbreviation and inline-code guards; `3.5`/`1,5` survive because a boundary
needs trailing whitespace; fenced code and tables swallow whole; headings and
bullets vanish, ordered markers speak their number ("1, "); links collapse to
labels, URLs to hosts, paths to basenames; run-ons break at commas or word
wraps before 280 chars. `speak()` now synthesizes segment N+1 only after
segment N's player exited — strictly sequential, audio never stacks — and
`agent_end`/`/say <text>` fire-and-forget a chain retired through a run token
by `/say stop`, disarming, or a newer utterance. `DANKVOICE_MAX` became a
per-answer budget of whole leading segments; the hand-rolled `stripForSpeech`
truncator is gone. No LLM "humanization" rewrite happens (the bundle's
`speech.enhanced` rewrites with the tiny/smol model; dankvoice speaks
mechanically cleaned text through the cloned voices). Tests: two new driver
scenarios plus two rewrites, 17/17 in `tests/dankvoice.bats`.
## 2026-10-18 — dankvoice: `/say` options menu with output-device routing (Refs #23)
Added `/say menu` (bare `/say` now opens it; `help` keeps the text listing), the