dankvoice: speak long answers as sequential sentence-sized requests
agent_end awaited one whole-answer synthesis; long answers hit the harness 30s extension-handler timeout (handler timed out after 30000ms) and the single frank-tts request was the stall. Port the built-in speak-aloud vocalizer's utterance segmenter (fHe in dist/cli.js) so segmentation matches the known-working stock path: [.!?…]-run boundaries, abbreviation/decimal/inline-code guards, fenced code and tables swallowed, markup cleaned. Play segment N to player exit before synthesizing N+1 (audio never stacks); fire-and-forget the chain so utterances outlive the handler timeout; a run token retires superseded chains. DANKVOICE_MAX becomes a per-answer budget of whole segments; stripForSpeech is gone. Refs #23 follow-up.
This commit is contained in:
@@ -9,8 +9,8 @@ tags:
|
||||
- voice
|
||||
- frank-tts
|
||||
- dankclaw
|
||||
verified: 2026-10-18
|
||||
timestamp: 2026-10-18
|
||||
verified: 2026-10-19
|
||||
timestamp: 2026-10-19
|
||||
---
|
||||
|
||||
# dankvoice
|
||||
@@ -54,9 +54,38 @@ player and device — the audible proof whenever a session "has say on" but
|
||||
stays silent.
|
||||
|
||||
Auto-speak is skipped in `print`/`json` mode so benches and pipelines stay
|
||||
silent. Markdown is stripped before synthesis — fenced code is dropped, links
|
||||
collapse to their label, bullets and table pipes disappear — and the utterance
|
||||
is truncated at a sentence boundary inside the cap.
|
||||
silent. A long answer is never one monolithic synthesis request; see
|
||||
[Segmentation](#segmentation) for how utterances are cut and played.
|
||||
|
||||
## Segmentation
|
||||
|
||||
Utterances are cut by a state machine ported from the harness's own
|
||||
speak-aloud vocalizer (the minified `fHe` class in the agent bundle's
|
||||
`dist/cli.js`), kept deliberately in lockstep with the stock path:
|
||||
|
||||
* A strong boundary is a run of `[.!?…]` followed by whitespace — so
|
||||
`!!!!!` and `...` are one boundary, never a pile of cuts.
|
||||
* Boundaries after `e.g.`, `etc.`, `Mr.` and friends, or inside inline
|
||||
code, don't count; decimals (`3.5`, `1,5`) survive because a boundary
|
||||
requires trailing whitespace.
|
||||
* Line starts are classified: table rows and fenced code are swallowed
|
||||
whole (their content is never read aloud), heading and bullet markers
|
||||
vanish, ordered-list markers speak their number ("1, …"), quote markers
|
||||
strip.
|
||||
* Per segment, markdown is cleaned: links collapse to their label, URLs
|
||||
to their host, deep paths to their last segment; emphasis markup dies.
|
||||
* Segments aim for one sentence; run-ons break at commas or word wraps
|
||||
before 280 characters.
|
||||
|
||||
Playback is strictly sequential: segment N+1 is synthesized only after
|
||||
segment N's player process exited, so audio never stacks. The chain runs
|
||||
detached from `agent_end` (fire-and-forget) and may legitimately outlive
|
||||
the harness's 30 s handler timeout — the old awaiting design produced a
|
||||
"handler timed out after 30000ms" toast on long answers. A new utterance,
|
||||
`/say stop`, or disarming auto-speak retires the live chain through a run
|
||||
token: the current player is killed and pending segments are skipped.
|
||||
`DANKVOICE_MAX` is a per-answer character budget: the leading whole
|
||||
segments that fit are spoken, the rest are dropped.
|
||||
|
||||
## Endpoint resolution
|
||||
|
||||
@@ -73,7 +102,7 @@ resolves, so no host address and no credential appears in this file. From
|
||||
| `DANKVOICE_CONFIG` | `~/.omp/cache/dankvoice.json` | persisted menu choices (voice, player, sink, cap, auto) |
|
||||
| `DANKVOICE_VOICE` | empty (server default) | default voice name |
|
||||
| `DANKVOICE_AUTO=1` | off | start with auto-speak armed |
|
||||
| `DANKVOICE_MAX` | `700` | spoken characters per utterance |
|
||||
| `DANKVOICE_MAX` | `700` | spoken character budget per answer (whole leading segments) |
|
||||
| `DANKVOICE_PLAYER` | detect | player binary, skipping detection |
|
||||
| `DANKVOICE_MODELS` | `~/.omp/agent/models.yml` | YAML to read the endpoint from |
|
||||
|
||||
@@ -86,7 +115,7 @@ if wireplumber is unreachable the menu offers only "system default". Each
|
||||
utterance is written to `$TMPDIR/dankvoice-*/say-<ms>.wav` and the file plus
|
||||
its directory are removed when the player process exits.
|
||||
|
||||
## Verified facts (2026-10-18)
|
||||
## Verified facts (2026-10-18, 2026-10-19)
|
||||
|
||||
* **Extension discovery** — any `.ts` file in `~/.omp/agent/extensions/` loads
|
||||
at startup: a probe file there printed `loaded 2026-10-19T02:36:55Z` from a
|
||||
@@ -118,6 +147,12 @@ its directory are removed when the player process exits.
|
||||
menu is used by later utterances and survives a fresh load via the settings
|
||||
file; a device chosen in the menu appears in the player invocation as
|
||||
`--target <sink>`.
|
||||
* **Segmented playback** — driver scenarios 2026-10-19: "First stop here.
|
||||
Second one goes!!! Really? Third ... done" produced exactly 3 sequential
|
||||
requests and 3 plays; "Model 3.5 ships today. Version 1,5 too." produced
|
||||
exactly 2 (decimals never split); the markdown fixture's fenced code and
|
||||
URL never reached the fixture endpoint; with `DANKVOICE_MAX=60` only the
|
||||
leading segment was spoken. `bats tests/dankvoice.bats` 17/17.
|
||||
|
||||
## Limits
|
||||
|
||||
@@ -141,6 +176,12 @@ its directory are removed when the player process exits.
|
||||
* `omp --help` (extension discovery flags, `say`, `plugin`) and
|
||||
`@earendil-works/pi-coding-agent` `dist/core/extensions/types.d.ts`
|
||||
(`ExtensionAPI.on/registerCommand`, `ExtensionMode = "tui" | "rpc" | "json" | "print"`).
|
||||
* The stock speak-aloud implementation — vocalizer pipeline `I6o` and
|
||||
utterance segmenter `fHe` — in `@oh-my-pi/pi-coding-agent`'s
|
||||
`dist/cli.js`, read off the minified bundle 2026-10-19; the enhanced
|
||||
rewrite ("with the tiny/smol model before synthesis") is that bundle's
|
||||
own `speech.enhanced` description. dankvoice ports the mechanical half
|
||||
only: no LLM rewrite happens before frank-tts.
|
||||
* Gitea issue #21 on `dank/command-center` (the work item this page ships with).
|
||||
* Gitea issue #23 on `dank/command-center` (the options-menu work item).
|
||||
|
||||
|
||||
@@ -12,6 +12,29 @@ timestamp: 2026-08-28
|
||||
# Bundle Change Log
|
||||
|
||||
|
||||
## 2026-10-19 — dankvoice: segmented sequential speech, copied from the bundle's vocalizer
|
||||
|
||||
Silent-answer incident follow-up: `agent_end` awaiting one whole-answer
|
||||
synthesis hit the harness's 30 s extension-handler timeout
|
||||
(`handler timed out after 30000ms`) on long answers, and the single frank-tts
|
||||
request for the entire answer was the stall. Instead of inventing a heuristic,
|
||||
ported the utterance-splitting state machine out of the built-in speak-aloud
|
||||
vocalizer (minified class `fHe` in `dist/cli.js`): strong boundaries are
|
||||
`[.!?…]+` runs followed by whitespace — `!!!!!`/`...` stay one cut — with
|
||||
abbreviation and inline-code guards; `3.5`/`1,5` survive because a boundary
|
||||
needs trailing whitespace; fenced code and tables swallow whole; headings and
|
||||
bullets vanish, ordered markers speak their number ("1, "); links collapse to
|
||||
labels, URLs to hosts, paths to basenames; run-ons break at commas or word
|
||||
wraps before 280 chars. `speak()` now synthesizes segment N+1 only after
|
||||
segment N's player exited — strictly sequential, audio never stacks — and
|
||||
`agent_end`/`/say <text>` fire-and-forget a chain retired through a run token
|
||||
by `/say stop`, disarming, or a newer utterance. `DANKVOICE_MAX` became a
|
||||
per-answer budget of whole leading segments; the hand-rolled `stripForSpeech`
|
||||
truncator is gone. No LLM "humanization" rewrite happens (the bundle's
|
||||
`speech.enhanced` rewrites with the tiny/smol model; dankvoice speaks
|
||||
mechanically cleaned text through the cloned voices). Tests: two new driver
|
||||
scenarios plus two rewrites, 17/17 in `tests/dankvoice.bats`.
|
||||
|
||||
## 2026-10-18 — dankvoice: `/say` options menu with output-device routing (Refs #23)
|
||||
|
||||
Added `/say menu` (bare `/say` now opens it; `help` keeps the text listing), the
|
||||
|
||||
Reference in New Issue
Block a user