dankvoice: humanize answers via the @tiny role and narrate thinking
Answers spoken with /say were verbatim markdown-cleaned text while the built-in vocalizer speaks prose rewritten by a small model. Port that enhancer (bundle Qyr/T6o: same prompt, 1536 tokens, 6 s, 4000-char window, null-fallback/empty-skip contract) and resolve its model from modelRoles.tiny exactly as the harness does, so auto-speak sounds like the enhanced TUI voice instead of a pager. agent_end now also speaks the last assistant message's thinking blocks as their own unit before the answer; narration-flagged thinking counts as answer text. Both knobs are menu-toggleable, persisted, and env-overridable; the fixture gains a chat route (echo/prose) and seven scenarios cover the round-trip, the 404 fallback, ordering, the kill switches and role resolution (22 green, 58 suite-wide; live smoke through real frank-zealot + frank-tts). Refs #27
This commit is contained in:
+67
-19
@@ -17,9 +17,11 @@ timestamp: 2026-10-19
|
||||
|
||||
`agent/extensions/dankvoice.ts` adds cloned-voice speech to the interactive
|
||||
harness: `/say` opens the options menu, `/say <text>` speaks through the
|
||||
speakers and `/say on` narrates the assistant's final answer after every
|
||||
turn, all served by **frank-tts** (Dankclaw :7772, the `speech` model role
|
||||
in [agent/models.yml](/core/index.md)).
|
||||
speakers and `/say on` narrates the assistant's thinking and final answer
|
||||
after every turn. The words are first rewritten into natural spoken prose by
|
||||
the **@tiny** role model (exactly what the built-in vocalizer's
|
||||
`speech.enhanced` does), and everything is served by **frank-tts**
|
||||
(Dankclaw :7772, the `speech` model role in [agent/models.yml](/core/index.md)).
|
||||
|
||||
## Why it exists
|
||||
|
||||
@@ -39,13 +41,13 @@ local speech and the cloned voices carry the real narration.
|
||||
|
||||
| Command | Effect |
|
||||
|---|---|
|
||||
| `/say` \| `/say menu` | options menu (`ctx.ui.select`): auto-speak toggle, voice, output device, player, spoken cap, test phrase |
|
||||
| `/say` \| `/say menu` | options menu (`ctx.ui.select`): auto-speak toggle, voice, output device, player, spoken cap, thinking-narration toggle, humanize toggle, test phrase |
|
||||
| `/say <text>` | synthesize once and play; interrupts whatever is playing |
|
||||
| `/say on` \| `/say off` | arm/disarm auto-speak of the assistant's final text at `agent_end`; the footer shows `🔊 <voice>` while armed |
|
||||
| `/say on` \| `/say off` | arm/disarm auto-speak of the thinking blocks and final text at `agent_end`; the footer shows `🔊 <voice>` while armed |
|
||||
| `/say voice <name>` | pick a voice; an empty name defers to the server's own default |
|
||||
| `/say voices` | list what frank-tts reports |
|
||||
| `/say stop` | cut the current utterance |
|
||||
| `/say status` \| `/say help` | endpoint, voice, player, device, cap |
|
||||
| `/say status` \| `/say help` | endpoint, voice, player, device, cap, thinking, humanize |
|
||||
|
||||
Menu choices persist to `~/.omp/cache/dankvoice.json` (runtime state, never
|
||||
tracked); the environment overrides the file, the file overrides defaults.
|
||||
@@ -54,7 +56,11 @@ player and device — the audible proof whenever a session "has say on" but
|
||||
stays silent.
|
||||
|
||||
Auto-speak is skipped in `print`/`json` mode so benches and pipelines stay
|
||||
silent. A long answer is never one monolithic synthesis request; see
|
||||
silent. Each spoken unit — thinking first when narrated, then the answer —
|
||||
is first rewritten into spoken prose by the @tiny model; when the enhancer
|
||||
(below) declines, the text goes to segmentation as-is, where fenced code and
|
||||
tables vanish, links collapse to their label and bullets and table pipes
|
||||
disappear. A long answer is never one monolithic synthesis request; see
|
||||
[Segmentation](#segmentation) for how utterances are cut and played.
|
||||
|
||||
## Segmentation
|
||||
@@ -85,7 +91,8 @@ the harness's 30 s handler timeout — the old awaiting design produced a
|
||||
`/say stop`, or disarming auto-speak retires the live chain through a run
|
||||
token: the current player is killed and pending segments are skipped.
|
||||
`DANKVOICE_MAX` is a per-answer character budget: the leading whole
|
||||
segments that fit are spoken, the rest are dropped.
|
||||
segments that fit are spoken, the rest are dropped — the thinking unit
|
||||
draws on it before the answer does.
|
||||
|
||||
## Endpoint resolution
|
||||
|
||||
@@ -102,9 +109,14 @@ resolves, so no host address and no credential appears in this file. From
|
||||
| `DANKVOICE_CONFIG` | `~/.omp/cache/dankvoice.json` | persisted menu choices (voice, player, sink, cap, auto) |
|
||||
| `DANKVOICE_VOICE` | empty (server default) | default voice name |
|
||||
| `DANKVOICE_AUTO=1` | off | start with auto-speak armed |
|
||||
| `DANKVOICE_MAX` | `700` | spoken character budget per answer (whole leading segments) |
|
||||
| `DANKVOICE_MAX` | `700` | spoken-character budget per answer (whole leading segments, shared by the thinking and answer units) |
|
||||
| `DANKVOICE_PLAYER` | detect | player binary, skipping detection |
|
||||
| `DANKVOICE_MODELS` | `~/.omp/agent/models.yml` | YAML to read the endpoint from |
|
||||
| `DANKVOICE_REWRITE_URL` | the @tiny role | chat base URL (`…/v1`) for the enhancer, bypassing role resolution |
|
||||
| `DANKVOICE_REWRITE_MODEL` | the @tiny role's model | chat model named in the rewrite body |
|
||||
| `DANKVOICE_REWRITE=0` | enhancer on when a route resolves | speak mechanical text |
|
||||
| `DANKVOICE_THINKING` | settings file, default on | `0`/`1` force thinking narration off/on |
|
||||
| `DANKVOICE_OMPCONFIG` | `~/.omp/agent/config.yml` | YAML to read the `modelRoles.tiny` role from |
|
||||
|
||||
Player detection order is `pw-play` → `paplay` → `ffplay` (PipeWire first:
|
||||
the workstations run WirePlumber). A saved output device routes the player
|
||||
@@ -115,7 +127,29 @@ if wireplumber is unreachable the menu offers only "system default". Each
|
||||
utterance is written to `$TMPDIR/dankvoice-*/say-<ms>.wav` and the file plus
|
||||
its directory are removed when the player process exits.
|
||||
|
||||
## Verified facts (2026-10-18, 2026-10-19)
|
||||
## Humanized narration (the @tiny role)
|
||||
|
||||
The built-in vocalizer never reads answers aloud verbatim when
|
||||
`speech.enhanced` is on: a small model rewrites each utterance into words a
|
||||
person would say. `dankvoice` ports that enhancer — same system prompt
|
||||
(verbatim from the bundle), same budgets (1536 max tokens, 6 s timeout,
|
||||
4000-char input window), same contract (a failed rewrite falls back to the
|
||||
mechanically cleaned text; an empty rewrite skips the unit as "nothing to
|
||||
say"). The target model is the harness's `@tiny` role, resolved the way the
|
||||
harness resolves it: `modelRoles.tiny` in `agent/config.yml` reads
|
||||
`provider (host)/model:effort`; the provider block in `agent/models.yml`
|
||||
supplies the `baseUrl`; the request is `POST {base}/chat/completions`. On
|
||||
this fleet that is **frank-zealot** (`llamacpp (dankclaw-3090)`,
|
||||
`:7771`) — the 6 s abort keeps a slow tiny model off the critical path.
|
||||
|
||||
Thinking blocks ride the same path: at `agent_end` the last assistant
|
||||
message's true `thinking` parts are narrated (rewritten first, as their own
|
||||
unit, before the answer), while blocks flagged `narration: true` — which
|
||||
providers use to address the user through the thinking channel — count as
|
||||
answer text. The menu, `/say status`, and `DANKVOICE_THINKING` toggle it;
|
||||
the choice persists in the settings file.
|
||||
|
||||
## Verified facts (2026-10-18, extended 2026-10-19)
|
||||
|
||||
* **Extension discovery** — any `.ts` file in `~/.omp/agent/extensions/` loads
|
||||
at startup: a probe file there printed `loaded 2026-10-19T02:36:55Z` from a
|
||||
@@ -152,7 +186,16 @@ its directory are removed when the player process exits.
|
||||
requests and 3 plays; "Model 3.5 ships today. Version 1,5 too." produced
|
||||
exactly 2 (decimals never split); the markdown fixture's fenced code and
|
||||
URL never reached the fixture endpoint; with `DANKVOICE_MAX=60` only the
|
||||
leading segment was spoken. `bats tests/dankvoice.bats` 17/17.
|
||||
leading segment was spoken.
|
||||
* **Humanize + thinking (2026-10-19, Refs #27)** — driver scenarios cover the
|
||||
rewrite round-trip (fixture chat route), the 404 → mechanical fallback,
|
||||
thinking narrated as its own unit before the answer, `DANKVOICE_THINKING=0`
|
||||
silencing it, and `narration: true` spoken once as the answer; the endpoint
|
||||
status line shows the resolved humanize model from the @tiny role. Live
|
||||
smoke against the real services: a markdown `/say` became 3 sequential
|
||||
prose segments (mechanical cleanup of that text yields at most 2), and a
|
||||
thinking+answer `agent_end` played thinking first. 22 green in
|
||||
`tests/dankvoice.bats`.
|
||||
|
||||
## Limits
|
||||
|
||||
@@ -160,8 +203,13 @@ its directory are removed when the player process exits.
|
||||
a compile-time enum in the harness bundle, and no setting routes it to
|
||||
frank-tts. Cloned voices live only on this file-producing/speaking path and
|
||||
on the `tts` tool. See [Handy](/tools/handy.md) for the STT half.
|
||||
* `agent_end` narrates the final message text only — mid-turn tool chatter is
|
||||
never spoken.
|
||||
* `agent_end` narrates thinking blocks and the final message only — mid-turn
|
||||
tool chatter is never spoken. Thinking covers the last assistant message;
|
||||
tool-loop gaps in earlier messages are skipped by design.
|
||||
* Humanized narration is serial: audio starts only after the @tiny rewrite
|
||||
returns (≤ 6 s per unit, then frank-tts synthesis). Say `/say off`-grade
|
||||
latency is the price of prose; `DANKVOICE_REWRITE=0` trades it back for
|
||||
instant mechanical speech.
|
||||
* frank-tts is LAN-only: no Caddy vhost fronts :7772, so this extension is
|
||||
usable on the home network.
|
||||
* Deployment, packs and the voice registry are owned by the
|
||||
@@ -176,14 +224,14 @@ its directory are removed when the player process exits.
|
||||
* `omp --help` (extension discovery flags, `say`, `plugin`) and
|
||||
`@earendil-works/pi-coding-agent` `dist/core/extensions/types.d.ts`
|
||||
(`ExtensionAPI.on/registerCommand`, `ExtensionMode = "tui" | "rpc" | "json" | "print"`).
|
||||
* The stock speak-aloud implementation — vocalizer pipeline `I6o` and
|
||||
utterance segmenter `fHe` — in `@oh-my-pi/pi-coding-agent`'s
|
||||
`dist/cli.js`, read off the minified bundle 2026-10-19; the enhanced
|
||||
rewrite ("with the tiny/smol model before synthesis") is that bundle's
|
||||
own `speech.enhanced` description. dankvoice ports the mechanical half
|
||||
only: no LLM rewrite happens before frank-tts.
|
||||
* The stock speak-aloud implementation — vocalizer pipeline `I6o`, utterance
|
||||
segmenter `fHe` and speech enhancer `Qyr` with its prompt `T6o` — in
|
||||
`@oh-my-pi/pi-coding-agent`'s `dist/cli.js`, read off the minified bundle
|
||||
2026-10-19: dankvoice ports the mechanical and the enhanced halves, so an
|
||||
LLM rewrite does happen before frank-tts.
|
||||
* Gitea issue #21 on `dank/command-center` (the work item this page ships with).
|
||||
* Gitea issue #23 on `dank/command-center` (the options-menu work item).
|
||||
* Gitea issue #27 on `dank/command-center` (the enhanced-narration work item).
|
||||
|
||||
## See Also
|
||||
|
||||
|
||||
@@ -12,6 +12,22 @@ timestamp: 2026-08-28
|
||||
# Bundle Change Log
|
||||
|
||||
|
||||
## 2026-10-19 — dankvoice: humanized narration and thinking aloud (Refs #27)
|
||||
|
||||
dankvoice answered in verbatim markdown-cleaned text while the built-in
|
||||
vocalizer speaks humanized prose — the bundle's `Qyr` enhancer (`speech.enhanced`,
|
||||
system prompt `T6o`) now lives in the extension: every spoken unit is rewritten
|
||||
by the @tiny role (resolved from `modelRoles.tiny` in config.yml + the provider
|
||||
baseUrl in models.yml → frank-zealot :7771) under the same budgets (1536 tokens,
|
||||
6 s, 4000-char window), with mechanical cleanup as the fallback and an empty
|
||||
rewrite skipping the unit. Auto-speak also narrates the last assistant message's
|
||||
thinking blocks before the answer (`narration: true` blocks count as answer
|
||||
text); both are menu-toggleable and persisted (`thinking` in the settings file),
|
||||
env levers `DANKVOICE_REWRITE_URL/MODEL/REWRITE=0/THINKING/OMPCONFIG`. The
|
||||
fixture grows a chat route with echo/prose modes; 7 new scenarios, 22 green.
|
||||
Live smoke: a markdown answer became 3 sequential prose segments through real
|
||||
zealot + frank-tts, thinking played before its answer.
|
||||
|
||||
## 2026-10-19 — dankvoice: segmented sequential speech, copied from the bundle's vocalizer
|
||||
|
||||
Silent-answer incident follow-up: `agent_end` awaiting one whole-answer
|
||||
|
||||
Reference in New Issue
Block a user