command-center/omp-docs/core/dankvoice.md

type, title, description, tags, verified, timestamp

type	title	description	tags	verified	timestamp
reference	dankvoice	omp extension that speaks assistant output with the fleet's cloned voices through frank-tts.	omp extension tts voice frank-tts dankclaw	2026-10-19	2026-10-19

dankvoice

agent/extensions/dankvoice.ts adds cloned-voice speech to the interactive harness: /say opens the options menu, /say <text> speaks through the speakers and /say on narrates the assistant's thinking and final answer after every turn. The words are first rewritten into natural spoken prose by the @tiny role model (exactly what the built-in vocalizer's speech.enhanced does), and everything is served by frank-tts (Dankclaw :7772, the speech model role in agent/models.yml).

Why it exists

Nothing built into the harness can pronounce Frank's cloned voices.

Surface	Engine	Verdict
TUI speak-aloud (speech.enabled, speech.voice)	bundled Kokoro ONNX worker, synthesizeStream(voice, speed, signal)	Kokoro voice enum only; never consults models.yml or the speech role
omp say [TEXT]	same local backend (--model = local model key)	omp say --voice narrator1 "test" → error: Expected --voice to be one of: af_heart, af_bella, af_nicole, af_kore, af_sarah, am_michael, am_fenrir, am_puck, bf_emma, bm_george, bm_fable (verified 2026-10-18)
tts tool / xd://tts device (modelRoles.speech)	frank-tts over OpenAI /v1/audio/speech	cloned voices work, but only as a file — nothing plays them

dankvoice is not a patch to that vocalizer (harness code, not ours); it is a second transport that happens to speak, so Kokoro stays available for quick local speech and the cloned voices carry the real narration.

Commands

Command	Effect
/say | /say menu	options menu (ctx.ui.select): auto-speak toggle, voice, output device, player, spoken cap, thinking-narration toggle, humanize toggle, test phrase
/say <text>	synthesize once and play; interrupts whatever is playing
/say on | /say off	arm/disarm auto-speak of the thinking blocks and final text at agent_end; the footer shows 🔊 <voice> while armed
/say voice <name>	pick a voice; an empty name defers to the server's own default
/say voices	list what frank-tts reports
/say stop	cut the current utterance
/say status | /say help	endpoint, voice, player, device, cap, thinking, humanize

Menu choices persist to ~/.omp/cache/dankvoice.json (runtime state, never tracked); the environment overrides the file, the file overrides defaults. "Speak a test phrase" synthesizes a short line through the current voice, player and device — the audible proof whenever a session "has say on" but stays silent.

Auto-speak is skipped in print/json mode so benches and pipelines stay silent. Each spoken unit — thinking first when narrated, then the answer — is first rewritten into spoken prose by the @tiny model; when the enhancer (below) declines, the text goes to segmentation as-is, where fenced code and tables vanish, links collapse to their label and bullets and table pipes disappear. A long answer is never one monolithic synthesis request; see Segmentation for how utterances are cut and played.

Segmentation

Utterances are cut by a state machine ported from the harness's own speak-aloud vocalizer (the minified fHe class in the agent bundle's dist/cli.js), kept deliberately in lockstep with the stock path:

• A strong boundary is a run of [.!?…] followed by whitespace — so !!!!! and ... are one boundary, never a pile of cuts.
• Boundaries after e.g., etc., Mr. and friends, or inside inline code, don't count; decimals (3.5, 1,5) survive because a boundary requires trailing whitespace.
• Line starts are classified: table rows and fenced code are swallowed whole (their content is never read aloud), heading and bullet markers vanish, ordered-list markers speak their number ("1, …"), quote markers strip.
• Per segment, markdown is cleaned: links collapse to their label, URLs to their host, deep paths to their last segment; emphasis markup dies.
• Segments aim for one sentence; run-ons break at commas or word wraps before 280 characters.

Playback is strictly sequential: segment N+1 is synthesized only after segment N's player process exited, so audio never stacks. The chain runs detached from agent_end (fire-and-forget) and may legitimately outlive the harness's 30 s handler timeout — the old awaiting design produced a "handler timed out after 30000ms" toast on long answers. A new utterance, /say stop, or disarming auto-speak retires the live chain through a run token: the current player is killed and pending segments are skipped. DANKVOICE_MAX is a per-answer character budget: the leading whole segments that fit are spoken, the rest are dropped — the thinking unit draws on it before the answer does.

Endpoint resolution

The base URL is read from agent/models.yml, from the provider block whose model row carries api: openai-speech — the same row the speech role resolves, so no host address and no credential appears in this file. From baseUrl: http://192.168.0.143:7772/v1 it derives POST …/v1/audio/speech, GET …/v1/audio/voices and GET …/health.

Environment	Default	Meaning
DANKVOICE_URL	models.yml	speech base URL (…/v1), bypasses the YAML read
DANKVOICE_MODEL	the id in that block (frank-tts)	model named in the request body
DANKVOICE_CONFIG	~/.omp/cache/dankvoice.json	persisted menu choices (voice, player, sink, cap, auto)
DANKVOICE_VOICE	empty (server default)	default voice name
DANKVOICE_AUTO=1	off	start with auto-speak armed
DANKVOICE_MAX	700	spoken-character budget per answer (whole leading segments, shared by the thinking and answer units)
DANKVOICE_PLAYER	detect	player binary, skipping detection
DANKVOICE_MODELS	~/.omp/agent/models.yml	YAML to read the endpoint from
DANKVOICE_REWRITE_URL	the @tiny role	chat base URL (…/v1) for the enhancer, bypassing role resolution
DANKVOICE_REWRITE_MODEL	the @tiny role's model	chat model named in the rewrite body
DANKVOICE_REWRITE=0	enhancer on when a route resolves	speak mechanical text
DANKVOICE_THINKING	settings file, default on	0/1 force thinking narration off/on
DANKVOICE_OMPCONFIG	~/.omp/agent/config.yml	YAML to read the modelRoles.tiny role from

Player detection order is pw-play → paplay → ffplay (PipeWire first: the workstations run WirePlumber). A saved output device routes the player explicitly — pw-play --target <sink> or paplay --device=<sink>; ffplay has no device concept and ignores it — while an unset device leaves the player's own default. Sinks are enumerated with wpctl list audio sinks; if wireplumber is unreachable the menu offers only "system default". Each utterance is written to $TMPDIR/dankvoice-*/say-<ms>.wav and the file plus its directory are removed when the player process exits.

Humanized narration (the @tiny role)

The built-in vocalizer never reads answers aloud verbatim when speech.enhanced is on: a small model rewrites each utterance into words a person would say. dankvoice ports that enhancer — same system prompt (verbatim from the bundle), same budgets (1536 max tokens, 6 s timeout, 4000-char input window), same contract (a failed rewrite falls back to the mechanically cleaned text; an empty rewrite skips the unit as "nothing to say"). The target model is the harness's @tiny role, resolved the way the harness resolves it: modelRoles.tiny in agent/config.yml reads provider (host)/model:effort; the provider block in agent/models.yml supplies the baseUrl; the request is POST {base}/chat/completions. On this fleet that is frank-zealot (llamacpp (dankclaw-3090), :7771) — the 6 s abort keeps a slow tiny model off the critical path.

The request body pins chat_template_kwargs: { enable_thinking: false }, and that field is load-bearing: the harness injects it for managed roles, but a raw fetch gets the model's default — thinking on — so the Qwen35 hybrid spends the budget on reasoning_content and answers content: "" (measured: 6.9 s / 6893 reasoning chars / empty content without the flag; 0.55 s / 369 clean chars with it). For a hybrid endpoint, empty content beside non-empty reasoning_content means the flag was ignored — a failed rewrite (mechanical fallback), never the legitimate "nothing to say".

Thinking blocks ride the same path: at agent_end the last assistant message's true thinking parts are narrated (rewritten first, as their own unit, before the answer), while blocks flagged narration: true — which providers use to address the user through the thinking channel — count as answer text. The menu, /say status, and DANKVOICE_THINKING toggle it; the choice persists in the settings file.

Verified facts (2026-10-18, extended 2026-10-19)

• Extension discovery — any .ts file in ~/.omp/agent/extensions/ loads at startup: a probe file there printed loaded 2026-10-19T02:36:55Z from a fresh omp -p run. omp --help documents --no-extensions (discovery off, explicit -e/--extension and --hook still load) and plugin install|link.
• Endpoint — resolving the live agent/models.yml yields speechUrl http://192.168.0.143:7772/v1/audio/speech, voicesUrl …/v1/audio/voices, healthUrl …/health, model frank-tts.
• Roster — GET /v1/audio/voices answers the bare JSON array ["narrator1","narrator2","narrator3"]; /health answers {"status":"ok","backend":"omni-http","voices":[…]}. GET /v1/voices and GET /v1/models are 404 on this shim, and an unknown voice answers 400 unknown voice 'zzz'; available voices: … with available_voices in the body — which is the fallback the extension uses if the roster route ever disappears.
• Omitting voice is legal — frank-tts fills its DEFAULT_VOICE since the 2026-10-18 voice-pack work, so an unset voice synthesizes instead of failing.
• Speech — /say "dankvoice live smoke test" produced a 99,884 B / 2.08 s WAV and /say voice narrator2 + /say "cloned voice check" a 69,164 B / 1.44 s WAV (durations from ffprobe), each handed to the player; temp directories were gone afterwards.
• Auto-speak — verified in a real TUI session: /say on armed the footer (🔊 narrator1), the next finished turn synthesized and played the answer (1.20 s WAV), and an unknown voice name answers with the roster instead of speaking. An agent_end carrying willContinue (the harness scheduled an auto-retry) is never narrated — only the terminal settle speaks.
• Menu — verified via driver scenarios 2026-10-18 (Refs #23): the menu lists auto-speak/voice/device/player/cap/test-phrase; a voice chosen in the menu is used by later utterances and survives a fresh load via the settings file; a device chosen in the menu appears in the player invocation as --target <sink>.
• Segmented playback — driver scenarios 2026-10-19: "First stop here. Second one goes!!! Really? Third ... done" produced exactly 3 sequential requests and 3 plays; "Model 3.5 ships today. Version 1,5 too." produced exactly 2 (decimals never split); the markdown fixture's fenced code and URL never reached the fixture endpoint; with DANKVOICE_MAX=60 only the leading segment was spoken.
• Humanize + thinking (2026-10-19, Refs #27) — driver scenarios cover the rewrite round-trip (fixture chat route), the 404 → mechanical fallback, thinking narrated as its own unit before the answer, DANKVOICE_THINKING=0 silencing it, and narration: true spoken once as the answer; the endpoint status line shows the resolved humanize model from the @tiny role. Live smoke against the real services: a markdown /say became 3 sequential prose segments (mechanical cleanup of that text yields at most 2), and a thinking+answer agent_end played thinking first. 22 green in tests/dankvoice.bats.
• Thinking-off fix (2026-10-19, Refs #32) — replaying the exact rewrite body against live frank-zealot: without chat_template_kwargs. enable_thinking: false the reply arrives as 6893 reasoning_content characters with content: "" in 6.9 s (twice the 6 s abort); with it, 369 clean characters in 0.55 s. Driver fixtures now pin the flag (thinkoff=false on the recorded chat call), turn reasoning-only replies into mechanical speech instead of silence, and time the menu's test phrase (~0 ms handler). 26 green in tests/dankvoice.bats.

Limits

• The TUI's own speak-aloud feature still narrates in Kokoro: its voice list is a compile-time enum in the harness bundle, and no setting routes it to frank-tts. Cloned voices live only on this file-producing/speaking path and on the tts tool. See Handy for the STT half.
• agent_end narrates thinking blocks and the final message only — mid-turn tool chatter is never spoken. Thinking covers the last assistant message; tool-loop gaps in earlier messages are skipped by design.
• Humanized narration is serial: audio starts only after the @tiny rewrite returns (≤ 6 s per unit, then frank-tts synthesis). Say /say off-grade latency is the price of prose; DANKVOICE_REWRITE=0 trades it back for instant mechanical speech.
• frank-tts is LAN-only: no Caddy vhost fronts :7772, so this extension is usable on the home network.
• Deployment, packs and the voice registry are owned by the serverAppConfigs repo (serverAppConfigs-docs/Dankclaw/Frank TTS.md), not by this one.

Citations

• Live probes on the workstation, 2026-10-18: omp say --voice narrator1 …, curl http://192.168.0.143:7772/health, /v1/audio/voices, /v1/audio/…/speech (invalid voice), ffprobe on the produced WAVs.
• omp --help (extension discovery flags, say, plugin) and @earendil-works/pi-coding-agent dist/core/extensions/types.d.ts (ExtensionAPI.on/registerCommand, ExtensionMode = "tui" | "rpc" | "json" | "print").
• The stock speak-aloud implementation — vocalizer pipeline I6o, utterance segmenter fHe and speech enhancer Qyr with its prompt T6o — in @oh-my-pi/pi-coding-agent's dist/cli.js, read off the minified bundle 2026-10-19: dankvoice ports the mechanical and the enhanced halves, so an LLM rewrite does happen before frank-tts.
• Gitea issue #21 on dank/command-center (the work item this page ships with).
• Gitea issue #23 on dank/command-center (the options-menu work item).
• Gitea issue #27 on dank/command-center (the enhanced-narration work item).
• Gitea issue #32 on dank/command-center (the latency/silence fix work item).

See Also

• Core Documentation — the live harness configuration.
• Handy — offline speech-to-text, the input half of voice.
• serverAppConfigs-docs/Dankclaw/Frank TTS.md — the frank-tts service runbook.