Files
command-center/agent/managed-skills/tts-synth-timeout-cap-run-triage/SKILL.md
T

3.2 KiB
Raw Blame History

name, description
name description
tts-synth-timeout-cap-run-triage Triage a dankvoice "The operation timed out." toast: prove/disprove a qwentts engine cap-run (2048-frame, no EOS) via Dankclaw journals before suspecting the client.

TTS synth-timeout triage

Symptom: omp shows Error: dankvoice: The operation timed out. — a real 60 s stall, not a client bug. SYNTH_TIMEOUT_MS = 60_000 (src/audio-io.ts) + the notify at utterance.ts:181 is the ONLY emitter of that text.

Known root cause (chronic as of 2026-10)

The pinned qwentts.cpp tts-server (frank-tts-engine, Dankclaw, container port 8880) can miss EOS and run to its hard-coded 2048-frame cap: ~164 s of audio from a short sentence, ~90–100 s wall. Measured ~5/day, 68 in 14 days. The engine is a serial FIFO (QWENTTS_MAX_BATCH=1), so every narration queued behind a cap-run also times out — the toast comes in bursts. Shim logs BrokenPipeError at wfile.write (client gone when buffered audio finally ships). No frame-cap/deadline flag exists (/app/tts-server --help); the shim can't cancel (engine buffers before writing); SHIM_TIMEOUT=600 deliberately outwaits it. Fix belongs in serverAppConfig (watchdog restart or upstream feature request), never in dankvoice.

Procedure

  1. Endpoint is alive? curl -fsS -o /dev/null -w "%{http_code} %{time_total}s\n" -m 10 http://192.168.0.143:7772/v1/audio/voices and time one synthesis with the session's actual voice (voice may be set per-directory in ~/.omp/cache/dankvoice.json byDir — read that first).
  2. Read the journals — from bash ssh, NOT the ssh://dankclaw/ device. The ssh:// device lands as a different user (blank) than the ssh config (dank@192.168.0.143) and sees no units, no sockets — false negatives everywhere:
    ssh -o BatchMode=yes dankclaw 'journalctl --user -u frank-tts --since "-6h" --no-pager | grep -B1 -A20 BrokenPipeError | tail'
    ssh -o BatchMode=yes dankclaw 'journalctl --user -u frank-tts-engine --since "-6h" --no-pager | grep -E "Generation done : (2048|[89][0-9]{2}) frames"'
    
    Signature = a facade BrokenPipeError whose timestamp is ~60 s after an engine Admit slot=0, and/or Generation done : 2048 frames. Window-correlate with the engine: --since "HH:MM" --until "HH:MM" and skip [Sample]/Generated noise.
  3. Chronicity count: journalctl --user -u frank-tts-engine --since "-14 days" --no-pager | grep -c "2048 frames".
  4. Verdict: cap-run confirmed → the client behaved correctly (toast + next utterance speaks). Route the fix to dank/serverAppConfigs (dankclaw/voiceservice/): host-side watchdog on the journal signature restarting frank-tts-engine.service, and/or upstream per-request frame budget.

Traps

  • systemctl --user / journalctl --user on Dankclaw require the dank user — rootless quadlet units live there.
  • zsh on Dankclaw eats === separators and unquoted URLs as globs: quote args or avoid echo ===.
  • tts-server is not on the exec PATH: use /app/tts-server.
  • Roster may contain only cloned voices (narrator1-4 retired) — a 400 on a stale voice name is a separate bug; timeouts are the cap-run.
  • Elevated RTF right after idle (~2 min gap) is iGPU clock ramp, not contention.