Merge pull request 'docs(tts): record the measured CPU engine A/B behind frank-tts' (#10) from docs/tts-engine-trials into main

serverAppConfigs-docs/Research/Local TTS and Voice Cloning.md

@@ -1,7 +1,7 @@
 ---
 type: research
 title: Local TTS and Voice Cloning
-description: How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the full local AI audio stack, and a recommended homelab deployment.
+description: How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the measured CPU engine A/B behind frank-tts and the arm A decision, the full local AI audio stack, and a recommended homelab deployment.
 tags:
   - research
   - tts
@@ -10,7 +10,7 @@ tags:
   - stt
   - vllm-omni
   - kokoro
-timestamp: 2026-10-18T00:00:00Z
+timestamp: 2026-10-18T16:05:00Z
 ---
 
 # Local TTS and Voice Cloning
@@ -47,13 +47,16 @@ greenfield: every fleet endpoint declares `input: [text, image]`
   local server needs a speech-role entry, not a patch.
 - **Fleet plan (2026-10-18): CPU-only on Dankclaw, not the 3090.** Gitea issue
   #2 rules out every GPU option (the 3090 has carried frank-zealot since
-  2026-10-17; both R9700s are frank-archon's), so the shape is a VoxCPM2 GGUF
+  2026-10-17; both R9700s are frank-archon's). The shape is a TTS engine on
-  engine on **8880** behind a voice-registry shim on **8890**
+  **8880** behind **frank-tts**, the fleet's speech endpoint on **7772**
-  (`scripts/voiceshim.env`, `dankclaw/voiceservice/`) that speaks OpenAI
+  (`scripts/frank-tts.env`, `dankclaw/voiceservice/`): one OpenAI
-  `/v1/audio/speech` and resolves voices by name. Nothing is deployed yet — on
+  `/v1/audio/speech` façade that resolves a voice *by name* and never shows a
-  2026-10-18 `voiceshim` has no unit installed and nothing answers on 8880 or
+  path. Nothing is deployed — no `frank-tts` unit and no container exist on the
-  8890. STT (Parakeet TDT 1.1B, CC-BY-4.0) stays on CPU. **No RTF or quality
+  host and nothing listens on 7772 or 8880 (checked 2026-10-18). STT (Parakeet
-  number has been measured for this shape**, so this page states none.
+  TDT 1.1B, CC-BY-4.0) stays on CPU. **The engine is chosen: arm A**
+  `qwentts.cpp` **resident `tts-server`, 1.7 B talker** — measured in
+  *Measured CPU engine A/B* below and picked on the live-output criterion in
+  [Decision: arm A for live output](#decision-arm-a-for-live-output-2026-10-18).
 
 ## Can Kokoro learn a new voice?
 
@@ -133,11 +136,11 @@ accepts only its built-in voice list, so a custom pack needs a patch.
 
 Paths for a Frank-cloned voice (2026-10-18):
 
-1. **Local server + speech role, no patch.** Register the local engine (or the
+1. **Local server + speech role, no patch.** Register frank-tts (port 7772) as
-   8890 voice shim) as a provider model on the `openai-speech` API and point
+   a provider model on the `openai-speech` API and point `modelRoles.speech` at
-   `modelRoles.speech` at it; the shim turns a voice *name* into a reference
+   it; the façade turns a voice *name* into that pack's reference clip and
-   clip and forwards it as base64 `reference_audio`
+   forwards it as base64 `reference_audio` (`scripts/frank-tts.env`,
-   (`scripts/voiceshim.env:19-35`). Not deployed as of 2026-10-18.
+   `TTS_BACKEND=omni-http`). Not deployed as of 2026-10-18.
 2. **xAI console custom voice — works today, zero code.** Record 2–3 min clean
    speech (≤120 s, 24 kHz 16-bit mono WAV) → console → `voice_id` → speech
    tool. Free (≤30 voices), cloud, US-only.
@@ -206,29 +209,161 @@ The CPU shape does not need a Python server at all:
   `VoxCPM2-Acoustic-F16.gguf`, Apache-2.0 card) or from the fork's own
   `tools/omni/voxcpm2/convert_voxcpm2_to_gguf.py`.
 
-Nothing has been measured on this fleet for any of these engines — no RTF, no
+Arm B (`llama-tts`) and arm A (`qwentts.cpp`) have both been measured on this
-latency, no MOS. That absence is the finding, not an omission.
+box — RTF, first-audio latency and RSS are in the next section. The other
+servers in the table above (vLLM-Omni, Kokoro-FastAPI, CosyVoice, F5-TTS,
+GPT-SoVITS) remain unmeasured here; that absence is still the finding.
+
+## Measured CPU engine A/B (2026-10-18)
+
+Both candidate upstreams for frank-tts were measured on Dankclaw's CPU — 8
+threads, no GPU touched, three runs per arm, seed 42, identical text, against
+the fleet's own audition voice packs.
+
+- **Arm A — [`ServeurpersoCom/qwentts.cpp`](https://github.com/ServeurpersoCom/qwentts.cpp)**
+  at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` (2026-10-16), weights
+  from [`Serveurperso/Qwen3-TTS-GGUF`](https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF):
+  `qwen-talker-1.7b-base-Q4_K_M.gguf` (1,219,245,248 B),
+  `qwen-talker-0.6b-base-Q4_K_M.gguf` (628,905,056 B),
+  `qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B) — byte counts verified
+  against the Hub API. Builds with zero compiler warnings; `GGML_BLAS=OFF` is
+  forced on this host (only `libopenblas.so.0` exists, no dev headers, so the
+  vendor `buildcpu.sh` cannot configure).
+- **Arm B — mainline `llama-tts`** with `--tts-speaker-file`, i.e. the
+  `llama-tts-cli` backend `scripts/frank-tts.env` already implements.
+
+RTF is **seconds of compute per second of audio** (lower is better; below 1 is
+faster than real time).
+
+| Arm | RTF | Run spread | Peak RSS |
+|---|---|---|---|
+| A · 0.6 B talker, CLI | 0.44 | ±0.002 | 3.4 GiB |
+| A · 1.7 B, CLI, uncloned | 0.57–0.58 | ±0.006 | 4.7 GiB |
+| A · 1.7 B, CLI, embedding-only clone | 0.63 | ±0.02 | n/m |
+| A · 1.7 B, CLI, ICL clone | 0.92 | ±0.01 | n/m |
+| A · 1.7 B, resident `tts-server`, warm | 0.55–0.57 | ±0.01 | 5.7 GiB |
+| A · same, ICL voice warm / **cold (first request)** | 0.67 / **1.65** | one-shot | 5.7 GiB |
+| B · mainline `llama-tts` | 0.93–0.99 | ±0.25 | n/m |
+
+**Read RTF only within an arm.** Conditioning changes clip length — the same
+seed yields 18.00 s of audio embedding-only and 15.12 s under ICL — so the
+denominator is the arm's own audio duration.
+
+Arm A's resident server is ~1.7× faster than arm B and about 40× tighter in
+run-to-run spread, and it already serves the HTTP shape frank-tts's façade
+speaks: `/health`, `/v1/models`, `/v1/audio/voices` (GET/POST/DELETE),
+`/v1/audio/speech` (`wav`/`pcm`), chunked streaming with the first audio byte
+at 86–451 ms and a server-side time-to-first-audio of 94–112 ms. Arm B has no
+HTTP path at all — that is what the façade's CLI backend exists for.
+
+Four laws came out of the trial, each worth a line of configuration:
+
+- **Boot-warm every voice.** The first request for a registered voice decodes
+  the whole reference to snapshot codec state: 6.2–8.0 s, first audio byte at
+  5,530 ms. Later requests restore that state in 0.5 ms. An engine unit must
+  run one throwaway synthesis per voice at start, or the first request of the
+  day pays five seconds.
+- **Speaker artifacts are reusable and exact.** `qwen-codec` turns a reference
+  WAV into `.spk` (8,192 B x-vector) + `.rvq` (5,456–6,864 B, 16 codebooks × T,
+  11-bit packed) in 1.8–2.1 s at 1.3–1.5 GiB peak; replaying them yields a
+  **byte-identical** WAV and saves 1.4 s per request. That pair is the
+  engine-artifact term the voice-pack law in `scripts/frank-tts.env` reserves a
+  slot for — arm A pins it as `.spk` + `.rvq` beside `reference.wav`.
+- **Streaming is not bit-identical to buffered**, whatever the upstream README
+  claims: same text, seed and voice, 92.2 % of samples differ (peak delta
+  1139/32768, −41.6 dB RMS — inaudible). A fixed seed does not pin output
+  across `response_format`, so never mix the two paths in a cache or a golden
+  fixture.
+- **A voice cannot be registered from an embedding alone**: `POST
+  /v1/audio/voices` accepts `wav_b64` or **both** `spk_b64` and `rvq_b64`
+  (`src/tts-server.h:480`), so an embedding-only voice has to be sent as the
+  WAV, or the lone-`spk` path has to be upstreamed.
+
+**Likeness is not measurable at this resolution.** x-vector cosine against each
+speaker's own reference: ICL 0.9886 / 0.9901, embedding-only 0.9844 / 0.9812,
+uncloned 0.9228 — but **two different speakers score 0.9497** on the same
+metric. The whole clone effect lives in a 0.04-wide band and the
+ICL-over-embedding gap is inside noise, so the four audition clips
+(`/home/blank/voice-audition/qwentts-*.mp3`, each against its `armB-*.mp3`
+mainline partner, guide in `LISTEN-engine.md`) are the instrument. **The
+operator heard them on 2026-10-18 and passed every clone**, which removed the
+last axis measurement could not reach; the engine was then picked on
+live-output grounds ([Decision](#decision-arm-a-for-live-output-2026-10-18)).
+The cosine
+extractor is still worth keeping: it is engine-neutral (it scores mainline
+output through the same encoder) and it separates an uncloned voice cleanly.
+
+Two operational details. The engine takes 8 threads and exposes **no `-t`
+flag** — `src/backend.h` uses `hardware_concurrency()/2` and prints the choice
+at load, so pinning a cpuset on the quadlet changes that number implicitly. And
+ICL needs a reference transcript that did not exist: the fleet has no ASR
+endpoint and no audio input, so the crops were transcribed with
+`vosk-small-en-us` inside the work dir, and two of three transcripts came back
+suspect ("demon souls" for *Dark Souls*) — a reason to curate `ref_text` into
+the pack manifest instead of transcribing on the host.
+
+## Decision: arm A for live output (2026-10-18)
+
+Every audition pair passed, so likeness stopped being the tiebreak. The
+question became **which engine is better for live output** — the harness
+speaking a reply, an alert read aloud, a click-to-speak — and on that criterion
+RTF only has to stay under 1.0 with margin: what separates the arms is **time to
+first audio** and the **spread**, because an arm whose RTF can cross 1.0 stalls
+a live stream mid-sentence.
+
+| | arm A · `qwentts.cpp` `tts-server` | arm B · mainline `llama-tts` |
+|---|---|---|
+| first audio byte | 86–451 ms | model load, per request |
+| warm RTF | 0.55–0.67 | 0.93–0.99 |
+| run spread | ±0.01 | ±0.25 |
+| HTTP | serves `/v1/audio/speech` | none — façade CLI only |
+| RAM | 5.7 GiB resident | per-request, freed after |
+
+**Chosen: arm A, 1.7 B talker, resident `tts-server` in a `frank-tts-engine`
+container, with `TTS_BACKEND=llama-tts-cli` kept in the repo as the rollback.**
+Three consequences are configuration, not opinion: the unit **boot-warms every
+voice** (5.5 s cold first byte otherwise), it **does not pin a cpuset** casually
+(threads are `hardware_concurrency()/2`, no flag), and every pack registers with
+the **WAV** because the engine rejects a lone embedding.
+
+Two follow-ups stay open on purpose, and neither blocks the deploy:
+
+- **Talker size.** The 0.6 B arm measured 0.44 RTF at 3.4 GiB and has never
+  been auditioned — every clip above is 1.7 B. It is the lever for a CPU
+  co-tenancy problem, not a preemptive saving.
+- **ICL vs embedding-only.** Both passed the ear test. Embedding-only is the
+  shipped default: 0.57–0.63 vs 0.67 warm, and it needs no `ref_text`, which
+  this fleet cannot produce honestly (the ASR crops came back suspect). Packs
+  still carry `.spk` + `.rvq`, so ICL remains a per-voice opt-in for batch work
+  like the podcast render, where total time matters more than first byte.
 
 ## Recommended deployment (Dankclaw)
 
 As decided in Gitea issue #2, this stays **CPU-only on Dankclaw**: the RTX 3090
 has carried frank-zealot since 2026-10-17 and both R9700s are frank-archon's,
-so no card may be asked to co-host TTS. The shape is one engine plus a thin
+so no card may be asked to co-host TTS. The shape is one engine plus one
-voice shim, and **nothing here is deployed as of 2026-10-18** — `systemctl
+façade, and **nothing here is deployed as of 2026-10-18**: `frank-tts` has
---user is-enabled voiceshim` reports `not-found` and nothing answers on 8880 or
+neither a unit nor a container on the host, and nothing listens on 7772 or
-8890.
+8880.
 
 | Service | Image / install | Port | CPU | Notes |
 |---|---|---|---|---|
-| TTS engine | `llama-tts-server` (`tc-mb/llama.cpp-omni`) on a VoxCPM2 GGUF; `vllm/vllm-omni` + Qwen3-TTS only if a GPU is ever deliberately freed | 8880 | engine decision still open | OpenAI `/v1/audio/speech`; `n_gpu_layers` is there for a GPU boot |
+| TTS engine | **arm A `qwentts.cpp` `tts-server`** on the Qwen3-TTS GGUFs (decided 2026-10-18, above); arm B mainline `llama-tts` stays in the repo as the `llama-tts-cli` rollback; `vllm/vllm-omni` + Qwen3-TTS only if a GPU is ever deliberately freed | 8880 (container-internal) | open, evidence above | arm A serves OpenAI `/v1/audio/speech` itself and boot-warms each voice; arm B has no HTTP path and runs through the façade's CLI backend |
-| Voice shim | `dankclaw/voiceservice/` (stdlib container) + `scripts/voiceshim.env` + `dankclaw/voiceservice/quadlet/voiceshim.container` | 8890 | yes | voice *name* → `reference.wav`, `voice`/`voice_id` on the wire, mp3 re-encode; backends `omni-http` (forwards base64 `reference_audio`) and `llama-tts-cli` (`--tts-speaker-file`, `nice 19`) |
+| frank-tts | `dankclaw/voiceservice/` (stdlib container) + `scripts/frank-tts.env` + its quadlet | 7772 | yes | voice *name* → `/voices/<name>/reference.wav`, `voice`/`voice_id` on the wire, mp3 re-encode; backends `omni-http` (forwards base64 `reference_audio`) and `llama-tts-cli` (`--tts-speaker-file`, `nice 19`) |
 | STT (English) | Parakeet TDT 1.1B behind a FastAPI wrapper, or whisper.cpp server | 8882 | yes | CC-BY-4.0; LibriSpeech-clean WER 1.39 (model card) |
 | STT (multilingual, opt.) | faster-whisper `large-v3` int8 | 8883 | yes | MIT |
 | VAD | Silero (embedded) | — | yes | MIT; model file 2 327 524 B |
 
-8880/8890 are what `scripts/voiceshim.env:35,49` carry (`TTS_UPSTREAM=
+7772 is frank-tts's host port — the next slot in the fleet's model-endpoint
-http://host.containers.internal:8880`, `SHIM_PORT=8890`); 8882-8883 for STT
+family (7770 frank-dragoon, 7771 frank-zealot). The engine behind it keeps 8880,
-keep clear of the fleet's LLM ports 7770/7771.
+but **not as a host-loopback publish**: netavark DNATs published ports only, and
+a bridge container cannot reach a `127.0.0.1` bind on the host — that is the
+lesson that 502'd hermes (`AGENTS.md`). `host.containers.internal`
+(169.254.1.2) is the gateway, so it reaches a host service only when that
+service binds a routable address; `scripts/frank-tts.env` names it as the
+engine's address, which holds for a `0.0.0.0` bind and not for a loopback one.
+The deployed pair therefore shares one quadlet network and the façade addresses
+the engine by container name.
 
 ## Voice-creation workflow (the "from sample files" answer)
 
@@ -284,6 +419,10 @@ keep clear of the fleet's LLM ports 7770/7771.
   https://huggingface.co/openbmb/VoxCPM2
 - llama.cpp-omni TTS server: https://github.com/tc-mb/llama.cpp-omni ·
   https://raw.githubusercontent.com/tc-mb/llama.cpp-omni/master/tools/omni/voxcpm2/README.md
+- qwentts.cpp (arm A): https://github.com/ServeurpersoCom/qwentts.cpp at commit
+  `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` · weights
+  https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF (byte counts against the
+  Hub API, 2026-10-18)
 - mainline `llama-tts`:
   https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/tts/README.md ·
   Pocket-TTS weights: https://huggingface.co/kyutai/pocket-tts (CC-BY-4.0)

serverAppConfigs-docs/log.md

@@ -2311,6 +2311,52 @@ every restart since the cutover.
   Kokoro-enum-only — so "the agent talks in my voice" needs that provider row or
   a `pi.registerProvider` extension, decided in phase 2.
 
+## 2026-10-18 — TTS engine picked: arm A `qwentts.cpp`, on the live-output criterion
+
+- Frank listened to the four audition pairs (`qwentts-*.mp3` against their
+  `armB-*.mp3` mainline partners) and **passed every clone**. That deleted the
+  only axis the A/B could not measure, and the question was reframed: **which
+  engine is better for live output**. On that criterion RTF only has to stay
+  under 1.0 with margin — the discriminators are time-to-first-audio and
+  run-to-run spread, because an arm whose RTF can cross 1.0 stalls a live
+  stream mid-sentence.
+- **Chosen: arm A `qwentts.cpp` @ `5916bf7`, the resident `tts-server`, 1.7 B
+  talker Q4_K_M + Q8_0 tokenizer, in a `frank-tts-engine` container on 8880.**
+  The `TTS_BACKEND=llama-tts-cli` path in `scripts/frank-tts.env` stays in the
+  repo as the rollback. The numbers that carried it: first audio byte 86–451 ms
+  against a per-request model load, warm RTF 0.55–0.67 against 0.93–0.99, spread
+  ±0.01 against ±0.25, and a native `/v1/audio/speech` against no HTTP path at
+  all. Three consequences are configuration, not preference: boot-warm every
+  voice, do not pin a cpuset (threads are `hardware_concurrency()/2`), register
+  every pack with its WAV (a lone embedding is rejected).
+- Two follow-ups deliberately **not** decided: the **0.6 B talker** (0.44 RTF,
+  3.4 GiB, never auditioned — every clip is 1.7 B) and **ICL vs
+  embedding-only** conditioning. Embedding-only is the shipped default because
+  it is cheaper and needs no `ref_text`, which this fleet cannot transcribe
+  honestly; `.spk` + `.rvq` still ride in every pack, so ICL remains a per-voice
+  opt-in for batch work where total time beats first byte.
+- **Addressing law corrected** in
+  [[Local TTS and Voice Cloning]]: the pair may not meet at a host-loopback
+  publish. netavark DNATs published ports only and a bridge container cannot
+  reach a host `127.0.0.1` bind — the lesson that 502'd hermes — so
+  `host.containers.internal:8880`, which `scripts/frank-tts.env` documents, is
+  correct only for a routable bind. The deployed pair shares one quadlet network
+  and the façade addresses the engine by container name.
+- **PR #12** (`feat/open-notebook-podcast` @ `114dd7a`, 8 commits, 403 tests) is
+  **merged** as `1eb8b21` (Frank's OK, given in session): open-notebook +
+  SurrealDB quadlets on the private bridge `10.89.4.0/24`, the API published on
+  **5056** because seerr holds `127.0.0.1:5055`, and `ESPERANTO_TTS_TIMEOUT`
+  moved 600 → **660** to sit one round-trip outside frank-tts's own
+  `SHIM_TIMEOUT`. Merged is not runnable: nothing listens on 7772 until the
+  voice stack lands, and the episode destination (`/home/dank/media/podcasts`)
+  stays interim pending Gitea #7.
+- Still in flight, not in this branch: `feat/frank-tts-engine` — the arm A image
+  build, the quadlet pair on one network, the boot-warm step, deployer /
+  planner / `test-changed.sh` routing, homepage and docs. This branch is also
+  where the two-branch overlap got resolved: merging `main` here conflicted on
+  `log.md` alone (both branches appended an entry) and took `index.md`,
+  `deploy-remote.sh`, `deploy-plan.py` and `test-changed.sh` clean.
+
 ## 2026-10-18 — open-notebook: the podcast generator's config lands (nothing deployed)
 
 - Goal: **open-notebook** as the fleet's podcast generator on Dankclaw — notes