--- type: research title: Local TTS and Voice Cloning description: How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the measured CPU engine A/B behind frank-tts and the arm A decision, the full local AI audio stack, and a recommended homelab deployment. tags: - research - tts - voice-cloning - audio - stt - vllm-omni - kokoro timestamp: 2026-10-18T16:05:00Z --- # Local TTS and Voice Cloning Research brief (2026-09-22) on creating a new local TTS voice from sample files, the open-weight voice-cloning landscape, the full local AI audio stack, and a recommended deployment for the fleet. Every claim is cited in [Citations](#citations). The fleet has no audio input today (that capability left with the retired NPU endpoint, returned with the Gemma4 audio projector on 2026-09-17, and left again with that retirement on 2026-10-01 — audio is still greenfield: every fleet endpoint declares `input: [text, image]` (`~/.omp/agent/models.yml:63,149,214`). ## Summary - **Kokoro-82M (OMP's built-in local TTS) cannot zero-shot clone a voice.** Voices are a fixed pack. A community fine-tune recipe exists (`semidark/kikiri-tts`): a ~51 h multi-speaker base stage + a few hundred single-speaker samples, then voice-pack extraction. Feasible on one GPU, but the output is Kokoro-styled, and OMP's local backend only accepts its built-in voice list (a custom pack needs a patch). - **The 2026 open-weight cloning landscape is strong.** Permissive-licensed leaders for a single-GPU homelab: **Chatterbox (MIT)**, **Qwen3-TTS (Apache-2.0)**, **CosyVoice 3 (Apache-2.0)**, **GLM-TTS (MIT weights)**, **VoxCPM2 (Apache-2.0)**, **F5-TTS (MIT)**, **GPT-SoVITS (MIT)**. - **OMP integration (re-verified 2026-10-18):** the speech tool is gated by `speechgen.enabled` and resolves models through the **`speech` role and its fallback chain** (`modelRoles.speech`; this workstation runs `local/kokoro`). Three APIs: local Kokoro ONNX (always WAV/PCM16), `xai-tts`, and **`openai-speech` — a generic OpenAI-compatible `/v1/audio/speech` client** (MP3 or WAV, `voice_id` forwarded only when explicitly set, 15 000 characters max; DeepInfra is now just a model on that API). The 2026-09-22 finding that OMP has *no* configurable OpenAI TTS endpoint is **retired**: a local server needs a speech-role entry, not a patch. - **Fleet plan (2026-10-18): CPU-only on Dankclaw, not the 3090.** Gitea issue #2 rules out every GPU option (the 3090 has carried frank-zealot since 2026-10-17; both R9700s are frank-archon's). The shape is a TTS engine on **8880** behind **frank-tts**, the fleet's speech endpoint on **7772** (`scripts/frank-tts.env`, `dankclaw/voiceservice/`): one OpenAI `/v1/audio/speech` façade that resolves a voice *by name* and never shows a path. Nothing is deployed — no `frank-tts` unit and no container exist on the host and nothing listens on 7772 or 8880 (checked 2026-10-18). STT (Parakeet TDT 1.1B, CC-BY-4.0) stays on CPU. **The engine is chosen: arm A** `qwentts.cpp` **resident `tts-server`, 1.7 B talker** — measured in *Measured CPU engine A/B* below and picked on the live-output criterion in [Decision: arm A for live output](#decision-arm-a-for-live-output-2026-10-18). ## Can Kokoro learn a new voice? Kokoro ([hexgrad/Kokoro](https://github.com/hexgrad/Kokoro), Apache-2.0, 82 M params, non-autoregressive) ships **fixed voice packs** — there is no reference-audio path, so zero-shot cloning is not an option. OMP's local backend runs `onnx-community/Kokoro-82M-v1.0-ONNX` offline, choosing a voice from a built-in list (`tts.localVoice`). The best-documented fine-tune path is [`semidark/kikiri-tts`](https://github.com/semidark/kikiri-tts) (Apache-2.0): | Stage | Data | Purpose | |---|---|---| | Multi-speaker base | ~51 h audio, 51 speakers | language adaptation / base fine-tune | | Single-speaker | 455–627 samples (~1–2 h of one voice) | voice specialization | | Extraction | — | voice-pack (.pt) extraction for inference | **Verdict:** fine-tuning Kokoro to a personal voice is possible on a 3090, but you must collect ~1–2 h of clean single-speaker audio, run the two-stage recipe, and patch OMP to load the custom pack. The effort/quality trade is worse than running a modern zero-shot cloner on the same GPU. ## 2026 voice-cloning landscape (permissive-licensed, single-GPU) Ranked (2026-09-22) for a 3090 homelab, English-primary, personal voice; the 2026-10-18 fleet decision below moved this work off the GPU entirely: 1. **Chatterbox (MIT)** — best license/quality/compute ratio; ~10 s reference (the README's own example is `your_10s_ref_clip.wav`); 110 M (Nano) to 0.5 B (Turbo/Multilingual); ~1–2 GB VRAM *(unverified 2026-10-18)*; Nano is documented for CPU — "3x faster than realtime on 8 CPU cores". *Top pick for a personal English voice.* No server in the README (library use only; wrap it yourself); every generated file carries Resemble's PerTh watermark. 2. **Qwen3-TTS-1.7B-Base (Apache-2.0)** — 3 s cloning, 10 languages, streaming, vLLM-Omni serving with a `POST /v1/audio/voices` upload *(unverified 2026-10-18 — the reachable docs do not list it)*, the closest thing to "named voice" semantics, official fine-tuning code. *Top pick for a OpenAI-compatible deployment.* 3. **CosyVoice 3 (Apache-2.0)** — 0.5 B, 9 languages + 18 zh dialects, 150 ms bi-streaming, FastAPI and gRPC servers. 4. **GLM-TTS (MIT model weights, Apache-2.0 code)** — CER 0.89 after multi-reward GRPO (down from 1.03), RL emotion control, zh/en; the prompt audio shipped in the repo is marked research-only. 5. **VoxCPM2 (Apache-2.0)** — 30 languages, 48 kHz, LoRA fine-tune from 5–10 min of audio, ~8 GB VRAM. *Best "fine-tune my voice" option.* 6. **F5-TTS (MIT)** — 0.3–0.5 B, en/zh; Docker + Triton. 7. **GPT-SoVITS (MIT)** — best few-shot fine-tune UX (1 min of data), WebUI, Docker images. Notable non-commercial / restricted: Higgs Audio v3, Breeze-TTS-2, Spark-TTS (non-commercial); IndexTTS (bilibili custom license), Fish Speech S2 (research license), XTTS v2 (CPML, non-commercial, vendor defunct); MegaTTS3 (WaveVAE encoder not released); Voxtral TTS (no official cloning — codec encoder not released). Excluded from cloning: Parler-TTS (style-text only), VibeVoice-TTS (code withdrawn 2025-09). ## OMP integration (definitive) The tool ships hidden and is enabled by `speechgen.enabled` (`~/.omp/agent/config.yml:222-223`); the harness docs list it as "Generate speech". Models come from the **`speech` role and its fallback chain**, not from a hardcoded backend list (`modelRoles.speech`, `~/.omp/agent/config.yml:16` = `local/kokoro` today): | API | Endpoint | Notes (from the installed bundle, 2026-10-18) | |---|---|---| | local-inference | on-device ONNX, `onnx-community/Kokoro-82M-v1.0-ONNX` (Apache-2.0) | no network, **always WAV/PCM16**, voice from the built-in list (`tts.localVoice`) | | `xai-tts` | `POST https://api.x.ai/v1/tts` | `ara`/`eve`/`leo`/`rex`/`sal` + custom voice IDs; MP3 or WAV | | `openai-speech` | `POST /audio/speech` | the generic OpenAI-compatible client — MP3 or WAV, `voice_id` forwarded **only when explicitly set**, 15 000-character cap on `input`; DeepInfra is one model entry on it (`https://api.deepinfra.com/v1/openai`) | The 2026-09-22 finding — "**no generic OpenAI-compatible TTS endpoint** … `deepinfra` is a hardcoded URL, not a configurable base URL" — is **retired**: `openai-speech` is that configurable client, selected by putting a model on the `speech` role. Still true from the original audit: the local Kokoro backend accepts only its built-in voice list, so a custom pack needs a patch. Paths for a Frank-cloned voice (2026-10-18): 1. **Local server + speech role, no patch.** Register frank-tts (port 7772) as a provider model on the `openai-speech` API and point `modelRoles.speech` at it; the façade turns a voice *name* into that pack's reference clip and forwards it as base64 `reference_audio` (`scripts/frank-tts.env`, `TTS_BACKEND=omni-http`). Not deployed as of 2026-10-18. 2. **xAI console custom voice — works today, zero code.** Record 2–3 min clean speech (≤120 s, 24 kHz 16-bit mono WAV) → console → `voice_id` → speech tool. Free (≤30 voices), cloud, US-only. 3. **No-patch workaround.** The `bash` tool can `curl` the local TTS server ad hoc, and the harness `tts` device writes a speech file to a path you give it. 4. **Kokoro fine-tune → OMP local backend.** Highest effort, Kokoro-styled. ## Full local AI audio stack **STT / ASR:** whisper.cpp (MIT, tiny→large-v3 2.9 GiB, all backends, built-in VAD); faster-whisper (MIT, CTranslate2 int8, ~4× faster); whisperX (BSD-2, word timestamps + diarization); **NVIDIA Parakeet TDT 1.1B** (CC-BY-4.0, LibriSpeech-clean WER 1.39% — best English open ASR); Parakeet 0.6B v3 / Canary 1B v2 (multilingual); Moonshine (MIT, on-device); VibeVoice-ASR-7B (MIT, 60-min, 50+ langs). **VAD:** Silero VAD (MIT, ~2 MB, <1 ms per 30 ms chunk; embedded in faster-whisper/whisperX). **TTS serving:** **vLLM-Omni** ([vllm-project/vllm-omni](https://github.com/vllm-project/vllm-omni), Apache-2.0) publishes the official images `vllm/vllm-omni` (CUDA) and `vllm/vllm-omni-rocm` (ROCm), both tagged `v0.28.0` (Docker Hub tag lists, 2026-10-18), and serves Qwen3-TTS (`docs/models/supported_models.md`). It serves the OpenAI speech path `POST /v1/audio/speech`; the `POST /v1/audio/voices` upload route is *(unverified 2026-10-18)* — the docs reachable today do not document it. Its gfx942/gfx950 validation note is a footnote scoped to MiniMax H3 diffusion, **not** to TTS, and the fleet runs no vLLM anywhere: the R9700s carry radiance as frank-archon (`scripts/serve-frank-archon.sh:2`). It is also no longer "the one server" with the OpenAI shape — llama.cpp's own `llama-tts-server` has it too (next section). Others: Kokoro-FastAPI (Docker, :8880, Apache-2.0), CosyVoice **FastAPI and gRPC** (`runtime/python/fastapi`, `runtime/python/grpc`; the README documents no WebSocket server), F5-TTS Gradio/Triton, GPT-SoVITS WebUI/api_v2, Fish S2 `tools/api_server.py` (own format, not OpenAI-shaped). **Post-processing:** SoX/ffmpeg (resample/normalize; 24 kHz mono 16-bit is the reference-clip standard); Demucs (`htdemucs`) for vocal separation before cloning; the Perth watermark is built into Chatterbox. ## llama.cpp-native TTS serving (verified 2026-10-18) The CPU shape does not need a Python server at all: - **`llama-tts-server`** ([`tc-mb/llama.cpp-omni`](https://github.com/tc-mb/llama.cpp-omni), MIT, "Omni inference in C/C++") is a standalone OpenAI-compatible TTS server that runs without an LLM model (build target `llama-tts-server`). `POST /v1/audio/speech` takes `model` (`"voxcpm"`, also accepts `"voxcpm2"`), `input`, `voice` — documented **"reserved"**, so do not build named-voice semantics on it — `response_format` (`wav`/`pcm`), `reference_audio` (base64 WAV for cloning), `seed` 42, `cfg_value` 2.0, `inference_timesteps` 10, `max_steps` 200, `temperature` 1.0. `POST /v1/audio/speech/stream` returns chunked WAV; `POST /v1/voxcpm2/init` hot-loads a runtime from `base_lm` + `acoustic` GGUFs with `n_gpu_layers` (`-1` = all); `GET /v1/audio/speech/models` lists what is loaded (`tools/omni/voxcpm2/README.md`:232, 321-354). - **Mainline `llama-tts`** (ggml-org/llama.cpp) has `--tts-speaker-file` for a speaker reference with **Qwen3-TTS** and with **Kyutai Pocket-TTS** — the latter requires it ("the model produces almost no audio without it"), and Pocket-TTS converts one `languages/` directory, never the repo root (`tools/tts/README.md`:20-58; `kyutai/pocket-tts` is CC-BY-4.0). - **VoxCPM2 GGUF provenance:** `openbmb/VoxCPM2` publishes **PyTorch weights only** (`model.safetensors`, no `.gguf` sibling — HF API 2026-10-18). The GGUFs the fork loads come from [`DennisHuang648/VoxCPM2-GGUF`](https://huggingface.co/DennisHuang648/VoxCPM2-GGUF) (`VoxCPM2-BaseLM-F16.gguf`, `VoxCPM2-BaseLM-Q8_0.gguf`, `VoxCPM2-Acoustic-F16.gguf`, Apache-2.0 card) or from the fork's own `tools/omni/voxcpm2/convert_voxcpm2_to_gguf.py`. Arm B (`llama-tts`) and arm A (`qwentts.cpp`) have both been measured on this box — RTF, first-audio latency and RSS are in the next section. The other servers in the table above (vLLM-Omni, Kokoro-FastAPI, CosyVoice, F5-TTS, GPT-SoVITS) remain unmeasured here; that absence is still the finding. ## Measured CPU engine A/B (2026-10-18) Both candidate upstreams for frank-tts were measured on Dankclaw's CPU — 8 threads, no GPU touched, three runs per arm, seed 42, identical text, against the fleet's own audition voice packs. - **Arm A — [`ServeurpersoCom/qwentts.cpp`](https://github.com/ServeurpersoCom/qwentts.cpp)** at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` (2026-10-16), weights from [`Serveurperso/Qwen3-TTS-GGUF`](https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF): `qwen-talker-1.7b-base-Q4_K_M.gguf` (1,219,245,248 B), `qwen-talker-0.6b-base-Q4_K_M.gguf` (628,905,056 B), `qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B) — byte counts verified against the Hub API. Builds with zero compiler warnings; `GGML_BLAS=OFF` is forced on this host (only `libopenblas.so.0` exists, no dev headers, so the vendor `buildcpu.sh` cannot configure). - **Arm B — mainline `llama-tts`** with `--tts-speaker-file`, i.e. the `llama-tts-cli` backend `scripts/frank-tts.env` already implements. RTF is **seconds of compute per second of audio** (lower is better; below 1 is faster than real time). | Arm | RTF | Run spread | Peak RSS | |---|---|---|---| | A · 0.6 B talker, CLI | 0.44 | ±0.002 | 3.4 GiB | | A · 1.7 B, CLI, uncloned | 0.57–0.58 | ±0.006 | 4.7 GiB | | A · 1.7 B, CLI, embedding-only clone | 0.63 | ±0.02 | n/m | | A · 1.7 B, CLI, ICL clone | 0.92 | ±0.01 | n/m | | A · 1.7 B, resident `tts-server`, warm | 0.55–0.57 | ±0.01 | 5.7 GiB | | A · same, ICL voice warm / **cold (first request)** | 0.67 / **1.65** | one-shot | 5.7 GiB | | B · mainline `llama-tts` | 0.93–0.99 | ±0.25 | n/m | **Read RTF only within an arm.** Conditioning changes clip length — the same seed yields 18.00 s of audio embedding-only and 15.12 s under ICL — so the denominator is the arm's own audio duration. Arm A's resident server is ~1.7× faster than arm B and about 40× tighter in run-to-run spread, and it already serves the HTTP shape frank-tts's façade speaks: `/health`, `/v1/models`, `/v1/audio/voices` (GET/POST/DELETE), `/v1/audio/speech` (`wav`/`pcm`), chunked streaming with the first audio byte at 86–451 ms and a server-side time-to-first-audio of 94–112 ms. Arm B has no HTTP path at all — that is what the façade's CLI backend exists for. Four laws came out of the trial, each worth a line of configuration: - **Boot-warm every voice.** The trial measured the first request for a registered voice at 6.2–8.0 s (first audio byte 5,530 ms) and later requests restoring the snapshot in 0.5 ms. **That ladder did not survive deployment:** measured 2026-10-18 on the built container against an empty registry — register, then time — the first synthesis of a freshly registered voice cost **1,883 ms** against **1,226–1,283 ms** for the requests after it, and a ~110-word request cost **9.4 s** then **9.8 s**, so request time tracks the audio generated rather than snapshot state. What still holds is the reason to warm at all: the registry is memory-only and re-registering does not warm, so an engine unit must run one throwaway synthesis per voice at start. It buys registration and about 0.6 s, not five seconds. - **Speaker artifacts are reusable and exact.** `qwen-codec` turns a reference WAV into `.spk` (8,192 B x-vector) + `.rvq` (5,456–6,864 B, 16 codebooks × T, 11-bit packed) in 1.8–2.1 s at 1.3–1.5 GiB peak; replaying them yields a **byte-identical** WAV and saves 1.4 s per request. That pair is the engine-artifact term the voice-pack law in `scripts/frank-tts.env` reserves a slot for — arm A pins it as `.spk` + `.rvq` beside `reference.wav`. - **Streaming is not bit-identical to buffered**, whatever the upstream README claims: same text, seed and voice, 92.2 % of samples differ (peak delta 1139/32768, −41.6 dB RMS — inaudible). A fixed seed does not pin output across `response_format`, so never mix the two paths in a cache or a golden fixture. - **A voice cannot be registered from an embedding alone**: `POST /v1/audio/voices` accepts `wav_b64` or **both** `spk_b64` and `rvq_b64` (`src/tts-server.h:480`), so an embedding-only voice has to be sent as the WAV, or the lone-`spk` path has to be upstreamed. **Likeness is not measurable at this resolution.** x-vector cosine against each speaker's own reference: ICL 0.9886 / 0.9901, embedding-only 0.9844 / 0.9812, uncloned 0.9228 — but **two different speakers score 0.9497** on the same metric. The whole clone effect lives in a 0.04-wide band and the ICL-over-embedding gap is inside noise, so the four audition clips (`/home/blank/voice-audition/qwentts-*.mp3`, each against its `armB-*.mp3` mainline partner, guide in `LISTEN-engine.md`) are the instrument. **The operator heard them on 2026-10-18 and passed every clone**, which removed the last axis measurement could not reach; the engine was then picked on live-output grounds ([Decision](#decision-arm-a-for-live-output-2026-10-18)). The cosine extractor is still worth keeping: it is engine-neutral (it scores mainline output through the same encoder) and it separates an uncloned voice cleanly. Two operational details. The engine takes 8 threads and exposes **no `-t` flag** — `src/backend.h` uses `hardware_concurrency()/2` and prints the choice at load, so pinning a cpuset on the quadlet changes that number implicitly. And ICL needs a reference transcript that did not exist: the fleet has no ASR endpoint and no audio input, so the crops were transcribed with `vosk-small-en-us` inside the work dir, and two of three transcripts came back suspect ("demon souls" for *Dark Souls*) — a reason to curate `ref_text` into the pack manifest instead of transcribing on the host. ## Decision: arm A for live output (2026-10-18) Every audition pair passed, so likeness stopped being the tiebreak. The question became **which engine is better for live output** — the harness speaking a reply, an alert read aloud, a click-to-speak — and on that criterion RTF only has to stay under 1.0 with margin: what separates the arms is **time to first audio** and the **spread**, because an arm whose RTF can cross 1.0 stalls a live stream mid-sentence. | | arm A · `qwentts.cpp` `tts-server` | arm B · mainline `llama-tts` | |---|---|---| | first audio byte | 86–451 ms | model load, per request | | warm RTF | 0.55–0.67 | 0.93–0.99 | | run spread | ±0.01 | ±0.25 | | HTTP | serves `/v1/audio/speech` | none — façade CLI only | | RAM | 5.7 GiB resident | per-request, freed after | **Chosen: arm A, 1.7 B talker, resident `tts-server` in a `frank-tts-engine` container, with `TTS_BACKEND=llama-tts-cli` kept in the repo as the rollback.** Three consequences are configuration, not opinion: the unit **boot-warms every voice** — the registry is memory-only, so an unwarm'd name is a 400 and the first synthesis otherwise costs about 0.6 s more than the ones after it, as corrected above —, it **does not pin a cpuset** casually (threads are `hardware_concurrency()/2`, no flag), and every pack registers with the **WAV** because the engine rejects a lone embedding. Two follow-ups stay open on purpose, and neither blocks the deploy: - **Talker size.** The 0.6 B arm measured 0.44 RTF at 3.4 GiB and has never been auditioned — every clip above is 1.7 B. It is the lever for a CPU co-tenancy problem, not a preemptive saving. - **ICL vs embedding-only.** Both passed the ear test. Embedding-only is the shipped default: 0.57–0.63 vs 0.67 warm, and it needs no `ref_text`, which this fleet cannot produce honestly (the ASR crops came back suspect). Packs still carry `.spk` + `.rvq`, so ICL remains a per-voice opt-in for batch work like the podcast render, where total time matters more than first byte. ## Recommended deployment (Dankclaw) As decided in Gitea issue #2, this stays **CPU-only on Dankclaw**: the RTX 3090 has carried frank-zealot since 2026-10-17 and both R9700s are frank-archon's, so no card may be asked to co-host TTS. The shape is one engine plus one façade, and **nothing here is deployed as of 2026-10-18**: `frank-tts` has neither a unit nor a container on the host, and nothing listens on 7772 or 8880. | Service | Image / install | Port | CPU | Notes | |---|---|---|---|---| | TTS engine | **arm A `qwentts.cpp` `tts-server`** on the Qwen3-TTS GGUFs (decided 2026-10-18, above); arm B mainline `llama-tts` stays in the repo as the `llama-tts-cli` rollback; `vllm/vllm-omni` + Qwen3-TTS only if a GPU is ever deliberately freed | 8880 (container-internal) | open, evidence above | arm A serves OpenAI `/v1/audio/speech` itself and boot-warms each voice; arm B has no HTTP path and runs through the façade's CLI backend | | frank-tts | `dankclaw/voiceservice/` (stdlib container) + `scripts/frank-tts.env` + its quadlet | 7772 | yes | voice *name* → `/voices//reference.wav`, `voice`/`voice_id` on the wire, mp3 re-encode; backends `omni-http` (forwards base64 `reference_audio`) and `llama-tts-cli` (`--tts-speaker-file`, `nice 19`) | | STT (English) | Parakeet TDT 1.1B behind a FastAPI wrapper, or whisper.cpp server | 8882 | yes | CC-BY-4.0; LibriSpeech-clean WER 1.39 (model card) | | STT (multilingual, opt.) | faster-whisper `large-v3` int8 | 8883 | yes | MIT | | VAD | Silero (embedded) | — | yes | MIT; model file 2 327 524 B | 7772 is frank-tts's host port — the next slot in the fleet's model-endpoint family (7770 frank-dragoon, 7771 frank-zealot). The engine behind it keeps 8880, but **not as a host-loopback publish**: netavark DNATs published ports only, and a bridge container cannot reach a `127.0.0.1` bind on the host — that is the lesson that 502'd hermes (`AGENTS.md`). `host.containers.internal` (169.254.1.2) is the gateway, so it reaches a host service only when that service binds a routable address; `scripts/frank-tts.env` names it as the engine's address, which holds for a `0.0.0.0` bind and not for a loopback one. The deployed pair therefore shares one quadlet network and the façade addresses the engine by container name. ## Voice-creation workflow (the "from sample files" answer) 1. Record 2–3 min clean speech (24 kHz 16-bit mono WAV; ffmpeg-normalize). 2. **Zero-shot:** keep the clip as `ref_audio` — Chatterbox/Qwen3-TTS/CosyVoice/ GLM-TTS clone from it per request (no training). 3. **Dedicated voice:** upload once to Qwen3-TTS via `/v1/audio/voices` (vLLM-Omni) and reference by ID; or LoRA-fine-tune VoxCPM2 (5–10 min audio); or the GPT-SoVITS 1-min fine-tune for the classic WebUI workflow. 4. **Kokoro fine-tune** only if the voice must live inside OMP's offline local backend — highest effort, Kokoro-styled output. ## Best practices - **Prefer zero-shot cloning over fine-tuning** for a personal voice: a modern cloner (Chatterbox/Qwen3-TTS) on a 10 s–3 min reference beats a Kokoro fine-tune on quality and effort. - **Normalize reference clips to 24 kHz 16-bit mono WAV** before use; extract clean voice with Demucs if the source is music/mixed. - **Serve through vLLM-Omni** when you want a single OpenAI-compatible endpoint for many TTS models — it is the only server with the standard `/v1/audio/speech` + `/v1/audio/voices` shape across the 2026 lineup. - **Check the license before serving publicly:** several strong cloners are non-commercial (Higgs v3, Breeze-TTS-2, Spark-TTS, Voxtral) or carry custom licenses (IndexTTS, Fish S2, XTTS v2). For a personal homelab voice, MIT / Apache-2.0 (Chatterbox, Qwen3-TTS, GLM-TTS, VoxCPM2) is the safe set. - **Keep TTS off the GPUs.** The 2026-09-22 advice to "co-locate on the 3090 at `--gpu-memory-utilization ~0.35`" is retired: that card belongs to frank-zealot (2026-10-17) and the R9700s to frank-archon, and Gitea issue #2 measures both as full. Run TTS on CPU; revisit only if the operator frees a card on purpose. - **Guard the endpoint like any other service:** if vhosted, it goes behind the Anubis → Authentik chain; non-browser clients (OMP's `curl`) must target the internal address, never the public hostname. ## Citations - OMP speech tool: https://omp.sh/docs/tools ("Generate speech", gated by `speechgen.enabled`; the old `https://omp.sh/docs/tools/tts` path returns 404 as of 2026-10-18) · installed binary `@oh-my-pi/pi-coding-agent/dist/cli.js` (the `openai-speech` API, the 15 000-character cap, the `voice_id` forwarding rule, local Kokoro WAV-only) - Kokoro: https://github.com/hexgrad/Kokoro · https://huggingface.co/spaces/hexgrad/Kokoro-TTS - Kokoro fine-tune: https://github.com/semidark/kikiri-tts - Chatterbox: https://github.com/resemble-ai/chatterbox - Qwen3-TTS: https://github.com/QwenLM/Qwen3-TTS · https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base - CosyVoice: https://github.com/FunAudioLLM/CosyVoice - GLM-TTS: https://github.com/zai-org/GLM-TTS · https://huggingface.co/zai-org/GLM-TTS - VoxCPM2: https://github.com/OpenBMB/VoxCPM · https://huggingface.co/openbmb/VoxCPM2 - llama.cpp-omni TTS server: https://github.com/tc-mb/llama.cpp-omni · https://raw.githubusercontent.com/tc-mb/llama.cpp-omni/master/tools/omni/voxcpm2/README.md - qwentts.cpp (arm A): https://github.com/ServeurpersoCom/qwentts.cpp at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` · weights https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF (byte counts against the Hub API, 2026-10-18) - mainline `llama-tts`: https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/tts/README.md · Pocket-TTS weights: https://huggingface.co/kyutai/pocket-tts (CC-BY-4.0) - VoxCPM2 GGUFs (not published by OpenBMB): https://huggingface.co/DennisHuang648/VoxCPM2-GGUF - GLM-TTS license split: model card `mit` (HF API 2026-10-18) vs code `LICENSE` (Apache-2.0): https://raw.githubusercontent.com/zai-org/GLM-TTS/main/LICENSE - F5-TTS: https://github.com/SWivid/F5-TTS - GPT-SoVITS: https://github.com/RVC-Boss/GPT-SoVITS - Dots.TTS: https://huggingface.co/dots-studio/dots.tts-base - IndexTTS: https://github.com/index-tts/index-tts - Fish Speech: https://github.com/fishaudio/fish-speech - MOSS-TTS-Nano: https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Nano - XTTS: https://github.com/coqui-ai/TTS - MegaTTS3: https://github.com/bytedance/MegaTTS3 - Higgs Audio: https://github.com/boson-ai/higgs-audio - Breeze-TTS-2: https://huggingface.co/BreezeBlue/Breeze-TTS-2 - Spark-TTS: https://github.com/SparkAudio/Spark-TTS - Voxtral TTS: https://huggingface.co/MistralAI/Voxtral-4B-TTS-2603 - Parler-TTS: https://github.com/huggingface/parler-tts - VibeVoice: https://github.com/microsoft/VibeVoice - vLLM-Omni: https://github.com/vllm-project/vllm-omni · https://docs.vllm.ai/projects/vllm-omni/en/latest/ - Kokoro-FastAPI: https://github.com/remsky/Kokoro-FastAPI - whisper.cpp: https://github.com/ggml-org/whisper.cpp · faster-whisper: https://github.com/SYSTRAN/faster-whisper · whisperX: https://github.com/m-bain/whisperX - Parakeet: https://huggingface.co/nvidia/parakeet-tdt-1.1b · Canary: https://huggingface.co/nvidia/canary-1b-v2 - Moonshine: https://github.com/moonshine-ai/moonshine - Silero VAD: https://github.com/snakers4/silero-vad - xAI custom voices: https://docs.x.ai/developers/model-capabilities/audio/custom-voices