serverAppConfigs/serverAppConfigs-docs/Research/Local TTS and Voice Cloning.md

---
type: research
title: Local TTS and Voice Cloning
description: How to create a new local TTS voice from sample files in oh-my-pi, the 2026 open-weight voice-cloning landscape, the measured CPU engine A/B behind frank-tts and the arm A decision, the full local AI audio stack, and a recommended homelab deployment.
tags:
  - research
  - tts
  - voice-cloning
  - audio
  - stt
  - vllm-omni
  - kokoro
timestamp: 2026-10-18T16:05:00Z
---

# Local TTS and Voice Cloning

Research brief (2026-09-22) on creating a new local TTS voice from sample
files, the open-weight voice-cloning landscape, the full local AI audio
stack, and a recommended deployment for the fleet. Every claim is cited in
[Citations](#citations). The fleet has no audio input today (that capability
left with the retired NPU endpoint, returned with the Gemma4 audio projector on
2026-09-17, and left again with that retirement on 2026-10-01 — audio is still
greenfield: every fleet endpoint declares `input: [text, image]`
(`~/.omp/agent/models.yml:63,149,214`).

## Summary

- **Kokoro-82M (OMP's built-in local TTS) cannot zero-shot clone a voice.**
  Voices are a fixed pack. A community fine-tune recipe exists
  (`semidark/kikiri-tts`): a ~51 h multi-speaker base stage + a few hundred
  single-speaker samples, then voice-pack extraction. Feasible on one GPU, but
  the output is Kokoro-styled, and OMP's local backend only accepts its
  built-in voice list (a custom pack needs a patch).
- **The 2026 open-weight cloning landscape is strong.** Permissive-licensed
  leaders for a single-GPU homelab: **Chatterbox (MIT)**, **Qwen3-TTS
  (Apache-2.0)**, **CosyVoice 3 (Apache-2.0)**, **GLM-TTS (MIT weights)**,
  **VoxCPM2 (Apache-2.0)**, **F5-TTS (MIT)**, **GPT-SoVITS (MIT)**.
- **OMP integration (re-verified 2026-10-18):** the speech tool is gated by
  `speechgen.enabled` and resolves models through the **`speech` role and its
  fallback chain** (`modelRoles.speech`; this workstation runs
  `local/kokoro`). Three APIs: local Kokoro ONNX (always WAV/PCM16), `xai-tts`,
  and **`openai-speech` — a generic OpenAI-compatible `/v1/audio/speech`
  client** (MP3 or WAV, `voice_id` forwarded only when explicitly set, 15 000
  characters max; DeepInfra is now just a model on that API). The 2026-09-22
  finding that OMP has *no* configurable OpenAI TTS endpoint is **retired**: a
  local server needs a speech-role entry, not a patch.
- **Fleet plan (2026-10-18): CPU-only on Dankclaw, not the 3090.** Gitea issue
  #2 rules out every GPU option (the 3090 has carried frank-zealot since
  2026-10-17; both R9700s are frank-archon's). The shape is a TTS engine on
  **8880** behind **frank-tts**, the fleet's speech endpoint on **7772**
  (`scripts/frank-tts.env`, `dankclaw/voiceservice/`): one OpenAI
  `/v1/audio/speech` façade that resolves a voice *by name* and never shows a
  path. Nothing is deployed — no `frank-tts` unit and no container exist on the
  host and nothing listens on 7772 or 8880 (checked 2026-10-18). STT (Parakeet
  TDT 1.1B, CC-BY-4.0) stays on CPU. **The engine is chosen: arm A**
  `qwentts.cpp` **resident `tts-server`, 1.7 B talker** — measured in
  *Measured CPU engine A/B* below and picked on the live-output criterion in
  [Decision: arm A for live output](#decision-arm-a-for-live-output-2026-10-18).

## Can Kokoro learn a new voice?

Kokoro ([hexgrad/Kokoro](https://github.com/hexgrad/Kokoro), Apache-2.0, 82 M
params, non-autoregressive) ships **fixed voice packs** — there is no
reference-audio path, so zero-shot cloning is not an option. OMP's local
backend runs `onnx-community/Kokoro-82M-v1.0-ONNX` offline, choosing a voice
from a built-in list (`tts.localVoice`).

The best-documented fine-tune path is
[`semidark/kikiri-tts`](https://github.com/semidark/kikiri-tts) (Apache-2.0):

| Stage | Data | Purpose |
|---|---|---|
| Multi-speaker base | ~51 h audio, 51 speakers | language adaptation / base fine-tune |
| Single-speaker | 455–627 samples (~1–2 h of one voice) | voice specialization |
| Extraction | — | voice-pack (.pt) extraction for inference |

**Verdict:** fine-tuning Kokoro to a personal voice is possible on a 3090, but
you must collect ~1–2 h of clean single-speaker audio, run the two-stage recipe,
and patch OMP to load the custom pack. The effort/quality trade is worse than
running a modern zero-shot cloner on the same GPU.

## 2026 voice-cloning landscape (permissive-licensed, single-GPU)

Ranked (2026-09-22) for a 3090 homelab, English-primary, personal voice; the
2026-10-18 fleet decision below moved this work off the GPU entirely:

1. **Chatterbox (MIT)** — best license/quality/compute ratio; ~10 s reference
   (the README's own example is `your_10s_ref_clip.wav`); 110 M (Nano) to 0.5 B
   (Turbo/Multilingual); ~1–2 GB VRAM *(unverified 2026-10-18)*; Nano is
   documented for CPU — "3x faster than realtime on 8 CPU cores". *Top pick for
   a personal English voice.* No server in the README (library use only; wrap
   it yourself); every generated file carries Resemble's PerTh watermark.
2. **Qwen3-TTS-1.7B-Base (Apache-2.0)** — 3 s cloning, 10 languages, streaming,
   vLLM-Omni serving with a `POST /v1/audio/voices` upload *(unverified
   2026-10-18 — the reachable docs do not list it)*, the closest thing to
   "named voice" semantics, official fine-tuning code. *Top pick for a
   OpenAI-compatible deployment.*
3. **CosyVoice 3 (Apache-2.0)** — 0.5 B, 9 languages + 18 zh dialects, 150 ms
   bi-streaming, FastAPI and gRPC servers.
4. **GLM-TTS (MIT model weights, Apache-2.0 code)** — CER 0.89 after
   multi-reward GRPO (down from 1.03), RL emotion control, zh/en; the prompt
   audio shipped in the repo is marked research-only.
5. **VoxCPM2 (Apache-2.0)** — 30 languages, 48 kHz, LoRA fine-tune from 5–10 min
   of audio, ~8 GB VRAM. *Best "fine-tune my voice" option.*
6. **F5-TTS (MIT)** — 0.3–0.5 B, en/zh; Docker + Triton.
7. **GPT-SoVITS (MIT)** — best few-shot fine-tune UX (1 min of data), WebUI,
   Docker images.

Notable non-commercial / restricted: Higgs Audio v3, Breeze-TTS-2, Spark-TTS
(non-commercial); IndexTTS (bilibili custom license), Fish Speech S2 (research
license), XTTS v2 (CPML, non-commercial, vendor defunct); MegaTTS3 (WaveVAE
encoder not released); Voxtral TTS (no official cloning — codec encoder not
released). Excluded from cloning: Parler-TTS (style-text only), VibeVoice-TTS
(code withdrawn 2025-09).

## OMP integration (definitive)

The tool ships hidden and is enabled by `speechgen.enabled`
(`~/.omp/agent/config.yml:222-223`); the harness docs list it as "Generate
speech". Models come from the **`speech` role and its fallback chain**, not
from a hardcoded backend list (`modelRoles.speech`,
`~/.omp/agent/config.yml:16` = `local/kokoro` today):

| API | Endpoint | Notes (from the installed bundle, 2026-10-18) |
|---|---|---|
| local-inference | on-device ONNX, `onnx-community/Kokoro-82M-v1.0-ONNX` (Apache-2.0) | no network, **always WAV/PCM16**, voice from the built-in list (`tts.localVoice`) |
| `xai-tts` | `POST https://api.x.ai/v1/tts` | `ara`/`eve`/`leo`/`rex`/`sal` + custom voice IDs; MP3 or WAV |
| `openai-speech` | `POST <provider baseUrl>/audio/speech` | the generic OpenAI-compatible client — MP3 or WAV, `voice_id` forwarded **only when explicitly set**, 15 000-character cap on `input`; DeepInfra is one model entry on it (`https://api.deepinfra.com/v1/openai`) |

The 2026-09-22 finding — "**no generic OpenAI-compatible TTS endpoint** …
`deepinfra` is a hardcoded URL, not a configurable base URL" — is **retired**:
`openai-speech` is that configurable client, selected by putting a model on the
`speech` role. Still true from the original audit: the local Kokoro backend
accepts only its built-in voice list, so a custom pack needs a patch.

Paths for a Frank-cloned voice (2026-10-18):

1. **Local server + speech role, no patch.** Register frank-tts (port 7772) as
   a provider model on the `openai-speech` API and point `modelRoles.speech` at
   it; the façade turns a voice *name* into that pack's reference clip and
   forwards it as base64 `reference_audio` (`scripts/frank-tts.env`,
   `TTS_BACKEND=omni-http`). Not deployed as of 2026-10-18.
2. **xAI console custom voice — works today, zero code.** Record 2–3 min clean
   speech (≤120 s, 24 kHz 16-bit mono WAV) → console → `voice_id` → speech
   tool. Free (≤30 voices), cloud, US-only.
3. **No-patch workaround.** The `bash` tool can `curl` the local TTS server ad
   hoc, and the harness `tts` device writes a speech file to a path you give it.
4. **Kokoro fine-tune → OMP local backend.** Highest effort, Kokoro-styled.

## Full local AI audio stack

**STT / ASR:** whisper.cpp (MIT, tiny→large-v3 2.9 GiB, all backends, built-in
VAD); faster-whisper (MIT, CTranslate2 int8, ~4× faster); whisperX (BSD-2, word
timestamps + diarization); **NVIDIA Parakeet TDT 1.1B** (CC-BY-4.0,
LibriSpeech-clean WER 1.39% — best English open ASR); Parakeet 0.6B v3 /
Canary 1B v2 (multilingual); Moonshine (MIT, on-device); VibeVoice-ASR-7B (MIT,
60-min, 50+ langs).

**VAD:** Silero VAD (MIT, ~2 MB, <1 ms per 30 ms chunk; embedded in
faster-whisper/whisperX).

**TTS serving:** **vLLM-Omni** ([vllm-project/vllm-omni](https://github.com/vllm-project/vllm-omni),
Apache-2.0) publishes the official images `vllm/vllm-omni` (CUDA) and
`vllm/vllm-omni-rocm` (ROCm), both tagged `v0.28.0` (Docker Hub tag lists,
2026-10-18), and serves Qwen3-TTS (`docs/models/supported_models.md`). It
serves the OpenAI speech path `POST /v1/audio/speech`; the
`POST /v1/audio/voices` upload route is *(unverified 2026-10-18)* — the docs
reachable today do not document it. Its gfx942/gfx950 validation note is a
footnote scoped to MiniMax H3 diffusion, **not** to TTS, and the fleet runs no
vLLM anywhere: the R9700s carry radiance as frank-archon
(`scripts/serve-frank-archon.sh:2`). It is also no longer "the one server" with
the OpenAI shape — llama.cpp's own `llama-tts-server` has it too (next
section). Others: Kokoro-FastAPI (Docker, :8880, Apache-2.0), CosyVoice
**FastAPI and gRPC** (`runtime/python/fastapi`, `runtime/python/grpc`; the
README documents no WebSocket server), F5-TTS Gradio/Triton, GPT-SoVITS
WebUI/api_v2, Fish S2 `tools/api_server.py` (own format, not OpenAI-shaped).

**Post-processing:** SoX/ffmpeg (resample/normalize; 24 kHz mono 16-bit is the
reference-clip standard); Demucs (`htdemucs`) for vocal separation before
cloning; the Perth watermark is built into Chatterbox.

## llama.cpp-native TTS serving (verified 2026-10-18)

The CPU shape does not need a Python server at all:

- **`llama-tts-server`** ([`tc-mb/llama.cpp-omni`](https://github.com/tc-mb/llama.cpp-omni),
  MIT, "Omni inference in C/C++") is a standalone OpenAI-compatible TTS server
  that runs without an LLM model (build target `llama-tts-server`).
  `POST /v1/audio/speech` takes `model` (`"voxcpm"`, also accepts `"voxcpm2"`),
  `input`, `voice` — documented **"reserved"**, so do not build named-voice
  semantics on it — `response_format` (`wav`/`pcm`), `reference_audio`
  (base64 WAV for cloning), `seed` 42, `cfg_value` 2.0, `inference_timesteps`
  10, `max_steps` 200, `temperature` 1.0. `POST /v1/audio/speech/stream`
  returns chunked WAV; `POST /v1/voxcpm2/init` hot-loads a runtime from
  `base_lm` + `acoustic` GGUFs with `n_gpu_layers` (`-1` = all);
  `GET /v1/audio/speech/models` lists what is loaded
  (`tools/omni/voxcpm2/README.md`:232, 321-354).
- **Mainline `llama-tts`** (ggml-org/llama.cpp) has `--tts-speaker-file` for a
  speaker reference with **Qwen3-TTS** and with **Kyutai Pocket-TTS** — the
  latter requires it ("the model produces almost no audio without it"), and
  Pocket-TTS converts one `languages/<name>` directory, never the repo root
  (`tools/tts/README.md`:20-58; `kyutai/pocket-tts` is CC-BY-4.0).
- **VoxCPM2 GGUF provenance:** `openbmb/VoxCPM2` publishes **PyTorch weights
  only** (`model.safetensors`, no `.gguf` sibling — HF API 2026-10-18). The
  GGUFs the fork loads come from
  [`DennisHuang648/VoxCPM2-GGUF`](https://huggingface.co/DennisHuang648/VoxCPM2-GGUF)
  (`VoxCPM2-BaseLM-F16.gguf`, `VoxCPM2-BaseLM-Q8_0.gguf`,
  `VoxCPM2-Acoustic-F16.gguf`, Apache-2.0 card) or from the fork's own
  `tools/omni/voxcpm2/convert_voxcpm2_to_gguf.py`.

Arm B (`llama-tts`) and arm A (`qwentts.cpp`) have both been measured on this
box — RTF, first-audio latency and RSS are in the next section. The other
servers in the table above (vLLM-Omni, Kokoro-FastAPI, CosyVoice, F5-TTS,
GPT-SoVITS) remain unmeasured here; that absence is still the finding.

## Measured CPU engine A/B (2026-10-18)

Both candidate upstreams for frank-tts were measured on Dankclaw's CPU — 8
threads, no GPU touched, three runs per arm, seed 42, identical text, against
the fleet's own audition voice packs.

- **Arm A — [`ServeurpersoCom/qwentts.cpp`](https://github.com/ServeurpersoCom/qwentts.cpp)**
  at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` (2026-10-16), weights
  from [`Serveurperso/Qwen3-TTS-GGUF`](https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF):
  `qwen-talker-1.7b-base-Q4_K_M.gguf` (1,219,245,248 B),
  `qwen-talker-0.6b-base-Q4_K_M.gguf` (628,905,056 B),
  `qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B) — byte counts verified
  against the Hub API. Builds with zero compiler warnings; `GGML_BLAS=OFF` is
  forced on this host (only `libopenblas.so.0` exists, no dev headers, so the
  vendor `buildcpu.sh` cannot configure).
- **Arm B — mainline `llama-tts`** with `--tts-speaker-file`, i.e. the
  `llama-tts-cli` backend `scripts/frank-tts.env` already implements.

RTF is **seconds of compute per second of audio** (lower is better; below 1 is
faster than real time).

| Arm | RTF | Run spread | Peak RSS |
|---|---|---|---|
| A · 0.6 B talker, CLI | 0.44 | ±0.002 | 3.4 GiB |
| A · 1.7 B, CLI, uncloned | 0.57–0.58 | ±0.006 | 4.7 GiB |
| A · 1.7 B, CLI, embedding-only clone | 0.63 | ±0.02 | n/m |
| A · 1.7 B, CLI, ICL clone | 0.92 | ±0.01 | n/m |
| A · 1.7 B, resident `tts-server`, warm | 0.55–0.57 | ±0.01 | 5.7 GiB |
| A · same, ICL voice warm / **cold (first request)** | 0.67 / **1.65** | one-shot | 5.7 GiB |
| B · mainline `llama-tts` | 0.93–0.99 | ±0.25 | n/m |

**Read RTF only within an arm.** Conditioning changes clip length — the same
seed yields 18.00 s of audio embedding-only and 15.12 s under ICL — so the
denominator is the arm's own audio duration.

Arm A's resident server is ~1.7× faster than arm B and about 40× tighter in
run-to-run spread, and it already serves the HTTP shape frank-tts's façade
speaks: `/health`, `/v1/models`, `/v1/audio/voices` (GET/POST/DELETE),
`/v1/audio/speech` (`wav`/`pcm`), chunked streaming with the first audio byte
at 86–451 ms and a server-side time-to-first-audio of 94–112 ms. Arm B has no
HTTP path at all — that is what the façade's CLI backend exists for.

Four laws came out of the trial, each worth a line of configuration:

- **Boot-warm every voice.** The trial measured the first request for a
  registered voice at 6.2–8.0 s (first audio byte 5,530 ms) and later requests
  restoring the snapshot in 0.5 ms. **That ladder did not survive deployment:**
  measured 2026-10-18 on the built container against an empty registry — register,
  then time — the first synthesis of a freshly registered voice cost **1,883 ms**
  against **1,226–1,283 ms** for the requests after it, and a ~110-word request
  cost **9.4 s** then **9.8 s**, so request time tracks the audio generated rather
  than snapshot state. What still holds is the reason to warm at all: the registry
  is memory-only and re-registering does not warm, so an engine unit must run one
  throwaway synthesis per voice at start. It buys registration and about 0.6 s,
  not five seconds.
- **Speaker artifacts are reusable and exact.** `qwen-codec` turns a reference
  WAV into `.spk` (8,192 B x-vector) + `.rvq` (5,456–6,864 B, 16 codebooks × T,
  11-bit packed) in 1.8–2.1 s at 1.3–1.5 GiB peak; replaying them yields a
  **byte-identical** WAV and saves 1.4 s per request. That pair is the
  engine-artifact term the voice-pack law in `scripts/frank-tts.env` reserves a
  slot for — arm A pins it as `.spk` + `.rvq` beside `reference.wav`.
- **Streaming is not bit-identical to buffered**, whatever the upstream README
  claims: same text, seed and voice, 92.2 % of samples differ (peak delta
  1139/32768, −41.6 dB RMS — inaudible). A fixed seed does not pin output
  across `response_format`, so never mix the two paths in a cache or a golden
  fixture.
- **A voice cannot be registered from an embedding alone**: `POST
  /v1/audio/voices` accepts `wav_b64` or **both** `spk_b64` and `rvq_b64`
  (`src/tts-server.h:480`), so an embedding-only voice has to be sent as the
  WAV, or the lone-`spk` path has to be upstreamed.

**Likeness is not measurable at this resolution.** x-vector cosine against each
speaker's own reference: ICL 0.9886 / 0.9901, embedding-only 0.9844 / 0.9812,
uncloned 0.9228 — but **two different speakers score 0.9497** on the same
metric. The whole clone effect lives in a 0.04-wide band and the
ICL-over-embedding gap is inside noise, so the four audition clips
(`/home/blank/voice-audition/qwentts-*.mp3`, each against its `armB-*.mp3`
mainline partner, guide in `LISTEN-engine.md`) are the instrument. **The
operator heard them on 2026-10-18 and passed every clone**, which removed the
last axis measurement could not reach; the engine was then picked on
live-output grounds ([Decision](#decision-arm-a-for-live-output-2026-10-18)).
The cosine
extractor is still worth keeping: it is engine-neutral (it scores mainline
output through the same encoder) and it separates an uncloned voice cleanly.

Two operational details. The engine takes 8 threads and exposes **no `-t`
flag** — `src/backend.h` uses `hardware_concurrency()/2` and prints the choice
at load, so pinning a cpuset on the quadlet changes that number implicitly. And
ICL needs a reference transcript that did not exist: the fleet has no ASR
endpoint and no audio input, so the crops were transcribed with
`vosk-small-en-us` inside the work dir, and two of three transcripts came back
suspect ("demon souls" for *Dark Souls*) — a reason to curate `ref_text` into
the pack manifest instead of transcribing on the host.

## Decision: arm A for live output (2026-10-18)

Every audition pair passed, so likeness stopped being the tiebreak. The
question became **which engine is better for live output** — the harness
speaking a reply, an alert read aloud, a click-to-speak — and on that criterion
RTF only has to stay under 1.0 with margin: what separates the arms is **time to
first audio** and the **spread**, because an arm whose RTF can cross 1.0 stalls
a live stream mid-sentence.

| | arm A · `qwentts.cpp` `tts-server` | arm B · mainline `llama-tts` |
|---|---|---|
| first audio byte | 86–451 ms | model load, per request |
| warm RTF | 0.55–0.67 | 0.93–0.99 |
| run spread | ±0.01 | ±0.25 |
| HTTP | serves `/v1/audio/speech` | none — façade CLI only |
| RAM | 5.7 GiB resident | per-request, freed after |

**Chosen: arm A, 1.7 B talker, resident `tts-server` in a `frank-tts-engine`
container, with `TTS_BACKEND=llama-tts-cli` kept in the repo as the rollback.**
Three consequences are configuration, not opinion: the unit **boot-warms every
voice** — the registry is memory-only, so an unwarm'd name is a 400 and the first
synthesis otherwise costs about 0.6 s more than the ones after it, as corrected
above —, it **does not pin a cpuset** casually (threads are
`hardware_concurrency()/2`, no flag), and every pack registers with the **WAV**
because the engine rejects a lone embedding.

Two follow-ups stay open on purpose, and neither blocks the deploy:

- **Talker size.** The 0.6 B arm measured 0.44 RTF at 3.4 GiB and has never
  been auditioned — every clip above is 1.7 B. It is the lever for a CPU
  co-tenancy problem, not a preemptive saving.
- **ICL vs embedding-only.** Both passed the ear test. Embedding-only is the
  shipped default: 0.57–0.63 vs 0.67 warm, and it needs no `ref_text`, which
  this fleet cannot produce honestly (the ASR crops came back suspect). Packs
  still carry `.spk` + `.rvq`, so ICL remains a per-voice opt-in for batch work
  like the podcast render, where total time matters more than first byte.

## Recommended deployment (Dankclaw)

As decided in Gitea issue #2, this stays **CPU-only on Dankclaw**: the RTX 3090
has carried frank-zealot since 2026-10-17 and both R9700s are frank-archon's,
so no card may be asked to co-host TTS. The shape is one engine plus one
façade, and **nothing here is deployed as of 2026-10-18**: `frank-tts` has
neither a unit nor a container on the host, and nothing listens on 7772 or
8880.

| Service | Image / install | Port | CPU | Notes |
|---|---|---|---|---|
| TTS engine | **arm A `qwentts.cpp` `tts-server`** on the Qwen3-TTS GGUFs (decided 2026-10-18, above); arm B mainline `llama-tts` stays in the repo as the `llama-tts-cli` rollback; `vllm/vllm-omni` + Qwen3-TTS only if a GPU is ever deliberately freed | 8880 (container-internal) | open, evidence above | arm A serves OpenAI `/v1/audio/speech` itself and boot-warms each voice; arm B has no HTTP path and runs through the façade's CLI backend |
| frank-tts | `dankclaw/voiceservice/` (stdlib container) + `scripts/frank-tts.env` + its quadlet | 7772 | yes | voice *name* → `/voices/<name>/reference.wav`, `voice`/`voice_id` on the wire, mp3 re-encode; backends `omni-http` (forwards base64 `reference_audio`) and `llama-tts-cli` (`--tts-speaker-file`, `nice 19`) |
| STT (English) | Parakeet TDT 1.1B behind a FastAPI wrapper, or whisper.cpp server | 8882 | yes | CC-BY-4.0; LibriSpeech-clean WER 1.39 (model card) |
| STT (multilingual, opt.) | faster-whisper `large-v3` int8 | 8883 | yes | MIT |
| VAD | Silero (embedded) | — | yes | MIT; model file 2 327 524 B |

7772 is frank-tts's host port — the next slot in the fleet's model-endpoint
family (7770 frank-dragoon, 7771 frank-zealot). The engine behind it keeps 8880,
but **not as a host-loopback publish**: netavark DNATs published ports only, and
a bridge container cannot reach a `127.0.0.1` bind on the host — that is the
lesson that 502'd hermes (`AGENTS.md`). `host.containers.internal`
(169.254.1.2) is the gateway, so it reaches a host service only when that
service binds a routable address; `scripts/frank-tts.env` names it as the
engine's address, which holds for a `0.0.0.0` bind and not for a loopback one.
The deployed pair therefore shares one quadlet network and the façade addresses
the engine by container name.

## Voice-creation workflow (the "from sample files" answer)

1. Record 2–3 min clean speech (24 kHz 16-bit mono WAV; ffmpeg-normalize).
2. **Zero-shot:** keep the clip as `ref_audio` — Chatterbox/Qwen3-TTS/CosyVoice/
   GLM-TTS clone from it per request (no training).
3. **Dedicated voice:** upload once to Qwen3-TTS via `/v1/audio/voices`
   (vLLM-Omni) and reference by ID; or LoRA-fine-tune VoxCPM2 (5–10 min audio);
   or the GPT-SoVITS 1-min fine-tune for the classic WebUI workflow.
4. **Kokoro fine-tune** only if the voice must live inside OMP's offline local
   backend — highest effort, Kokoro-styled output.

## Best practices

- **Prefer zero-shot cloning over fine-tuning** for a personal voice: a modern
  cloner (Chatterbox/Qwen3-TTS) on a 10 s–3 min reference beats a Kokoro
  fine-tune on quality and effort.
- **Normalize reference clips to 24 kHz 16-bit mono WAV** before use; extract
  clean voice with Demucs if the source is music/mixed.
- **Serve through vLLM-Omni** when you want a single OpenAI-compatible endpoint
  for many TTS models — it is the only server with the standard
  `/v1/audio/speech` + `/v1/audio/voices` shape across the 2026 lineup.
- **Check the license before serving publicly:** several strong cloners are
  non-commercial (Higgs v3, Breeze-TTS-2, Spark-TTS, Voxtral) or carry custom
  licenses (IndexTTS, Fish S2, XTTS v2). For a personal homelab voice, MIT /
  Apache-2.0 (Chatterbox, Qwen3-TTS, GLM-TTS, VoxCPM2) is the safe set.
- **Keep TTS off the GPUs.** The 2026-09-22 advice to "co-locate on the 3090 at
  `--gpu-memory-utilization ~0.35`" is retired: that card belongs to
  frank-zealot (2026-10-17) and the R9700s to frank-archon, and Gitea issue #2
  measures both as full. Run TTS on CPU; revisit only if the operator frees a
  card on purpose.
- **Guard the endpoint like any other service:** if vhosted, it goes behind the
  Anubis → Authentik chain; non-browser clients (OMP's `curl`) must target the
  internal address, never the public hostname.

## Citations

- OMP speech tool: https://omp.sh/docs/tools ("Generate speech", gated by
  `speechgen.enabled`; the old `https://omp.sh/docs/tools/tts` path returns 404
  as of 2026-10-18) · installed binary
  `@oh-my-pi/pi-coding-agent/dist/cli.js` (the `openai-speech` API, the
  15 000-character cap, the `voice_id` forwarding rule, local Kokoro WAV-only)
- Kokoro: https://github.com/hexgrad/Kokoro ·
  https://huggingface.co/spaces/hexgrad/Kokoro-TTS
- Kokoro fine-tune: https://github.com/semidark/kikiri-tts
- Chatterbox: https://github.com/resemble-ai/chatterbox
- Qwen3-TTS: https://github.com/QwenLM/Qwen3-TTS ·
  https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base
- CosyVoice: https://github.com/FunAudioLLM/CosyVoice
- GLM-TTS: https://github.com/zai-org/GLM-TTS ·
  https://huggingface.co/zai-org/GLM-TTS
- VoxCPM2: https://github.com/OpenBMB/VoxCPM ·
  https://huggingface.co/openbmb/VoxCPM2
- llama.cpp-omni TTS server: https://github.com/tc-mb/llama.cpp-omni ·
  https://raw.githubusercontent.com/tc-mb/llama.cpp-omni/master/tools/omni/voxcpm2/README.md
- qwentts.cpp (arm A): https://github.com/ServeurpersoCom/qwentts.cpp at commit
  `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` · weights
  https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF (byte counts against the
  Hub API, 2026-10-18)
- mainline `llama-tts`:
  https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/tts/README.md ·
  Pocket-TTS weights: https://huggingface.co/kyutai/pocket-tts (CC-BY-4.0)
- VoxCPM2 GGUFs (not published by OpenBMB):
  https://huggingface.co/DennisHuang648/VoxCPM2-GGUF
- GLM-TTS license split: model card `mit` (HF API 2026-10-18) vs code
  `LICENSE` (Apache-2.0):
  https://raw.githubusercontent.com/zai-org/GLM-TTS/main/LICENSE
- F5-TTS: https://github.com/SWivid/F5-TTS
- GPT-SoVITS: https://github.com/RVC-Boss/GPT-SoVITS
- Dots.TTS: https://huggingface.co/dots-studio/dots.tts-base
- IndexTTS: https://github.com/index-tts/index-tts
- Fish Speech: https://github.com/fishaudio/fish-speech
- MOSS-TTS-Nano: https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Nano
- XTTS: https://github.com/coqui-ai/TTS
- MegaTTS3: https://github.com/bytedance/MegaTTS3
- Higgs Audio: https://github.com/boson-ai/higgs-audio
- Breeze-TTS-2: https://huggingface.co/BreezeBlue/Breeze-TTS-2
- Spark-TTS: https://github.com/SparkAudio/Spark-TTS
- Voxtral TTS: https://huggingface.co/MistralAI/Voxtral-4B-TTS-2603
- Parler-TTS: https://github.com/huggingface/parler-tts
- VibeVoice: https://github.com/microsoft/VibeVoice
- vLLM-Omni: https://github.com/vllm-project/vllm-omni ·
  https://docs.vllm.ai/projects/vllm-omni/en/latest/
- Kokoro-FastAPI: https://github.com/remsky/Kokoro-FastAPI
- whisper.cpp: https://github.com/ggml-org/whisper.cpp ·
  faster-whisper: https://github.com/SYSTRAN/faster-whisper ·
  whisperX: https://github.com/m-bain/whisperX
- Parakeet: https://huggingface.co/nvidia/parakeet-tdt-1.1b ·
  Canary: https://huggingface.co/nvidia/canary-1b-v2
- Moonshine: https://github.com/moonshine-ai/moonshine
- Silero VAD: https://github.com/snakers4/silero-vad
- xAI custom voices:
  https://docs.x.ai/developers/model-capabilities/audio/custom-voices