Open-weight zero-shot cloning TTS — landscape report (scout TtsLandscape2026, recovered from aborted yield)

State of open-weight zero-shot voice cloning TTS (as of mid/late 2026): Breeze TTS 2 (1,215 Elo, non-commercial, 3B) tops the Artificial Analysis Speech Arena; for a homelab with a 14B GGUF on a 24GB 3090, the realistic clone candidates that fit alongside the LLM are F5-TTS, CosyVoice 3, Chatterbox, IndexTTS-2.5, Qwen3-TTS 1.7B, and VoxCPM2; the biggest server is vLLM-Omni (OpenAI /v1/audio/speech, Qwen3-TTS has a voice-upload registry). XTTS v2 still works personally via the idiap fork but its weights are permanently non-commercial. Strix Halo 395 has measured llama.cpp numbers (Vulkan > ROCm) and 128GB unified memory makes coexistence trivial, though gfx1151 needs ROCm 7.x for PyTorch-based engines.

1. Pinggy ranking (Aug 2026, AI Speech Arena Elo) [https://pinggy.io/blog/best_open_source_self_hosted_text_to_speech_models/]

#	Model	Elo	License	Cloning?	Hardware note
1	Breeze TTS 2 (BreezeBlue, 3B)	1,215	Weights: non-commercial (code Apache 2.0)	Yes, + voice design/direction	12GB min (~7.7GiB eager), 24GB fast path; RTF 0.32; <40ms TTFA (H100)
2	Fish Audio S2 Pro (4B+400M)	1,125	Non-commercial (Fish Research License; arXiv paper CC BY 4.0 is the paper's license, not weights')	Yes	SGLang; RTF 0.195, ~100ms TTFA on 1 H200; no published VRAM floor
3	Step Audio EditX (3B)	1,102	Apache 2.0 (highest-ranked commercially usable)	Zero-shot	12GB min (tested on L40S); vLLM; clips <30s; 6 languages; last commit April 2026
4	Voxtral TTS (Mistral, ~4B)	1,082	CC BY-NC 4.0	Yes, ~3s, no transcript needed	16GB VRAM; vLLM Omni 0.18.0+; RTF 0.103 c1 / 0.302 c32 on H200
5	Magpie-Multilingual 357M (NVIDIA)	1,066 (v2602)	NVIDIA Open Model License (commercial OK)	No (explicitly removed)	9 languages; 20s/inference cap; 5 fixed speakers
6	Kokoro 82M	1,060	Apache 2.0	No	<2GB VRAM or CPU ~4GB RAM (ONNX); 54 preset voices (8 langs per pinggy, 9 per offlinetts guide)
7	Maya1 (3B)	1,045	Apache 2.0	No (text-described voice design)	16GB+; claimed sub-100ms streaming; English only
8	Higgs Audio V3 (4B)	1,042	Code Apache 2.0 / weights non-commercial (Boson)	Yes, 8k-token context	SGLang-Omni ~15 req/s, RTF 0.26 (H100); no VRAM floor; 100+ languages
9	Chatterbox (0.5B)	1,020	MIT	Yes, ~5s	23 langs (v3 ~25); Turbo variant 350M, 75ms, ~6x RTF; no VRAM floor; PerTh neural watermark on all output
10	Zonos-v0.1 (1.6B)	1,000	Apache 2.0	Yes, 10-30s sample	6GB+; ~2x RTF on 4090; hybrid needs RTX 3000+; still beta

Unranked but important: Qwen3-TTS (0.6B/1.7B, Apache 2.0, 10 langs, 3s cloning, 1.7B Base ~3M downloads, CustomVoice ~2.4M; no Elo score so it isn't lineable up against Breeze/Fish), VoxCPM2 (2B, Apache 2.0, 30 langs + 9 dialects, 48kHz, ~8GB VRAM RTF 0.30 on 4090, 1.84% WER en, 0.97% CER zh-hard, llama.cpp-omni GGUF runs on CPU with no PyTorch), GLM-TTS (MIT weights, RL-trained, 0.89 CER, clone from 3-10s, repo quiet since Apr 2026). Two traps: VibeVoice 7B was MIT-licensed Aug 2025 but withdrawn 10 days later for 'inconsistent use' — community mirrors are lawful (license can't be revoked), 969 Elo persists; don't confuse with VibeVoice-ASR (speech recognition). There is no DeepSeek TTS (V4 is text-only, MIT, April 2026). Two leaderboards: Provider Voices (model+makers' voices; Breeze #1 at 1215) vs Controlled Voice (same 8 cloned voices, so it measures cloning ability; Voxtral #1 at 1010, then Breeze 1002, Fish 1000) — 'Breeze's own voices are excellent; its cloning is ordinary'.

2. Top cloning contenders — clone requirements, license, size, 2026 quality/latency

F5-TTS (SWivid, v1, 330M) [https://github.com/SWivid/F5-TTS][https://huggingface.co/api/models/SWivid/F5-TTS]: zero-shot clone from 5s reference (per offlinetts guide; repo supports --ref_audio+--ref_text, transcript optional via ASR, which costs extra GPU memory). License split (the same trap pinggy warns about): repo badge says MIT, but the HF model card for the weights says license:cc-by-nc-4.0 (non-commercial) — the weights' license is the binding one; personal use fine, commercial not. L20 (24GB, near-3090 class) measured: RTF 0.0394 at concurrency 2, 253ms avg latency (client-server Triton-TRT-LLM); PyTorch offline RTF 0.1467. 4 languages (EN/ZH/JA/KO).
Chatterbox (Resemble AI, 0.5B, v3 June 2026) [https://www.resemble.ai/learn/models/chatterbox; pinggy]: clone from ~5s; MIT; 22-25 languages; ~200ms latency (Turbo 75ms, ~6x RTF); blind win rates published incl. losses: beats ElevenLabs Turbo v2.5 65.3%, VibeVoice 7B 59.1%, ties Cartesia Sonic 3 49.8%; every output carries PerTh neural watermarking. Repo last commit July 2026; 26k stars / 1.8M downloads.
IndexTTS-2.5 (0.8B, Aug 10 2026) [https://github.com/index-tts/index-tts][https://raw.githubusercontent.com/index-tts/index-tts/main/LICENSE]: clone from a single reference clip (no explicit length documented). License = 'bilibili Model Use License': royalty-free; commercial only if you exceed 100M MAU or 1B RMB revenue in the prior period — i.e. personal/home use unrestricted, commercial also effectively fine for a homelab. 5 languages (ZH/EN/JA/ES/AR), emotion + speed + pinyin/phoneme control; 'production deployment supported via vLLM' recipe (https://recipes.vllm.ai/IndexTeam/IndexTTS-2.5) plus in-repo Triton TRT server. CV3-Eval: SS 77.1-77.9 (best speaker similarity in the table), WER 3.93-5.12.
CosyVoice 3 (Fun-CosyVoice3-0.5B-2512, Alibaba, Dec 2025) [https://github.com/QwenAudio/CosyVoice][https://huggingface.co/FunAudioLLM/Fun-CosyVoice3-0.5B-2512]: Apache 2.0 (repo + HF card); zero-shot clone — vLLM-Omni requires ref_audio+ref_text on every request, no voice presets. Reference clip ~3-10s [inference from DeepWiki's description of upstream zero-shot mode + r/LocalLLaMA 'up to 10s']; 9 languages + 18 Chinese dialects + cross-lingual clone; streaming as low as 150ms; 0.5B params (very small); eval table: 0.81% CER zh / 1.68% WER en (RL checkpoint), SS 75-78. H200-specific optimized deploy profile needs 140GiB device memory; generic profile for smaller GPUs [https://raw.githubusercontent.com/vllm-project/vllm-omni/main/docs/user_guide/examples/online_serving/text_to_speech.md].
Qwen3-TTS (0.6B/1.7B, Alibaba, Jan 22 2026) [https://github.com/QwenLM/Qwen3-TTS]: Apache 2.0 (weights+tokenizers); Base variants do 3-second rapid voice clone (ref_audio+ref_text, or x_vector_only_mode=True skips transcript, quality reduced); also VoiceDesign (text-described voice, no reference) and CustomVoice (9 premium speakers). 10 languages, trained 5M+ hours; end-to-end streaming latency as low as 97ms; benchmark: Seed-TTS WER 0.77/1.24 (12Hz 1.7B/0.6B) — claims lowest WER among the table vs MiniMax and ElevenLabs. 1.7B Base weights 4.54GB, 0.6B 2.52GB; recommends FlashAttention-2 to cut VRAM (bf16, so ~6-8GB total on a 1.7B run is a reasonable estimate [inference]). vLLM support officially day-0 but 'only offline inference' per the repo at read time; current vLLM-Omni docs list Qwen3-TTS with PCM+WebSocket streaming and a voice registry upload endpoint.
Voxtral TTS (Mistral, ~4B, Mar 23 2026): CC BY-NC 4.0 (non-commercial, fine for personal use); clone from ~3s with no transcript (infers prosody/emotion from the voice prompt); best open-weight clone on the Controlled Voice Arena (1,010); 9 languages; 16GB VRAM; RTF 0.103 H200 c1, 0.302 c32. No WER/similarity published (Mistral says they prioritized human-eval).
Breeze TTS 2 (3B): non-commercial weights (written authorization from RESONIA, INC. for commercial); cloning is one of 3 modes (clone / voice-design / voice-direction); cloning is 'ordinary' on the controlled board (1,002, three-way tie); 12GB min/24GB fast path; RTF 0.32.
Fish Audio S2 Pro: non-commercial; clone via ref_audio+ref_text on vLLM-Omni; 80+ languages; RTF 0.195 H200; no VRAM floor published.
Step Audio EditX: Apache 2.0; zero-shot clone; 3B; 12GB; 14 emotions, 30+ styles, 20+ paralinguistic tags; 30s per-inference cap; highest-ranked Apache-2.0 model (1,102).

3. OpenAI-compatible /v1/audio/speech servers today

There is no Ollama equivalent for TTS (Ollama issue #11021 open since June 2025, unimplemented) [pinggy]; what exists is a standardized API shape: 'almost everything now speaks OpenAI's POST /v1/audio/speech dialect'.

vLLM-Omni [https://raw.githubusercontent.com/vllm-project/vllm-omni/main/docs/user_guide/examples/online_serving/text_to_speech.md] — the de-facto standard: vllm serve <model> --omni exposes POST /v1/audio/speech plus /v1/audio/voices, batch and WebSocket/PCM streaming, launchable with --omni. Supported: AuK/AuK-Flash, CosyVoice3 (clone only, ref_audio+ref_text every request, no presets), Fish Drama 3, Fish S2 Pro (clone, ref params), Gepard-1.0, higgs-audio v2, GLM-TTS (clone, ref params), OmniVoice (offline only), Qwen3-TTS (clone on Base; ✓ voice presets + /v1/audio/voices upload — the one server with a true cloned-voice registry the voice param resolves to), VoxCPM2 (clone), Voxtral TTS (presets, clone gated upstream). Sample rate per model: 44.1k Fish, 48k VoxCPM2, 22.05k Gepard, 24k others. Note: plain vLLM does NOT do TTS text->audio; that's only in the vllm-omni subproject.
LocalAI [https://github.com/mudler/LocalAI; pinggy]: broadest coverage — Coqui, Piper, Qwen3-TTS, VibeVoice, Chatterbox, fish-speech; OpenAI-compatible.
Kokoro-FastAPI (remsky, 5.6k stars, Apache 2.0) [https://github.com/remsky/Kokoro-FastAPI]: Dockerized OpenAI-compatible /v1/audio/speech for Kokoro-82M; voice = preset id (54 voices, 8-9 langs); 'clone-tuning' via Inno-Kokoro voicepacks — reference-adapted voicepacks, i.e. a fine-tuning-style workaround rather than raw zero-shot cloning [inference]; CPU, NVIDIA, AMD ROCm (experimental), Apple MPS images; prebuilt multi-arch.
Speaches (speaches-ai, 3.7k stars, MIT) [https://github.com/speaches-ai/speaches]: OpenAI-compatible (audio endpoints + Realtime API), but TTS = Kokoro + Piper only, so no cloning; master branch quiet since April 2026 (pinggy).
chatterbox-tts-api (travisvn, 690 stars, AGPL-3.0 — watch for your stack) [https://github.com/travisvn/chatterbox-tts-api]: FastAPI OpenAI-compatible server for Chatterbox (22 langs, voice cloning); has Voice Library Management — upload/manage/use custom voices by name = cloned-voice registry semantics.
CosyVoice3-API (hsiang-han, community) [https://github.com/hsiang-han/CosyVoice3-API]: 'Pure OpenAI-compatible TTS API powered by CosyVoice 3, zero-shot voice cloning with 3-second reference audio'.
F5-TTS: no native /v1/audio/speech; ships a Triton/TRT-LLM client-server (gRPC/HTTP) in src/f5_tts/runtime/triton_trtllm, plus Gradio and CLI. Community wrappers not verified.
IndexTTS: production deployment via the vLLM recipe (https://recipes.vllm.ai/IndexTeam/IndexTTS-2.5, cited in repo); in-repo Triton TRT server + Gradio webui. Whether the vLLM recipe exposes /v1/audio/speech specifically is unverified [inference: it's a vLLM serving recipe, so likely the standard endpoint].
VibeVoice: weights withdrawn but MIT, so community servers are lawful: vibevoice-community/VibeVoice-API ('OpenAI-compatible API for VibeVoice', community), marhensa/vibevoice-realtime-openai-api (VibeVoice-Realtime-0.5B, Open WebUI config). The 7B clone-capable weights live on community mirrors.
XTTS v2 / OpenVoice / 'EresBiT': no canonical xtts-v2-fastapi server found — only personal/community wrappers (e.g. Hammad112's XTTS v2 FastAPI platform). OpenVoice v2 (MIT, 954 Elo) is dormant since 2025 (pinggy) but is the only model with a decoupled tone-color converter; no active OpenAI server for it was found. For 'EresBiT': a dedicated 2026 OpenAI-compatible EresBiT/EresBert server was not found in searches [searched, not found]; the nearest live things are the community eres-bert voice-conversion tools (Chinese-ecosystem, voice-clone focused) and LocalAI's Coqui backend. Treat 'EresBiT' as [inference: no canonical 2026 server; name may be community/project-specific].

Does the OpenAI voice parameter map to a cloned-voice registry? Only two flavors genuinely do: vLLM-Omni+Qwen3-TTS (presets + /v1/audio/voices upload) and chatterbox-tts-api (upload/manage voices by name). Others: voice = preset name (Kokoro-FastAPI, Speaches, VibeVoice servers, CosyVoice3's 'default'), or cloning bypasses voice entirely via extra ref_audio+ref_text fields (CosyVoice3, GLM-TTS, Fish on vLLM-Omni).

4. Hardware reality

(a) 24GB RTX 3090 + 14B GGUF

14B dense at Q4-class ≈ 14e9 × ~4.8 bits / 8 ≈ 8.4-8.7GB weights; plus ~2-4GB KV cache (context dependent) and engine overhead → ~12-16GB [inference, arithmetic]. That leaves ~8-12GB free.
Fits with margin (VRAM estimates for small models are mostly [inference] since vendors don't publish floors; F5-TTS RTF is measured on a 24GB L20, 3090-class):
- F5-TTS (330M): ~1-3GB [inference]; measured RTF 0.0394 / 253ms latency at c2 on L20.
- CosyVoice 3 (0.5B): ~1-3GB [inference]; generic (non-H200) vLLM deploy config exists.
- Chatterbox 0.5B / 0.35B Turbo: ~1-2GB [inference; no vendor floor].
- IndexTTS-2.5 (0.8B): ~2-4GB [inference; BF16/FP16 halves VRAM per repo docs].
- Qwen3-TTS 1.7B (bf16 + FlashAttention-2): ~6-8GB [inference; 4.54GB weights documented].
- VoxCPM2 (2B): ~8GB documented, RTF 0.30 on a 4090 [pinggy] — borderline with the LLM at Q4; llama.cpp-omni GGUF can also run it CPU-side to free GPU.
- Kokoro 82M: <2GB documented [pinggy] — always fits; no cloning though.
Does NOT fit: Breeze TTS 2 (7.7GiB eager / 14.4GiB fast), Voxtral (~16GB), Step EditX (12GB floor), Fish S2 Pro (no floor but 4B+400M, SGLang — expect 12-16GB+ [inference]), Higgs V3 (4B, no floor, ~12-16GB [inference]), Zonos2 (16GB+ at FP16; GGUF for CPU). Practical conclusion: on the 3090 the good cloning pairings with a 14B are F5-TTS, CosyVoice 3, Chatterbox, IndexTTS-2.5, and Qwen3-TTS 1.7B; VoxCPM2 is borderline; the top-3 Elo cloners (Breeze/Fish/Voxtral) are too big to coexist. If the LLM workload pauses, a 16GB TTS (Voxtral/Step) is also viable.

(b) AMD Strix Halo / Ryzen AI Max+ 395 (128GB unified, gfx1151)

No VRAM partitioning: 128GB unified is shared by CPU and 'GPU', so 14B GGUF + any TTS engine is a RAM-budget question, not a VRAM one. 14B Q4 (~8.5GB) is a small fraction of 128GB.
Measured LLM baseline on this exact class of box (Beelink GTR9 Pro, 128GB) [https://github.com/botAGI/strix-halo-llm-benchmarks]: Qwen3.6-35B-A3B Q4_K_M, llama.cpp server — Vulkan 15.9 ms/token (~63 tok/s) c1 vs ROCm 18.7 ms/token (~53 tok/s) c1; TTFA 210ms (c1) → 338ms (c4) → 865ms (c8) on Vulkan; 3-hour sustained c4: 100% completion, 1.6% drift. Vulkan beats ROCm in this config — supports the 'ROCm spotty' premise; the ROCm backend works but isn't the fast path here.
TTS-specific: F5-TTS README documents exactly this gotcha: 'RDNA 3.5 and RDNA 4 GPUs (gfx1151/gfx1201) require ROCm 7.x; ROCm 6.x causes HIP error: invalid device function' (issue #1236) — so PyTorch/CUDA-style engines on this APU need the ROCm 7.x torch build. Kokoro-FastAPI ships an AMD ROCm (experimental, x86_64 only) image and a CPU image; a 3090-class card note: F5-TTS's ROCm 7.x pytorch wheels (torch==2.9.1+rocm7.2) are the documented path. CosyVoice/Chatterbox/XTTS ROCm-on-gfx1151 numbers: not found in sources [searched, not found] — treat as [inference: use ROCm 7.x torch + PyTorch, or fall back to CPU/Vulkan where supported (llama.cpp-omni supports Vulkan)].
CPU-only paths (all fine on a 395 — 128GB, 32 cores of memory bandwidth): Kokoro: ~2.4x real-time on CPU, <4GB RAM (ONNX) [pinggy, third-party benchmark]; VoxCPM2: runs as GGUF on CPU/Metal/CUDA/Vulkan via llama.cpp-omni with no PyTorch at all — 'Kokoro has been the only real CPU option in this space for a year; this is the first serious model to join it' (pinggy); F5-TTS/Chatterbox/CosyVoice CPU speed: no published numbers [inference: expect slower than GPU; F5-TTS's PyTorch RTF 0.1467 is GPU-L20, CPU would be worse].

5. XTTS v2 in 2026

Status: Coqui the company shut down January 2024 without relicensing anything [pinggy, cites the shutdown post]. The original coqui-ai/TTS repo is dead (~45k stars, unmaintained); the idiap/coqui-ai-TTS fork is actively maintained — 2.3k stars, 5,314 commits on dev, released as PyPI coqui-tts, v0.27.0 adds a caching mechanism for cloned voices, v0.20.0 shipped XTTSv2 with 17 languages and <200ms streaming [https://github.com/idiap/coqui-ai-TTS]. Code is MPL-2.0.
License (the part that matters): code is MPL-2.0, but the weights are under the Coqui Public Model License, 'only non-commercial use of a machine learning model and its outputs' [coqui-ai/TTS discussions 4145/4304; pinggy; coqui/XTTS-v2 HF LICENSE]. The CPML is non-negotiable and nobody who can sell a commercial license exists anymore — Coqui is gone and the cpml URL is a dead 404 [pinggy]. So: fine for Frank's personal homelab agent; not a viable commercial building block (unlike e.g. Chatterbox's MIT or IndexTTS's near-unrestricted Bilibili license).
Capability today: 17 languages; clone from a short clip (~3s per pinggy, ~6s per HF coqui/XTTS-v2 card and localaimaster — sources disagree on the exact figure); <200ms streaming. Quality: 920 Elo, dead-last on the 2026 leaderboard (vs Step EditX 1,102 Apache-2.0 and Chatterbox 1,020 MIT, both newer and permissively licensed). It's a graveyard-tier model (pinggy: 'the bottom of the leaderboard is a graveyard' — XTTS is listed among four models to skip, alongside dead OpenVoice, StyleTTS2 and MetaVoice). Still, it's the most battle-tested for personal use and LocalAI ships a Coqui backend. Recommendation: pick it only for its 17-language breadth or existing integrations; otherwise Chatterbox (MIT, ~5s clone, 2026 quality) or F5-TTS (5s clone, RTF 0.04, non-commercial OK for personal use) dominate it in 2026.

Bottom line for a /v1/audio/speech server on a 3090 next to a 14B GGUF

Personal/cloning quality tier, fits the budget: Qwen3-TTS 1.7B via vLLM-Omni (Apache 2.0, 3s clone, native /v1/audio/speech + /v1/audio/voices registry, 97ms streaming, ~6-8GB [inference]) — the best match for an omp harness wanting a voice parameter that resolves to a saved clone; or Chatterbox v3 via travisvn/chatterbox-tts-api (MIT model, AGPL server, voice library upload, ~1-2GB [inference]). For minimum footprint: F5-TTS (5s clone, measured RTF 0.04, ~1-2GB [inference], weights CC-BY-NC = personal-use only) or CosyVoice 3 (Apache 2.0, 0.5B, 3-10s reference). The top Elo cloners (Breeze/Fish/Voxtral) are non-commercial (OK for personal) but 12-16GB+ and won't coexist with the 14B on a 24GB card.
On the Strix Halo 395: anything above coexists trivially with 128GB unified; use llama.cpp/Vulkan paths where available (VoxCPM2 GGUF via llama.cpp-omni), ROCm 7.x for PyTorch engines (F5-TTS explicitly requires it on gfx1151; ROCm 6.x crashes), CPU-only fallback = Kokoro (~2.4x RT) or VoxCPM2 GGUF. [inference on CPU speeds for the PyTorch models, no vendor numbers.]

19 KiB Raw Blame History Unescape Escape