feat(frank-tts): cloned-voice speech endpoint — OpenAI-shaped facade, voice packs by name

Phase 1 of the voice-over goal (Gitea issue #2). One stdlib-only HTTP facade in
front of whichever TTS engine wins the engine decision: POST /v1/audio/speech
with voice=<name> answers audio in THAT voice, where a name is real exactly when
/voices/<name>/reference.wav exists — an unknown name is a 400 listing the
voices that are, never a silent fallback (a wrong-voice endpoint stays wrong for
weeks). voice_id works as the alias the omp harness sends; response_format=mp3
is really mp3 (ffmpeg re-encode); oversized input is refused with 414 instead of
half-narrated; /health reports the backend and the resolved voice list.

Two backends behind the one surface, chosen by TTS_BACKEND: omni-http forwards to
an OpenAI-shaped upstream carrying the voice as base64 reference_audio, and
llama-tts-cli runs a local binary per request with --tts-speaker-file at nice 19
so narration never starves the text endpoints. Both are driven in the seam
through stubs — no GPU, no containers.

Named frank-tts, a first-class fleet model endpoint: port 7772 in the 7770/7771
family, alias naming per the StarCraft law, rootless boot-persistent self-healing
quadlet with a read-only /home/dank/voices mount. Voice-pack law recorded in the
env header: reference.wav PLUS the engine's own speaker artifact (required, so a
voice is reusable in other runtimes), plus manifest.json provenance with offsets
always past the first 10 s of the source file.

Not deployed: the engine decision, the pack builder, deployer/homepage/Prometheus
registration and the omp models.yml runner row are phase 2.

dankclaw/voiceservice/Containerfile

@@ -0,0 +1,43 @@
+# frank-tts — the cloned-voice speech façade (dankclaw/voiceservice/app.py).
+#
+# Standard library only, so there is nothing to pip-install and nothing to
+# compile: the image is a python runtime plus ffmpeg (the one binary the app
+# shells out to, for response_format=mp3) plus one COPY of app.py. Rebuild to
+# ship a new app.py.
+#
+# Build on dankclaw (deploy-remote.sh build_compose_image frank-tts does this,
+# including --format docker, which the fleet's other local builds also pass):
+#   podman build --format docker -t franksdank/frank-tts:latest \
+#       -f dankclaw/voiceservice/Containerfile .
+#
+# BASE PINNED BY DIGEST, like every other image this fleet runs from a
+# registry: a tag moves under a rebuild and a voice endpoint that changes
+# python or ffmpeg mid-week is a debug session nobody asked for. The digest is
+# the multi-arch INDEX of python:3.12-alpine as published 2026-10-18 (amd64 +
+# arm64 on linux); Dankclaw resolves the amd64 entry.
+#
+# WHY 3.12: the fleet's ceiling for this app is python <= 3.12 (single file,
+# no build step); the alpine flavour also keeps the image in the tens of MB
+# and ships busybox `wget`, which is what the quadlet unit's HealthCmd runs —
+# so no HEALTHCHECK belongs here (the probe lives in the unit, as in
+# alertmanager/prometheus; only distroless bases need it baked in).
+FROM docker.io/library/python:3.12-alpine@sha256:1b668429b3511ab407d8e00648891631b0b1a4d7e15e3ca70f38ab5b91ad4ab4
+
+# ffmpeg: mp3 output. The app always asks the upstream for wav and re-encodes
+# on the way out, so ffmpeg is the only external program in the picture.
+RUN apk add --no-cache ffmpeg
+
+# Unbuffered stdout: the unit's journal must show the "listening on" line (and
+# any backend error) the moment it happens, not when the buffer fills.
+ENV PYTHONUNBUFFERED=1
+
+COPY dankclaw/voiceservice/app.py /app/app.py
+
+# Non-root. The voice packs are bind-mounted read-only, so this user only ever
+# reads them — they must be world-readable (the pack builder writes 644/755).
+# The one thing it writes is its per-request temp dir under /tmp.
+RUN adduser -D -u 10000 -h /app -s /sbin/nologin shim
+USER shim
+
+EXPOSE 7772
+CMD ["python3", "/app/app.py"]

dankclaw/voiceservice/app.py

@@ -0,0 +1,413 @@
+#!/usr/bin/env python3
+"""frank-tts — the fleet's OpenAI-compatible speech façade (component: the shim).
+
+One local endpoint that answers ``voice=<name>`` with audio in a cloned voice.
+Consumers (the omp harness, OpenWebUI, open-notebook) speak the OpenAI
+``POST /v1/audio/speech`` shape and never learn a file path: a voice is a
+directory under ``$VOICES_DIR`` and ``reference.wav`` inside it is what makes
+the name real. A name that has no reference is not a voice — the shim answers
+400 with the list that is, instead of silently narrating in a fallback voice
+(silent fallback is how a wrong-voice endpoint stays wrong for weeks).
+
+Two backends sit behind the one surface, chosen by ``TTS_BACKEND``:
+
+``omni-http``
+    Forward to an OpenAI-shaped upstream TTS server (``$TTS_UPSTREAM``) and
+    carry the cloned voice with the request as ``reference_audio`` (base64 of
+    ``reference.wav``). The upstream does the inference; this process only
+    resolves the voice and, when asked, re-encodes to mp3 with ``ffmpeg``.
+
+``llama-tts-cli``
+    Run a local binary per request (``$LLAMA_TTS_BIN``), hand it the
+    reference with ``--tts-speaker-file``, read the WAV it writes. ``nice -n
+    19`` because this box also serves text and a voice request must never
+    starve it.
+
+Everything comes from the environment, read at REQUEST time (no
+import-time-only globals): the same module can be imported with an injected
+``Config`` or run with ``python3 app.py`` and the unit's env file.
+
+Runtime law: single process, standard library only, no build step. Audio
+bytes never reach the log — request lines and error text only.
+"""
+
+from __future__ import annotations
+
+import argparse
+import base64
+import json
+import os
+import re
+import subprocess
+import sys
+import tempfile
+import urllib.error
+import urllib.parse
+import urllib.request
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+
+WAV_CONTENT_TYPE = "audio/wav"
+MP3_CONTENT_TYPE = "audio/mpeg"
+
+# A voice name is a plain directory name. Slashes and `..` are refused here so
+# a client-supplied name can never escape $VOICES_DIR.
+VOICE_NAME_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}")
+
+
+class ShimError(Exception):
+    """A failure with the HTTP status the client should see."""
+
+    def __init__(self, status, message):
+        super().__init__(message)
+        self.status = status
+        self.message = message
+
+
+class Config:
+    """Every knob of the shim. Constructed from the environment per request by
+    default, or injected into ``create_server`` for tests and embedding."""
+
+    def __init__(self, voices_dir, backend, upstream, upstream_model, llama_bin,
+                 llama_model, llama_mmproj, llama_lang, llama_threads, port,
+                 max_chars, timeout):
+        self.voices_dir = Path(voices_dir)
+        self.backend = backend
+        self.upstream = upstream
+        self.upstream_model = upstream_model
+        self.llama_bin = llama_bin
+        self.llama_model = llama_model
+        self.llama_mmproj = llama_mmproj
+        self.llama_lang = llama_lang
+        self.llama_threads = llama_threads
+        self.port = port
+        self.max_chars = max_chars
+        self.timeout = timeout
+
+    @classmethod
+    def from_env(cls, environ=None):
+        env = os.environ if environ is None else environ
+        return cls(
+            voices_dir=env.get("VOICES_DIR", "/voices"),
+            backend=(env.get("TTS_BACKEND") or "omni-http").strip().lower(),
+            upstream=(env.get("TTS_UPSTREAM") or "http://host.containers.internal:8880").rstrip("/"),
+            upstream_model=env.get("TTS_UPSTREAM_MODEL") or "voxcpm2",
+            llama_bin=env.get("LLAMA_TTS_BIN") or "",
+            llama_model=env.get("LLAMA_TTS_MODEL") or "",
+            llama_mmproj=env.get("LLAMA_TTS_MMPROJ") or "",
+            llama_lang=env.get("LLAMA_TTS_LANG") or "english",
+            llama_threads=_int(env, "LLAMA_TTS_THREADS", 8),
+            port=_int(env, "SHIM_PORT", 7772),
+            max_chars=_int(env, "SHIM_MAX_CHARS", 15000),
+            timeout=float(_int(env, "SHIM_TIMEOUT", 600)),
+        )
+
+
+def _int(env, name, default):
+    try:
+        return int(str(env.get(name, "")).strip())
+    except (TypeError, ValueError):
+        return default
+
+
+def list_voices(voices_dir):
+    """Voice names in ``voices_dir``: directories that hold a ``reference.wav``.
+
+    A missing or unreadable directory is an empty fleet, not an error — the
+    health endpoint must still answer so the unit reports its state.
+    """
+    try:
+        entries = sorted(p.name for p in voices_dir.iterdir()
+                         if p.is_dir() and (p / "reference.wav").is_file())
+    except OSError:
+        return []
+    return entries
+
+
+def resolve_voice(voices_dir, voice):
+    """``$VOICES_DIR/<voice>/reference.wav`` for a valid name, else ``None``."""
+    if not isinstance(voice, str) or not VOICE_NAME_RE.fullmatch(voice):
+        return None
+    reference = voices_dir / voice / "reference.wav"
+    return reference if reference.is_file() else None
+
+
+def to_mp3(wav_bytes, timeout):
+    """Re-encode WAV bytes to mp3 with the image's ``ffmpeg`` (pipes, no temp
+    file: the audio never lands on disk to be left behind)."""
+    argv = ["ffmpeg", "-hide_banner", "-loglevel", "error", "-i", "pipe:0",
+            "-f", "mp3", "pipe:1"]
+    try:
+        proc = subprocess.run(argv, input=wav_bytes, capture_output=True,
+                              timeout=timeout)
+    except FileNotFoundError:
+        raise ShimError(500, "response_format=mp3 needs the ffmpeg binary, which is not on PATH")
+    except subprocess.TimeoutExpired:
+        raise ShimError(502, "ffmpeg mp3 conversion timed out after %gs" % timeout)
+    if proc.returncode != 0 or not proc.stdout:
+        raise ShimError(502, "ffmpeg mp3 conversion failed (exit %s): %s"
+                        % (proc.returncode, _tail(proc.stderr)))
+    return proc.stdout
+
+
+def _tail(raw):
+    """Last line of a subprocess' stderr, for an error message. Text only —
+    audio bytes are never echoed, and never logged."""
+    text = (raw or b"").decode("utf-8", "replace").strip()
+    return text.splitlines()[-1][:300] if text else "no output"
+
+
+def _error(message, voices=None):
+    body = {"error": {"message": message, "type": "invalid_request_error"}}
+    if voices is not None:
+        body["error"]["available_voices"] = voices
+    return body
+
+
+class SpeechHandler(BaseHTTPRequestHandler):
+    protocol_version = "HTTP/1.1"
+    server_version = "frank-tts"
+    sys_version = ""
+
+    # -- plumbing ----------------------------------------------------------
+    def config(self):
+        return getattr(self.server, "config", None) or Config.from_env()
+
+    def log_message(self, fmt, *args):
+        # One prefix for the journal. Request lines and outcomes only: a
+        # synthesized text, a base64 reference voice and the audio bytes
+        # itself never reach stderr, in success or in failure.
+        sys.stderr.write("frank-tts: %s - %s\n"
+                         % (self.address_string(), fmt % args))
+
+    def log_error(self, fmt, *args):
+        # The base class routes refused requests through a separate sink; keep
+        # them on the same line format so the journal reads uniformly.
+        self.log_message(fmt, *args)
+
+    def _json(self, status, obj, head=False):
+        self._bytes(status, json.dumps(obj).encode("utf-8"), "application/json", head)
+
+    def _bytes(self, status, data, content_type, head=False):
+        self.send_response(status)
+        self.send_header("Content-Type", content_type)
+        self.send_header("Content-Length", str(len(data)))
+        self.end_headers()
+        if not head:
+            self.wfile.write(data)
+
+    def _drain(self):
+        """Read and discard a body this route will not answer. HTTP/1.1
+        keep-alive makes an unread body the NEXT request line, so a client
+        that reuses the connection would see a parse error it never sent."""
+        if getattr(self, "_body_read", False):
+            return  # already consumed; a second read would block the response
+        try:
+            length = int(self.headers.get("Content-Length") or 0)
+        except ValueError:
+            return
+        if length > 0:
+            self.rfile.read(length)
+
+    def _body(self):
+        """Parse the JSON request body. Never returned to a log or an error."""
+        try:
+            length = int(self.headers.get("Content-Length") or 0)
+        except ValueError:
+            raise ValueError("Content-Length is not a number")
+        raw = self.rfile.read(length) if length > 0 else b""
+        self._body_read = True
+        if not raw:
+            raise ValueError("expected a JSON body")
+        parsed = json.loads(raw.decode("utf-8"))
+        if not isinstance(parsed, dict):
+            raise ValueError("expected a JSON object")
+        return parsed
+
+    # -- routes ------------------------------------------------------------
+    def do_GET(self, head=False):
+        # HEAD shares the route table: `wget --spider` (the fleet's probe
+        # shape) asks for HEAD, and a 501 there would read as a dead service.
+        path = urllib.parse.urlparse(self.path).path
+        cfg = self.config()
+        if path == "/health":
+            self._json(200, {"status": "ok", "backend": cfg.backend,
+                             "voices": list_voices(cfg.voices_dir)}, head)
+        elif path == "/v1/audio/voices":
+            self._json(200, list_voices(cfg.voices_dir), head)
+        else:
+            self._json(404, _error("no such endpoint: %s" % path), head)
+
+    def do_HEAD(self):
+        self.do_GET(head=True)
+
+    def do_POST(self):
+        path = urllib.parse.urlparse(self.path).path
+        if path != "/v1/audio/speech":
+            self._drain()
+            self._json(404, _error("no such endpoint: %s" % path))
+            return
+        cfg = self.config()
+        try:
+            body = self._body()
+        except (ValueError, UnicodeDecodeError) as exc:
+            self._drain()
+            self._json(400, _error("malformed request body: %s" % exc))
+            return
+
+        text = body.get("input")
+        if not isinstance(text, str) or not text.strip():
+            self._json(400, _error("`input` must be a non-empty string"))
+            return
+        if len(text) > cfg.max_chars:
+            self._json(414, _error("input is %d characters, over the %d limit "
+                                   "(SHIM_MAX_CHARS); split it or raise the limit"
+                                   % (len(text), cfg.max_chars)))
+            return
+
+        fmt = str(body.get("response_format") or "wav").strip().lower()
+        if fmt not in ("wav", "mp3"):
+            self._json(400, _error("unsupported response_format %r; this "
+                                   "endpoint serves wav and mp3" % fmt))
+            return
+
+        # `voice_id` is the alias the omp harness forwards for `voice`.
+        voice = body.get("voice") or body.get("voice_id")
+        reference = resolve_voice(cfg.voices_dir, voice)
+        if reference is None:
+            available = list_voices(cfg.voices_dir)
+            listed = ", ".join(available) if available else "(none: no voice pack with a reference.wav in %s)" % cfg.voices_dir
+            self._json(400, _error("unknown voice %r; available voices: %s"
+                                   % (voice, listed), available))
+            return
+
+        model = str(body.get("model") or cfg.upstream_model)
+        try:
+            if cfg.backend == "omni-http":
+                audio = self._speech_http(cfg, model, text, str(voice), reference)
+            elif cfg.backend == "llama-tts-cli":
+                audio = self._speech_cli(cfg, text, reference)
+            else:
+                raise ShimError(500, "unknown TTS_BACKEND %r (expected omni-http or llama-tts-cli)" % cfg.backend)
+            if fmt == "mp3":
+                audio = to_mp3(audio, cfg.timeout)
+        except ShimError as exc:
+            self._json(exc.status, _error(exc.message))
+            return
+
+        content_type = MP3_CONTENT_TYPE if fmt == "mp3" else WAV_CONTENT_TYPE
+        self._bytes(200, audio, content_type)
+        self.log_message("speech voice=%s format=%s chars=%d -> %d bytes %s",
+                         voice, fmt, len(text), len(audio), content_type)
+
+    # -- backends ----------------------------------------------------------
+    def _speech_http(self, cfg, model, text, voice, reference):
+        """Forward to the upstream with the cloned voice riding along as
+        base64 ``reference_audio``. The upstream is asked for wav always; mp3
+        is our own re-encode on the way out."""
+        try:
+            payload = {
+                "model": model,
+                "input": text,
+                "voice": voice,
+                "response_format": "wav",
+                "reference_audio": base64.b64encode(reference.read_bytes()).decode("ascii"),
+            }
+        except OSError as exc:
+            raise ShimError(500, "cannot read %s: %s" % (reference, exc))
+
+        url = cfg.upstream + "/v1/audio/speech"
+        request = urllib.request.Request(
+            url, data=json.dumps(payload).encode("utf-8"),
+            headers={"Content-Type": "application/json"}, method="POST")
+        try:
+            with urllib.request.urlopen(request, timeout=cfg.timeout) as response:
+                audio = response.read()
+        except urllib.error.HTTPError as exc:
+            detail = exc.read(512).decode("utf-8", "replace").strip()
+            raise ShimError(502, "upstream %s returned HTTP %d: %s"
+                            % (url, exc.code, detail or "no detail"))
+        except urllib.error.URLError as exc:
+            raise ShimError(502, "upstream %s unreachable: %s" % (url, exc.reason))
+        except (TimeoutError, OSError) as exc:
+            raise ShimError(502, "upstream %s failed: %s" % (url, exc))
+        if not audio:
+            raise ShimError(502, "upstream %s returned no audio" % url)
+        return audio
+
+    def _speech_cli(self, cfg, text, reference):
+        """Run the local TTS binary once, in a temp dir that is always removed
+        — also on failure, or a crash-looping binary fills /tmp."""
+        if not cfg.llama_bin or not cfg.llama_model:
+            raise ShimError(500, "TTS_BACKEND=llama-tts-cli needs LLAMA_TTS_BIN and LLAMA_TTS_MODEL")
+        work = tempfile.TemporaryDirectory(prefix="frank-tts-")
+        try:
+            out_path = Path(work.name) / "speech.wav"
+            argv = ["nice", "-n", "19", "env", "OMP_NUM_THREADS=%d" % cfg.llama_threads,
+                    cfg.llama_bin, "-m", cfg.llama_model]
+            if cfg.llama_mmproj:
+                argv += ["-mm", cfg.llama_mmproj]
+            argv += ["--tts-lang", cfg.llama_lang,
+                     "--tts-speaker-file", str(reference),
+                     "-p", text,
+                     "--output", str(out_path)]
+            try:
+                proc = subprocess.run(argv, cwd=work.name, capture_output=True,
+                                      timeout=cfg.timeout)
+            except FileNotFoundError as exc:
+                raise ShimError(500, "cannot run the TTS command: %s" % exc)
+            except subprocess.TimeoutExpired:
+                raise ShimError(502, "TTS binary timed out after %gs" % cfg.timeout)
+            if proc.returncode != 0:
+                raise ShimError(502, "TTS binary failed (exit %s): %s"
+                                % (proc.returncode, _tail(proc.stderr)))
+            if not out_path.is_file():
+                raise ShimError(502, "TTS binary exited 0 but wrote no %s" % out_path.name)
+            return out_path.read_bytes()
+        finally:
+            work.cleanup()
+
+
+class SpeechServer(ThreadingHTTPServer):
+    """Threads so a health probe answers while a long synthesis runs; daemon
+    threads so SIGTERM is never held hostage by a stuck request."""
+
+    daemon_threads = True
+    allow_reuse_address = True
+    config = None
+
+
+def create_server(config=None, host="0.0.0.0", port=None):
+    """Bind the shim. ``config``/``port`` are the injectable seams: tests pass
+    a Config with a temp VOICES_DIR and ``port=0`` for an ephemeral port."""
+    cfg = Config.from_env() if config is None else config
+    server = SpeechServer((host, cfg.port if port is None else port), SpeechHandler)
+    server.config = cfg
+    return server
+
+
+def main(argv=None):
+    parser = argparse.ArgumentParser(
+        prog="frank-tts", description="OpenAI-compatible speech endpoint for cloned voice packs")
+    parser.add_argument("--host", default="0.0.0.0",
+                        help="bind address (default: all interfaces)")
+    parser.add_argument("--port", type=int, default=None,
+                        help="bind port (default: $SHIM_PORT, 0 = ephemeral)")
+    args = parser.parse_args(argv)
+
+    server = create_server(host=args.host, port=args.port)
+    cfg = server.config
+    bound = server.server_address[1]
+    print("frank-tts: backend=%s voices_dir=%s listening on http://%s:%d"
+          % (cfg.backend, cfg.voices_dir, args.host, bound), flush=True)
+    try:
+        server.serve_forever()
+    except KeyboardInterrupt:
+        pass
+    finally:
+        server.server_close()
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

dankclaw/voiceservice/quadlet/frank-tts.container

@@ -0,0 +1,71 @@
+# Quadlet unit: frank-tts (rootless, boot-persistent).
+#
+# Install path on dankclaw: ~/.config/containers/systemd/frank-tts.container
+# Unit name: frank-tts.service (container name: frank-tts)
+#
+# WHAT IT IS. The fleet's one OpenAI-compatible speech endpoint: consumers
+# (omp, OpenWebUI, open-notebook) POST /v1/audio/speech with voice=<name> and
+# get audio in that cloned voice. A voice is a directory under
+# /home/dank/voices holding reference.wav; the shim refuses any name without
+# one (400 + the list that does exist) rather than narrating in a fallback
+# voice nobody chose. The engine behind it is a separate decision —
+# TTS_BACKEND in scripts/frank-tts.env picks between forwarding to an
+# OpenAI-shaped upstream on host :8880 and running a local CLI binary per
+# request.
+#
+# IMAGE. Built locally from dankclaw/voiceservice/Containerfile by
+# deploy-remote.sh (build_compose_image), like loki/alloy: python 3.12-alpine
+# pinned by digest + ffmpeg, app.py copied in, running as uid 10000.
+# Pull=never and the localhost/ prefix are required — this host's quadlet
+# path cannot resolve a locally built tag from local storage otherwise.
+#
+# NETWORK. Bridge (dankclaw_default), NOT host: nothing here needs a shared
+# loopback namespace, and the comfyui/loki/authentik units are bridge too.
+# Consequences both ways: consumers reach it on the published 7772, and the
+# shim reaches the host-side TTS engine through host.containers.internal
+# (never the LAN IP — a bridge netns cannot route to the host's own address
+# for every bind shape, and the fleet's law is the gateway name). NOTE for
+# whoever starts the :8880 engine: host.containers.internal reaches the host
+# network namespace, so the engine must bind an address that namespace
+# accepts; a 127.0.0.1-only bind is not it (the 2026-09-17 signal-relay hop
+# needed the host LAN bind for the same reason).
+#
+# VOICES. /home/dank/voices is host state, never repo state, and is mounted
+# READ-ONLY: this endpoint has no business writing voice packs. Because the
+# container runs as uid 10000, the packs must be world-readable (644 files,
+# 755 directories), which is how the pack builder writes them.
+#
+# NO GPU. The shim is a façade: audio conversion via ffmpeg on the CPU and,
+# for the CLI backend, whatever CPU threads LLAMA_TTS_THREADS allows, at nice
+# 19. No /dev/kfd, no /dev/dri, no keep-groups — unlike comfyui, whose unit
+# carries them deliberately.
+[Unit]
+Description=frank-tts (cloned-voice OpenAI-compatible TTS facade)
+After=network-online.target
+Wants=network-online.target
+
+[Container]
+ContainerName=frank-tts
+Image=localhost/franksdank/frank-tts:latest
+Pull=never
+Network=dankclaw_default
+PublishPort=7772:7772
+EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts.env
+Volume=/home/dank/voices:/voices:ro
+# The probe is a probe, not a voice test: /health answers 200 with the backend
+# and the resolved voice list, so a mounted-but-empty /voices still reads
+# healthy while a dead app does not. busybox wget ships in the alpine base;
+# keep the command argument-only — the podman 5.x generator splits HealthCmd
+# on spaces (see the comfyui unit for the quoting trap).
+HealthCmd=wget --spider -q http://127.0.0.1:7772/health
+HealthInterval=30s
+HealthTimeout=10s
+HealthRetries=3
+HealthStartPeriod=30s
+
+[Service]
+Restart=always
+RestartSec=10
+
+[Install]
+WantedBy=default.target