serverAppConfigs/serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md

type, description, tags, timestamp

type	description	tags	timestamp
reference	The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes.	dankclaw tts voice quadlet cpu	2026-10-18

Frank TTS Engine

frank-tts-engine is the arm A text-to-speech engine on Dankclaw: qwentts.cpp's resident tts-server HTTP process, holding the Qwen3-TTS 1.7 B base talker (Q4_K_M) and the 12 Hz tokenizer (Q8_0) in RAM once and synthesizing from there. It listens on container port 8880 and is deliberately not a fleet endpoint — the endpoint consumers address is frank-tts on 7772, which resolves voice names, re-encodes to mp3 and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page.

Deployed 2026-10-18. The chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified, 18 s), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit active with no drop-ins, container healthy 31 s after the build, 4.0 GiB resident on a host that kept 17 GiB available, 3 voice(s) registered and warm in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a file, not an argument (Troubleshooting), and STEP 3/13 RUN useradd is the first honest proof of the image's non-root user.

Host: Dankclaw (192.168.0.143), rootless Podman, user dank — Port: 8880, never published — Network: the private systemd-frank-tts-engine bridge (10.89.5.0/24) and no other — Deployer: frank-tts-engine — Consumers: frank-tts (7772) and, through it, every voice request in the fleet including Open Notebook's podcast renders.

Facts (verified 2026-10-18 against the pinned upstream commit, the Hub, and the CPU trial on this host)

Item	Value
Engine	tts-server from qwentts.cpp, pinned at commit 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4 (dankclaw/voiceservice/Containerfile.engine). The source is not vendored: the image clones that commit and its recorded ggml submodule, so the build needs outbound HTTPS and the cfhash label is what keeps an unchanged re-deploy offline
Build line	cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF && cmake --build build -j 8 — the exact line the 2026-10-18 trial ran on this box. GGML_BLAS=OFF is forced: Dankclaw has libopenblas.so.0 and no BLAS dev headers, so upstream's own Dockerfile and buildcpu.sh (both of which pass BLAS on) cannot be copied. No -DGGML_NATIVE is passed, and ggml defaults it on, so the binary is -march=native for the machine that built it: build this image on the host that runs it
Image	built locally as franksdank/frank-tts-engine:latest, two stages from one digest-pinned base docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac (index digest resolved 2026-10-18). Runtime stage: libgomp1 (OpenMP is on), curl (the unit's HealthCmd runs as /bin/sh -c, and the warmer speaks HTTP), uid 10000, LD_LIBRARY_PATH=/app because ggml builds shared by default and the link-time RPATH names the build tree
Talker	qwen-talker-1.7b-base-Q4_K_M.gguf, 1 219 245 248 B, sha256 ea393ebaf2167ea23ce9fc18b093822851358a950d7075cd47ab4f6ce23e887d (the Hub's LFS pointer oid)
Tokenizer	qwen-tokenizer-12hz-Q8_0.gguf, 291 150 624 B, sha256 1883beeed99348fc35e23dd225e9082f93f6f8c109330a33d935baa8acdbfd94
Source of both	https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF, fetched by scripts/download-frank-tts-engine-qwentts.sh into /home/dank/models/Qwen3-TTS-GGUF, mounted read-only at /models. It has to be the base variant: the engine refuses every voice registration unless the model reports type base, so on customvoice/voicedesign the cloned-voice product does not exist. A 0.6 B talker (628 905 056 B) exists and is not the decided artifact
Memory	5.7 GiB peak RSS with both files resident (2026-10-18 trial). No LimitMEMORY/MemoryMax: a limit would turn a busy moment into an OOM kill in the middle of an utterance
Concurrency	--max-batch 1: a serialised FIFO. Warm RTF on this box is 0.55–0.67 against real time 1.0, and that margin is the entire headroom a live stream has; two concurrent syntheses split the same cores and the second caller's first byte additionally waits for the first utterance to finish
Network	systemd-frank-tts-engine (10.89.5.0/24, dankclaw/voiceservice/quadlet/frank-tts-engine.network) and nothing else, with no PublishPort at all. frank-tts joins the same bridge and dials http://frank-tts-engine:8880
Voices	/home/dank/voices mounted read-only at /voices, and it is the warmer that reads it, not the server: tts-server receives reference audio as base64 in a registration body and keeps its registry in memory only
Health	GET /health → {"status":"ok"}, served by the HTTP thread rather than the synthesis queue, so a busy engine still reads healthy. Probed by the unit (HealthCmd, 30 s interval, 120 s start period) inside the container's namespace, because there is no published port to probe from outside
Routes	GET /health, GET /v1/models, `GET

Why this engine, and not the alternatives

The criterion Frank set on 2026-10-18 was "whatever is better for live output", and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. RTF only has to stay under 1.0 with margin; what separates the arms is first-byte latency and spread. Measured on this host the same day (CPU; the full A/B is the section "Decision: arm A for live output (2026-10-18)" in , which is the source of these numbers):

Measure	Resident tts-server (arm A, chosen)	Mainline per-request CLI path (arm B)
first audio byte	86–451 ms end to end; 94–112 ms server-side	model load, per request
warm real-time factor while streaming	0.55–0.67	0.93–0.99
run-to-run spread	±0.01	±0.25
HTTP surface	serves /v1/audio/speech itself	none — façade CLI only

The spread is the whole argument: at ±0.01 the engine finishes synthesizing 1.5–1.8× faster than the audio plays, every run; at ±0.25 some runs fall behind real time and a live listener hears the stream stall mid-sentence. The rollback path stays in the repo untouched — TTS_BACKEND=llama-tts-cli in scripts/frank-tts.env still selects the per-request CLI engine, and that file's header documents what it costs.

Three laws this service lives under

1. The registry law → the unit warms at boot. The voice registry is memory only: a restart forgets every voice, and re-registering one does not warm it — only synthesizing does. A boot that registers nothing leaves the façade answering 400 for every name, which is why frank-tts-engine.container carries

ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh
TimeoutStartSec=900

and the warm pass runs on every start including boot. What it buys is registration (mandatory) plus a measurably smaller first request — and the size of that saving is not the number the trial published. Re-measured on the deployed build, 2026-10-18, against an empty registry (register, then time the synthesis): the first request for narrator1 cost 1 883 ms, the second 1 283 ms, the third 1 226 ms — about 0.6 s of saved silence. A ~110-word request cost 9.4 s and then 9.8 s, so request time tracks the audio generated, not snapshot state. The trial's 6.2–8.0 s / first audio byte 5 530 ms / later requests 0.5 ms ladder does not reproduce here; the 0.5 ms figure is an internal state-restore cost that was never an end-to-end request time, and the honest claim is the ~0.6 s above. The - prefix is load-bearing: with Restart=always, a warmer that exits non-zero would restart the container and crash-loop the endpoint forever. The hook is podman exec rather than a host script because this container publishes nothing — there is no address the host could dial — and a consequence worth knowing: an exec inherits the unit's EnvironmentFile, so the warmer's knobs (VOICES_DIR, HEALTH_WAIT, WARM_TEXT, REQUEST_TIMEOUT) live in scripts/frank-tts-engine.env beside the engine's own. The deployer runs the same hook a second time so a failed warm becomes a deploy result rather than only a journal line, and so the per-voice numbers land on the console:

frank-tts-engine-warm: voice narrator1 cold=1402ms warm=1812ms
frank-tts-engine-warm: 3 voice(s) registered and warm

2. The thread law → nothing is ever pinned. tts-server in this build has no -t/--threads flag: src/backend.h computes the CPU thread count as std::thread::hardware_concurrency() / 2 (its comment: GEMM shares SIMD units across hyperthreads). Thread count therefore follows what the kernel reports, not what a cgroup allows — so CPUQuota=, CPUAffinity=, taskset or a slice silently changes the engine's GEMM tiling and its timings, with no log line. Nothing in this fleet pins this container. If a neighbour starves it, fix the neighbour.

3. Streaming is not buffered. At the same seed, response_format=pcm (chunked, codec chunks pushed as they decode) and response_format=wav (buffered) differ in 92.2 % of samples, −41.6 dB RMS. Nothing may compare the two paths, cache one against the other, or assume a seed pins output across response_format. frank-tts always asks for wav; the warmer asks for wav; the test seam is forbidden from pinning either.

The voice registry contract

A voice becomes known to the engine through POST /v1/audio/voices, and only these shapes are accepted:

• {"name": …, "wav_b64": …} — the reference clip travels as base64 WAV (the engine resamples to 24 kHz internally). This is what the boot warmer sends.
• {"name": …, "spk_b64": …, "rvq_b64": …} — the extracted embedding plus the reference codec codes. spk_b64 alone is rejected (src/tts-server.h:480): it is wav_b64 or both of the pair.

Whether a registered voice is embedding-only or in-context-learning is then decided by ref_text: tools/tts-server.cpp:258 selects ICL only when both ref_text and the reference codes are present. Registering {name, wav_b64} with no ref_text therefore yields the x-vector voice — the shipped default (0.57–0.63 warm RTF against ICL's 0.67) and the only one this fleet can produce honestly, because the audition crops have no trustworthy transcript to put in ref_text. The ICL door stays open for batch work like the podcast render: a pack that wants it needs its text in the pack plus one key in the payload built by dankclaw/voiceservice/tts-engine-warm.sh.

The packs themselves are host state at /home/dank/voices/<name>/reference.wav (24 kHz mono PCM), alongside the reference.spk / reference.rvq extraction and a manifest. Both halves of the voice stack read that directory — frank-tts for its name list, the warmer for its registrations — and tts-server never opens it. Two naming facts, both from the source: pack names must be lowercase, because tts_voice_name lowercases every name engine-side while the façade matches the directory name exactly (the warmer refuses anything outside [a-z0-9._-]); and GET /v1/audio/voices at this commit returns an empty JSON object rather than the voice list (the handler builds the array and never attaches it to the document), so use the façade's /v1/voices for "which voices exist" and treat the engine's listing route as broken, not empty.

Status visibility

The engine is scraped by nothing in Prometheus and has no homepage card, and both are shape decisions rather than gaps. It exposes no metrics endpoint; and since it publishes no port, there is no address the host's status checker (scripts/homepage-status.py, which dials host:port) or the dankclaw_default Prometheus container could reach. That absence is the point of the network shape, not a cost of it (see Container Network Namespaces for the netavark rule that makes a host-loopback publish useless to a peer container anyway). The pair's public signal is the endpoint in front of it: Frank TTS carries the card and is what a consumer can dial. Its up-state is visible in four places, and the runbook uses all of them:

• systemctl --user status frank-tts-engine — the unit, including whatever the warm hook logged on the way to active.
• podman healthcheck run frank-tts-engine — runs the unit's own HealthCmd against /health on demand and exits non-zero when the engine does not answer. This is also what scripts/deploy-remote.sh gates the deploy on (wait_container_healthy), precisely because there is no URL to gate on.
• podman exec frank-tts-engine curl -fsS http://127.0.0.1:8880/health — loopback is valid inside the container's namespace, which is where the address lives.
• The journal of the warm pass: journalctl --user -u frank-tts-engine, whose per-voice cold= / warm= lines are the only signal that voices are actually registered — a live HTTP listener with an empty registry still looks healthy.

The façade on 7772 is the pair's public surface and carries the probe. Note the honest limit there: a façade that answers /health does not prove the engine behind it is alive — it reports its backend and its voice list, not an upstream ping. That is exactly why the warm pass is part of the deploy and not only of the boot.

Files this service owns

Path	Role
dankclaw/voiceservice/Containerfile.engine	two-stage build of the pinned commit, proven cmake line, digest-pinned base, ships both scripts
dankclaw/voiceservice/tts-engine-start.sh	image entrypoint (/app/start.sh): env → flags, refuses to boot without weights
dankclaw/voiceservice/tts-engine-warm.sh	boot warmer (/app/warm-voices.sh): register from reference.wav (body via a temp file), speak a fixed sentence twice, report cold/warm
dankclaw/voiceservice/quadlet/frank-tts-engine.container	the unit: bridge only, no publish, env file, two read-only mounts, health probe, warm hook, Restart=always
dankclaw/voiceservice/quadlet/frank-tts-engine.network	the private 10.89.5.0/24 bridge, with the subnet registry
scripts/frank-tts-engine.env	paths and numbers only, with the thread law and the registry law in its header
scripts/download-frank-tts-engine-qwentts.sh	weight fetcher: no resume, byte count + LFS oid check, --verify-only
scripts/deploy-remote.sh → deploy_frank_tts_engine	weights gate → build → quadlet → restart → container health → warm
tests/frank_tts_engine.bats	the seam: generator output, entrypoint argv, warmer against a stub engine, planner and routing maps

Deploy record (host steps, in the order they ran)

Run 2026-10-18 as ./scripts/deploy.sh dankclaw --service frank-tts-engine, then --service frank-tts. Steps 1–6 are this service and the façade; only step 7 is still open.

1. Fetch the weights — scripts/download-frank-tts-engine-qwentts.sh (1.4 GiB, byte + sha256 verified, idempotent; --verify-only is what the deployer calls).
2. Deploy the engine — ./scripts/deploy.sh dankclaw --service frank-tts-engine (--service is the only way to name one service; deploy-remote.sh is the remote half and takes <target> <git-revision> [service]..., so never call it by hand with a bare service name). It builds the image (needs github.com once), installs the unit and the network file, gates on the container's health status, then warms.
3. Confirm the endpoint is alive — podman exec frank-tts-engine curl -fsS http://127.0.0.1:8880/health → {"status":"ok"}, and systemctl --user status frank-tts-engine showing the unit active with the warm pass behind it.
4. Confirm the voices are known — the warmer's journal lines name every pack, and once the façade is up curl http://127.0.0.1:7772/v1/audio/voices is the authoritative list (the façade has exactly three routes: /health, /v1/audio/voices, /v1/audio/speech).
5. Measure the ladder instead of trusting the trial — two identical requests for one voice back to back. The design expected 6.2-8.0 s then 0.5 ms; the deployed container gave 1,883 ms then 1,226-1,283 ms, so the hook buys ~0.6 s and stays only for the registry reason (law 1). The retired numbers must not be quoted.
6. Then the façade on 7772 (its own page), after the engine answered — that order is the whole point of step 6: publishing the façade in front of a dead engine would make every consumer fail at the first voice line.
7. Open: Open Notebook is still not installed, so nothing renders a podcast yet; the voice leg it waits on now exists.

Troubleshooting

Symptom	Meaning	Move
Unit restarts every 10 s, journal says cannot read /models/…	weights missing or the mount is empty	run the fetcher; re-deploy
Unit active, but every voice name returns 400 from the façade	the engine restarted and lost its in-memory registry, or the warm pass failed	journalctl --user -u frank-tts-engine for the warmer's lines; podman exec frank-tts-engine /app/warm-voices.sh
Every voice logs registration failed (HTTP no response), with /usr/bin/curl: Argument list too long on the line above it	the hook handed the registration body to curl as one argument: the kernel caps a single argument at 131 072 bytes (MAX_ARG_STRLEN), and a shipped reference.wav base64's to about 1.27 MB	the body goes through a temp file and --data-binary @file (2026-10-18). Any pack over ~96 KB trips this, so no small fixture can see it — the seam ships a 300 KB pack for exactly that reason
Everything is ~2× slower after a systemd tweak	a cpuset/cap pin changed hardware_concurrency() and the engine resized its thread pools	remove the pin (law 2)
502 from the façade with frank-tts upstream request failed	the façade cannot resolve or reach the engine	podman network inspect systemd-frank-tts-engine and podman inspect frank-tts --format '{{json .NetworkSettings.Networks}}' — the façade must be attached to the bridge; a 127.0.0.1:8880 or host.containers.internal:8880 upstream in scripts/frank-tts.env is the old shape and cannot work
A voice is missing from the list	its directory has no reference.wav, or the name is not lowercase	fix the pack; both are refused loudly by design
podman exec in the deploy says no such file or directory for /app/warm-voices.sh	the image predates the warmer	re-deploy: since 2026-10-18 build_compose_image hashes every file the image COPYs, not only Containerfile.engine, so editing the warmer rebuilds it — see
The warm pass logs registered, but the warm synthesis failed for EVERY voice, while a hand-run /app/warm-voices.sh succeeds	WARM_TEXT arrived with its quotes. podman's --env-file does no shell parsing — it takes the literal remainder of the line — so the env-file reader got "Warm up, one two three." where source gives Warm up, one two three.; the quotes plus a backslash then make the synthesis body invalid JSON	the hook strips one matched pair and escapes what is left; prove the value with podman exec frank-tts-engine env | grep WARM_TEXT. The seam pins both readers, because a test that only sources the env file cannot see this
A 502 for /v1/voices where the engine answers 200 on /health from the same boot	the engine restarted and its in-memory registry is empty again — /health is model-loaded, NOT voices-registered	re-run podman exec frank-tts-engine /app/warm-voices.sh; that is the same distinction the frank-tts healthcheck enforces