Frank
ea2fc6440b
The cutover PR merged the runtime; this carries the surface that has to be true afterwards. - Engine page: the shape is now `--device Vulkan0` on the gfx1151 as a co-tenant beside frank-dragoon; the registry/thread/streaming laws gain a fourth — the device law — whose numbers stay in the verdict report, and the deploy record gets the cutover steps including the `glslc` configure gate. The CPU-era arm/rollback prose keeps the qualitative argument and drops the dead numbers per the purpose test. - Endpoint page: "Where the engine computes" now says iGPU + co-tenancy, and states the NPU verdict in one paragraph instead of a research tour. - Quadlet comment: it claimed supplementary groups are what grant render access. On this host the udev rule world-writes the render nodes and the container's `id` shows no groups at all, so keep-groups is belt-and-braces; the RSS note now matches the device shape instead of the CPU trial. - Verdict report: gate 2 (contention) recorded as measured after the cutover, and the build-stage prediction corrected — the package is `glslc`, not `libvulkan-dev glslang-dev`. - index.md regenerated (one description changed).
135 lines
7.5 KiB
INI
135 lines
7.5 KiB
INI
# Quadlet unit: frank-tts-engine (rootless, boot-persistent).
|
|
#
|
|
# Install path on dankclaw: ~/.config/containers/systemd/frank-tts-engine.container
|
|
# Unit name: frank-tts-engine.service (container name: frank-tts-engine)
|
|
#
|
|
# WHAT IT IS. The arm A TTS engine: qwentts.cpp (pinned commit
|
|
# 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4) built into an image by
|
|
# dankclaw/voiceservice/Containerfile.engine, running its resident `tts-server`
|
|
# with the 1.7 B base talker Q4_K_M and the Q8_0 12 Hz tokenizer loaded ONCE at
|
|
# startup. It is deliberately NOT a fleet endpoint: the endpoint consumers
|
|
# address is **frank-tts on 7772** ([[Frank TTS]]), which resolves voice names,
|
|
# re-encodes to mp3 and owns the timeout budget. Runbook: [[Frank TTS Engine]].
|
|
#
|
|
# WHY THIS ENGINE (Frank's call, 2026-10-18, criterion "whatever is better for
|
|
# live output"): server-side time-to-first-audio 94-112 ms (86-451 ms end to
|
|
# end through the façade), streaming that stays ahead of real time at warm RTF
|
|
# 0.55-0.67, and a run-to-run spread of +-0.01 — against the mainline CLI path's
|
|
# 0.93-0.99 at +-0.25, which crosses 1.0 often enough to stall a live stream.
|
|
# The A/B that decided it is "Decision: arm A for live output" in
|
|
# [[Local TTS and Voice Cloning]].
|
|
#
|
|
# IMAGE. Built locally by deploy-remote.sh (build_compose_image), like
|
|
# loki/alloy/frank-tts. `Pull=never` plus the `localhost/` prefix are required:
|
|
# this host's quadlet path cannot resolve a locally built tag from local storage
|
|
# otherwise. The build is a network operation (it clones the pinned commit) and
|
|
# is skipped when the Containerfile hash label already matches.
|
|
#
|
|
# NETWORK, AND WHY IT PUBLISHES NOTHING. The container sits on exactly one
|
|
# network: the private `frank-tts-engine.network` (10.89.5.0/24). The façade
|
|
# joins the same network and dials `http://frank-tts-engine:8880` — the
|
|
# container DNS name, on the network that owns it. There is no `PublishPort` of
|
|
# any kind, not even `127.0.0.1:8880`, and the reason is the lesson that 502'd
|
|
# hermes: netavark DNATs published ports only, so a host-loopback publish is not
|
|
# a path to a peer container — the façade could never have used it, and a bridge
|
|
# container cannot reach the host's own loopback bind either (see
|
|
# [[Container Network Namespaces]]). A publish would therefore buy exactly one
|
|
# thing: an extra reachable surface on an engine that has no authentication, on
|
|
# a box whose CPU is also carrying the text endpoints. Diagnostics do not need
|
|
# it either — `podman exec frank-tts-engine curl http://127.0.0.1:8880/health`
|
|
# speaks to the server from inside its own namespace, which is where the address
|
|
# is valid.
|
|
#
|
|
# MEMORY / THE RENDER NODE. Host RSS is ~1.7 GiB, not the size of the GGUFs:
|
|
# the Vulkan backend copies both tensors into device memory at load, so they
|
|
# stop counting in the process's RSS (measured after the 2026-10-19 cutover —
|
|
# `podman stats` / `ps -o rss`). No memory limit is set: the box has the room
|
|
# and a limit would convert a busy moment into an OOM kill mid-utterance.
|
|
#
|
|
# `AddDevice=/dev/dri` is what makes the iGPU visible; without it the Vulkan
|
|
# ICD enumerates nothing and the server dies on device lookup after it has
|
|
# already read both GGUFs. `GroupAdd=keep-groups` is belt-and-braces, not the
|
|
# thing that works here: the units podman spawns carry no supplementary groups
|
|
# at all, and this host grants render access a different way — the udev rule
|
|
# that makes the render nodes world-writable (same rule ComfyUI rides, see
|
|
# dankclaw/comfyui). Measured: `id` inside this container lists only the
|
|
# container's own group, and `tts-server` still holds renderD129.
|
|
# The node is NOT named: this host is hybrid, renderD128 is the NVIDIA card and
|
|
# renderD129 the amdgpu iGPU, so a node number in the unit is a guess that a
|
|
# re-probe can invalidate, while the directory hands the container every render
|
|
# node and the driver picks by name.
|
|
#
|
|
# /dev/kfd stays absent on purpose: that is the ROCm compute interface, owned by
|
|
# the engines that use HIP. A Vulkan endpoint has no use for it, and handing a
|
|
# container a second way onto the GPU is surface for nothing.
|
|
#
|
|
# THE VOICE MOUNT IS READ-ONLY AND IS FOR THE WARMER, NOT THE ENGINE. A voice
|
|
# becomes known to the engine only through POST /v1/audio/voices, and the
|
|
# registration is MEMORY ONLY — it does not survive a restart, which is exactly
|
|
# why the warm hook below runs on every start. That hook runs INSIDE this
|
|
# container (see ExecStartPost), so it reads the packs at /voices; `tts-server`
|
|
# itself never opens the directory, it receives the reference audio as base64 in
|
|
# the registration body. The façade mounts the same host directory read-only for
|
|
# its own name list, so one directory stays the single source of a voice name.
|
|
#
|
|
# REGISTRY LAW, THE REASON EXECSTARTPOST EXISTS. The voice registry is memory
|
|
# only: a restart forgets every voice, and re-registering does not warm one —
|
|
# only synthesizing does — so a boot without this hook leaves the façade
|
|
# answering 400 for every name. The per-voice cold penalty is real but small, and
|
|
# not the trial's 6.2-8.0 s: measured on the built container 2026-10-18 against an
|
|
# empty registry, the first synthesis of a freshly registered voice cost 1,883 ms
|
|
# against 1,226-1,283 ms for the requests after it. The warm pass runs serially
|
|
# (two short syntheses per voice, ~5 s each) and `TimeoutStartSec=900` keeps a growing library from
|
|
# tripping systemd's default 90 s start deadline. The leading `-` on
|
|
# ExecStartPost is load-bearing: with `Restart=always`, a warmer that exits
|
|
# non-zero (a pack missing its reference.wav, the engine still loading past the
|
|
# warmer's own patience) would restart the whole container and crash-loop the
|
|
# endpoint forever. A failed warm must be a journal line, not a restart.
|
|
#
|
|
# The hook is `podman exec`, not a host script, because there is no address the
|
|
# host could dial: this container publishes nothing (above). Exec-ing into it
|
|
# also means the warmer inherits this unit's EnvironmentFile, so every knob it
|
|
# has lives in scripts/frank-tts-engine.env beside the engine's own, and it sees
|
|
# the same /voices and 127.0.0.1:8880 the engine does.
|
|
#
|
|
# HEALTH. `GET /health` answers `{"status":"ok"}` from the HTTP thread, not the
|
|
# synthesis queue, so a busy engine still reads healthy and a dead one does not.
|
|
# HealthStartPeriod covers the first model load. The command is argument-only
|
|
# and needs /bin/sh — the generator wraps HealthCmd as CMD-SHELL — so the image
|
|
# installs curl (see the opposite case in dankclaw/loki, whose distroless base
|
|
# has no shell).
|
|
|
|
[Unit]
|
|
Description=frank-tts-engine (arm A qwentts.cpp TTS engine, internal :8880)
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
|
|
[Container]
|
|
ContainerName=frank-tts-engine
|
|
Image=localhost/franksdank/frank-tts-engine:latest
|
|
Pull=never
|
|
Network=frank-tts-engine.network
|
|
# No PublishPort, at all: the only reachable address this server has is the
|
|
# bridge address the façade resolves by name. Diagnostics use `podman exec`.
|
|
EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts-engine.env
|
|
Volume=/home/dank/models/Qwen3-TTS-GGUF:/models:ro
|
|
Volume=/home/dank/voices:/voices:ro
|
|
# Render access for the iGPU (see MEMORY / THE RENDER NODE above).
|
|
AddDevice=/dev/dri
|
|
GroupAdd=keep-groups
|
|
HealthCmd=curl -fsS http://127.0.0.1:8880/health
|
|
HealthInterval=30s
|
|
HealthTimeout=10s
|
|
HealthRetries=3
|
|
HealthStartPeriod=120s
|
|
|
|
[Service]
|
|
Restart=always
|
|
RestartSec=10
|
|
# Model load (tens of seconds) plus two short syntheses per registered voice.
|
|
TimeoutStartSec=900
|
|
ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh
|
|
|
|
[Install]
|
|
WantedBy=default.target
|