The arm A Qwen-TTS engine as a boot-persistent endpoint on Dankclaw
(frank-tts-engine, port 7772 inside a private bridge, no host port published),
the deploy wiring for it, and a same-day audit of the pieces that had never
actually been run.
Closes nothing yet: Gitea #2 phase 1 is not finished by this PR — the code
and its gates are here, the host is not (see Deploy below).
Commits
8d76854
the engine as a quadlet unit: Containerfile.engine, start hook, in-image boot warmer, .network bridge, env file, fetcher
8fefe6e
deploy wiring for the 7772 endpoint (planner, remote handler, homepage law) + voice-naming rules written down
c1087f2
plain wikilink to the merged voice-pack page
d135ae7
useradd, not adduser — the pinned ubuntu:24.04 base ships no adduser, so the build died at that RUN line
d336bda
the boot warmer survived only one of the two readers of its env file: podman's --env-file does not strip shell quotes, so every voice came up cold
59e021d
build_compose_image hashed only the Containerfile, so editing a COPY'd file rebuilt nothing — deploy reported success and served the old code
7178e1c
the two stale-image doors + the warm-text law in the vault; Voice Packs.md step 4 corrected against the engine's own source
Why the three fixes matter
All three were silent, which is what the audit was for:
adduser → build-time, loud, but the image had never been built, so nothing
had ever reached it. Proven in the base on Dankclaw: command -v adduser
prints nothing; useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin tts answers uid=10000(tts) gid=999(tts).
WARM_TEXT arrives with its quotes through --env-file (podman takes the
literal remainder of the line — no shell parsing), which made the warm
synthesis body invalid JSON. The unit stayed active, the voices registered,
and every voice was cold. The hook now unwraps exactly one matched pair — only
when the inner content also carries a quote or backslash, so a value that
legitimately contains quotes is never mangled — and JSON-escapes what is left.
The stale cfhash: editing app.py or tts-engine-warm.sh left the label
matching, podman build never ran, /health answered 200, and the container
ran previous code. Each case arm now names the files its Containerfile COPYs and hashes them with it. comfyui/loki/alloy have no extras, so their
hash is byte-unchanged and they do not rebuild on this commit.
Verification
./tests/run.sh → 443 passed (bats, all seams).
The new engine seam drives build_compose_image against a stub podman and
asserts observable output — did the deploy ask podman to build, or did it
skip? — through five steps: first deploy builds, unchanged tree does not, a
warm-hook edit rebuilds the engine image and not the façade, an app.py
edit rebuilds the façade. Reverted scripts/deploy-remote.sh to confirm it
goes red on the old hash, at exactly the warm-hook step.
A test reads scripts/frank-tts-engine.envboth ways — bash source and
the way podman's --env-file does — and asserts the engine receives the same
sentence in both shapes, including a value carrying its own quotes. That test
is the reason the quoting bug cannot come back; a test that only sources the
file could not see it, which is how it survived the first deploy.
Each fix commit was left green on its own (bats tests/frank_tts_engine.bats:
26 at the warm commit, 27 after the deployer commit) so the branch bisects.
Playwright e2e is not run here: this PR publishes no new host port and
touches no vhost, and the engine is not deployed. The gate runs on deploy.
First run builds the image — a ~10 min git clone --recursive of the pinned
engine source — then installs the user quadlet, starts it, and runs the warmer
through podman exec. Nothing listens on 7772 today, so this is the step that
makes open-notebook's podcast render able to run at all. No GPU involved: the
engine is CPU/OpenMP, so it co-tenants with frank-dragoon on RAM and cores,
not on the iGPU.
## What this branch is
The **arm A Qwen-TTS engine as a boot-persistent endpoint** on Dankclaw
(`frank-tts-engine`, port 7772 inside a private bridge, no host port published),
the deploy wiring for it, and a same-day audit of the pieces that had never
actually been run.
Closes nothing yet: **Gitea #2 phase 1 is not finished by this PR** — the code
and its gates are here, the host is not (see *Deploy* below).
## Commits
| | |
|---|---|
| `8d76854` | the engine as a quadlet unit: `Containerfile.engine`, start hook, in-image boot warmer, `.network` bridge, env file, fetcher |
| `8fefe6e` | deploy wiring for the 7772 endpoint (planner, remote handler, homepage law) + voice-naming rules written down |
| `c1087f2` | plain wikilink to the merged voice-pack page |
| `d135ae7` | **`useradd`, not `adduser`** — the pinned ubuntu:24.04 base ships no `adduser`, so the build died at that `RUN` line |
| `d336bda` | the boot warmer survived only one of the two readers of its env file: podman's `--env-file` does not strip shell quotes, so every voice came up cold |
| `59e021d` | `build_compose_image` hashed only the `Containerfile`, so editing a `COPY`'d file rebuilt **nothing** — deploy reported success and served the old code |
| `7178e1c` | the two stale-image doors + the warm-text law in the vault; `Voice Packs.md` step 4 corrected against the engine's own source |
## Why the three fixes matter
All three were **silent**, which is what the audit was for:
- `adduser` → build-time, loud, but the image had never been built, so nothing
had ever reached it. Proven in the base on Dankclaw: `command -v adduser`
prints nothing; `useradd --system --uid 10000 --home-dir /app --shell
/usr/sbin/nologin tts` answers `uid=10000(tts) gid=999(tts)`.
- `WARM_TEXT` arrives **with its quotes** through `--env-file` (podman takes the
literal remainder of the line — no shell parsing), which made the warm
synthesis body invalid JSON. The unit stayed `active`, the voices registered,
and every voice was cold. The hook now unwraps exactly one matched pair — only
when the inner content also carries a quote or backslash, so a value that
legitimately contains quotes is never mangled — and JSON-escapes what is left.
- The stale `cfhash`: editing `app.py` or `tts-engine-warm.sh` left the label
matching, `podman build` never ran, `/health` answered 200, and the container
ran previous code. Each case arm now names the files its `Containerfile`
`COPY`s and hashes them with it. comfyui/loki/alloy have no extras, so their
hash is byte-unchanged and they do not rebuild on this commit.
## Verification
- `./tests/run.sh` → **443 passed** (bats, all seams).
- The new engine seam drives `build_compose_image` against a **stub podman** and
asserts observable output — *did the deploy ask podman to build, or did it
skip?* — through five steps: first deploy builds, unchanged tree does not, a
warm-hook edit rebuilds the **engine** image and not the façade, an `app.py`
edit rebuilds the façade. Reverted `scripts/deploy-remote.sh` to confirm it
goes **red** on the old hash, at exactly the warm-hook step.
- A test reads `scripts/frank-tts-engine.env` **both** ways — bash `source` and
the way podman's `--env-file` does — and asserts the engine receives the same
sentence in both shapes, including a value carrying its own quotes. That test
is the reason the quoting bug cannot come back; a test that only sources the
file could not see it, which is how it survived the first deploy.
- Each fix commit was left green on its own (`bats tests/frank_tts_engine.bats`:
26 at the warm commit, 27 after the deployer commit) so the branch bisects.
- Playwright e2e is **not** run here: this PR publishes no new host port and
touches no vhost, and the engine is not deployed. The gate runs on deploy.
## Deploy (after merge, your call)
```
./scripts/deploy.sh dankclaw --service frank-tts-engine
```
First run builds the image — a ~10 min `git clone --recursive` of the pinned
engine source — then installs the user quadlet, starts it, and runs the warmer
through `podman exec`. Nothing listens on 7772 today, so this is the step that
makes open-notebook's podcast render able to run at all. No GPU involved: the
engine is CPU/OpenMP, so it co-tenants with `frank-dragoon` on RAM and cores,
not on the iGPU.
Refs: Gitea #2 (voice-over phase 1), [[Frank TTS Engine]], [[deployment]].
Repo state only: nothing deployed, no host touched, no container started.
- `frank-tts-engine` on Dankclaw: `qwentts.cpp` commit
5916bf7c79dbb4e66fc059cdc507a79a23a28bb4, resident `tts-server`, 1.7 B base
talker Q4_K_M + Q8_0 tokenizer, container port 8880. It is deliberately NOT a
fleet endpoint — `frank-tts` on 7772 is the endpoint consumers address.
- Publishes NOTHING. A host-loopback publish would have been useless to the
façade (netavark DNATs published ports only, so a bridge container cannot
reach a host loopback bind — the shape that 502'd hermes) and would have been
the one LAN-reachable surface of an unauthenticated synthesiser. Both voice
containers sit on one committed quadlet network, `frank-tts-engine.network`
pinned to 10.89.5.0/24 (next free /24 after open-notebook's 10.89.4.0/24), and
peers address the engine by container name. Diagnostics and the deployer's
health gate run inside the netns (`podman exec`, `wait_container_healthy`).
- Cold-voice law honoured in the unit: the first request for a registered voice
decodes its whole reference to snapshot codec state (6.2-8.0 s, first audio
byte 5 530 ms), later ones restore in 0.5 ms, and the registry is memory-only.
`ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh`
with `TimeoutStartSec=900`; the `-` prefix keeps a failed warm from
crash-looping the endpoint under `Restart=always`. It logs cold=/warm= per
voice, so the under-a-second second request is proven on every boot.
- Thread law recorded: `tts-server` has no `--threads`; src/backend.h uses
hardware_concurrency()/2, so a cpuset pin silently changes the thread count.
- Image builds from the pinned commit with the proven line (GGML_BLAS=OFF
forced by this host, every GPU backend off, -j 8, no -DGGML_NATIVE because
ggml defaults to -march=native for the building host). Two stages; the runtime
stage COPYs both the entrypoint and the warmer, because a container that
publishes nothing is reached with `podman exec`, not a port.
- `scripts/download-frank-tts-engine-qwentts.sh` fetches the two GGUFs with byte
and LFS-oid checks (talker 1 219 245 248 B, tokenizer 291 150 624 B); the
deployer gates on --verify-only instead of pulling 1.4 GiB mid-deploy.
- Registered in SERVICE_ORDER, DANKCLAW_SERVICES, the deploy-remote.sh dispatch,
its pre-deploy requirement, deploy.sh's regex, the planner's path map (the
whole `dankclaw/voiceservice/` tree mapped to nothing before this), and
test-changed.sh ABOVE the `dankclaw/*.container` case whose `*` crosses `/`.
- New seam tests/frank_tts_engine.bats (19): the Quadlet generator's own output
for network/publish/boot/warm shape, entrypoint argv driven from the tracked
env file, the warmer against a stub tts-server over a real socket
(registration keys, ref_text absent, late engine, hung synthesis, name
refusal), the fetcher's verifier, the pinned cmake line, and both routing
tables. Suite: 422 passed.
- Vault: [[Frank TTS Engine]] linked from the index; log.md entry for the day.
Stage 2 of the voice stack. Repo state only: nothing deployed, no host touched,
no container started.
- `frank-tts` is now registered everywhere a dankclaw service has to be:
`SERVICE_ORDER` (after `frank-tts-engine`, before the consumers), the planner's
dankclaw set, `deploy-remote.sh`'s allowlist + dispatch + a
`deploy_frank_tts` handler, and `deploy.sh`'s dispatch regex — plus the
`build_compose_image` arm naming `dankclaw/voiceservice/Containerfile`, without
which the build silently returns 0 and the unit's `Pull=never` tag never
exists.
- The unit joins TWO networks, in order: `dankclaw_default` first (default route,
where consumers and open-notebook live), then the voice bridge. `TTS_UPSTREAM`
moves from `host.containers.internal:8880` to `http://frank-tts-engine:8880` —
the actual fix for a shape that could never work, since a bridge container
cannot reach a host-loopback bind and the engine publishes nothing at all.
- `Wants=`/`After=` on the engine, never `Requires=`: the `llama-tts-cli` rollback
backend needs no upstream, so a broken engine may delay this endpoint but must
not take its unit down.
- The handler installs the engine's quadlet pair too, because a referenced
`Network=` file the generator cannot find means no façade unit is produced at
all, and `--only frank-tts` has no other way to get it. It creates
`/home/dank/voices` as the deploy user (rootless podman would auto-create a
missing bind source root-owned, which reads as "no voices"), and gates on
podman's recorded health.
- Voice naming is now written down where it can be found: a voice is a lowercase
directory name matching `^[a-z0-9][a-z0-9._-]*$`, at most 64 chars. That shape
is the intersection of two rules that disagree — the shim would accept mixed
case, the engine lowercases names — so a mixed-case pack resolves on one side
only, and the boot warmer refuses it loudly. `narrator1/2/3` are load-bearing
for open-notebook.
- Homepage card for the pair, `wip: down` and hidden, probing `127.0.0.1:7772`;
the engine gets no card because it publishes nothing and could never be probed.
- **No Prometheus job, deliberately.** No `/metrics` exists on the shim, this
Prometheus 3 refuses HTML/JSON bodies (the existing `comfyui` and `services`
probes are documented down for exactly that), and `LlmEndpointDown` is scoped
`up{job=~"frank-.*"}` — so a `/health` job would page on Signal for a healthy
endpoint. Recorded in the page instead of quietly skipped.
- Vault: new [[Frank TTS]] page linked from the index; [[Frank TTS Engine]] now
points at it for the pair's public signal; log entry for the day.
- Seam grows to 25 tests: the generator's rendering of the pair (both
attachments, first-network order, one publish, Wants-not-Requires, bounded
per-unit blocks), the env file's upstream checked as a network JOIN rather than
a string, planner separation and order, the handler's steps, the card's gating,
routing that selects both seams for either deployable.
The runtime stage of Containerfile.engine ran `adduser --system …`. The pinned
ubuntu:24.04 base ships no adduser, so the build died at that RUN line with
crun's "executable file not found in $PATH".
Verified in the base itself on Dankclaw, 2026-10-18: `command -v adduser`
prints nothing, and the exact replacement line — `useradd --system --uid 10000
--home-dir /app --shell /usr/sbin/nologin tts` — answers `uid=10000(tts)
gid=999(tts)`. Same account as before: numeric uid/gid 10000, non-login shell,
no password entry. Its one warning ("uid 10000 is greater than SYS_UID_MAX
999") is cosmetic and is what keeps the uid equal to the façade's, so the
comment says so before someone "fixes" it by dropping --system.
`WARM_TEXT` reaches the container **with its quotes**. The unit reads the env
file through podman's `--env-file`, which takes the literal remainder of the
line — no shell parsing — while bash `source` strips one layer. The hook
interpolated the raw value into the synthesis body, so with the quotes intact
the JSON was invalid and every voice logged "registered, but the warm synthesis
failed" while a hand-run hook (bash shape) succeeded. Reproduced both ways
against a real `podman run --env-file`, not by reading the man page.
The hook now strips exactly one matched pair — only when both ends are the same
quote AND the inner content carries another quote or a backslash, so a value
that legitimately contains quotes is never mangled — and then JSON-escapes what
is left, which is what makes a sentence like `He said "hi\" once"` arrive as
itself instead of corrupting the body.
The seam gains a test that reads `scripts/frank-tts-engine.env` BOTH ways —
`source` and the way `podman --env-file` does — and asserts the engine receives
the same sentence either way, in both shapes. A test that only sources the file
cannot see this bug, which is how it survived the first deploy.
build_compose_image skips `podman build` when the installed image's `cfhash`
label matches a hash taken from the checkout now — the skip is what keeps
frank-tts-engine's ten-minute recursive source clone off every re-deploy. The
hash covered the Containerfile ALONE, so any change to a file the image COPYs
was invisible: edit dankclaw/voiceservice/app.py or tts-engine-warm.sh, re-run
the deploy, and podman reports the existing image is fine. The redeployed
container runs the previous code, /health answers 200, and the e2e gate has no
way to see it. It bit the boot warmer twice this week.
Each case arm now names its own extra=(…) — the paths its Containerfile COPYs —
and the label hashes the Containerfile plus those. The two arms are kept apart
deliberately: the façade bakes app.py and tts-start.sh, the engine bakes
tts-engine-start.sh and tts-engine-warm.sh, and comfyui/loki/alloy have no extra
files, so their hash is unchanged and they do not rebuild on this commit.
Hashing rather than word-splitting because these are repo-relative paths under
a fixed checkout root.
The seam drives the real function against a stub podman and asserts observable
output — did the deploy ask podman to build, or did it skip? — in five steps:
first deploy builds, unchanged tree does not, editing the engine's COPY'd warm
hook rebuilds the ENGINE image and not the façade, editing app.py rebuilds the
façade. Reverted scripts/deploy-remote.sh to prove it red on the old hash: the
warm-hook step is exactly where it fails.
Three pages, all from faults found this session:
- `LLM/deployment.md` gains "Locally built images": which images have no
registry digest, what the `cfhash` skip is for, why it must cover every COPY'd
file (the silent shape: build skipped, /health green, old code serving), and
the base-image half of the law — the engine's `useradd`, because the pinned
ubuntu:24.04 has no `adduser`.
- `Dankclaw/Frank TTS Engine.md` troubleshooting gains the two lines an operator
actually needs: every voice cold on a fresh boot with a hand-run hook
succeeding (= `WARM_TEXT` arrived quoted, `podman exec … env | grep WARM_TEXT`
proves it), and a 502 from the façade beside a 200 `/health` from the engine
(model loaded ≠ voices registered). The stale-warmer row is re-keyed to the
new rebuild rule.
- `Dankclaw/Voice Packs.md` step 4 was wrong: it told the next agent to restart
the engine *instead of* the warm hook because the hook "only registers packs it
has not seen". It registers every pack on every run and the engine replaces a
name it already holds — `tools/tts-server.cpp:148` at the pinned commit:
"Re-registering a name replaces the previous entry." A restart is what an
engine bump needs (the registry is memory-only), never worse, only slower.
- `Dankclaw/Frank TTS.md` deploy command re-keyed to `./scripts/deploy.sh
dankclaw --service frank-tts`, the way it is actually deployed; the raw
`deploy-remote.sh` form is plumbing.
Three faults, all of the reporting-success-while-doing-nothing species: the
build that could not start (no adduser in the base), the warm pass that could
not warm (podman's --env-file keeps the quotes), and the image that would never
rebuild (cfhash over the Containerfile alone). Records the proofs, the two
behaviour seams, and what stays undone: nothing is deployed, #2 phase 1 stays
open until 7772 answers.
The vault link check reads [[Frank TTS Engine]] literally, so wrapping it made
the target "Frank\nTTS Engine" and the unresolved-link seam failed. Rewrapped,
not reworded.
dank
merged commit 747c6656d5 into main2026-10-18 18:30:22 -07:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
What this branch is
The arm A Qwen-TTS engine as a boot-persistent endpoint on Dankclaw
(
frank-tts-engine, port 7772 inside a private bridge, no host port published),the deploy wiring for it, and a same-day audit of the pieces that had never
actually been run.
Closes nothing yet: Gitea #2 phase 1 is not finished by this PR — the code
and its gates are here, the host is not (see Deploy below).
Commits
8d76854Containerfile.engine, start hook, in-image boot warmer,.networkbridge, env file, fetcher8fefe6ec1087f2d135ae7useradd, notadduser— the pinned ubuntu:24.04 base ships noadduser, so the build died at thatRUNlined336bda--env-filedoes not strip shell quotes, so every voice came up cold59e021dbuild_compose_imagehashed only theContainerfile, so editing aCOPY'd file rebuilt nothing — deploy reported success and served the old code7178e1cVoice Packs.mdstep 4 corrected against the engine's own sourceWhy the three fixes matter
All three were silent, which is what the audit was for:
adduser→ build-time, loud, but the image had never been built, so nothinghad ever reached it. Proven in the base on Dankclaw:
command -v adduserprints nothing;
useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin ttsanswersuid=10000(tts) gid=999(tts).WARM_TEXTarrives with its quotes through--env-file(podman takes theliteral remainder of the line — no shell parsing), which made the warm
synthesis body invalid JSON. The unit stayed
active, the voices registered,and every voice was cold. The hook now unwraps exactly one matched pair — only
when the inner content also carries a quote or backslash, so a value that
legitimately contains quotes is never mangled — and JSON-escapes what is left.
cfhash: editingapp.pyortts-engine-warm.shleft the labelmatching,
podman buildnever ran,/healthanswered 200, and the containerran previous code. Each case arm now names the files its
ContainerfileCOPYs and hashes them with it. comfyui/loki/alloy have no extras, so theirhash is byte-unchanged and they do not rebuild on this commit.
Verification
./tests/run.sh→ 443 passed (bats, all seams).build_compose_imageagainst a stub podman andasserts observable output — did the deploy ask podman to build, or did it
skip? — through five steps: first deploy builds, unchanged tree does not, a
warm-hook edit rebuilds the engine image and not the façade, an
app.pyedit rebuilds the façade. Reverted
scripts/deploy-remote.shto confirm itgoes red on the old hash, at exactly the warm-hook step.
scripts/frank-tts-engine.envboth ways — bashsourceandthe way podman's
--env-filedoes — and asserts the engine receives the samesentence in both shapes, including a value carrying its own quotes. That test
is the reason the quoting bug cannot come back; a test that only sources the
file could not see it, which is how it survived the first deploy.
bats tests/frank_tts_engine.bats:26 at the warm commit, 27 after the deployer commit) so the branch bisects.
touches no vhost, and the engine is not deployed. The gate runs on deploy.
Deploy (after merge, your call)
First run builds the image — a ~10 min
git clone --recursiveof the pinnedengine source — then installs the user quadlet, starts it, and runs the warmer
through
podman exec. Nothing listens on 7772 today, so this is the step thatmakes open-notebook's podcast render able to run at all. No GPU involved: the
engine is CPU/OpenMP, so it co-tenants with
frank-dragoonon RAM and cores,not on the iGPU.
Refs: Gitea #2 (voice-over phase 1), Frank TTS Engine, deployment.
The runtime stage of Containerfile.engine ran `adduser --system …`. The pinned ubuntu:24.04 base ships no adduser, so the build died at that RUN line with crun's "executable file not found in $PATH". Verified in the base itself on Dankclaw, 2026-10-18: `command -v adduser` prints nothing, and the exact replacement line — `useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin tts` — answers `uid=10000(tts) gid=999(tts)`. Same account as before: numeric uid/gid 10000, non-login shell, no password entry. Its one warning ("uid 10000 is greater than SYS_UID_MAX 999") is cosmetic and is what keeps the uid equal to the façade's, so the comment says so before someone "fixes" it by dropping --system.