--- type: reference description: "How a voice pack is built: the files every pack must carry, the source-material gate, adding a voice, engine-bump drift." tags: - dankclaw - frank-tts - voices - tts timestamp: 2026-10-18 --- # Voice Packs A **voice pack** is the unit of identity for **frank-tts**: one directory under `/home/dank/voices`, named by the voice, carrying the reference audio that makes the name real plus the pinned engine's own speaker artifacts and a provenance record. `voice=narrator1` on `POST /v1/audio/speech` resolves to `/home/dank/voices/narrator1/reference.wav` and to nothing else — a name without that file is a 400 that lists the names that do exist, because a silent fallback voice is how a wrong-voice endpoint stays wrong for weeks. **Host:** Dankclaw (192.168.0.143) — **Pack root:** `/home/dank/voices`, host state that stays out of the repo, mounted read-only at `/voices` by the façade unit — **Builder:** `scripts/build-voice-pack.sh` with tunables in `scripts/voice-packs.env` — **Seam:** `tests/voice_pack_unit.bats` — **Engine:** arm A `qwentts.cpp`, decided in . Packs are **world-readable on purpose** (`drwxr-xr-x`, owner `dank`): the façade container runs as uid 10000 and reads the mount from outside its own uid map. ## The required files | File | What it is | Who reads it | |------|------------|--------------| | `reference.wav` | the crop: 24 kHz mono `pcm_s16le`, **18–25 s** of clean single-speaker audio | the façade (it is the existence gate for the name) and the engine's boot warmer, which base64s it and registers the voice with no transcript | | `reference.spk` | the engine's x-vector: **8,192 B** = `enc_dim` 2048 × f32 | the engine, for the ICL/`--ref-spk` path | | `reference.rvq` | the engine's RVQ code stream: **5,456–6,864 B**, 16 codebooks × T frames packed at 11 bits = 22 B per 12.5 Hz frame | the engine, for the ICL/`--ref-rvq` path | | `manifest.json` | provenance: source path and sha256, crop start/end offsets, sample rate, duration, the tool and engine commit that made the artifacts, and the `voice` name | humans, and the builder itself on its second gate assertion | `qwen-codec.log` rides along with the extractor's own record of the run; the façade ignores anything it does not name and never deletes anything. Division of enforcement, which is deliberate and not a gap: the **façade** gates on `reference.wav` because that is the only engine-agnostic term (see the voice-pack law in `scripts/frank-tts.env`); the **builder** is what refuses to emit a pack without its engine artifacts; the **engine** registers a voice from the WAV alone because its own `POST /v1/audio/voices` refuses a lone embedding — it wants `wav_b64` or **both** `spk_b64` and `rvq_b64`. So the artifacts are in the pack to make ICL a per-voice opt-in, not to boot it. ## How a pack is built ```bash # everything except the extractor: gate, crop, probe, plan, the manifest it would write scripts/build-voice-pack.sh --dry-run narrator1 /home/dank/voices # the real thing, on the host that holds the engine build scripts/build-voice-pack.sh narrator1 /home/dank/voices ``` Crops come from `scripts/voice-packs.env` — per voice: source path, offset, duration, and the source's pinned sha256 — and any of them can be overridden on the command line (`--source`, `--offset`, `--duration`, `--sha256`, `--tool`). `--all` walks `VOICE_PACKS`. The builder assumes no GPU, no container, no fleet endpoint and no network: `ffmpeg` cuts the crop, the pinned extractor makes the artifacts, and both run `nice -n 19` so a build never competes with the text endpoints on that box. The extractor is `qwen-codec` in the `qwentts.cpp` tree at `VOICE_PACK_ENGINE_COMMIT` (`5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`). Its usage text, read out of that tree and not guessed: ``` Usage: qwen-codec --model [-i ] [--talker ] [--format ] Required: --model Codec GGUF (qwen-tokenizer-12hz-*.gguf) Optional: --device Compute device: CUDA0, Vulkan0, MTL0, CPU... (default: best available) -i Input. WAV -> encode, .rvq -> decode --talker Talker GGUF (Base only). Encode also extracts the speaker embedding and writes it next to the .rvq as a .spk file --format WAV output format: wav16, wav24, wav32 (default: wav16) Output is auto-named next to input : clip.wav -> clip.rvq, clip.rvq -> clip.wav. ``` Three consequences shape the builder. **There is no `-o`**, so the pack's WAV is staged under its final stem and the artifacts land beside it with nothing to rename. **`--talker` must be a _Base_ talker** — the customvoice and voicedesign GGUFs carry no speaker encoder and the tool says `has no speaker encoder (Base only)`. **Encode truncates to the hop boundary**, so the `.rvq` size is arithmetic, not chance: `22 × floor(samples / 1920)` bytes, which is where the recorded 5,456–6,864 B window comes from and why the **effective crop window is 19.84–25 s** even though the WAV law says 18–25 s. A crop shorter than that is refused before the extractor runs, with the number in the message. The build is staged and published, never assembled in place: `reference.wav` is the file the façade reads, so it moves into position **last**, after the artifacts and the manifest. A failed extractor therefore leaves a pack directory that is not a voice. Nothing is ever deleted — an existing pack is refused unless `--force`, and `--force` moves the whole old directory to `/home/dank/voices/.AGENTTRASH/-/` and builds fresh beside it. Measured on the shipped `narrator1` crop (2026-10-18, `--dry-run` against the real source): 98.739–118.636 s → `955,378 B` of 24 kHz mono `pcm_s16le`, 19.897 s, planned `.rvq` 5,456 B — the same byte counts the trial produced by hand, and the extractor itself costs 1.8–2.1 s and 1.3–1.5 GiB peak per voice. ## The 10-second gate Project-wide source-material invariant: **the first 10 seconds of any sample file used to generate voices is forbidden material.** An offset that starts inside it is refused with exit 1 and a message that names the invariant, before a directory is created: ``` ERROR: source-material invariant violated: the crop must start past the first 10 s of the source file, and 'narrator1' asks for 4.5 s into … — nothing was written. ``` The same gate is then asserted **from the manifest** that was just written — `.crop.offset_seconds >= .invariant.forbidden_source_seconds`, plus the recorded sha256 against the source it claims — so a pack cannot reach disk carrying a forbidden crop even if the offsets in the env file are edited into nonsense. `10` is not a knob: `scripts/voice-packs.env` carries the offsets, the sample rate, the windows and the engine facts, but the forbidden region is a constant in the builder. `tests/voice_pack_unit.bats` pins both halves (a refusal that creates nothing, and one legal crop that builds the whole pack) and applies the invariant to every crop in the tracked pack list. ## Adding a fourth voice 1. Pick the source file and hash it: `sha256sum /path/to/sample.wav`. A crop offset is a claim about one exact file, so the digest goes in the env with it. 2. Find a speech-dense stretch **past 10 s**. The trial's recipe, applied once: `ffmpeg -ss 60 -t 600 -i SRC -af silencedetect=noise=-35dB:d=0.6 -f null -` build the runs of non-silence, keep runs ≥ 18 s, take the longest (earliest on ties), and cap at 25 s. Aim for 20–25 s: shorter than 19.84 s and the `.rvq` falls under the floor. 3. Add the four lines to `scripts/voice-packs.env` and the name to `VOICE_PACKS`: ``` VOICE_PACKS="narrator1 narrator2 narrator3 narrator4" VOICE_PACK_NARRATOR4_SOURCE=/home/dank/voice-sources/audiosample4.m4b VOICE_PACK_NARRATOR4_OFFSET=142.31 VOICE_PACK_NARRATOR4_DURATION=24 VOICE_PACK_NARRATOR4_SHA256=<64 hex> ``` The name must be lowercase `[a-z0-9._-]`: the façade matches the directory name verbatim while `tts-server` lowercases names internally, so an uppercase name would be reported one way by the façade and another by the engine. The builder refuses one anyway. 4. `scripts/build-voice-pack.sh --dry-run narrator4 /home/dank/voices`, read the plan, then drop `--dry-run`. 5. `./tests/run.sh` (the pack-list invariant test covers the new row for free). 6. Register it with the running engine. The engine's voice registry is memory-only and warm-on-boot, but a restart re-decodes **every** voice (6–8 s each), so adding one pack uses the hook instead: ```bash podman exec frank-tts-engine /app/warm-voices.sh curl -s localhost:7772/v1/audio/voices # the façade's list, not the engine's ``` The hook walks `/voices` and registers **every** pack that has a `reference.wav`, on every run — the engine replaces a name it already holds (`tools/tts-server.cpp:148` at the pinned commit: *"Re-registering a name replaces the previous entry."*) — then synthesizes twice per voice and logs `voice cold=…ms warm=…ms` to the unit's journal. The two numbers should be close; see law 1 of [[Frank TTS Engine]] for what they measure. The rootless unit restart — `systemctl --user restart frank-tts-engine.service` — is for when the unit or its env changed, not for a new pack. ## When the extractor's output size changes on an engine bump The builder treats the two artifact sizes as properties of the pinned engine, so an engine bump that changes them **fails the build**: `reference.spk is N B, expected 8192 B` or `reference.rvq is N B, expected …`. That is the failure being loud, not a bug. Work through it in order: 1. Re-read the tool's own numbers rather than trusting this page: `qwen-codec` prints `SpeakerEncoder] Loaded: enc_dim=…` and `Pipeline] Ready: hop … , … codebooks @ … Hz` on every run, and the pinned tree's `tools/qwen-codec.cpp` documents the packing. `enc_dim × 4` is the `.spk` `codebooks × code_bits / 8` bytes per frame × frames is the `.rvq`. 2. Decide whether the new number is intended. A changed `enc_dim` means the old `.spk` files are unreadable by the new engine, so **every** pack must be rebuilt, not just the failing one. 3. Update the four arithmetic lines in `scripts/voice-packs.env` (`VOICE_PACK_SPK_BYTES`, `VOICE_PACK_RVQ_MIN_BYTES`, `VOICE_PACK_RVQ_MAX_BYTES` and `VOICE_PACK_RVQ_BYTES_PER_FRAME` if the codebook packing changed) **and** `VOICE_PACK_ENGINE_COMMIT`, `VOICE_PACK_TOOL`, `VOICE_PACK_CODEC_MODEL` `VOICE_PACK_TALKER_MODEL` to the new build. An env diff that moves the sizes without moving the commit is the mistake to avoid. 4. Rebuild all three packs with `--force` (the retired directories keep the old bytes readable beside their replacements until a human clears them), then re-run the warm hook: `podman exec frank-tts-engine /app/warm-voices.sh`. That hook registers **every** pack it finds on every run, and the engine replaces a name it already holds — `tools/tts-server.cpp:148` at the pinned commit says so in its own words: *"Re-registering a name replaces the previous entry."* So the rebuilt `reference.wav` is what the running engine speaks afterwards, and the hook's cold/warm pass is what re-snapshots it. A restart (`systemctl --user restart frank-tts-engine.service`) is what an ENGINE bump needs — the registry is memory-only and lives in that process — and the unit runs the same hook on every start anyway, so a restart is never worse, only slower. 5. `./tests/voice_pack_unit.bats` — the stub in that seam reproduces the packing law, so it needs the same arithmetic update as the env file. The seam's engine-bump test is exactly this scenario with a wrong `.spk` size, and its assertion — no `reference.wav`, no manifest — is the property that matters. ## Host state (census 2026-10-18, read-only) - `/home/dank/voices/` holds `narrator1`, `narrator2`, `narrator3`, each with `reference.wav`, a `reference16k.wav`, and a `manifest.json` **in the trial's own schema** (`kind: voice-reference-pack`, a silence-scan record). They carry **no** `reference.spk` / `reference.rvq`: the trial's artifacts live in its own work directory (`/home/dank/voice-audition/qwentts/voices/`, and only for narrator1 and narrator2). The builder's first real run is therefore a migration, and it should be a `--force` run per voice so the old directories are retired rather than left half-updated. - The three source files (`/home/blank/audiosample.mp3` `audiosample2.m4b`, `audiosample3.m4b`, digests pinned in `scripts/voice-packs.env`) live on the workstation, not on Dankclaw; the builder reads them from `$VOICE_PACK_SOURCE_DIR=/home/dank/voice-sources`, so they have to be copied there once before a build. - `VOICE_PACK_TOOL` points at the trial's build tree (`/home/dank/voice-audition/qwentts/qwentts.cpp/build/qwen-codec`, checked out at exactly `5916bf7c`), and the two GGUFs live at `/home/dank/models/Qwen3-TTS-GGUF/`, the engine slice's fetch target — that path and its fetch state belong to [[Frank TTS Engine]]. - Whether the façade (7772) and the engine (8880) are up and serving is deploy state on [[Frank TTS]] / [[Frank TTS Engine]] — read it there, not from this census. ## Known gaps, on purpose - **No transcript in the manifest.** ICL wants a `ref_text` that matches the crop, and the trial's machine transcripts came back wrong ("Demon Souls" for *Dark Souls*). Transcribing on the host is not a fix, so the pack carries none and the engine registers the WAV-only voice, which is what its boot warmer does anyway. When ICL becomes a per-voice opt-in, the curated transcript belongs in this manifest — a human-supplied field, not something the builder can derive. - **No 16 kHz variant.** The packs the trial left behind carry `reference16k.wav` for the `llama-tts-cli` rollback engine; arm A is native 24 kHz, so the builder emits one reference. `--force` retires the whole directory including that variant, so if the rollback engine is ever revived: `ffmpeg -i reference.wav -ar 16000 -c:a pcm_s16le reference16k.wav`. - **No loudness pass.** The trial's pipeline also measured and normalised (two-pass `loudnorm`, `astats`, `ebur128`, spectrograms) to a `-16 LUFS` spec. None of that is in the pack law, which asks for 24 kHz mono inside the duration window, so the builder does only that; a future engine that is loudness-sensitive should get the pass as an explicit, documented step.