docs(voice-packs): runbook, index link, and the log entry

The page a new operator needs: what the three required files are and who
reads each one, how a pack is built (with the extractor's usage text
verbatim, because the flags are not guessable and there is no -o), the
10-second gate and where it fires twice, how to add a fourth voice
including the warm hook that registers a pack without restarting the
engine, and the engine-bump procedure for when the artifact sizes move.
Plus the host census as of 2026-10-18 — the trial packs carry no
artifacts and their manifest is the trial's own schema, so the builder's
first real run is a --force migration — and the three gaps left open on
purpose: no curated ICL transcript, no 16 kHz variant, no loudness pass.

Linked from the Dankclaw services list; log entry appended oldest-first.

serverAppConfigs-docs/Dankclaw/Voice Packs.md

@@ -0,0 +1,251 @@
+---
+type: reference
+description: "How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material gate scripts/build-voice-pack.sh enforces, how to add a fourth voice, and what to do when a pinned engine bump changes the extractor's output sizes."
+tags:
+  - dankclaw
+  - frank-tts
+  - voices
+  - tts
+timestamp: 2026-10-18
+---
+
+# Voice Packs
+
+A **voice pack** is the unit of identity for **frank-tts**: one directory under
+`/home/dank/voices`, named by the voice, carrying the reference audio that makes
+the name real plus the pinned engine's own speaker artifacts and a provenance
+record. `voice=narrator1` on `POST /v1/audio/speech` resolves to
+`/home/dank/voices/narrator1/reference.wav` and to nothing else — a name without
+that file is a 400 that lists the names that do exist, because a silent fallback
+voice is how a wrong-voice endpoint stays wrong for weeks.
+
+**Host:** Dankclaw (192.168.0.143) — **Pack root:** `/home/dank/voices`, host
+state that stays out of the repo, mounted read-only at `/voices` by the façade
+unit — **Builder:** `scripts/build-voice-pack.sh` with tunables in
+`scripts/voice-packs.env` — **Seam:** `tests/voice_pack_unit.bats` — **Engine:**
+arm A `qwentts.cpp`, decided in [[Local TTS and Voice Cloning]].
+
+Packs are **world-readable on purpose** (`drwxr-xr-x`, owner `dank`): the façade
+container runs as uid 10000 and reads the mount from outside its own uid map.
+
+## The required files
+
+| File | What it is | Who reads it |
+|------|------------|--------------|
+| `reference.wav` | the crop: 24 kHz mono `pcm_s16le`, **18–25 s** of clean single-speaker audio | the façade (it is the existence gate for the name) and the engine's boot warmer, which base64s it and registers the voice with no transcript |
+| `reference.spk` | the engine's x-vector: **8,192 B** = `enc_dim` 2048 × f32 | the engine, for the ICL/`--ref-spk` path |
+| `reference.rvq` | the engine's RVQ code stream: **5,456–6,864 B**, 16 codebooks × T frames packed at 11 bits = 22 B per 12.5 Hz frame | the engine, for the ICL/`--ref-rvq` path |
+| `manifest.json` | provenance: source path and sha256, crop start/end offsets, sample rate, duration, the tool and engine commit that made the artifacts, and the `voice` name | humans, and the builder itself on its second gate assertion |
+
+`qwen-codec.log` rides along with the extractor's own record of the run; the
+façade ignores anything it does not name and never deletes anything.
+
+Division of enforcement, which is deliberate and not a gap: the **façade** gates
+on `reference.wav` because that is the only engine-agnostic term (see the
+voice-pack law in `scripts/frank-tts.env`); the **builder** is what refuses to
+emit a pack without its engine artifacts; the **engine** registers a voice from
+the WAV alone because its own `POST /v1/audio/voices` refuses a lone embedding —
+it wants `wav_b64` or **both** `spk_b64` and `rvq_b64`. So the artifacts are in
+the pack to make ICL a per-voice opt-in, not to boot it.
+
+## How a pack is built
+
+```bash
+# everything except the extractor: gate, crop, probe, plan, the manifest it would write
+scripts/build-voice-pack.sh --dry-run narrator1 /home/dank/voices
+
+# the real thing, on the host that holds the engine build
+scripts/build-voice-pack.sh narrator1 /home/dank/voices
+```
+
+Crops come from `scripts/voice-packs.env` — per voice: source path, offset,
+duration, and the source's pinned sha256 — and any of them can be overridden on
+the command line (`--source`, `--offset`, `--duration`, `--sha256`, `--tool`).
+`--all` walks `VOICE_PACKS`. The builder assumes no GPU, no container, no fleet
+endpoint and no network: `ffmpeg` cuts the crop, the pinned extractor makes the
+artifacts, and both run `nice -n 19` so a build never competes with the text
+endpoints on that box.
+
+The extractor is `qwen-codec` in the `qwentts.cpp` tree at
+`VOICE_PACK_ENGINE_COMMIT` (`5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`). Its
+usage text, read out of that tree and not guessed:
+
+```
+Usage: qwen-codec --model <gguf> [-i <input>] [--talker <gguf>] [--format <fmt>]
+
+Required:
+  --model <gguf>          Codec GGUF (qwen-tokenizer-12hz-*.gguf)
+
+Optional:
+  --device <name>         Compute device: CUDA0, Vulkan0, MTL0, CPU...
+                          (default: best available)
+  -i <path>               Input. WAV -> encode, .rvq -> decode
+  --talker <gguf>         Talker GGUF (Base only). Encode also extracts the speaker
+                          embedding and writes it next to the .rvq as a .spk file
+  --format <fmt>          WAV output format: wav16, wav24, wav32 (default: wav16)
+
+Output is auto-named next to input : clip.wav -> clip.rvq, clip.rvq -> clip.wav.
+```
+
+Three consequences shape the builder. **There is no `-o`**, so the pack's WAV is
+staged under its final stem and the artifacts land beside it with nothing to
+rename. **`--talker` must be a _Base_ talker** — the customvoice and voicedesign
+GGUFs carry no speaker encoder and the tool says `has no speaker encoder (Base
+only)`. **Encode truncates to the hop boundary**, so the `.rvq` size is arithmetic,
+not chance: `22 × floor(samples / 1920)` bytes, which is where the recorded
+5,456–6,864 B window comes from and why the **effective crop window is
+19.84–25 s** even though the WAV law says 18–25 s. A crop shorter than that is
+refused before the extractor runs, with the number in the message.
+
+The build is staged and published, never assembled in place: `reference.wav` is
+the file the façade reads, so it moves into position **last**, after the
+artifacts and the manifest. A failed extractor therefore leaves a pack directory
+that is not a voice. Nothing is ever deleted — an existing pack is refused unless
+`--force`, and `--force` moves the whole old directory to
+`/home/dank/voices/.AGENTTRASH/<voice>-<UTC>/` and builds fresh beside it.
+
+Measured on the shipped `narrator1` crop (2026-10-18, `--dry-run` against the
+real source): 98.739–118.636 s → `955,378 B` of 24 kHz mono `pcm_s16le`, 19.897
+s, planned `.rvq` 5,456 B — the same byte counts the trial produced by hand, and
+the extractor itself costs 1.8–2.1 s and 1.3–1.5 GiB peak per voice.
+
+## The 10-second gate
+
+Project-wide source-material invariant: **the first 10 seconds of any sample
+file used to generate voices is forbidden material.** An offset that starts
+inside it is refused with exit 1 and a message that names the invariant, before
+a directory is created:
+
+```
+ERROR: source-material invariant violated: the crop must start past the first
+10 s of the source file, and 'narrator1' asks for 4.5 s into … — nothing was
+written.
+```
+
+The same gate is then asserted **from the manifest** that was just written —
+`.crop.offset_seconds >= .invariant.forbidden_source_seconds`, plus the recorded
+sha256 against the source it claims — so a pack cannot reach disk carrying a
+forbidden crop even if the offsets in the env file are edited into nonsense.
+`10` is not a knob: `scripts/voice-packs.env` carries the offsets, the sample
+rate, the windows and the engine facts, but the forbidden region is a constant in
+the builder. `tests/voice_pack_unit.bats` pins both halves (a refusal that
+creates nothing, and one legal crop that builds the whole pack) and applies the
+invariant to every crop in the tracked pack list.
+
+## Adding a fourth voice
+
+1. Pick the source file and hash it: `sha256sum /path/to/sample.wav`. A crop
+   offset is a claim about one exact file, so the digest goes in the env with it.
+2. Find a speech-dense stretch **past 10 s**. The trial's recipe, applied once:
+   `ffmpeg -ss 60 -t 600 -i SRC -af silencedetect=noise=-35dB:d=0.6 -f null -`,
+   build the runs of non-silence, keep runs ≥ 18 s, take the longest (earliest
+   on ties), and cap at 25 s. Aim for 20–25 s: shorter than 19.84 s and the
+   `.rvq` falls under the floor.
+3. Add the four lines to `scripts/voice-packs.env` and the name to
+   `VOICE_PACKS`:
+
+   ```
+   VOICE_PACKS="narrator1 narrator2 narrator3 narrator4"
+
+   VOICE_PACK_NARRATOR4_SOURCE=/home/dank/voice-sources/audiosample4.m4b
+   VOICE_PACK_NARRATOR4_OFFSET=142.31
+   VOICE_PACK_NARRATOR4_DURATION=24
+   VOICE_PACK_NARRATOR4_SHA256=<64 hex>
+   ```
+
+   The name must be lowercase `[a-z0-9._-]`: the façade matches the directory
+   name verbatim while `tts-server` lowercases names internally, so an uppercase
+   name would be reported one way by the façade and another by the engine. The
+   builder refuses one anyway.
+4. `scripts/build-voice-pack.sh --dry-run narrator4 /home/dank/voices`, read the
+   plan, then drop `--dry-run`.
+5. `./tests/run.sh` (the pack-list invariant test covers the new row for free).
+6. Register it with the running engine. The engine's voice registry is
+   memory-only and warm-on-boot, but a restart re-decodes **every** voice (6–8 s
+   each), so adding one pack uses the hook instead:
+
+   ```bash
+   podman exec frank-tts-engine /app/warm-voices.sh
+   curl -s localhost:7772/v1/audio/voices      # the façade's list, not the engine's
+   ```
+
+   The hook walks `/voices`, registers any pack with a `reference.wav` it has not
+   seen, and logs `voice <name> cold=…ms warm=…ms` to the unit's journal. The
+   rootless unit restart — `systemctl --user restart frank-tts-engine.service` —
+   is for when the unit or its env changed, not for a new pack.
+
+## When the extractor's output size changes on an engine bump
+
+The builder treats the two artifact sizes as properties of the pinned engine, so
+an engine bump that changes them **fails the build**: `reference.spk is N B,
+expected 8192 B` or `reference.rvq is N B, expected …`. That is the failure being
+loud, not a bug. Work through it in order:
+
+1. Re-read the tool's own numbers rather than trusting this page: `qwen-codec`
+   prints `SpeakerEncoder] Loaded: enc_dim=…` and `Pipeline] Ready: hop … , …
+   codebooks @ … Hz` on every run, and the pinned tree's
+   `tools/qwen-codec.cpp` documents the packing. `enc_dim × 4` is the `.spk`;
+   `codebooks × code_bits / 8` bytes per frame × frames is the `.rvq`.
+2. Decide whether the new number is intended. A changed `enc_dim` means the old
+   `.spk` files are unreadable by the new engine, so **every** pack must be
+   rebuilt, not just the failing one.
+3. Update the four arithmetic lines in `scripts/voice-packs.env`
+   (`VOICE_PACK_SPK_BYTES`, `VOICE_PACK_RVQ_MIN_BYTES`, `VOICE_PACK_RVQ_MAX_BYTES`,
+   and `VOICE_PACK_RVQ_BYTES_PER_FRAME` if the codebook packing changed) **and**
+   `VOICE_PACK_ENGINE_COMMIT`, `VOICE_PACK_TOOL`, `VOICE_PACK_CODEC_MODEL`,
+   `VOICE_PACK_TALKER_MODEL` to the new build. An env diff that moves the sizes
+   without moving the commit is the mistake to avoid.
+4. Rebuild all three packs with `--force` (the retired directories keep the old
+   bytes readable beside their replacements until a human clears them), then
+   restart the engine rather than using the warm hook: the hook only registers
+   packs it has **not** seen, and every voice here is already registered against
+   the old artifacts. `systemctl --user restart frank-tts-engine.service`
+   re-warms them all from the new bytes.
+5. `./tests/voice_pack_unit.bats` — the stub in that seam reproduces the packing
+   law, so it needs the same arithmetic update as the env file. The seam's
+   engine-bump test is exactly this scenario with a wrong `.spk` size, and its
+   assertion — no `reference.wav`, no manifest — is the property that matters.
+
+## Host state (census 2026-10-18, read-only)
+
+- `/home/dank/voices/` holds `narrator1`, `narrator2`, `narrator3`, each with
+  `reference.wav`, a `reference16k.wav`, and a `manifest.json` **in the trial's
+  own schema** (`kind: voice-reference-pack`, a silence-scan record). They carry
+  **no** `reference.spk` / `reference.rvq`: the trial's artifacts live in its own
+  work directory (`/home/dank/voice-audition/qwentts/voices/`, and only for
+  narrator1 and narrator2). The builder's first real run is therefore a
+  migration, and it should be a `--force` run per voice so the old directories
+  are retired rather than left half-updated.
+- The three source files (`/home/blank/audiosample.mp3`,
+  `audiosample2.m4b`, `audiosample3.m4b`, digests pinned in
+  `scripts/voice-packs.env`) live on the workstation, not on Dankclaw; the
+  builder reads them from `$VOICE_PACK_SOURCE_DIR=/home/dank/voice-sources`, so
+  they have to be copied there once before a build.
+- `VOICE_PACK_TOOL` points at the trial's build tree
+  (`/home/dank/voice-audition/qwentts/qwentts.cpp/build/qwen-codec`, checked out
+  at exactly `5916bf7c`), and the two GGUFs at
+  `/home/dank/models/Qwen3-TTS-GGUF/`, which is where the engine slice's
+  download script puts them — that directory does not exist yet, so until it
+  does the models have to come from the trial's `models/` directory.
+- Nothing is deployed: nothing listens on 7772 or 8880 today, so building packs
+  is possible on that host while serving them is not.
+
+## Known gaps, on purpose
+
+- **No transcript in the manifest.** ICL wants a `ref_text` that matches the
+  crop, and the trial's machine transcripts came back wrong ("Demon Souls" for
+  *Dark Souls*). Transcribing on the host is not a fix, so the pack carries none
+  and the engine registers the WAV-only voice, which is what its boot warmer
+  does anyway. When ICL becomes a per-voice opt-in, the curated transcript
+  belongs in this manifest — a human-supplied field, not something the builder
+  can derive.
+- **No 16 kHz variant.** The packs the trial left behind carry
+  `reference16k.wav` for the `llama-tts-cli` rollback engine; arm A is native
+  24 kHz, so the builder emits one reference. `--force` retires the whole
+  directory including that variant, so if the rollback engine is ever revived:
+  `ffmpeg -i reference.wav -ar 16000 -c:a pcm_s16le reference16k.wav`.
+- **No loudness pass.** The trial's pipeline also measured and normalised
+  (two-pass `loudnorm`, `astats`, `ebur128`, spectrograms) to a `-16 LUFS` spec.
+  None of that is in the pack law, which asks for 24 kHz mono inside the
+  duration window, so the builder does only that; a future engine that is
+  loudness-sensitive should get the pass as an explicit, documented step.

serverAppConfigs-docs/index.md

@@ -80,6 +80,7 @@ macOS mini) as quadlets inside its podman machine — not on Dankserver.
 - [[Seerr]] — unified *arr request manager (guard-chained; fronts FrankBook-Pro's stack)
 - [[Marreta]] — content-cleaning proxy (guard-chained; shares SearXNG's Gluetun egress)
 - [[Open Notebook]] — podcast generator (scripts by frank-zealot, voices by frank-tts); repo-owned config, not deployed
+- [[Voice Packs]] — how a frank-tts voice is built: reference.wav + the pinned engine's speaker artifacts, cut past the 10-second source-material gate
 
 ## Infrastructure
 

serverAppConfigs-docs/log.md

@@ -2441,3 +2441,56 @@ every restart since the cutover.
   `~/.harbor/compose.opennotebook.yml` pins the deprecated `-single` image,
   carries no encryption key and injects the whole harbor env — left untouched,
   read-only, as the record of what not to copy.
+
+## 2026-10-18 — voice-pack builder: the 10-second source-material gate becomes code
+
+- Third slice of the voice stack (after the façade and the engine decision):
+  `scripts/build-voice-pack.sh` turns a source recording plus a crop offset into
+  a deployable pack — `reference.wav` (24 kHz mono `pcm_s16le`, 18–25 s), the
+  engine's own `reference.spk` (8,192 B) + `reference.rvq` (5,456–6,864 B) made
+  from it, and `manifest.json` provenance. Tracked tunables in
+  `scripts/voice-packs.env`: pack list, per-voice source/offset/duration/pinned
+  source sha256, extractor path and engine commit, artifact arithmetic. New runbook
+  [[Voice Packs]].
+- **The gate is the product.** The project-wide invariant — no voice material
+  from the first 10 s of a sample file — had only ever been a sentence in
+  `scripts/frank-tts.env`. An offset inside it now exits 1 before a directory is
+  created, and the same assertion runs again against the manifest that was just
+  written, next to the recorded sha256, so a pack cannot carry a forbidden crop
+  even if the env file is edited into nonsense. `10` stays a constant in the
+  builder; the env file carries offsets, never the law.
+- **The extractor's contract was read, not guessed:** `qwen-codec` in qwentts.cpp
+  `5916bf7c` takes `--model <codec gguf> --talker <base talker gguf> -i <wav>`
+  with an optional `--device`, has **no `-o`** and names both artifacts next to
+  the input with the extension swapped. So the WAV is staged under its final stem
+  and `reference.wav` — the file the façade gates on — is moved into place
+  **last**, after the artifacts and the manifest: a failed extractor leaves a
+  directory that is not a voice. `--force` retires an existing pack whole to
+  `.AGENTTRASH/` instead of touching bytes; nothing in this script deletes.
+- Sizes are arithmetic, so they are checked: 16 codebooks × 11 bits = 22 B per
+  12.5 Hz frame (`22 × floor(samples/1920)`) and `enc_dim 2048 × f32` = 8,192 B.
+  That makes the **effective crop window 19.84–25 s**, narrower than the WAV
+  law's 18–25 s, and a too-short crop is refused before the extractor runs. An
+  engine bump that changes either number fails the build by design; the bump
+  procedure is in [[Voice Packs]].
+- Verified against reality, not against the model: the pinned `narrator1` crop
+  (98.739–118.636 s of `audiosample.mp3`, sha256 `2299510…`) comes out of a
+  `--dry-run` at 955,378 B / 19.897 s with 5,456 B of `.rvq` planned — byte-for-byte
+  the trial's hand-cut numbers — and all three pinned source digests match the
+  files on disk.
+- Seam: `tests/voice_pack_unit.bats` (13 tests) — refusal inside the forbidden
+  region creates nothing, a legal crop builds the whole pack, `ffprobe` on the
+  generated reference, manifest round-trip through `jq` with a sha that matches
+  the source, re-run-without-`--force` leaves every byte alone, `--force` retires
+  rather than deletes, `--dry-run` needs no extractor, a wrong-sized artifact
+  from a bumped engine cannot publish a voice, and every crop in the tracked pack
+  list clears the invariant. `qwen-codec` is stood in by a stub with the real
+  tool's I/O shape, so no engine, no GPU, no network, no fleet host. Suite green:
+  **416/416** (`./tests/run.sh`, wall 77 s) — 403 before this branch.
+- Deliberately NOT done: no transcript in the manifest (ICL wants a curated
+  `ref_text`; the trial's machine transcripts were wrong, and the engine
+  registers WAV-only voices anyway), no 16 kHz variant (arm A is native 24 kHz),
+  no loudness pass (not in the pack law), and no pack built on the host — the
+  three sources still live on the workstation and the engine GGUFs have not been
+  fetched to `/home/dank/models/Qwen3-TTS-GGUF` yet, so the first real run is a
+  host step.