fix(open-notebook): size the TTS clock outside frank-tts's own ceiling

The env file first landed with ESPERANTO_TTS_TIMEOUT equal to SHIM_TIMEOUT=600.
Equality is a race: both budgets expire at the same instant, so the case the
number existed to prevent — the worker abandoning a synthesis frank-tts is still
finishing — is exactly what happens on the tail of a long line. Nested budgets
go inside-out, which is the rule scripts/frank-tts.env states for its own
LLAMA_TIMEOUT; the client waits one round-trip longer than the server, so a
frank-tts give-up arrives labelled as frank-tts's error. 660.

The seam's assertion moves from equality to `>` for the same reason, and now
also holds ESPERANTO_LLM_TIMEOUT under upstream's 600 s web-UI patience — above
it the failure surfaces as a UI timeout with no name on it. The scriptwriting
number itself stays at the app's 180 s default: frank-zealot answers an outline
or a transcript well inside it, which corrects the earlier commit message's
claim that both timeouts were raised past upstream's 300 s.
This commit is contained in:
Frank
2026-10-18 16:31:49 -07:00
parent e2393bacad
commit 114dd7a93e
4 changed files with 29 additions and 15 deletions
+7 -6
View File
@@ -38,11 +38,12 @@
# LINE, not per episode: one episode is dozens of POSTs to /v1/audio/speech # LINE, not per episode: one episode is dozens of POSTs to /v1/audio/speech
# (the app synthesises each speaker turn separately and concatenates the clips # (the app synthesises each speaker turn separately and concatenates the clips
# with ffmpeg afterwards), and a CPU engine takes single-digit seconds for a # with ffmpeg afterwards), and a CPU engine takes single-digit seconds for a
# line, with the tail of a long turn well past the 300 s default. 600 matches # line, with the tail of a long turn well past the 300 s default. 660 sits one
# SHIM_TIMEOUT in scripts/frank-tts.env — the shim's own per-request ceiling — # round-trip OUTSIDE SHIM_TIMEOUT=600 in scripts/frank-tts.env — the shim's own
# so open-notebook never abandons a synthesis frank-tts is still finishing. An # per-request ceiling — so a frank-tts give-up arrives as frank-tts's error and
# abandoned request does not cancel the engine process; it just leaves the # not as open-notebook abandoning a synthesis still in flight. An abandoned
# episode stuck in "generating" while the audio arrives. # request does not cancel the engine process; it just leaves the episode stuck in
# "generating" while the audio arrives.
# #
# THE SCRIPTWRITER. ESPERANTO_LLM_TIMEOUT is the app's own default (180 s), kept # THE SCRIPTWRITER. ESPERANTO_LLM_TIMEOUT is the app's own default (180 s), kept
# visible beside the TTS number because both belong to one podcast run: the # visible beside the TTS number because both belong to one podcast run: the
@@ -87,7 +88,7 @@ TTS_BATCH_SIZE=1
OPEN_NOTEBOOK_WORKER_MAX_TASKS=1 OPEN_NOTEBOOK_WORKER_MAX_TASKS=1
# Timeouts: see "THE TTS CLOCK" and "THE SCRIPTWRITER" above. # Timeouts: see "THE TTS CLOCK" and "THE SCRIPTWRITER" above.
ESPERANTO_TTS_TIMEOUT=600 ESPERANTO_TTS_TIMEOUT=660
ESPERANTO_LLM_TIMEOUT=180 ESPERANTO_LLM_TIMEOUT=180
# Heavy extraction runtimes stay OFF, which is also what keeps boot fast: each # Heavy extraction runtimes stay OFF, which is also what keeps boot fast: each
@@ -88,9 +88,9 @@ None of it blocks anything: model rows are added **by name** ("If a model isn't
## The CPU guard, and why both numbers are 1 ## The CPU guard, and why both numbers are 1
Defaults are `TTS_BATCH_SIZE=5` and `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` (`docs/5-CONFIGURATION/environment-reference.md:129-130`) — five parallel synthesis requests from five parallel jobs. frank-tts spawns an engine process per request (about 8 GiB resident, 8 threads, from the frank-tts CPU trial of 2026-10-18; `scripts/frank-tts.env` pins `LLAMA_TTS_THREADS=8`), on a machine that also serves frank-dragoon and frank-zealot. Five of those is ~40 GiB and forty threads competing with two LLM endpoints for the same cores. `scripts/open-notebook.env:86-87` sets both to 1: one episode at a time, one voice line at a time. Upstream's own guidance is the same direction ("Set 1 for a local model on a single GPU", "Lower it … for rate-limited or single-threaded TTS servers"). Defaults are `TTS_BATCH_SIZE=5` and `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` (`docs/5-CONFIGURATION/environment-reference.md:129-130`) — five parallel synthesis requests from five parallel jobs. frank-tts spawns an engine process per request (about 8 GiB resident, 8 threads, from the frank-tts CPU trial of 2026-10-18; `scripts/frank-tts.env` pins `LLAMA_TTS_THREADS=8`), on a machine that also serves frank-dragoon and frank-zealot. Five of those is ~40 GiB and forty threads competing with two LLM endpoints for the same cores. `scripts/open-notebook.env:87-88` sets both to 1: one episode at a time, one voice line at a time. Upstream's own guidance is the same direction ("Set 1 for a local model on a single GPU", "Lower it … for rate-limited or single-threaded TTS servers").
The clock next to it: `ESPERANTO_TTS_TIMEOUT=600` (`:90`) is patience **per dialogue line**, not per episode — a three-minute episode is dozens of separate POSTs, and the tail of a long turn on a CPU engine goes past the 300 s default. 600 matches `SHIM_TIMEOUT` in `scripts/frank-tts.env`, frank-tts's own per-request ceiling, so the worker never abandons a synthesis the shim is still finishing; an abandoned request does not cancel the engine process, it just strands the episode in "generating". The clock next to it: `ESPERANTO_TTS_TIMEOUT` (`:91`) is patience **per dialogue line**, not per episode — a three-minute episode is dozens of separate POSTs, and the tail of a long turn on a CPU engine goes past upstream's 300 s default. It is set to **660, one round-trip outside `SHIM_TIMEOUT=600`** in `scripts/frank-tts.env` (frank-tts's own per-request ceiling): nested budgets go inside-out — the same rule that file states for `LLAMA_TIMEOUT` — so a frank-tts give-up arrives labelled as frank-tts's error instead of the worker's own timeout, and re-measuring frank-tts cannot silently strand this number below it. An abandoned request does not cancel the engine process; it only strands the episode in "generating".
## Secrets, and the difference between blank and unset ## Secrets, and the difference between blank and unset
+7 -2
View File
@@ -2345,8 +2345,13 @@ every restart since the cutover.
to `TTS_BATCH_SIZE=5` × `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` — a five-way to `TTS_BATCH_SIZE=5` × `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` — a five-way
self-DoS. Both are pinned to `1` in `scripts/open-notebook.env`, and the self-DoS. Both are pinned to `1` in `scripts/open-notebook.env`, and the
worker knob is only real because the image's own supervisord expands worker knob is only real because the image's own supervisord expands
`${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5}` (`supervisord.conf:25`). Both `${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5}` (`supervisord.conf:25`). The TTS clock
`ESPERANTO_*` timeouts go above upstream's 300 s for CPU synthesis. goes to **660 — one round-trip outside** frank-tts's own `SHIM_TIMEOUT=600`, so
a shim give-up reads as the shim's error rather than the worker's; the seam
asserts the direction (`>`) against the frank-tts env instead of restating a
number. `ESPERANTO_LLM_TIMEOUT` stays at the app's 180 s default: frank-zealot
answers an outline or a transcript well inside it, and upstream says to keep it
under 600 so a slow call fails with a name on it.
- **What a credential needs in order to exist:** `OPEN_NOTEBOOK_ENCRYPTION_KEY` - **What a credential needs in order to exist:** `OPEN_NOTEBOOK_ENCRYPTION_KEY`
must be set before the UI can save *any* provider credential, and must be set before the UI can save *any* provider credential, and
`OPEN_NOTEBOOK_PASSWORD` must stay **absent** to remain auth-off — the app `OPEN_NOTEBOOK_PASSWORD` must stay **absent** to remain auth-off — the app
+13 -5
View File
@@ -130,15 +130,23 @@ DB_PIN="sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723"
# request on a box that also serves two LLM endpoints. # request on a box that also serves two LLM endpoints.
grep -qxF 'TTS_BATCH_SIZE=1' "$ENVF" grep -qxF 'TTS_BATCH_SIZE=1' "$ENVF"
grep -qxF 'OPEN_NOTEBOOK_WORKER_MAX_TASKS=1' "$ENVF" grep -qxF 'OPEN_NOTEBOOK_WORKER_MAX_TASKS=1' "$ENVF"
# Per dialogue line, matched to frank-tts's own per-request ceiling so the # Per dialogue line, and strictly beyond frank-tts's own per-request
# worker never abandons a synthesis the shim is still finishing. # ceiling: nested budgets go inside-out (the rule scripts/frank-tts.env
# states for LLAMA_TIMEOUT), so a shim give-up surfaces as frank-tts's
# error instead of the worker's own timeout. The seam reads the ceiling
# rather than restating it, so re-measuring frank-tts cannot leave this
# file silently behind.
tts="$(sed -n 's/^ESPERANTO_TTS_TIMEOUT=//p' "$ENVF")" tts="$(sed -n 's/^ESPERANTO_TTS_TIMEOUT=//p' "$ENVF")"
shim="$(sed -n 's/^SHIM_TIMEOUT=//p' "$REPO_ROOT/scripts/frank-tts.env")" shim="$(sed -n 's/^SHIM_TIMEOUT=//p' "$REPO_ROOT/scripts/frank-tts.env")"
[ -n "$tts" ] || { echo "no ESPERANTO_TTS_TIMEOUT in the env file"; false; } [ -n "$tts" ] || { echo "no ESPERANTO_TTS_TIMEOUT in the env file"; false; }
[ -n "$shim" ] || { echo "no SHIM_TIMEOUT in scripts/frank-tts.env"; false; } [ -n "$shim" ] || { echo "no SHIM_TIMEOUT in scripts/frank-tts.env"; false; }
[ "$tts" = "$shim" ] || { [ "$tts" -gt "$shim" ] || {
echo "ESPERANTO_TTS_TIMEOUT=$tts does not match frank-tts SHIM_TIMEOUT=$shim"; false; } echo "ESPERANTO_TTS_TIMEOUT=$tts is not beyond frank-tts SHIM_TIMEOUT=$shim"; false; }
grep -qE '^ESPERANTO_LLM_TIMEOUT=[0-9]+$' "$ENVF" # Upstream: keep the scriptwriting call under the web UI's ~600 s patience,
# or the failure arrives as a UI timeout with no name on it.
llm="$(sed -n 's/^ESPERANTO_LLM_TIMEOUT=//p' "$ENVF")"
[[ "$llm" =~ ^[0-9]+$ ]] && [ "$llm" -lt 600 ] || {
echo "ESPERANTO_LLM_TIMEOUT=$llm is not a number under 600"; false; }
} }
@test "secrets stay declared-but-empty in the tracked env file" { @test "secrets stay declared-but-empty in the tracked env file" {