fix(open-notebook): size the TTS clock outside frank-tts's own ceiling
The env file first landed with ESPERANTO_TTS_TIMEOUT equal to SHIM_TIMEOUT=600. Equality is a race: both budgets expire at the same instant, so the case the number existed to prevent — the worker abandoning a synthesis frank-tts is still finishing — is exactly what happens on the tail of a long line. Nested budgets go inside-out, which is the rule scripts/frank-tts.env states for its own LLAMA_TIMEOUT; the client waits one round-trip longer than the server, so a frank-tts give-up arrives labelled as frank-tts's error. 660. The seam's assertion moves from equality to `>` for the same reason, and now also holds ESPERANTO_LLM_TIMEOUT under upstream's 600 s web-UI patience — above it the failure surfaces as a UI timeout with no name on it. The scriptwriting number itself stays at the app's 180 s default: frank-zealot answers an outline or a transcript well inside it, which corrects the earlier commit message's claim that both timeouts were raised past upstream's 300 s.
This commit is contained in:
@@ -38,11 +38,12 @@
|
|||||||
# LINE, not per episode: one episode is dozens of POSTs to /v1/audio/speech
|
# LINE, not per episode: one episode is dozens of POSTs to /v1/audio/speech
|
||||||
# (the app synthesises each speaker turn separately and concatenates the clips
|
# (the app synthesises each speaker turn separately and concatenates the clips
|
||||||
# with ffmpeg afterwards), and a CPU engine takes single-digit seconds for a
|
# with ffmpeg afterwards), and a CPU engine takes single-digit seconds for a
|
||||||
# line, with the tail of a long turn well past the 300 s default. 600 matches
|
# line, with the tail of a long turn well past the 300 s default. 660 sits one
|
||||||
# SHIM_TIMEOUT in scripts/frank-tts.env — the shim's own per-request ceiling —
|
# round-trip OUTSIDE SHIM_TIMEOUT=600 in scripts/frank-tts.env — the shim's own
|
||||||
# so open-notebook never abandons a synthesis frank-tts is still finishing. An
|
# per-request ceiling — so a frank-tts give-up arrives as frank-tts's error and
|
||||||
# abandoned request does not cancel the engine process; it just leaves the
|
# not as open-notebook abandoning a synthesis still in flight. An abandoned
|
||||||
# episode stuck in "generating" while the audio arrives.
|
# request does not cancel the engine process; it just leaves the episode stuck in
|
||||||
|
# "generating" while the audio arrives.
|
||||||
#
|
#
|
||||||
# THE SCRIPTWRITER. ESPERANTO_LLM_TIMEOUT is the app's own default (180 s), kept
|
# THE SCRIPTWRITER. ESPERANTO_LLM_TIMEOUT is the app's own default (180 s), kept
|
||||||
# visible beside the TTS number because both belong to one podcast run: the
|
# visible beside the TTS number because both belong to one podcast run: the
|
||||||
@@ -87,7 +88,7 @@ TTS_BATCH_SIZE=1
|
|||||||
OPEN_NOTEBOOK_WORKER_MAX_TASKS=1
|
OPEN_NOTEBOOK_WORKER_MAX_TASKS=1
|
||||||
|
|
||||||
# Timeouts: see "THE TTS CLOCK" and "THE SCRIPTWRITER" above.
|
# Timeouts: see "THE TTS CLOCK" and "THE SCRIPTWRITER" above.
|
||||||
ESPERANTO_TTS_TIMEOUT=600
|
ESPERANTO_TTS_TIMEOUT=660
|
||||||
ESPERANTO_LLM_TIMEOUT=180
|
ESPERANTO_LLM_TIMEOUT=180
|
||||||
|
|
||||||
# Heavy extraction runtimes stay OFF, which is also what keeps boot fast: each
|
# Heavy extraction runtimes stay OFF, which is also what keeps boot fast: each
|
||||||
|
|||||||
@@ -88,9 +88,9 @@ None of it blocks anything: model rows are added **by name** ("If a model isn't
|
|||||||
|
|
||||||
## The CPU guard, and why both numbers are 1
|
## The CPU guard, and why both numbers are 1
|
||||||
|
|
||||||
Defaults are `TTS_BATCH_SIZE=5` and `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` (`docs/5-CONFIGURATION/environment-reference.md:129-130`) — five parallel synthesis requests from five parallel jobs. frank-tts spawns an engine process per request (about 8 GiB resident, 8 threads, from the frank-tts CPU trial of 2026-10-18; `scripts/frank-tts.env` pins `LLAMA_TTS_THREADS=8`), on a machine that also serves frank-dragoon and frank-zealot. Five of those is ~40 GiB and forty threads competing with two LLM endpoints for the same cores. `scripts/open-notebook.env:86-87` sets both to 1: one episode at a time, one voice line at a time. Upstream's own guidance is the same direction ("Set 1 for a local model on a single GPU", "Lower it … for rate-limited or single-threaded TTS servers").
|
Defaults are `TTS_BATCH_SIZE=5` and `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` (`docs/5-CONFIGURATION/environment-reference.md:129-130`) — five parallel synthesis requests from five parallel jobs. frank-tts spawns an engine process per request (about 8 GiB resident, 8 threads, from the frank-tts CPU trial of 2026-10-18; `scripts/frank-tts.env` pins `LLAMA_TTS_THREADS=8`), on a machine that also serves frank-dragoon and frank-zealot. Five of those is ~40 GiB and forty threads competing with two LLM endpoints for the same cores. `scripts/open-notebook.env:87-88` sets both to 1: one episode at a time, one voice line at a time. Upstream's own guidance is the same direction ("Set 1 for a local model on a single GPU", "Lower it … for rate-limited or single-threaded TTS servers").
|
||||||
|
|
||||||
The clock next to it: `ESPERANTO_TTS_TIMEOUT=600` (`:90`) is patience **per dialogue line**, not per episode — a three-minute episode is dozens of separate POSTs, and the tail of a long turn on a CPU engine goes past the 300 s default. 600 matches `SHIM_TIMEOUT` in `scripts/frank-tts.env`, frank-tts's own per-request ceiling, so the worker never abandons a synthesis the shim is still finishing; an abandoned request does not cancel the engine process, it just strands the episode in "generating".
|
The clock next to it: `ESPERANTO_TTS_TIMEOUT` (`:91`) is patience **per dialogue line**, not per episode — a three-minute episode is dozens of separate POSTs, and the tail of a long turn on a CPU engine goes past upstream's 300 s default. It is set to **660, one round-trip outside `SHIM_TIMEOUT=600`** in `scripts/frank-tts.env` (frank-tts's own per-request ceiling): nested budgets go inside-out — the same rule that file states for `LLAMA_TIMEOUT` — so a frank-tts give-up arrives labelled as frank-tts's error instead of the worker's own timeout, and re-measuring frank-tts cannot silently strand this number below it. An abandoned request does not cancel the engine process; it only strands the episode in "generating".
|
||||||
|
|
||||||
## Secrets, and the difference between blank and unset
|
## Secrets, and the difference between blank and unset
|
||||||
|
|
||||||
|
|||||||
@@ -2345,8 +2345,13 @@ every restart since the cutover.
|
|||||||
to `TTS_BATCH_SIZE=5` × `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` — a five-way
|
to `TTS_BATCH_SIZE=5` × `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` — a five-way
|
||||||
self-DoS. Both are pinned to `1` in `scripts/open-notebook.env`, and the
|
self-DoS. Both are pinned to `1` in `scripts/open-notebook.env`, and the
|
||||||
worker knob is only real because the image's own supervisord expands
|
worker knob is only real because the image's own supervisord expands
|
||||||
`${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5}` (`supervisord.conf:25`). Both
|
`${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5}` (`supervisord.conf:25`). The TTS clock
|
||||||
`ESPERANTO_*` timeouts go above upstream's 300 s for CPU synthesis.
|
goes to **660 — one round-trip outside** frank-tts's own `SHIM_TIMEOUT=600`, so
|
||||||
|
a shim give-up reads as the shim's error rather than the worker's; the seam
|
||||||
|
asserts the direction (`>`) against the frank-tts env instead of restating a
|
||||||
|
number. `ESPERANTO_LLM_TIMEOUT` stays at the app's 180 s default: frank-zealot
|
||||||
|
answers an outline or a transcript well inside it, and upstream says to keep it
|
||||||
|
under 600 so a slow call fails with a name on it.
|
||||||
- **What a credential needs in order to exist:** `OPEN_NOTEBOOK_ENCRYPTION_KEY`
|
- **What a credential needs in order to exist:** `OPEN_NOTEBOOK_ENCRYPTION_KEY`
|
||||||
must be set before the UI can save *any* provider credential, and
|
must be set before the UI can save *any* provider credential, and
|
||||||
`OPEN_NOTEBOOK_PASSWORD` must stay **absent** to remain auth-off — the app
|
`OPEN_NOTEBOOK_PASSWORD` must stay **absent** to remain auth-off — the app
|
||||||
|
|||||||
@@ -130,15 +130,23 @@ DB_PIN="sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723"
|
|||||||
# request on a box that also serves two LLM endpoints.
|
# request on a box that also serves two LLM endpoints.
|
||||||
grep -qxF 'TTS_BATCH_SIZE=1' "$ENVF"
|
grep -qxF 'TTS_BATCH_SIZE=1' "$ENVF"
|
||||||
grep -qxF 'OPEN_NOTEBOOK_WORKER_MAX_TASKS=1' "$ENVF"
|
grep -qxF 'OPEN_NOTEBOOK_WORKER_MAX_TASKS=1' "$ENVF"
|
||||||
# Per dialogue line, matched to frank-tts's own per-request ceiling so the
|
# Per dialogue line, and strictly beyond frank-tts's own per-request
|
||||||
# worker never abandons a synthesis the shim is still finishing.
|
# ceiling: nested budgets go inside-out (the rule scripts/frank-tts.env
|
||||||
|
# states for LLAMA_TIMEOUT), so a shim give-up surfaces as frank-tts's
|
||||||
|
# error instead of the worker's own timeout. The seam reads the ceiling
|
||||||
|
# rather than restating it, so re-measuring frank-tts cannot leave this
|
||||||
|
# file silently behind.
|
||||||
tts="$(sed -n 's/^ESPERANTO_TTS_TIMEOUT=//p' "$ENVF")"
|
tts="$(sed -n 's/^ESPERANTO_TTS_TIMEOUT=//p' "$ENVF")"
|
||||||
shim="$(sed -n 's/^SHIM_TIMEOUT=//p' "$REPO_ROOT/scripts/frank-tts.env")"
|
shim="$(sed -n 's/^SHIM_TIMEOUT=//p' "$REPO_ROOT/scripts/frank-tts.env")"
|
||||||
[ -n "$tts" ] || { echo "no ESPERANTO_TTS_TIMEOUT in the env file"; false; }
|
[ -n "$tts" ] || { echo "no ESPERANTO_TTS_TIMEOUT in the env file"; false; }
|
||||||
[ -n "$shim" ] || { echo "no SHIM_TIMEOUT in scripts/frank-tts.env"; false; }
|
[ -n "$shim" ] || { echo "no SHIM_TIMEOUT in scripts/frank-tts.env"; false; }
|
||||||
[ "$tts" = "$shim" ] || {
|
[ "$tts" -gt "$shim" ] || {
|
||||||
echo "ESPERANTO_TTS_TIMEOUT=$tts does not match frank-tts SHIM_TIMEOUT=$shim"; false; }
|
echo "ESPERANTO_TTS_TIMEOUT=$tts is not beyond frank-tts SHIM_TIMEOUT=$shim"; false; }
|
||||||
grep -qE '^ESPERANTO_LLM_TIMEOUT=[0-9]+$' "$ENVF"
|
# Upstream: keep the scriptwriting call under the web UI's ~600 s patience,
|
||||||
|
# or the failure arrives as a UI timeout with no name on it.
|
||||||
|
llm="$(sed -n 's/^ESPERANTO_LLM_TIMEOUT=//p' "$ENVF")"
|
||||||
|
[[ "$llm" =~ ^[0-9]+$ ]] && [ "$llm" -lt 600 ] || {
|
||||||
|
echo "ESPERANTO_LLM_TIMEOUT=$llm is not a number under 600"; false; }
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "secrets stay declared-but-empty in the tracked env file" {
|
@test "secrets stay declared-but-empty in the tracked env file" {
|
||||||
|
|||||||
Reference in New Issue
Block a user