serverAppConfigs/serverAppConfigs-docs/Dankclaw/Open Notebook.md

type, description, tags, timestamp

type	description	tags	timestamp
reference	The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house.	dankclaw open-notebook podcasts quadlet	2026-10-18

Open Notebook

open-notebook (lfnovo/open-notebook) is the notebook layer that turns existing notes and sources into a spoken episode: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by frank-zealot, the voices are rendered by frank-tts, and nothing leaves the LAN.

It is NOT deployed. This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no open-notebook or open-notebook-surrealdb container exists on Dankclaw (podman ps -a census, 2026-10-18), and the homepage card ships wip: "down" with showLink: false, so the card is hidden and reported wip-down. Deploying it is a deliberate next step, not a pending one.

Host: Dankclaw (rootless Podman, user dank) — Ports: 8502 UI and 5056 API, published on the LAN, no vhost and no Caddy route — Deployer: open-notebook (one service, two units) — Docs: https://github.com/lfnovo/open-notebook

Facts (verified 2026-10-18 against the pinned images, upstream source, and the live host)

Item	Value
App image	docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212 (open-notebook.container:79) — the tag upstream's own docker-compose.yml:27 pins for the multi-container shape. v1-latest is a moving tag, so the manifest-list digest (resolved from the registry 2026-10-18) is what makes this reproducible
Database image	docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723 (open-notebook-surrealdb.container:66) — that digest is SurrealDB 2.7.0 (image built 2026-09-23; measured by running the digest: surreal version)
Do NOT use	the -single tags (v1-latest-single). That variant bundles SurrealDB inside the app image and is deprecated upstream; here it would give the host a second, hidden database
Ports	container 8502 (Next.js UI) + 5055 (FastAPI API) → published 0.0.0.0:8502:8502 and 0.0.0.0:5056:5055 (open-notebook.container:83-84). Host 5055 is taken: seerr publishes 127.0.0.1:5055 (dankclaw/seerr/quadlet/seerr.container, and ss -ltn on Dankclaw shows 127.0.0.1:5055 live, 2026-10-18). The API therefore leaves on 5056, which is what forces API_URL (below)
Database port	8000 inside its own namespace, never published (open-notebook-surrealdb.container has no PublishPort). Upstream publishes 127.0.0.1:8000 for debugging (docker-compose.yml:18); this unit does not, and the seam asserts that it stays that way
Data	open-notebook-data:/app/data (named volume: uploads, LangGraph SQLite checkpoints, extraction caches) plus the interim host bind /home/dank/media/podcasts:/app/data/podcasts (open-notebook.container:87-88); database in the open-notebook-db named volume
Networks	app: dankclaw_default first, then the private systemd-open-notebook (10.89.4.0/24, open-notebook.network:33); database: the private bridge only. Order is load-bearing — the generated --network flags keep file order, so the default route rides the network where host.containers.internal is proven
Units	dankclaw/open-notebook/quadlet/open-notebook.container → open-notebook.service; open-notebook-surrealdb.container → open-notebook-surrealdb.service; both Restart=always, RestartSec=10 / 5, [Install] WantedBy=default.target — boot-persistent per Quadlet Lifecycle (generated units cannot be enabled on this host; the [Install] section is what the v1 generator honors at user-manager start)
Health	app: curl -fsS http://127.0.0.1:5055/health (open-notebook.container:89); database: /surreal is-ready (open-notebook-surrealdb.container:73)
Env	scripts/open-notebook.env (repo, knobs) then /home/dank/Containers/open-notebook/env (host, secrets, mode 600) — two EnvironmentFile= lines per unit, repo file first because the last file wins
Deployer	deploy_open_notebook in scripts/deploy-remote.sh: seed secrets → ensure both digest pins are in local storage → install both units + the network → mkdir -p /home/dank/media/podcasts → restart database, wait for its container health → restart app → wait_for_health open-notebook http://127.0.0.1:5056/health 180

What the image actually runs

One container, three processes under supervisord (supervisord.conf upstream): uvicorn api.main:app --port 5055 (:12), the job worker (:25), and the Next.js frontend (:39). Two consequences the config depends on:

• One unit owns the whole app. There is no separate worker or frontend container to scale or restart, and the frontend refuses to start until the API answers (wait-for-api.sh && node server.js, supervisord.conf:39) — so gating the API gates the UI.
• The worker's concurrency comes from the environment, not a flag baked into the image: --max-tasks ${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5} (supervisord.conf:25). Setting it in the env file is a real change, not decoration.

Also from the image itself: WORKDIR /app and EXPOSE 8502 5055 (Dockerfile:84, :126), no USER line (the app runs as container root, which in this host's rootless map lands as dank — the Seerr UID note measured 0 → 1000 on this machine), and curl and ffmpeg installed in the runtime layer (Dockerfile:74-76) — curl makes the health probe possible without a shell wrapper, ffmpeg is how the per-line clips become one file.

DATA_FOLDER is hard-coded "./data" (open_notebook/config.py:5) and PODCASTS_FOLDER is "./data/podcasts" (:19); the database stores each episode's audio path relative to that folder (episodes/<uuid>/audio/<uuid>.mp3), and the download route is GET /api/podcasts/episodes/{id}/audio. That is why covering one subdirectory with a bind mount is sufficient — nothing in the database has to change if the mount moves.

Provisioning the providers (after first boot; UI or REST, never a unit)

Credentials are stored in the database, encrypted, and are not expressible in env or a Quadlet file. The sequence is one-time, in Manage → Models, and both entries use provider type OpenAI Compatible (upstream: any server that implements /v1/audio/speech works this way — docs/5-CONFIGURATION/local-tts.md:5).

1. TTS credential — base URL http://frank-tts:7772/v1. The /v1 is part of the URL: the client posts to <base_url>/audio/speech, so a base URL without /v1 404s on every dialogue line (local-tts.md:72-80, the base-URL table, always carries /v1). frank-tts resolves by container name because the app is attached to dankclaw_default, the network frank-tts's unit joins (dankclaw/voiceservice/quadlet/frank-tts.container:51-52). If frank-tts is ever moved off that network, the published-port form is http://192.168.0.143:7772/v1 — a published port is dialable at the host's LAN address from any netns. The API-key field stays empty; frank-tts does not authenticate.
2. LLM credential — base URL http://host.containers.internal:7771/v1, model row frank-zealot. frank-zealot is a host process on 0.0.0.0:7771 (ss -ltn on Dankclaw shows the listener with no container owning it, 2026-10-18), and on this netavark host a bridge container reaches the host's LAN IP only at a published port — which is why Prometheus scrapes the same endpoint as host.containers.internal:7771 (shared/prometheus/prometheus.yml:89-111, "not their own LAN addresses"). See Container Network Namespaces. http://192.168.0.143:7771/v1 is the variant to try only if the gateway name ever stops resolving; it is expected to be refused from a bridge netns.
3. Model rows are created by name. Discovery is optional (see the next section): type the model id and confirm the add. One base URL per credential, so the TTS and LLM credentials are necessarily two separate configurations.

Verify reachability before saving, from inside the container, because both addresses are netns-dependent:

podman exec open-notebook curl -sS http://frank-tts:7772/health
podman exec open-notebook curl -sS http://host.containers.internal:7771/v1/models

The blocker moved; it did not vanish. frank-tts and frank-tts-engine are deployed (2026-10-18) and answer on 7772, so the voice leg works end to end — podman exec open-notebook curl -sS http://frank-tts:7772/health is the check to run the day the app exists. What still blocks a render is open-notebook itself: not installed on this host yet. When it is, the two provider rows above are the only wiring, and the seeded speaker profiles carrying OpenAI voice names are the first trap.

Why frank-zealot and not frank-dragoon: podcast scriptwriting asks the model for a JSON-schema response (with a plain-JSON fallback). The decision rests on each endpoint's documented request surface (frank-zealot, frank-dragoon): zealot's is a known-good target for that call shape, dragoon's structured-output behaviour is unverified, so it is the unproven alternative, not the fallback. ESPERANTO_LLM_TIMEOUT=180 (the app's own default) covers an outline or transcript call on zealot comfortably; upstream's note is to keep it under 600 so a slow call fails with a clear error instead of a UI timeout.

Speakers and voices: 1–4, and the IDs are free text

A speaker profile carries one to four speakers (docs/3-USER-GUIDE/creating-podcasts.md:27), each with a personality and a Voice ID that is typed exactly and passed through to the TTS request untouched (:29) — there is no whitelist, no enum, no validation against a voice list. So the fleet's voice-pack names work verbatim:

Speaker	Voice ID to type
Speaker 1	narrator1
Speaker 2	narrator2
Speaker 3	narrator3

Those are the pack names frank-tts resolves from /home/dank/voices (scripts/frank-tts.env header: a voice is real exactly when /voices/<name>/reference.wav exists). Two traps, both from upstream's shipped defaults:

• The three seeded speaker profiles carry OpenAI voice names (nova, alloy, …) and no voice model at all — they show a "Setup required" badge, and generation fails with a voice error until they are edited (creating-podcasts.md:13, :33; local-tts.md:89). Add a narrator1/narrator2 pair (or a third/fourth) rather than editing the seeded ones in place.
• Two speakers with the same voice are hard to follow (creating-podcasts.md:89). Distinct packs per speaker is the point of the 1–4 shape.

Two buttons that fail against frank-tts, and why that is cosmetic

• Test Connection probes GET <base_url>/models (open_notebook/ai/connection_tester.py:208-229). frank-tts serves exactly two GET routes — /health and /v1/audio/voices — and 404s every other path (dankclaw/voiceservice/app.py:228-238). The button reports a failure against a healthy server.
• Discover Models does the same GET <base>/models (open_notebook/ai/model_discovery.py:26-28) and finds nothing to list.
• The TTS test path also synthesises with a hard-coded voice alloy for openai_compatible (connection_tester.py:328-336), which is not a fleet voice pack and would be refused even if the probe reached frank-tts.

None of it blocks anything: model rows are added by name ("If a model isn't listed, type its id and click Add", local-tts.md:82) and the Voice ID field is free text. The failure is a missing /models route on a deliberately minimal shim, not a broken credential. The honest end-to-end test is a real generation job, or the podman exec … curl /v1/audio/speech call frank-tts's own seam documents.

The CPU guard, and why both numbers are 1

Defaults are TTS_BATCH_SIZE=5 and OPEN_NOTEBOOK_WORKER_MAX_TASKS=5 (docs/5-CONFIGURATION/environment-reference.md:129-130) — five parallel synthesis requests from five parallel jobs. frank-tts spawns an engine process per request (about 8 GiB resident, 8 threads, from the frank-tts CPU trial of 2026-10-18; scripts/frank-tts.env pins LLAMA_TTS_THREADS=8), on a machine that also serves frank-dragoon and frank-zealot. Five of those is ~40 GiB and forty threads competing with two LLM endpoints for the same cores. scripts/open-notebook.env:87-88 sets both to 1: one episode at a time, one voice line at a time. Upstream's own guidance is the same direction ("Set 1 for a local model on a single GPU", "Lower it … for rate-limited or single-threaded TTS servers").

The clock next to it: ESPERANTO_TTS_TIMEOUT (:91) is patience per dialogue line, not per episode — a three-minute episode is dozens of separate POSTs, and the tail of a long turn on a CPU engine goes past upstream's 300 s default. It is set to 660, one round-trip outside SHIM_TIMEOUT=600 in scripts/frank-tts.env (frank-tts's own per-request ceiling): nested budgets go inside-out — the same rule that file states for LLAMA_TIMEOUT — so a frank-tts give-up arrives labelled as frank-tts's error instead of the worker's own timeout, and re-measuring frank-tts cannot silently strand this number below it. An abandoned request does not cancel the engine process; it only strands the episode in "generating".

Secrets, and the difference between blank and unset

OPEN_NOTEBOOK_PASSWORD unset means authentication off (environment-reference.md:46) — every API route, including the episode-audio download, answers anyone who can reach the published port. That is a real exposure on a LAN-published service with no vhost and no guard chain in front of it, which is why deploy_open_notebook refuses to start either unit unless the seeded host file carries a value for all four secret keys (OPEN_NOTEBOOK_ENCRYPTION_KEY, OPEN_NOTEBOOK_PASSWORD, SURREAL_PASSWORD, SURREAL_PASS). OPEN_NOTEBOOK_ENCRYPTION_KEY is separately load-bearing: it is required to save any provider credential, and changing it later makes stored keys undecryptable (:44).

The tracked file declares the secrets empty and the host file supplies them; podman applies --env-file arguments in order and the last one wins (verified against the generated ExecStart line, 2026-10-18). Reordering the two EnvironmentFile= lines in a unit would blank every credential — that is the trap the seam's ordering assertion exists for.

The database has the same trap in a different spelling: the app reads SURREAL_PASSWORD, the surreal CLI reads SURREAL_PASS (its start --help prints [env: SURREAL_PASS=]; measured against the pinned digest). One value, written twice, by the seed function — so the pair cannot be given different root passwords. Both arrive through the environment, not through Exec=: --pass on the command line would put the database password in /proc/<pid>/cmdline, readable by any ps on the host. Measured with the pinned image: env-only credentials produce Initialising credentials user=…, the root user is created on first boot, a correct login authenticates and a wrong password is refused — no --user/--pass flags needed, and no --bind either, because the image's own SURREAL_BIND=0.0.0.0:8000 is what makes the RPC reachable from another container at all.

Episode destination: interim, and reversible (issue #7)

Episode files currently exist in two places by design. The app's own state stays in the open-notebook-data named volume; finished audio additionally lands on the host, because /home/dank/media/podcasts is bind-mounted over /app/data/podcasts (open-notebook.container:88). That is the interim answer to where episodes live: the destination is an open decision tracked as Gitea issue #7, labelled human-only — agents must not work it. Until Frank settles it, episode MP3s are ordinary files in his home directory, readable without podman unshare, which the database volume would require (measured with the pinned SurrealDB image).

Reversibility is one line: delete the Volume=/home/dank/media/podcasts:/app/data/podcasts line and the episodes go back to living solely inside the named volume. Nothing else in the config assumes the bind. See Storage Policy for the repo's storage conventions this interim state deliberately bends.

Deploying, and the digest-pull step

# on Dankclaw, as dank — the pins are Pull=never, so fetch them first
podman pull docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212

deploy_open_notebook runs the same two pulls when the image is absent from local storage, so the manual step is only needed to pre-seed an offline host or to bump a pin. Then scripts/deploy.sh --target dankclaw --service open-notebook. Both Pull=never and the pin comment live in the unit files; upgrading means editing the digest, not repulling a tag — see Deployment Flow.

Rollback

• Nothing deployed (today's state): revert the commits. There is no host state to undo — no unit, no container, no volume, no /home/dank/media/podcasts directory beyond what the deployer would create.
• Deployed: systemctl --user stop open-notebook.service open-notebook-surrealdb.service, then move the three files (open-notebook.container, open-notebook-surrealdb.container, open-notebook.network) out of ~/.config/containers/systemd/ into ~/.AGENTTRASH/ — never rm — and systemctl --user daemon-reload. The generated units disappear with their sources.
• Kept on purpose: the open-notebook-data and open-notebook-db named volumes and /home/dank/media/podcasts. Episode audio and the notebook database are user data; removing them is a separate, deliberate act, and episode files are host files precisely so they can outlive the container.
• Homepage: the card is wip: "down" + showLink: false, so retiring the service needs no card surgery beyond setting it back to that state.

Citations

• Repo: dankclaw/open-notebook/quadlet/{open-notebook.container,open-notebook-surrealdb.container,open-notebook.network}, scripts/open-notebook.env, dankclaw/.env.example, scripts/deploy-plan.py, scripts/deploy-remote.sh (seed_open_notebook_env, ensure_pinned_image, deploy_open_notebook), scripts/deploy.sh, scripts/test-changed.sh, tests/open_notebook.bats, shared/homepage/services.json, dankclaw/seerr/quadlet/seerr.container (the 5055 collision), dankclaw/voiceservice/{quadlet/frank-tts.container,app.py}, scripts/frank-tts.env, shared/prometheus/prometheus.yml
• Upstream lfnovo/open-notebook @ main, read 2026-10-18: docker-compose.yml:3,10,11,18,27,29-30,39-43, Dockerfile:69-76,84,126,140,158, supervisord.conf:12,23-25,39, api/main.py:73,227-230,239-241,414, api/credentials_service.py:418-420,552, api/routers/credentials.py:426-429,433-437, open_notebook/config.py:5,19, open_notebook/podcasts/audio_paths.py, open_notebook/ai/connection_tester.py:208-238,326-336, open_notebook/ai/model_discovery.py:26-28, docs/3-USER-GUIDE/creating-podcasts.md:13,27,29,33,89, docs/5-CONFIGURATION/environment-reference.md:44,46,106,110,129,130, docs/5-CONFIGURATION/local-tts.md:5,45,72-82,89
• Registry (2026-10-18): lfnovo/open_notebook tags (v1-latest, latest, the -single set) and manifest-list digests; surrealdb/surrealdb v2 → 2.7.0
• Local measurements (2026-10-18, workstation, pinned digests): surreal start --help (credential and bind options carry [env: SURREAL_*]); running the pinned SurrealDB digest with env-only credentials (credential init in the log, correct login authenticates, wrong password refused, /surreal is-ready → OK exit 0, named volume owned by the mapped uid); the v1-compatible Quadlet generator dry-run of all three unit files (two --env-file in order, two --network in order, named volume + host bind, both health probes, --pull never)
• Host evidence (Dankclaw, 2026-10-18, read-only): ss -ltn (127.0.0.1:5055 held by seerr, 0.0.0.0:7771 a host listener, nothing on 7772/8502/5056/8000), podman ps -a (no open-notebook container, dankclaw_default present), subnet registry from dankclaw/open-webui/quadlet/open-webui.network