Files
serverAppConfigs/serverAppConfigs-docs/Dankclaw/Open Notebook.md
T
Frank 0930510f9b docs(vault): the embedding lane has a consumer, and two shapes that bite
open-notebook's embedding default now points at qwen3-embedding-0.6b on frank-dragoon's XDNA2 lane, proven through the app's own paths (model test 1024 dims, worker 'Successfully embedded', vector search ranking two unrelated notes 0.659/0.292 and 0.906/0.318). Records the two app-side traps: defaults store the model ID and a name is accepted then fails in the worker, and POST /api/embed 500s on an upstream RecordID typing bug after the vector is already written. frank-dragoon's lane section moves from 'off until the mount fix' to armed-and-serving with the boot-ledger contract and the measured idle RSS; the page's stale 'not on dankclaw_default' reachability claims are retired.
2026-10-19 22:46:57 -07:00

178 lines
32 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
type: reference
description: "The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house."
tags:
- dankclaw
- open-notebook
- podcasts
- quadlet
timestamp: 2026-10-19
---
# Open Notebook
open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook)) is the notebook layer that turns existing notes and sources into a **spoken episode**: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by **frank-archon**, the voices are rendered by **frank-tts**, and nothing leaves the LAN.
**Deployed 2026-10-19.** Both units installed and `active`, both containers healthy, SurrealDB migrated to v25 on first boot, and the first episode rendered end to end: a note in, an outline, 23 dialogue turns, 23 clips, and one **747-second MP3 on the host bind, produced in 301 seconds** — about 2.5× real time with every concurrency knob at 1. The download route answers `200 audio/mpeg` with the bearer token and **401 without it**. It is guarded like every other user service — `https://notebook.franksdank.cloud`, Anubis → Authentik → the app's own login — and the API is no longer published on the LAN at all.
**Host:** Dankclaw (rootless Podman, user `dank`) — **User path:** `https://notebook.franksdank.cloud` (guarded) — **Published:** `0.0.0.0:8502` (the only upstream Caddy dials) and `127.0.0.1:5056` (the deployer's health gate, nothing else) — **Deployer:** `open-notebook` (one service, two units) — **Docs:** https://github.com/lfnovo/open-notebook
## Facts (images and upstream source verified 2026-10-18; live-host columns re-read 2026-10-19 after deploy)
| Item | Value |
|------|-------|
| App image | `docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212` (`open-notebook.container:79`) — the tag upstream's own `docker-compose.yml:27` pins for the multi-container shape. `v1-latest` is a moving tag, so the manifest-list digest (resolved from the registry 2026-10-18) is what makes this reproducible |
| Database image | `docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723` (`open-notebook-surrealdb.container:66`) — that digest is **SurrealDB 2.7.0** (image built 2026-09-23; measured by running the digest: `surreal version`) |
| Do NOT use | the `-single` tags (`v1-latest-single`). That variant bundles SurrealDB inside the app image and is deprecated upstream; here it would give the host a second, hidden database |
| Ports | container 8502 (Next.js UI) + 5055 (FastAPI API) → published `0.0.0.0:8502:8502` and `127.0.0.1:5056:5055` (`open-notebook.container`). **Host 5055 is taken**: seerr publishes `127.0.0.1:5055` (`dankclaw/seerr/quadlet/seerr.container`, and `ss -ltn` on Dankclaw shows it live). The API therefore leaves on 5056, and on loopback only: the vhost is its sole user path and the image's own Next.js layer rewrites `/api/*` to it, so nothing outside the container needs it |
| User path | `https://notebook.franksdank.cloud` → `anubis:8923` → post-auth listener `@notebook` → `192.168.0.143:8502` (`dankclaw/Caddyfile`); Authentik provider `notebook-proxy` (`external_host` the vhost with its trailing slash, `internal_host http://caddy:8017`); LAN record in `dankclaw/pihole/addn-hosts` |
| Database port | 8000 inside its own namespace, **never published** (`open-notebook-surrealdb.container` has no `PublishPort`). Upstream publishes `127.0.0.1:8000` for debugging (`docker-compose.yml:18`); this unit does not, and the seam asserts that it stays that way |
| Data | `open-notebook-data:/app/data` (named volume: uploads, LangGraph SQLite checkpoints, extraction caches) **plus** the interim host bind `/home/dank/media/podcasts:/app/data/podcasts` (`open-notebook.container:87-88`); database in the `open-notebook-db` named volume |
| Networks | app: `dankclaw_default` first, then the private `systemd-open-notebook` (10.89.4.0/24, `open-notebook.network:33`); database: the private bridge only. Order is load-bearing — the generated `--network` flags keep file order, so the default route rides the network where `host.containers.internal` is proven |
| Units | `dankclaw/open-notebook/quadlet/open-notebook.container` → `open-notebook.service`; `open-notebook-surrealdb.container` → `open-notebook-surrealdb.service`; both `Restart=always`, `RestartSec=10` / `5`, `[Install] WantedBy=default.target` — boot-persistent per [[Quadlet Lifecycle]] (generated units cannot be `enable`d on this host; the `[Install]` section is what the v1 generator honors at user-manager start) |
| Health | app: `curl -fsS http://127.0.0.1:5055/health` (`open-notebook.container:89`); database: `/surreal is-ready` (`open-notebook-surrealdb.container:73`) |
| Env | `scripts/open-notebook.env` (repo, knobs) then `/home/dank/Containers/open-notebook/env` (host, secrets, mode 600) — two `EnvironmentFile=` lines per unit, repo file **first** because the last file wins |
| Deployer | `deploy_open_notebook` in `scripts/deploy-remote.sh`: seed secrets → ensure both digest pins are in local storage → install both units + the network → `mkdir -p /home/dank/media/podcasts` → restart database, wait for its container health → restart app → `wait_for_health open-notebook http://127.0.0.1:5056/health 180` |
## The guard chain, and the single origin
`notebook.franksdank.cloud` follows the fleet's exposure policy in this order, and the order is the law ([[Network Architecture]]):
1. **Provider first.** Authentik proxy provider `notebook-proxy` existed before any route pointed at the app: `external_host https://notebook.franksdank.cloud/` (exact vhost, trailing slash), `internal_host http://caddy:8017`, the three flows copied from `embedded-outpost`, Application `Open Notebook` (`notebook-guard`), and the provider attached to the embedded outpost's provider set. Flip the vhost first and authenticated traffic lands on the outpost's error page. Verify after the change that the outpost logged no `Failed to fetch outpost from API` (the null-flow wedge) and that an already-guarded vhost still answers.
2. **Then the route, in one deploy:** the guarded site block (`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}` — the whole request, so no path ever reaches the app unchallenged) plus the post-auth `@notebook` handler on `http://:8017`, plus the LAN record in `dankclaw/pihole/addn-hosts`. The public Porkbun A record is a host action, not a repo file.
3. **One upstream, not two.** `reverse_proxy 192.168.0.143:8502`: the image's own Next.js layer rewrites `/api/*` to the FastAPI beside it (`next.config.ts`, `INTERNAL_API_URL` default `http://localhost:5055`), so the API needs no route and no outward face — hence `127.0.0.1:5056`. Caddy dials the published LAN port rather than a container name because this Caddy lives in root's Podman and cannot see a rootless container at all ([[Container Network Namespaces]]). SSE (`/api/search/ask`, chat) streams through the same bare proxy that carries OpenWebUI's chat at `ai.franksdank.cloud`; no `flush_interval` or timeout directive is needed, and adding a route timeout is precisely what would cut it.
4. **`API_URL` must name the vhost.** `GET /config` (the Next route) answers in priority order: `API_URL` env → autodetect from `Host` + `X-Forwarded-Proto` → `http://localhost:5055`. Autodetect appends **`:5055`**, a port that belongs to seerr on this host, so leaving `API_URL` unset hands the browser an address nothing answers and every call fails under a UI that loaded fine. `CORS_ORIGINS` is set to the same single origin only to keep the app from defaulting to `*`. `scripts/open-notebook.env` carries both.
The app's own login stays on behind the chain. It is not redundant: publishing the API on loopback does **not** hide it from this host's other containers, which can dial `open-notebook:5055` by name on `dankclaw_default`. The chain decides *who from outside* gets in; `OPEN_NOTEBOOK_PASSWORD` decides what anything that already reached the port may do.
## What the image actually runs
One container, three processes under supervisord (`supervisord.conf` upstream): `uvicorn api.main:app --port 5055` (`:12`), the job worker (`:25`), and the Next.js frontend (`:39`). Two consequences the config depends on:
- **One unit owns the whole app.** There is no separate worker or frontend container to scale or restart, and the frontend refuses to start until the API answers (`wait-for-api.sh && node server.js`, `supervisord.conf:39`) — so gating the API gates the UI.
- **The worker's concurrency comes from the environment**, not a flag baked into the image: `--max-tasks ${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5}` (`supervisord.conf:25`). Setting it in the env file is a real change, not decoration.
Also from the image itself: `WORKDIR /app` and `EXPOSE 8502 5055` (`Dockerfile:84`, `:126`), **no `USER` line** (the app runs as container root, which in this host's rootless map lands as `dank` — the [[Seerr]] UID note measured `0 → 1000` on this machine), and `curl` **and** `ffmpeg` installed in the runtime layer (`Dockerfile:74-76`) — curl makes the health probe possible without a shell wrapper, ffmpeg is how the per-line clips become one file.
`DATA_FOLDER` is hard-coded `"./data"` (`open_notebook/config.py:5`) and `PODCASTS_FOLDER` is `"./data/podcasts"` (`:19`); the database stores each episode's audio path **relative** to that folder (`episodes/<uuid>/audio/<uuid>.mp3`), and the download route is `GET /api/podcasts/episodes/{id}/audio`. That is why covering one subdirectory with a bind mount is sufficient — nothing in the database has to change if the mount moves.
## What is provisioned (REST, 2026-10-19)
Credentials live in the database, encrypted with `OPEN_NOTEBOOK_ENCRYPTION_KEY`, and are **not expressible in env or a Quadlet file** — the UI (**Manage → Models**) or the REST API are the only doors. Both use provider type `openai_compatible`, because any server that answers `/v1/audio/speech` or `/v1/chat/completions` works through it (`docs/5-CONFIGURATION/local-tts.md:5`).
| Record | Value |
|--------|-------|
| Credential `frank-tts` | modality `text_to_speech`, base URL `http://frank-tts:7772/v1`, no API key (the shim does not authenticate) |
| Credential `frank-archon` | modality `language`, base URL `http://192.168.0.235:7770/v1` |
| Credential `frank-dragoon NPU` | modality `embedding`, base URL `http://frank-dragoon-halogen:8731/v1`, dummy API key (the lane does not authenticate) — a container name on `dankclaw_default`, which is the address shape that clears the URL guard below |
| Model row `voxcpm2` | type `text_to_speech` on the frank-tts credential. The name is decorative: frank-tts ignores the request's `model` field and synthesizes whatever `voice` names ([[Frank TTS]]) |
| Model row `frank-archon` | type `language`; `PUT /api/models/defaults` points chat, transformation and large-context at it |
| Model row `qwen3-embedding-0.6b` | type `embedding` on the NPU-lane credential; `default_embedding_model` points at it, so every note save embeds through the XDNA2 lane ([[frank-dragoon]]) |
| Speaker profile `dank-duo` | two speakers, Frank → `voice_id: narrator1`, Comrade → `narrator2`, `voice_model` = the TTS row |
| Episode profile `dank-daily` | `num_segments: 4`, `speaker_config` = `dank-duo`, outline and transcript LLM = the archon row, briefing that ends "Never invent numbers." |
Order that works over REST: credentials → model rows (`POST /api/models` with `credential: <credential id>`) → `PUT /api/models/defaults` → speaker profile (`voice_model: <model id>`) → episode profile (`speaker_config` / `outline_llm` / `transcript_llm`). `POST /api/models/{id}/test` proves each row: the language row answered with a real completion from frank-archon; the TTS row reached frank-tts and came back `unknown voice 'alloy'; available voices: narrator1, narrator2, narrator3` — the app's own test path hard-codes an OpenAI voice, so that message *is* the proof the leg works.
**`POST /api/podcasts/generate` takes profile NAMES, not record ids.** `episode_profile: "dank-daily"` works; the `episode_profile:…` id returns a bare `500 Failed to submit podcast generation job`, and the actual reason (`Episode profile '<id>' not found`) appears only in `podman logs open-notebook`.
**The scriptwriter is frank-archon, not the planned frank-zealot, because two rules collide on the address.** Measured from inside the app's netns 2026-10-19:
- `192.168.0.143:7771` (frank-zealot, a host-netns listener) and `192.168.0.143:7770` (frank-dragoon, a **root**-published port) both refuse the connection; `host.containers.internal:7771` answers. Full census: [[Container Network Namespaces]].
- The app refuses to **store** that one working address: its URL guard rejects any base URL that resolves inside link-local `169.254.0.0/16` (`open_notebook/utils/url_validation.py`, "Link-local addresses are not allowed (metadata endpoints)"), unconditionally — no env var touches it, and `AddHost=` cannot dodge it because the check runs on the resolved address.
`http://192.168.0.235:7770/v1` — another machine, plain RFC1918 — was, when this was first written, the only fleet LLM address that passed both tests; a container name on `dankclaw_default` passes them as well (the embedding credential below is stored that way), but frank-archon stays the scriptwriter for the reason in the next paragraph. Cost: frank-archon runs `--max-num-seqs 2` and the operator session itself lives there, so a render is a slow-down for whoever else is talking to that endpoint, not a stall; `ESPERANTO_LLM_TIMEOUT=180` covers an outline or transcript call comfortably.
**frank-dragoon can write transcripts, but only greedily — and open-notebook cannot ask for greedy.** Measured 2026-10-19 against the live endpoint: `response_format` as `json_schema` or `json_object` returns schema-valid JSON at `temperature: 0` and a **400** at any sampled temperature — halogen decodes greedy under a schema, and because its sampler default is server-side, omitting `temperature` is sampled, not greedy. That 400 is the trap: esperanto's `is_json_schema_unsupported_error` matches the words *not implemented* in it, so `_SchemaFallbackModel` in `podcast_creator/nodes.py` reads it as "this endpoint has no schema support", downgrades to `json_object`, gets the same 400, and **fails the run** — esperanto's own rescue (drop `response_format` entirely) fires only on the different message `'response_format.type' must be 'json_schema'`. The full law is on [[frank-dragoon]]. And the fix is not reachable from here: a model record is `{credential, name, provider, type}` and a model response adds only timestamps (`GET /openapi.json`, `ModelCreate`/`ModelResponse`) — **there is no per-row config field**, no way to attach `temperature: 0` to one row from the UI or the API. Making `frank-dragoon` the writer would therefore need something in the path that rewrites request bodies, which is a heavier tenant than the contention it removes. **frank-archon stays the scriptwriter.** Reachability stopped being the objection on 2026-10-19 — the endpoint joined `dankclaw_default`, which is exactly why the embedding lane below is dialable by name — but no network change reaches the no-per-row-config wall.
**Embeddings run on the XDNA2 lane — provisioned 2026-10-19.** `qwen3-embedding-0.6b` answers `/v1/embeddings` at `http://frank-dragoon-halogen:8731/v1`, the flash container's own bridge address; a container name on `dankclaw_default` resolves to `10.89.1.39`, plain RFC1918, so it clears the URL guard that ruled out `host.containers.internal` above. The lane's shape (one program, the token ceiling, what it costs the Flash model) is owned by [[frank-dragoon]]; the two shapes that bite on the app side are open-notebook's own:
- **`PUT /api/models/defaults` stores the record ID, not the name.** `"default_embedding_model": "qwen3-embedding-0.6b"` is accepted and echoed back, and then every embed dies in the worker with `Model with ID qwen3-embedding-0.6b not found`. The value has to be the row's `model:…` id — the other defaults in that same object are ids, so mirror them.
- **`POST /api/embed` answers 500 while succeeding.** Upstream's `EmbedResponse.command_id` declares `str` and the worker hands it a `RecordID`, so FastAPI raises on serialization *after* the vector is written. The truth is the worker line `Successfully embedded note … in 0.19s`, never the HTTP code. Saving a note (`POST /api/notes`) submits `embed_note` by itself, so that route is only needed to backfill old rows.
Proof from the app's own paths, 2026-10-19: `POST /api/models/{id}/test` → `Embedding dimensions: 1024`; two notes with nothing in common, then `POST /api/search` with `type: "vector"` (the enum is `text` or `vector`; the score field is `similarity`) — "a yellow waterbird beside a glowing beacon tower" put the grebe note at **0.659** and the boiler note at 0.292, and "quarterly depreciation schedule for industrial boilers" inverted it to **0.906** / 0.318. Embed cost 0.19–0.43 s per note. Semantic search, source ingestion and notebook-context questions are live; podcast generation never needed any of it.
**Secrets never go in a command line.** The API's bearer token *is* `OPEN_NOTEBOOK_PASSWORD`, and a `-H "Authorization: Bearer …"` argument is world-readable in `/proc/<pid>/cmdline`; put it in a mode-600 curl config and pass `-K`:
```bash
PW=$(grep -m1 '^OPEN_NOTEBOOK_PASSWORD=' /home/dank/Containers/open-notebook/env | cut -d= -f2- | tr -d '"')
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: Bearer %s"\n' "$PW" > "$cfg"
curl -s -K "$cfg" http://127.0.0.1:5056/api/models | head -c 400
rm -f "$cfg"
```
## Speakers and voices: 1–4, and the IDs are free text
A **speaker profile** carries one to four speakers (`docs/3-USER-GUIDE/creating-podcasts.md:27`), each with a personality and a **Voice ID that is typed exactly and passed through to the TTS request untouched** (`:29`) — there is no whitelist, no enum, no validation against a voice list. So the fleet's voice-pack names work verbatim:
| Speaker | Voice ID to type |
|---------|------------------|
| Speaker 1 | `narrator1` |
| Speaker 2 | `narrator2` |
| Speaker 3 | `narrator3` |
Those are the pack names frank-tts resolves from `/home/dank/voices` (`scripts/frank-tts.env` header: a voice is real exactly when `/voices/<name>/reference.wav` exists). Two traps, both from upstream's shipped defaults:
- The three seeded speaker profiles carry **OpenAI voice names** (`nova`, `alloy`, …) and **no voice model at all** — they show a "Setup required" badge, and generation fails with a voice error until they are edited (`creating-podcasts.md:13`, `:33`; `local-tts.md:89`). Add a `narrator1`/`narrator2` pair (or a third/fourth) rather than editing the seeded ones in place.
- Two speakers with the same voice are hard to follow (`creating-podcasts.md:89`). Distinct packs per speaker is the point of the 1–4 shape.
## Two buttons that fail against frank-tts, and why that is cosmetic
- **Test Connection** probes `GET <base_url>/models` (`open_notebook/ai/connection_tester.py:208-229`). frank-tts serves exactly two GET routes — `/health` and `/v1/audio/voices` — and 404s every other path (`dankclaw/voiceservice/app.py:228-238`). The button reports a failure against a healthy server.
- **Discover Models** does the same `GET <base>/models` (`open_notebook/ai/model_discovery.py:26-28`) and finds nothing to list.
- The TTS test path also synthesises with a **hard-coded voice `alloy`** for `openai_compatible` (`connection_tester.py:328-336`), which is not a fleet voice pack and would be refused even if the probe reached frank-tts.
None of it blocks anything: model rows are added **by name** ("If a model isn't listed, type its id and click Add", `local-tts.md:82`) and the Voice ID field is free text. The failure is a missing `/models` route on a deliberately minimal shim, not a broken credential. The honest end-to-end test is a real generation job, or the `podman exec … curl /v1/audio/speech` call frank-tts's own seam documents.
## The CPU guard, and why both numbers are 1
Defaults are `TTS_BATCH_SIZE=5` and `OPEN_NOTEBOOK_WORKER_MAX_TASKS=5` (`docs/5-CONFIGURATION/environment-reference.md:129-130`) — five parallel synthesis requests from five parallel jobs. frank-tts spawns an engine process per request (about 8 GiB resident, 8 threads, from the frank-tts CPU trial of 2026-10-18; `scripts/frank-tts.env` pins `LLAMA_TTS_THREADS=8`), on a machine that also serves frank-dragoon and frank-zealot. Five of those is ~40 GiB and forty threads competing with two LLM endpoints for the same cores. `scripts/open-notebook.env:87-88` sets both to 1: one episode at a time, one voice line at a time. Upstream's own guidance is the same direction ("Set 1 for a local model on a single GPU", "Lower it … for rate-limited or single-threaded TTS servers").
The clock next to it: `ESPERANTO_TTS_TIMEOUT` (`:91`) is patience **per dialogue line**, not per episode — a three-minute episode is dozens of separate POSTs, and the tail of a long turn on a CPU engine goes past upstream's 300 s default. It is set to **660, one round-trip outside `SHIM_TIMEOUT=600`** in `scripts/frank-tts.env` (frank-tts's own per-request ceiling): nested budgets go inside-out — the same rule that file states for `LLAMA_TIMEOUT` — so a frank-tts give-up arrives labelled as frank-tts's error instead of the worker's own timeout, and re-measuring frank-tts cannot silently strand this number below it. An abandoned request does not cancel the engine process; it only strands the episode in "generating".
What the guard costs, measured on that first render (the deployed shape): 23 dialogue clips, one POST each, **10–11 s per clip** with both knobs at 1 → 301 s of wall clock for 747 s of speech, i.e. **2.5× real time**, box still responsive. That ratio is the budget to hold any future TTS engine to: halve the per-clip time, halve the episode.
## Secrets, and the difference between blank and unset
`OPEN_NOTEBOOK_PASSWORD` unset means **authentication off** (`environment-reference.md:46`) — every API route, including the episode-audio download, answers anyone who can reach it. Moving the API to loopback did not remove that exposure, it only removed the LAN from it: any container on `dankclaw_default` dials `open-notebook:5055` by name, so an unset password is an open episode library to every container on this host, guarded or not. That is why `deploy_open_notebook` refuses to start either unit unless the seeded host file carries a value for all four secret keys (`OPEN_NOTEBOOK_ENCRYPTION_KEY`, `OPEN_NOTEBOOK_PASSWORD`, `SURREAL_PASSWORD`, `SURREAL_PASS`). `OPEN_NOTEBOOK_ENCRYPTION_KEY` is separately load-bearing: it is **required to save any provider credential**, and changing it later makes stored keys undecryptable (`:44`).
The tracked file declares the secrets **empty** and the host file supplies them; podman applies `--env-file` arguments in order and the last one wins (verified against the generated `ExecStart` line, 2026-10-18). Reordering the two `EnvironmentFile=` lines in a unit would blank every credential — that is the trap the seam's ordering assertion exists for.
The database has the same trap in a different spelling: the app reads `SURREAL_PASSWORD`, the `surreal` CLI reads `SURREAL_PASS` (its `start --help` prints `[env: SURREAL_PASS=]`; measured against the pinned digest). One value, written twice, by the seed function — so the pair cannot be given different root passwords. Both arrive through the environment, **not** through `Exec=`: `--pass` on the command line would put the database password in `/proc/<pid>/cmdline`, readable by any `ps` on the host. Measured with the pinned image: env-only credentials produce `Initialising credentials user=…`, the root user is created on first boot, a correct login authenticates and a wrong password is refused — no `--user`/`--pass` flags needed, and no `--bind` either, because the image's own `SURREAL_BIND=0.0.0.0:8000` is what makes the RPC reachable from another container at all.
## Episode destination: interim, and reversible (issue #7)
**Episode files currently exist in two places by design.** The app's own state stays in the `open-notebook-data` named volume; everything the render produces additionally lands on the host, because `/home/dank/media/podcasts` is bind-mounted over `/app/data/podcasts` (`open-notebook.container:88`). The first render confirmed the layout: `episodes/<uuid>/clips/0001.mp3 … 00NN.mp3` (one file per dialogue line, written as the worker goes) and `episodes/<uuid>/audio/<uuid>.mp3` (the ffmpeg concatenation). A half-finished render is therefore visible and greppable from the host without `podman unshare`. That is the **interim** answer to where episodes live: the destination is an open decision tracked as **Gitea issue #7, labelled `human-only` — agents must not work it.** Until Frank settles it, episode MP3s are ordinary files in his home directory.
Reversibility is one line: delete the `Volume=/home/dank/media/podcasts:/app/data/podcasts` line and the episodes go back to living solely inside the named volume. Nothing else in the config assumes the bind. See [[Storage Policy]] for the repo's storage conventions this interim state deliberately bends.
## Deploying, and the digest-pull step
```bash
# on Dankclaw, as dank — the pins are Pull=never, so fetch them first
podman pull docker.io/surrealdb/surrealdb:v2@sha256:e5d7387e3727bbe2f7cd6d869e48cd34777272258572ead37c9656b25f745723
podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f74b9b8246eca8c5480e5dd1babfa92db1038ff2f30212
```
`deploy_open_notebook` runs the same two pulls when the image is absent from local storage, so the manual step is only needed to pre-seed an offline host or to bump a pin. Then `scripts/deploy.sh --target dankclaw --service open-notebook`. Both `Pull=never` and the pin comment live in the unit files; upgrading means **editing the digest**, not repulling a tag — see [[Deployment Flow]].
## Rollback
- **Take it off the air, in the reverse of the deploy order:** delete the guarded site block and the `@notebook` handler from `dankclaw/Caddyfile` and deploy `caddy`; then in Authentik delete the `notebook-guard` Application and the `notebook-proxy` provider and drop it from the embedded outpost's provider set. A **route with no provider 502s** while a provider with no route is inert — that asymmetry is why the chain unwinds Caddy-first.
- **Stop the app:** `systemctl --user stop open-notebook.service open-notebook-surrealdb.service`, then move the three files (`open-notebook.container`, `open-notebook-surrealdb.container`, `open-notebook.network`) out of `~/.config/containers/systemd/` into `~/.AGENTTRASH/` — never `rm` — and `systemctl --user daemon-reload`. The generated units disappear with their sources.
- **Kept on purpose:** the `open-notebook-data` and `open-notebook-db` named volumes and `/home/dank/media/podcasts`. Episode audio and the notebook database are user data; removing them is a separate, deliberate act, and episode files are host files precisely so they can outlive the container.
- **Provider credentials are database rows**, encrypted with `OPEN_NOTEBOOK_ENCRYPTION_KEY`; no repo file mirrors them. Regenerating that key while the database volume survives makes every stored credential undecryptable and forces re-entry — so a rollback that keeps the volume keeps the key file with it.
- **Homepage:** the card test in `tests/open_notebook.bats` pins the live state (`showLink: true`, link = the vhost), so retiring the service edits that test in the same commit as the card flip back to `wip: "down"`.
## Citations
- Repo: `dankclaw/open-notebook/quadlet/{open-notebook.container,open-notebook-surrealdb.container,open-notebook.network}`, `scripts/open-notebook.env`, `dankclaw/.env.example`, `scripts/deploy-plan.py`, `scripts/deploy-remote.sh` (`seed_open_notebook_env`, `ensure_pinned_image`, `deploy_open_notebook`), `scripts/deploy.sh`, `scripts/test-changed.sh`, `tests/open_notebook.bats`, `shared/homepage/services.json`, `dankclaw/seerr/quadlet/seerr.container` (the 5055 collision), `dankclaw/voiceservice/{quadlet/frank-tts.container,app.py}`, `scripts/frank-tts.env`, `shared/prometheus/prometheus.yml`
- Upstream `lfnovo/open-notebook` @ main, read 2026-10-18: `docker-compose.yml:3,10,11,18,27,29-30,39-43`, `Dockerfile:69-76,84,126,140,158`, `supervisord.conf:12,23-25,39`, `api/main.py:73,227-230,239-241,414`, `api/credentials_service.py:418-420,552`, `api/routers/credentials.py:426-429,433-437`, `open_notebook/config.py:5,19`, `open_notebook/podcasts/audio_paths.py`, `open_notebook/ai/connection_tester.py:208-238,326-336`, `open_notebook/ai/model_discovery.py:26-28`, `docs/3-USER-GUIDE/creating-podcasts.md:13,27,29,33,89`, `docs/5-CONFIGURATION/environment-reference.md:44,46,106,110,129,130`, `docs/5-CONFIGURATION/local-tts.md:5,45,72-82,89`
- Registry (2026-10-18): `lfnovo/open_notebook` tags (`v1-latest`, `latest`, the `-single` set) and manifest-list digests; `surrealdb/surrealdb` `v2` → 2.7.0
- Local measurements (2026-10-18, workstation, pinned digests): `surreal start --help` (credential and bind options carry `[env: SURREAL_*]`); running the pinned SurrealDB digest with env-only credentials (credential init in the log, correct login authenticates, wrong password refused, `/surreal is-ready` → `OK` exit 0, named volume owned by the mapped uid); the v1-compatible Quadlet generator dry-run of all three unit files (two `--env-file` in order, two `--network` in order, named volume + host bind, both health probes, `--pull never`)
- Host evidence (Dankclaw, 2026-10-18, read-only): `ss -ltn` (`127.0.0.1:5055` held by seerr, `0.0.0.0:7771` a host listener, nothing on 7772/8502/5056/8000), `podman ps -a` (no open-notebook container, `dankclaw_default` present), subnet registry from `dankclaw/open-webui/quadlet/open-webui.network`
- Deploy + provisioning (Dankclaw, 2026-10-19): first-boot log — migrations 1→25, `Credentials were provided, and no root users were found. The root user 'open_notebook' will be created`, `api`/`worker`/`frontend` all RUNNING under supervisord; `GET :8502/config` → the `API_URL` value; `GET /api/providers` (the `openai_compatible` modality list); `GET /openapi.json` schemas for `CreateCredentialRequest` / `ModelCreate` / `SpeakerProfileCreate` / `PodcastGenerateRequest`; the POSTs listed above; the note-save worker error `No embedding model configured.`
- Embedding lane (Dankclaw, 2026-10-19): `POST /api/credentials` → `credential:7fdqz09tt0c4xp68x7cm`, `POST /api/models` → `model:p3ki598dydfinu93tbny`, `POST /api/models/{id}/test` → `Embedding dimensions: 1024`; worker lines `Successfully embedded note … in 0.43s` / `0.19s`; the name-not-id failure `Model with ID qwen3-embedding-0.6b not found`; the `/api/embed` 500 body (`EmbedResponse.command_id`, `input_value=RecordID(table_name=command…)`); `POST /api/search` `similarity` 0.6590/0.2921 and 0.9059/0.3176 for the two-note pair; `podman logs frank-dragoon-halogen` showing `POST /v1/embeddings 200` from `10.89.1.21`. Both probe notes deleted afterwards.
- First render (2026-10-19): job `command:9rdojq5ne7dj9caicq20`, `execution_time: 301.21 s`, `episode:7ft03thj98sn0ek9xf29`; `episodes/d5fed29b-33e9-40ac-b46a-e0537c56c256/clips/0001.mp3 … 0023.mp3`; `…/audio/d5fed29b-….mp3` = 11,957,420 bytes, mp3 24 kHz mono, `ffprobe` duration 747.31 s; `GET /api/podcasts/episodes/{id}/audio` → 200 `audio/mpeg` with the bearer, 401 without.
- Upstream code read inside the pinned image (2026-10-19): `frontend/next.config.ts:26-43` (the `/api/*` rewrite and the one-port claim), `frontend/src/app/config/route.ts:122-173` (`/config` priority order; autodetect appends `:5055`), `frontend/src/app/api/_sse-proxy.ts` (SSE proxied with `no-transform`), `open_notebook/utils/url_validation.py` (link-local refusal, `is_blocked_ip`, no environment override anywhere in the file).
- Guard chain (2026-10-19): provider `notebook-proxy` created through `podman exec -i authentik-server ak shell` with the flows copied from `embedded-outpost` (pk 1), Application `Open Notebook`/`notebook-guard`, embedded-outpost provider set 9 → 10; afterwards the outpost logged zero `Failed to fetch outpost from API` and `seerr`/`grafana` still answered 200. Before the route existed, `notebook.franksdank.cloud` resolved to nothing (`000`).