From 317f36b456a89d5e3f561dc002875462c37b6456 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 02:55:50 -0800 Subject: [PATCH 1/7] feat(audiobookshelf): bootstrap the account, library and scan from the deployer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A freshly deployed Audiobookshelf answers /healthcheck and is otherwise dead: no account, no library, no scan. The UI registration step is gone by design — the doors are POST /init (200 with plain text "OK", not JSON) and /status (isInit), and there is no /setup route. That is an API-only bootstrap, so it must be code. scripts/audiobookshelf-init.sh is idempotent: /status -> /init when unregistered, POST /login for the bearer, POST /api/libraries for the Audiobooks library on /audiobooks, then POST /api/libraries/{id}/scan. The deployer runs it only after the health gate has answered. Credentials live in ~/.config/secrets/audiobookshelf (mode 600): the bearer rides an Authorization header and the password only a POST body — never a URL or argv, where any ps on the host would print them. The seam drives an inline stub server that logs every request line, and asserts: fresh bootstrap creates account, library and scan; a second run creates nothing new and still fires the incremental scan; no secret appears in any logged request line; a server that refuses is reported as "refused: " with exit 1 rather than success; ABS_SCAN outside 0|1 fails before a single request; the deployer orders init after health. Measured on Dankserver: run against the already-bootstrapped host it is a no-op that re-fires the scan; the library then scans (1,500+ books, every item with audio tracks, none invalid). --- scripts/audiobookshelf-init.sh | 188 +++++++++++++++++++++++++++++++ scripts/deploy-remote.sh | 6 + scripts/test-changed.sh | 6 + tests/audiobookshelf_init.bats | 200 +++++++++++++++++++++++++++++++++ 4 files changed, 400 insertions(+) create mode 100755 scripts/audiobookshelf-init.sh create mode 100755 tests/audiobookshelf_init.bats diff --git a/scripts/audiobookshelf-init.sh b/scripts/audiobookshelf-init.sh new file mode 100755 index 0000000..c1ebd8b --- /dev/null +++ b/scripts/audiobookshelf-init.sh @@ -0,0 +1,188 @@ +#!/usr/bin/env bash +# audiobookshelf-init.sh — make an Audiobookshelf server usable, from nothing. +# +# Why this is a script and not a browser session (live 2026-11-02, Dankserver): +# a freshly deployed Audiobookshelf answers /healthcheck with 200 while it has no +# user, no library and no scan — "healthy" says nothing about usefulness, and a +# service that only works after someone remembers to click through a first-run +# wizard is not deployed, it is installed. Everything below is the API the React +# client itself calls, so the same procedure runs unattended and lands in git. +# +# The API facts this script depends on, read off the pinned image rather than +# guessed (dist-server/server/Server.js inside the digest this repo pins): +# * the creation route is POST /init with {"newRoot":{"username","password"}}. +# There is NO /setup route in 2.37.x — POST /setup answers "Unauthorized", +# which reads exactly like an auth fault and is instead a missing route. +# * /status (no /api prefix) reports {"isInit": }; every route +# under /api needs the bearer token that POST /login returns. +# * the library count is NOT a field on the library DTO; the door is +# GET /api/libraries/{id}/items -> {"total": N}. +# +# The credential is generated once into a mode-600 host file and never enters +# the repo, the URL, or a process argument list. Same pattern as calibre-server. +# +# usage: audiobookshelf-init.sh # create what is missing, then scan +# ABS_SCAN=0 audiobookshelf-init.sh # bootstrap without a scan +# +# environment (defaults are the deployed shape): +# ABS_URL server base URL (http://127.0.0.1:8000) +# ABS_SECRET credential file (/home/dank/.config/secrets/audiobookshelf) +# ABS_LIBRARY library name to ensure (Audiobooks) +# ABS_FOLDER folder to bind inside the server (/audiobooks) +# ABS_SCAN 1 triggers a scan (default), 0 skips it +# +# exit 0 every step is satisfied (created or already present) +# exit 1 a step was refused or the server misbehaved (named on stderr) +# exit 2 usage / unreadable credential file +set -euo pipefail + +ABS_URL=${ABS_URL:-http://127.0.0.1:8000} +ABS_SECRET=${ABS_SECRET:-/home/dank/.config/secrets/audiobookshelf} +ABS_LIBRARY=${ABS_LIBRARY:-Audiobooks} +ABS_FOLDER=${ABS_FOLDER:-/audiobooks} +ABS_SCAN=${ABS_SCAN:-1} + +die() { printf 'refused: %s\n' "$*" >&2; exit 1; } + +case "$ABS_SCAN" in + 0 | 1) ;; + *) printf 'refused: ABS_SCAN wants 0 or 1, got %s\n' "$ABS_SCAN" >&2; exit 2 ;; +esac + +# ---- credential ------------------------------------------------------------- +umask 077 +if [ ! -f "$ABS_SECRET" ]; then + mkdir -p "$(dirname "$ABS_SECRET")" + user=${ABS_ADMIN_USER:-frank} + # `tr … | head -c 24` is the obvious line and it is a trap under + # `set -o pipefail`: head exits at 24 bytes, tr takes SIGPIPE, the pipeline + # status is 141 and set -e kills the script before the credential is ever + # written (measured 2026-11-02). od reads a FIXED byte count and exits on its + # own, so no consumer closes a pipe early; bash slices the length. + pass=$(od -An -tx1 -N 32 /dev/urandom | tr -d ' \n') + pass=${pass:0:24} + [ "${#pass}" -eq 24 ] || die "could not generate a password (no /dev/urandom)" + printf 'ABS_ADMIN_USER=%s\nABS_ADMIN_PASSWORD=%s\n' "$user" "$pass" > "$ABS_SECRET" + printf 'credential: created %s\n' "$ABS_SECRET" +fi +[ -r "$ABS_SECRET" ] || die "$ABS_SECRET exists but is not readable by $(id -un)" +chmod 600 "$ABS_SECRET" + +# ---- everything else, over HTTP --------------------------------------------- +# python3 rather than curl so the password and the bearer token never appear in +# any argv (they would be world-readable in /proc//cmdline otherwise). +ABS_URL="$ABS_URL" ABS_SECRET="$ABS_SECRET" ABS_LIBRARY="$ABS_LIBRARY" \ +ABS_FOLDER="$ABS_FOLDER" ABS_SCAN="$ABS_SCAN" python3 - <<'PY' +import json, os, sys, urllib.error, urllib.request + +BASE = os.environ["ABS_URL"].rstrip("/") +LIB = os.environ["ABS_LIBRARY"] +FOLDER = os.environ["ABS_FOLDER"] +SCAN = os.environ["ABS_SCAN"] == "1" +cred = {} +for line in open(os.environ["ABS_SECRET"]): + if "=" in line: + k, _, v = line.strip().partition("=") + cred[k] = v +user = cred.get("ABS_ADMIN_USER") or "frank" +password = cred.get("ABS_ADMIN_PASSWORD") or "" +if not password: + sys.exit("refused: %s has no ABS_ADMIN_PASSWORD" % os.environ["ABS_SECRET"]) + + +def call(method, path, body=None, token=None, want_json=True): + data = None + headers = {"X-Requested-With": "audiobookshelf-init"} + if body is not None: + data = json.dumps(body).encode() + headers["Content-Type"] = "application/json" + if token: + headers["Authorization"] = "Bearer " + token + req = urllib.request.Request(BASE + path, data=data, headers=headers, method=method) + try: + with urllib.request.urlopen(req, timeout=60) as r: + raw = r.read() + if not raw or not want_json: + return r.status, raw + try: + return r.status, json.loads(raw) + except ValueError: + # Express answers sendStatus() with the STATUS TEXT as the body: + # POST /init and the scan route both reply "OK" (measured on the + # pinned 2.37.2 image). A successful call must never be judged by + # whether its body happens to parse as JSON. + return r.status, raw.decode("utf-8", "replace")[:200] + except urllib.error.HTTPError as e: + raw = e.read() + try: + payload = json.loads(raw) + except Exception: + payload = raw.decode("utf-8", "replace")[:200] + return e.code, payload + except urllib.error.URLError as e: + # Not an HTTP answer at all: nothing listening, DNS dead, connection + # refused. HTTPError is a URLError subclass, so this arm must come after. + return 0, "unreachable (%s)" % e.reason + + +status, st = call("GET", "/status") +if status != 200 or not isinstance(st, dict): + sys.exit("refused: %s/status answered %s (%s) — is the service up?" % (BASE, status, st)) +if st.get("isInit"): + print("account: already registered") +else: + code, body = call("POST", "/init", {"newRoot": {"username": user, "password": password}}) + if code != 200: + sys.exit("refused: POST /init answered %s (%s)" % (code, body)) + print("account: created (%s)" % user) + +code, login = call("POST", "/login", {"username": user, "password": password}) +if code != 200 or not isinstance(login, dict) or not login.get("user", {}).get("token"): + sys.exit("refused: login answered %s (%s) — the credential file does not match the server" + % (code, login)) +token = login["user"]["token"] +print("token: obtained (%s)" % login["user"].get("type", "user")) + +code, libs = call("GET", "/api/libraries", token=token) +if code != 200 or not isinstance(libs, dict) or "libraries" not in libs: + sys.exit("refused: GET /api/libraries answered %s (%s)" % (code, libs)) +lib = next((l for l in libs["libraries"] if l.get("name") == LIB), None) +if lib: + print("library: exists %s (%s)" % (LIB, lib["id"])) +else: + body = { + "name": LIB, + "provider": "audiobook", + "icon": "book", + "folders": [{"fullPath": FOLDER}], + "settings": { + "coverEncoding": "jpg", + "skipMatchingMediaWithAsin": False, + "skipMatchingMediaWithIsbn": False, + "autoDetectCrossoverAudiobooks": False, + "ignoreAudiobookMetadata": False, + "useAudiobookFileMetadata": False, + }, + "providerMetadata": {}, + } + code, made = call("POST", "/api/libraries", body, token=token) + if code != 200: + sys.exit("refused: POST /api/libraries answered %s (%s)" % (code, made)) + lib = made + print("library: created %s (%s) folder %s" % (LIB, lib["id"], FOLDER)) + +if SCAN: + code, body = call("POST", "/api/libraries/%s/scan" % lib["id"], {}, token=token, + want_json=False) + if code != 200: + sys.exit("refused: scan answered %s (%s)" % (code, body)) + print("scan: accepted") +else: + print("scan: skipped (ABS_SCAN=0)") + +code, items = call("GET", "/api/libraries/%s/items?limit=1" % lib["id"], token=token) +if code == 200 and isinstance(items, dict): + print("items: %s (a fresh scan raises this from 0; watch it, not /healthcheck)" + % items.get("total")) +PY +exit 0 diff --git a/scripts/deploy-remote.sh b/scripts/deploy-remote.sh index 2c1bbfc..5160157 100755 --- a/scripts/deploy-remote.sh +++ b/scripts/deploy-remote.sh @@ -1318,6 +1318,12 @@ deploy_audiobookshelf() { # /healthcheck answers once the HTTP server is bound; the library scan that # follows runs in the background and must not gate the deploy. wait_for_health audiobookshelf http://127.0.0.1:8000/healthcheck 90 + # A server with no root user and no library is "healthy" and useless, so the + # bootstrapper runs after the health gate and its failure fails the deploy: + # it creates the account (credential in the mode-600 host secret file), the + # Audiobooks library over the /audiobooks bind, and fires the scan. Every + # step is idempotent, so a re-deploy costs one incremental scan. + scripts/audiobookshelf-init.sh } # The content server's single admin account and the desktop's basic-auth diff --git a/scripts/test-changed.sh b/scripts/test-changed.sh index 35fdfe1..29bd6a2 100755 --- a/scripts/test-changed.sh +++ b/scripts/test-changed.sh @@ -104,6 +104,12 @@ for path in "${CHANGED[@]}"; do dankserver/jellyfin/quadlet/*|dankserver/plex/quadlet/*) add_test tests/media_overlay.bats ;; + # The Audiobookshelf bootstrapper (account + library + scan over the API) + # and the deployer hook that runs it after the health gate. + scripts/audiobookshelf-init.sh|tests/audiobookshelf_init.bats) + add_test tests/audiobookshelf_init.bats + add_test tests/deployment_flow.bats + ;; shared/homepage/*) add_test tests/homepage_caddy.bats add_test tests/homepage_config.bats diff --git a/tests/audiobookshelf_init.bats b/tests/audiobookshelf_init.bats new file mode 100755 index 0000000..8a7f92e --- /dev/null +++ b/tests/audiobookshelf_init.bats @@ -0,0 +1,200 @@ +#!/usr/bin/env bats +# Seam: scripts/audiobookshelf-init.sh — the Audiobookshelf bootstrapper. +# +# A freshly deployed Audiobookshelf answers /healthcheck 200 while it has no +# user, no library and no scan, so "healthy" proves nothing about usefulness. +# This seam pins what the bootstrapper must make true, against a stub that +# speaks the same API as the pinned image (POST /init + POST /login at the +# root, bearer-authenticated /api/*, library count only on /items): +# 1. an unregistered server gains the account, the library and a scan, and the +# credential lands mode 600 with both keys; +# 2. a second run creates nothing new (exactly one /init, one library) and +# still fires the incremental scan; +# 3. neither the bearer token nor the password ever rides a URL — a secret in +# a URL or an argv is world-readable in /proc//cmdline, which is how +# an HF token leaked into a transcript on 2026-09-16; +# 4. a server that cannot be reached is refused by URL with exit 1, never +# reported as success (a green deploy over a dead API is a lie); +# 5. ABS_SCAN outside 0/1 is a usage error raised before anything is touched; +# 6. the deployer runs the bootstrapper only after the health gate answered. + +load helpers + +setup() { + INIT="$REPO_ROOT/scripts/audiobookshelf-init.sh" + TMP="$(mktemp -d)" + PORT=18937 + cat > "$TMP/stub.py" <<'PY' +import json, sys +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + +port, logf = int(sys.argv[1]), sys.argv[2] +state = {"init": False, "user": None, "pw": None, "libs": []} + + +def log(line): + with open(logf, "a") as f: + f.write(line + "\n") + + +class H(BaseHTTPRequestHandler): + def log_message(self, *a): + pass + + def send(self, code, body, as_json=False): + data = json.dumps(body).encode() if as_json else str(body).encode() + self.send_response(code) + self.send_header("Content-Type", "application/json" if as_json else "text/plain") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + + def body(self): + n = int(self.headers.get("Content-Length") or 0) + return json.loads(self.rfile.read(n) or b"{}") + + def authed(self): + return self.headers.get("Authorization") == "Bearer stub-token" + + def do_GET(self): + path = self.path.split("?")[0] + log("GET %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no")) + if path == "/status": + return self.send(200, {"app": "audiobookshelf", "isInit": state["init"]}, True) + if not self.authed(): + return self.send(401, "Unauthorized") + if path == "/api/libraries": + return self.send(200, {"libraries": state["libs"]}, True) + if path.endswith("/items"): + return self.send(200, {"total": 42, "results": []}, True) + return self.send(404, "nope") + + def do_POST(self): + path = self.path.split("?")[0] + payload = self.body() + log("POST %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no")) + if path == "/init": + if state["init"]: + return self.send(500, "already initialized") + root = payload.get("newRoot", {}) + state["init"] = True + state["user"] = root.get("username") + state["pw"] = root.get("password") + log("INIT %s" % root.get("username")) + return self.send(200, "OK") + if path == "/login": + if not state["init"]: + return self.send(401, "Unauthorized") + if (payload.get("username"), payload.get("password")) != (state["user"], state["pw"]): + return self.send(401, "Unauthorized") + return self.send(200, {"user": {"username": state["user"], "type": "root", + "id": "u1", "token": "stub-token"}}, True) + if not self.authed(): + return self.send(401, "Unauthorized") + if path == "/api/libraries": + lib = {"id": "lib-1", "name": payload.get("name"), "folders": payload.get("folders", [])} + state["libs"].append(lib) + log("LIBRARY %s %s" % (lib["name"], lib["folders"])) + return self.send(200, lib, True) + if path.endswith("/scan"): + log("SCAN") + return self.send(200, "OK") + return self.send(404, "nope") + + +ThreadingHTTPServer(("127.0.0.1", port), H).serve_forever() +PY + python3 "$TMP/stub.py" "$PORT" "$TMP/requests.log" > "$TMP/stub.out" 2>&1 & + STUB_PID=$! + for _ in $(seq 1 50); do + curl -s -o /dev/null "http://127.0.0.1:$PORT/status" && break + sleep 0.1 + done +} + +teardown() { + [ -n "${STUB_PID:-}" ] && kill "$STUB_PID" 2>/dev/null || true + rm -rf "$TMP" +} + +boot() { + env ABS_URL="http://127.0.0.1:$PORT" ABS_SECRET="$TMP/cred" \ + ABS_ADMIN_USER=frank ABS_LIBRARY=Audiobooks ABS_FOLDER=/audiobooks \ + ABS_SCAN="${SCAN:-1}" bash "$INIT" +} + +@test "an unregistered server gains the account, the library and the scan" { + run boot + [ "$status" -eq 0 ] + printf '%s\n' "${lines[@]}" | grep -qx "credential: created $TMP/cred" + printf '%s\n' "${lines[@]}" | grep -q "^account: created (frank)" + printf '%s\n' "${lines[@]}" | grep -q "^token: obtained (root)" + printf '%s\n' "${lines[@]}" | grep -q "^library: created Audiobooks (lib-1)" + printf '%s\n' "${lines[@]}" | grep -q "^scan: accepted" + printf '%s\n' "${lines[@]}" | grep -q "^items: 42" + # The state roots of this service hold an account credential, so the file + # that carries it must never be group- or world-readable. + [ "$(stat -c %a "$TMP/cred")" = "600" ] + grep -qx 'ABS_ADMIN_USER=frank' "$TMP/cred" + grep -Eq '^ABS_ADMIN_PASSWORD=[0-9a-f]{24}$' "$TMP/cred" + # The folder the library indexes is the bind inside the container, not the + # host path — a library pointed at /mnt/entropy would answer 0 items and + # look like an empty audiobook collection. + grep -q "LIBRARY Audiobooks \[{'fullPath': '/audiobooks'}\]" "$TMP/requests.log" +} + +@test "a second run creates nothing new and still fires the incremental scan" { + boot > /dev/null + [ -e "$TMP/requests.log" ] + run boot + [ "$status" -eq 0 ] + printf '%s\n' "${lines[@]}" | grep -q "^account: already registered" + printf '%s\n' "${lines[@]}" | grep -q "^library: exists Audiobooks (lib-1)" + printf '%s\n' "${lines[@]}" | grep -q "^scan: accepted" + # No second credential, no second account, no duplicate library: the deployer + # may run this on every re-deploy of the service. + ! printf '%s\n' "${lines[@]}" | grep -q "credential: created" + [ "$(grep -c '^INIT ' "$TMP/requests.log")" -eq 1 ] + [ "$(grep -c '^LIBRARY ' "$TMP/requests.log")" -eq 1 ] + [ "$(grep -c '^SCAN$' "$TMP/requests.log")" -eq 2 ] +} + +@test "the bearer token and the password never ride a URL" { + boot + # The stub logs the request line (method + path). A secret showing up there + # means it travelled as a query parameter — visible in access logs, shell + # history and /proc//cmdline. + ! awk '{print $2}' "$TMP/requests.log" | grep -q 'stub-token' + pw=$(awk -F= '/^ABS_ADMIN_PASSWORD=/{print $2}' "$TMP/cred") + [ -n "$pw" ] + ! awk '{print $2}' "$TMP/requests.log" | grep -q -- "$pw" + # and the authenticated routes only answered once the header was present + grep -q '^GET /api/libraries auth=yes$' "$TMP/requests.log" +} + +@test "a server that does not answer is refused by URL, not reported as success" { + run env ABS_URL="http://127.0.0.1:1" ABS_SECRET="$TMP/cred" bash "$INIT" + [ "$status" -eq 1 ] + [[ "$output" == *refused:* ]] + [[ "$output" == *"127.0.0.1:1"* ]] +} + +@test "ABS_SCAN outside 0/1 is a usage error raised before anything is touched" { + run env ABS_URL="http://127.0.0.1:$PORT" ABS_SECRET="$TMP/cred" ABS_SCAN=2 \ + bash "$INIT" + [ "$status" -eq 2 ] + [[ "$output" == *ABS_SCAN* ]] + [ ! -e "$TMP/cred" ] +} + +@test "the deployer bootstraps only after the health gate has answered" { + body=$(awk '/^deploy_audiobookshelf\(\)/,/^}/' "$REPO_ROOT/scripts/deploy-remote.sh") + [ -n "$body" ] + printf '%s\n' "$body" | grep -q \ + 'wait_for_health audiobookshelf http://127\.0\.0\.1:8000/healthcheck' + printf '%s\n' "$body" | grep -q 'scripts/audiobookshelf-init\.sh' + printf '%s\n' "$body" | awk ' + /wait_for_health audiobookshelf/ { health = NR } + /audiobookshelf-init\.sh/ { init = NR } + END { exit !(health && init && init > health) }' +} -- 2.54.0 From 48ef6d34cdfc2bc656800ad820b09dfa64915fa7 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 02:55:55 -0800 Subject: [PATCH 2/7] docs(audiobookshelf): the first run is a script, and its re-run is the scan trigger MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The page claimed the account can only be created from a LAN address; measured, the door is POST /init and it answers plain text. Replaced with the bootstrap the deployer now runs, and with the doors a consumer actually needs: /status reports isInit with no /api prefix, the item count rides /api/libraries/{id}/items -> total (the library DTO has no totals), and the list DTO carries media.numTracks while /api/items/{id} returns media.audioFiles. New law: the folder watcher cannot cross the rootless podman bind — the same gap as Jellyfin's auto-scan note — so files added on the host stay invisible until a scan runs, and the deployer's re-run of the bootstrapper is the cheap trigger. Testing Policy lists the new seam; index.md regenerated from frontmatter. --- .../Architecture/Testing Policy.md | 2 +- .../Dankserver/Audiobookshelf.md | 60 ++++++++++++++----- serverAppConfigs-docs/index.md | 2 +- 3 files changed, 48 insertions(+), 16 deletions(-) diff --git a/serverAppConfigs-docs/Architecture/Testing Policy.md b/serverAppConfigs-docs/Architecture/Testing Policy.md index 7f132ce..69ce731 100644 --- a/serverAppConfigs-docs/Architecture/Testing Policy.md +++ b/serverAppConfigs-docs/Architecture/Testing Policy.md @@ -23,7 +23,7 @@ This repository tests its configuration at defined seams. The tests run with `ba | Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) | | Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free | | E2E (browser) | `tests/e2e/` + `tests/e2e_scaffold.bats` | Playwright drives the live guard chain; the bats seam asserts structure and credential hygiene. See [[E2E Suite]] | -| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service | +| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{audiobookshelf_init,anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service | ## Focused pre-commit testing diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index e856ca0..e079d20 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -1,6 +1,6 @@ --- type: reference -description: "Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, first-run account, laws." +description: "Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, the API bootstrap, laws." tags: - dankserver - audiobooks @@ -37,19 +37,32 @@ manager) — **Internal-only:** LAN address, no vhost, no DNS name. | Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` | | Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` | -## First run +## Bootstrap -1. Open `http://192.168.0.235:8000` **from the LAN**. ABS accepts the initial - account creation only from a local-network address; a remote address gets a - refusal, not a form. -2. Create the owner account, then add one **Audi Book** library with folder - `/audiobooks`. The path is the in-container name of the read-only bind. -3. The first scan reads the whole tree once. It runs in the background — the - deploy gate is already green by then, and `IOWeight=10` keeps it off the - critical path. +`scripts/audiobookshelf-init.sh` — run by the deployer after the health gate, so a +deployed ABS is a *usable* ABS: `/healthcheck` answers 200 on a server with no +user, no library and no scan, and nothing else in the deploy notices the +difference. The script is idempotent; re-running it costs one incremental scan. -Nothing in this repo holds an ABS credential: the account is created in the -browser and lives in the `config` bind on the SSD. +| Step | Door | Printed | +|------|------|---------| +| credential | `~/.config/secrets/audiobookshelf` (mode 600, generated once, host-side) | `credential: created …` | +| account | `POST /init` with `{"newRoot":{"username","password"}}` | `account: created (frank)` / `already registered` | +| session | `POST /login` → bearer token | `token: obtained (root)` | +| library | `POST /api/libraries`, folder `/audiobooks`, provider `audiobook` | `library: created Audiobooks ()` / `exists` | +| scan | `POST /api/libraries//scan` | `scan: accepted` | +| proof | `GET /api/libraries//items?limit=1` | `items: ` | + +Sign in at `http://192.168.0.235:8000` as `frank`; the password is +`awk -F= '/^ABS_ADMIN_PASSWORD=/{print $2}' ~/.config/secrets/audiobookshelf` on +Dankserver. Change it in **Settings → Users → frank → Password**; the secret file +then holds a stale value and only matters again if the `config` bind is ever +destroyed (a fresh DB makes the script create a new owner from it). + +The scan of 1,103 shelves is slow and background; watch `items:` climb from 0, +never `/healthcheck`. It also reads files that are genuinely broken and logs +`[AudioFileScanner] Invalid data found when processing input` per file — that is +the file, not the bind: the same line count grows while the total climbs. ## Laws @@ -95,10 +108,29 @@ browser and lives in the `config` bind on the SSD. in front. Any public vhost requires the Anubis + Authentik chain first, and the outpost proxy provider has to exist before the vhost flips (external exposure policy in `AGENTS.md`). +- **There is no `/setup` route; the creation door is `POST /init`.** `/status` + (no `/api` prefix) reports `isInit` = "a root user exists", `POST /init` with + `{"newRoot":{"username","password"}}` creates it, `POST /login` returns the + bearer every `/api/*` route wants. `POST /setup` answers `Unauthorized` — that + is a missing route wearing an auth message. Read off `dist-server/server/Server.js` + inside the pinned digest rather than from a forum. +- **A 200 may carry plain text.** `res.sendStatus(200)` answers the body `OK`, and + `/init` and the scan route use it; a client that parses every success as JSON + crashes on the one call that matters (`tests/audiobookshelf_init.bats` pins it). +- **The item count is not a field on the library DTO.** The door is + `GET /api/libraries//items?limit=1` → `total`. A poller that guesses a + plausible name (`numItems`) prints `0` forever beside a scan that is working. +- **A fresh scan is the only thing that notices new audio.** `inotify` does not + cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that + law — so the in-container watcher sees nothing a host process files onto the + array, and the deployer's re-run of the bootstrapper is the cheap scan trigger. ## Verification `./tests/run.sh` covers this unit through `tests/dankserver_quadlet.bats` (boot persistence, the SSD state roots, the `:ro` array bind, the mount gate, the -`wget` health check) and `tests/media_overlay.bats` (only [[Plex]] and -[[Jellyfin]] may reference the metadata view). +`wget` health check), `tests/audiobookshelf_init.bats` (the bootstrapper against a +stub of the API above: create-once, idempotent re-run, no secret in a URL, refusal +by URL when the server does not answer, and the deployer calling it only after the +health gate) and `tests/media_overlay.bats` (only [[Plex]] and [[Jellyfin]] may +reference the metadata view). diff --git a/serverAppConfigs-docs/index.md b/serverAppConfigs-docs/index.md index 20673f6..44e044e 100644 --- a/serverAppConfigs-docs/index.md +++ b/serverAppConfigs-docs/index.md @@ -44,7 +44,7 @@ is a locator: one row, one clause, no numbers. | Page | Owns | |---|---| -| [[Audiobookshelf]] | Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, first-run account, laws. | +| [[Audiobookshelf]] | Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, the API bootstrap, laws. | | [[Calibre]] | Calibre is the e-book library manager that backs Calibre-Web. | | [[Calibre-Web]] | Calibre-Web is a web application that browses, reads, and downloads eBooks from an existing Calibre library database. | | [[Copyparty]] | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. | -- 2.54.0 From 56dfc6d0b90a2c7c6e54d5dac394293ba7da9235 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 02:57:25 -0800 Subject: [PATCH 3/7] docs(audiobookshelf): the scanner refusal names the file, not the bind The first full scan logged "Invalid data found when processing input" per file and carried on. That line is about the file: the scanner skips it, so the book never appears in the library at all. Census of the 43 refused files: 38 were 0 bytes, 3 held WAV bytes behind a .mp3 name, 2 were unreadable. The check to run first is a byte count, not a server investigation. Full suite: 818 tests, exit 0. --- serverAppConfigs-docs/Dankserver/Audiobookshelf.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index e079d20..f490b24 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -124,6 +124,12 @@ the file, not the bind: the same line count grows while the total climbs. cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that law — so the in-container watcher sees nothing a host process files onto the array, and the deployer's re-run of the bootstrapper is the cheap scan trigger. +- **`Invalid data found when processing input` names the file, not the bind.** The + scanner logs it per file and skips the file — the book never appears — so a + missing title beside those lines is library debris, not a server fault. Census of + the refusals on the first full scan: 38 of 43 files were 0 bytes, 3 were WAV + bytes wearing a `.mp3` name, 2 were unreadable. `stat -c %s` before any + investigation of the server. ## Verification -- 2.54.0 From 006bbcef2820b7295f7d97f90392e3cfe89b94bb Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 03:18:04 -0800 Subject: [PATCH 4/7] docs(audiobookshelf): paging the items endpoint never terminates itself Two censuses this session reported exactly 1,500 and exactly 4,000 books for one library mid-scan. Cause: an out-of-range offset returns the last item instead of an empty page, so a loop written "until the page is empty" manufactures its own total, and total itself moves while a scan is running. Stop on total; count after the pass settles. --- serverAppConfigs-docs/Dankserver/Audiobookshelf.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index f490b24..153aee5 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -117,9 +117,13 @@ the file, not the bind: the same line count grows while the total climbs. - **A 200 may carry plain text.** `res.sendStatus(200)` answers the body `OK`, and `/init` and the scan route use it; a client that parses every success as JSON crashes on the one call that matters (`tests/audiobookshelf_init.bats` pins it). -- **The item count is not a field on the library DTO.** The door is - `GET /api/libraries//items?limit=1` → `total`. A poller that guesses a - plausible name (`numItems`) prints `0` forever beside a scan that is working. +- **The item count is not a field on the library DTO, and paging never ends + itself.** The door is `GET /api/libraries//items?limit=1` → `total`; a poller + that guesses a plausible name (`numItems`) prints `0` forever beside a scan that + is working. An out-of-range `offset` returns the last item rather than an empty + page, so a census that loops "until empty" invents its own total — two runs + reported exactly 1,500 and 4,000 for one library — and `total` itself moves while + a scan runs. Stop on `total`, and take the count after the pass settles. - **A fresh scan is the only thing that notices new audio.** `inotify` does not cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that law — so the in-container watcher sees nothing a host process files onto the -- 2.54.0 From f5415667141f6ebca853569c9a4c66eae01d089d Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 03:34:28 -0800 Subject: [PATCH 5/7] docs(audiobookshelf): a book is the shallowest audio folder, and it owns the subtree Measured on the live library: 6 stray files at the root of _unsorted made the whole collection dump ONE item of 5,693 tracks, with 542 real books folded into it; no scan subdivides an existing item. Also pinned: the items endpoint hands back ceil(total/limit)*limit rows because an out-of-range offset repeats the tail (that is where "exactly 1,500" and "exactly 4,000" came from), and DELETE /api/items/:id?hard=1 runs fs.remove() on the item's folder - pointed at a covering item that is the whole subtree, so un-swallowing is a DB-only delete after the stray root files are filed. --- .../Dankserver/Audiobookshelf.md | 23 ++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index 153aee5..5ce6c80 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -121,13 +121,30 @@ the file, not the bind: the same line count grows while the total climbs. itself.** The door is `GET /api/libraries//items?limit=1` → `total`; a poller that guesses a plausible name (`numItems`) prints `0` forever beside a scan that is working. An out-of-range `offset` returns the last item rather than an empty - page, so a census that loops "until empty" invents its own total — two runs - reported exactly 1,500 and 4,000 for one library — and `total` itself moves while - a scan runs. Stop on `total`, and take the count after the pass settles. + page, so a census that loops "until empty" invents its own total: `limit=500` + against a `total` of 4,628 hands back exactly 5,000 rows — the collected count + is `ceil(total / limit) × limit`, which is why two runs reported "exactly 1,500" + and "exactly 4,000" for one library. Stop on `total`, count *unique ids*, and + take the number after the pass settles (`total` itself moves during a scan). - **A fresh scan is the only thing that notices new audio.** `inotify` does not cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that law — so the in-container watcher sees nothing a host process files onto the array, and the deployer's re-run of the bootstrapper is the cheap scan trigger. +- **A book is the shallowest folder that holds audio — and that item owns the + subtree beneath it.** A folder with audio in its own file list becomes ONE book + and the scanner folds everything below it into that book's tracks instead of + making items: 6 stray files at the root of `_unsorted` turned the whole + collection dump into a single item of 5,693 tracks, titled after whichever file + supplied metadata, with 542 real books inside it. So "hundreds of books are + missing" is a layout fault, not a scan fault — and no scan, forced or not, + subdivides an existing item. Census the layout the server's way: a directory is + a book folder iff its *own* files include an audio extension. +- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** Read + `LibraryItemController.delete`: `hardDelete = req.query.hard == 1` and then + `fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole + swallowed subtree — `hard=1` there deletes every audiobook under it off the + array. Un-swallowing is: file the loose root files with the journal-guarded + mover, then delete the stale covering item with NO query string. - **`Invalid data found when processing input` names the file, not the bind.** The scanner logs it per file and skips the file — the book never appears — so a missing title beside those lines is library debris, not a server fault. Census of -- 2.54.0 From f71c7ccce40de6f7f300b4016b49b3c5e9985861 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 03:41:53 -0800 Subject: [PATCH 6/7] docs(audiobookshelf): the grouping rule, not "shallowest folder with audio" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Corrects the law pushed earlier today. The mechanism is `groupFileItemsIntoLibraryItemDirs` (utils/scandir.js), called on a live `recurseFiles` walk of the tree EVERY scan: each file attaches to the FIRST directory along its path that already has a group, so one media file at a collection root owns everything beneath it. Measured: one 949 MB `.cbr` hid 652 book folders behind a single 5,693-track item. - "Media file" is audio OR ebook (`audiobooksOnly=false` -> SupportedAudioTypes ∪ SupportedEbookTypes, 17 audio + 6 ebook extensions). An audio-only census cannot see the file that hides an Audiobooks library. - Deleting the covering item does not fix a shadow: grouping is recomputed from disk and the next scan rebuilds the item (measured: `1 Added`, same path). - Un-shadow by MOVE: one file per folder is ABS's own single-book layout. - `?force=1` is not a discovery arm (both arms walk the disk; force re-scans every existing item, 1:33 against 0:04). - Paging corrected: `offset` is ignored outright, `page` is the parameter. - `DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()` of the whole swallowed subtree. Never pass `hard`. Shape table gains the layout row and the move journal; the stale "1,103 shelves" scan count is destroyed. --- .../Dankserver/Audiobookshelf.md | 59 ++++++++++++------- 1 file changed, 38 insertions(+), 21 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index 5ce6c80..837370b 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -36,6 +36,7 @@ manager) — **Internal-only:** LAN address, no vhost, no DNS name. | Self-healing | `Restart=always`, `RestartSec=5`, `WantedBy=default.target` | | Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` | | Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` | +| Library layout | one book per folder — ABS groups by the first directory holding a media file (see Laws); every move is journalled at `/mnt/entropy/media-relayout-audiobooks.jsonl` | ## Bootstrap @@ -59,7 +60,7 @@ Dankserver. Change it in **Settings → Users → frank → Password**; the secr then holds a stale value and only matters again if the `config` bind is ever destroyed (a fresh DB makes the script create a new owner from it). -The scan of 1,103 shelves is slow and background; watch `items:` climb from 0, +A full scan is slow and runs in the background; watch the `items:` total climb, never `/healthcheck`. It also reads files that are genuinely broken and logs `[AudioFileScanner] Invalid data found when processing input` per file — that is the file, not the bind: the same line count grows while the total climbs. @@ -120,31 +121,47 @@ the file, not the bind: the same line count grows while the total climbs. - **The item count is not a field on the library DTO, and paging never ends itself.** The door is `GET /api/libraries//items?limit=1` → `total`; a poller that guesses a plausible name (`numItems`) prints `0` forever beside a scan that - is working. An out-of-range `offset` returns the last item rather than an empty - page, so a census that loops "until empty" invents its own total: `limit=500` - against a `total` of 4,628 hands back exactly 5,000 rows — the collected count - is `ceil(total / limit) × limit`, which is why two runs reported "exactly 1,500" - and "exactly 4,000" for one library. Stop on `total`, count *unique ids*, and - take the number after the pass settles (`total` itself moves during a scan). + is working. Paging is on `page` (0-based) plus `limit`, and **`offset` is + ignored**: `limit=500&offset=500` returns the same 500 ids as `offset=0`, while + `page=1` shares none of them. A census that pages on `offset` therefore reads + `ceil(total / limit) × limit` rows containing only `limit` distinct items — the + exact origin of the "exactly 1,500" and "exactly 4,000" counts two runs + reported for one library. Page on `page`, count *unique ids*, stop on `total`, + and take the number after the pass settles (`total` moves during a scan). - **A fresh scan is the only thing that notices new audio.** `inotify` does not cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that law — so the in-container watcher sees nothing a host process files onto the array, and the deployer's re-run of the bootstrapper is the cheap scan trigger. -- **A book is the shallowest folder that holds audio — and that item owns the - subtree beneath it.** A folder with audio in its own file list becomes ONE book - and the scanner folds everything below it into that book's tracks instead of - making items: 6 stray files at the root of `_unsorted` turned the whole - collection dump into a single item of 5,693 tracks, titled after whichever file - supplied metadata, with 542 real books inside it. So "hundreds of books are - missing" is a layout fault, not a scan fault — and no scan, forced or not, - subdivides an existing item. Census the layout the server's way: a directory is - a book folder iff its *own* files include an audio extension. -- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** Read - `LibraryItemController.delete`: `hardDelete = req.query.hard == 1` and then +- **One stray media file at a collection root turns the whole collection into one + book.** Every scan re-walks the tree from disk (`scanFolder` → `recurseFiles`), + and `utils/scandir.js:groupFileItemsIntoLibraryItemDirs` assigns each file to the + FIRST directory along its path that already has a group: the first file seen at a + root owns every deeper file, and the folders below it never become items. One + 949 MB `.cbr` sitting at `_unsorted`'s root collapsed 652 book folders into a + single item of 5,693 tracks, titled after whichever file supplied metadata. + "Hundreds of books are missing" is a layout fault, not a scan fault. +- **"Media file" means audio OR ebook, so an audio-only census cannot see the file + that is hiding the library.** With `audiobooksOnly=false` (the deployed shape) + the predicate is `SupportedAudioTypes` ∪ `SupportedEbookTypes`: 20 audio + extensions (including `mp4`, `mpg`, `webm`) plus `epub pdf mobi azw3 cbr cbz`. A + census reporting "that root holds no loose audio" can be correct and still + useless — the file doing the shadowing here was a comic archive. +- **Deleting the covering item does not un-shadow anything.** Grouping is recomputed + from disk, so the next scan rebuilds the item from the same stray file (measured: + `DELETE`, then scan → `1 Added`, the item back at the same path). The fix is the + MOVE: give every stray root file a folder of its own (one file per folder is + ABS's own single-book layout; formats of one book share a folder), through the + journal-guarded mover, then delete the stale item so the old path is not kept, + then scan. +- **`?force=1` is not a discovery arm.** Both arms of `POST /libraries/:id/scan` + walk the disk; force additionally re-scans every item that already exists + (measured 1:33 against 0:04 on ~4,600 items). Finding new folders needs neither, + so the deployer's cheap trigger stays on the plain arm. +- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** In + `LibraryItemController.delete`, `hardDelete = req.query.hard == 1` and then `fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole - swallowed subtree — `hard=1` there deletes every audiobook under it off the - array. Un-swallowing is: file the loose root files with the journal-guarded - mover, then delete the stale covering item with NO query string. + swallowed subtree — `hard=1` there deletes every book beneath it off the array. + Assert the item's `path` first, and never pass `hard`. - **`Invalid data found when processing input` names the file, not the bind.** The scanner logs it per file and skips the file — the book never appears — so a missing title beside those lines is library debris, not a server fault. Census of -- 2.54.0 From ad9563412de56ab671de1dcbfff3badafedb5477 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 03:59:14 -0800 Subject: [PATCH 7/7] docs(audiobookshelf): what a residual is made of MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four laws from finishing the un-shadow, correcting one: - A book whose files ALL fail to probe never becomes an item, so a disk-vs-DB folder gap beside `Invalid data` lines is corrupt media, not a lost book. A format-only ffprobe clears files the scanner rejects — do not trust it. - `Disc 01`/`CD 1` subfolders are parts of one item (110 of them here, under a 192-track book); the extension census counts them, ABS does not. - `.cbr`/`.cbz` group like media but `parseComicMetadata` throws `ERAR_BAD_ARCHIVE`, so those items land media-less: the file, not the scanner. - Scan completion is a NEW scan uuid; `podman logs --since` is a sliding window and ages its own evidence out. - `?force=1` is now recorded as neither discovery nor repair: a forced full re-scan left item count, path set and media-less count identical. Verification names the five book folders that need re-acquiring. --- .../Dankserver/Audiobookshelf.md | 41 +++++++++++++++---- 1 file changed, 32 insertions(+), 9 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index 837370b..d7e15a2 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -153,21 +153,37 @@ the file, not the bind: the same line count grows while the total climbs. ABS's own single-book layout; formats of one book share a folder), through the journal-guarded mover, then delete the stale item so the old path is not kept, then scan. -- **`?force=1` is not a discovery arm.** Both arms of `POST /libraries/:id/scan` - walk the disk; force additionally re-scans every item that already exists - (measured 1:33 against 0:04 on ~4,600 items). Finding new folders needs neither, - so the deployer's cheap trigger stays on the plain arm. +- **`?force=1` is neither a discovery arm nor a repair arm.** Both arms of + `POST /libraries/:id/scan` walk the disk, so finding new folders needs neither + and the deployer's cheap trigger stays on the plain arm. Force additionally + re-scans every item that already exists (measured 1:33 against 0:04 on ~4,600 + items) and repaired nothing on a library whose residuals were file-level: item + count, path set and media-less count identical before and after. - **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** In `LibraryItemController.delete`, `hardDelete = req.query.hard == 1` and then `fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole swallowed subtree — `hard=1` there deletes every book beneath it off the array. Assert the item's `path` first, and never pass `hard`. - **`Invalid data found when processing input` names the file, not the bind.** The - scanner logs it per file and skips the file — the book never appears — so a - missing title beside those lines is library debris, not a server fault. Census of - the refusals on the first full scan: 38 of 43 files were 0 bytes, 3 were WAV - bytes wearing a `.mp3` name, 2 were unreadable. `stat -c %s` before any - investigation of the server. + scanner logs it per file and skips that file; a book whose files ALL fail never + becomes an item at all, so the disk-vs-DB folder census shows it as a book with + no item. That is corrupt media, not a lost book. Never clear a suspect file with + a lenient probe: `ffprobe -show_entries format=duration` happily returns three + hours for the WAV bytes wearing a `.mp3` name that the scanner refuses. Read the + log's file paths instead, and `stat -c %s` first — 0-byte files are the common + case, and a truncated download reports `moov atom not found`. +- **A `Disc 01` / `CD 1` subfolder is part of a book, not a book.** The extension + census counts those folders; ABS deliberately does not — its grouping attaches + their files to the ancestor item (measured: 110 such folders under one item + carrying all 192 tracks). Read the basename before believing a disk-vs-DB gap. +- **`.cbr`/`.cbz` group like media and never play.** They sit in the grouping + predicate, so they hide books, but the comic parser throws `ERAR_BAD_ARCHIVE` + (`parseComicMetadata` → `CbrComicBookExtractor`) and the item lands with no + media. A media-less row beside those lines is the file, not the scanner. +- **A scan is finished when a NEW scan uuid completes.** Match + `Library scan "" completed in ...`. `podman logs --since 30m` is a sliding + window: the set of visible completion lines changes on its own as old lines age + out, so a poller keyed on that hash reads a mid-scan number and calls it done. ## Verification @@ -178,3 +194,10 @@ stub of the API above: create-once, idempotent re-run, no secret in a URL, refus by URL when the server does not answer, and the deployer calling it only after the health gate) and `tests/media_overlay.bats` (only [[Plex]] and [[Jellyfin]] may reference the metadata view). + +Five book folders hold media no scanner can read, so they stay out of the library: +three 2024 Nebula nominees in the `_unsorted` box (two `.m4b` with no `moov` atom, +one 36-file `.mp3` set that is not MPEG audio at all), `Anthony Heald/Bounty +Hunter Wars - Book 1` and `Star Wars Full Cast Audio Drama/Dark Empire III` — the +latter two also exist in readable form under their author folders. Fix by +re-acquiring the files; the scanner names every one of them in its log. -- 2.54.0