From f71c7ccce40de6f7f300b4016b49b3c5e9985861 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 03:41:53 -0800 Subject: [PATCH] docs(audiobookshelf): the grouping rule, not "shallowest folder with audio" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Corrects the law pushed earlier today. The mechanism is `groupFileItemsIntoLibraryItemDirs` (utils/scandir.js), called on a live `recurseFiles` walk of the tree EVERY scan: each file attaches to the FIRST directory along its path that already has a group, so one media file at a collection root owns everything beneath it. Measured: one 949 MB `.cbr` hid 652 book folders behind a single 5,693-track item. - "Media file" is audio OR ebook (`audiobooksOnly=false` -> SupportedAudioTypes ∪ SupportedEbookTypes, 17 audio + 6 ebook extensions). An audio-only census cannot see the file that hides an Audiobooks library. - Deleting the covering item does not fix a shadow: grouping is recomputed from disk and the next scan rebuilds the item (measured: `1 Added`, same path). - Un-shadow by MOVE: one file per folder is ABS's own single-book layout. - `?force=1` is not a discovery arm (both arms walk the disk; force re-scans every existing item, 1:33 against 0:04). - Paging corrected: `offset` is ignored outright, `page` is the parameter. - `DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()` of the whole swallowed subtree. Never pass `hard`. Shape table gains the layout row and the move journal; the stale "1,103 shelves" scan count is destroyed. --- .../Dankserver/Audiobookshelf.md | 59 ++++++++++++------- 1 file changed, 38 insertions(+), 21 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index 5ce6c80..837370b 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -36,6 +36,7 @@ manager) — **Internal-only:** LAN address, no vhost, no DNS name. | Self-healing | `Restart=always`, `RestartSec=5`, `WantedBy=default.target` | | Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` | | Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` | +| Library layout | one book per folder — ABS groups by the first directory holding a media file (see Laws); every move is journalled at `/mnt/entropy/media-relayout-audiobooks.jsonl` | ## Bootstrap @@ -59,7 +60,7 @@ Dankserver. Change it in **Settings → Users → frank → Password**; the secr then holds a stale value and only matters again if the `config` bind is ever destroyed (a fresh DB makes the script create a new owner from it). -The scan of 1,103 shelves is slow and background; watch `items:` climb from 0, +A full scan is slow and runs in the background; watch the `items:` total climb, never `/healthcheck`. It also reads files that are genuinely broken and logs `[AudioFileScanner] Invalid data found when processing input` per file — that is the file, not the bind: the same line count grows while the total climbs. @@ -120,31 +121,47 @@ the file, not the bind: the same line count grows while the total climbs. - **The item count is not a field on the library DTO, and paging never ends itself.** The door is `GET /api/libraries//items?limit=1` → `total`; a poller that guesses a plausible name (`numItems`) prints `0` forever beside a scan that - is working. An out-of-range `offset` returns the last item rather than an empty - page, so a census that loops "until empty" invents its own total: `limit=500` - against a `total` of 4,628 hands back exactly 5,000 rows — the collected count - is `ceil(total / limit) × limit`, which is why two runs reported "exactly 1,500" - and "exactly 4,000" for one library. Stop on `total`, count *unique ids*, and - take the number after the pass settles (`total` itself moves during a scan). + is working. Paging is on `page` (0-based) plus `limit`, and **`offset` is + ignored**: `limit=500&offset=500` returns the same 500 ids as `offset=0`, while + `page=1` shares none of them. A census that pages on `offset` therefore reads + `ceil(total / limit) × limit` rows containing only `limit` distinct items — the + exact origin of the "exactly 1,500" and "exactly 4,000" counts two runs + reported for one library. Page on `page`, count *unique ids*, stop on `total`, + and take the number after the pass settles (`total` moves during a scan). - **A fresh scan is the only thing that notices new audio.** `inotify` does not cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that law — so the in-container watcher sees nothing a host process files onto the array, and the deployer's re-run of the bootstrapper is the cheap scan trigger. -- **A book is the shallowest folder that holds audio — and that item owns the - subtree beneath it.** A folder with audio in its own file list becomes ONE book - and the scanner folds everything below it into that book's tracks instead of - making items: 6 stray files at the root of `_unsorted` turned the whole - collection dump into a single item of 5,693 tracks, titled after whichever file - supplied metadata, with 542 real books inside it. So "hundreds of books are - missing" is a layout fault, not a scan fault — and no scan, forced or not, - subdivides an existing item. Census the layout the server's way: a directory is - a book folder iff its *own* files include an audio extension. -- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** Read - `LibraryItemController.delete`: `hardDelete = req.query.hard == 1` and then +- **One stray media file at a collection root turns the whole collection into one + book.** Every scan re-walks the tree from disk (`scanFolder` → `recurseFiles`), + and `utils/scandir.js:groupFileItemsIntoLibraryItemDirs` assigns each file to the + FIRST directory along its path that already has a group: the first file seen at a + root owns every deeper file, and the folders below it never become items. One + 949 MB `.cbr` sitting at `_unsorted`'s root collapsed 652 book folders into a + single item of 5,693 tracks, titled after whichever file supplied metadata. + "Hundreds of books are missing" is a layout fault, not a scan fault. +- **"Media file" means audio OR ebook, so an audio-only census cannot see the file + that is hiding the library.** With `audiobooksOnly=false` (the deployed shape) + the predicate is `SupportedAudioTypes` ∪ `SupportedEbookTypes`: 20 audio + extensions (including `mp4`, `mpg`, `webm`) plus `epub pdf mobi azw3 cbr cbz`. A + census reporting "that root holds no loose audio" can be correct and still + useless — the file doing the shadowing here was a comic archive. +- **Deleting the covering item does not un-shadow anything.** Grouping is recomputed + from disk, so the next scan rebuilds the item from the same stray file (measured: + `DELETE`, then scan → `1 Added`, the item back at the same path). The fix is the + MOVE: give every stray root file a folder of its own (one file per folder is + ABS's own single-book layout; formats of one book share a folder), through the + journal-guarded mover, then delete the stale item so the old path is not kept, + then scan. +- **`?force=1` is not a discovery arm.** Both arms of `POST /libraries/:id/scan` + walk the disk; force additionally re-scans every item that already exists + (measured 1:33 against 0:04 on ~4,600 items). Finding new folders needs neither, + so the deployer's cheap trigger stays on the plain arm. +- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** In + `LibraryItemController.delete`, `hardDelete = req.query.hard == 1` and then `fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole - swallowed subtree — `hard=1` there deletes every audiobook under it off the - array. Un-swallowing is: file the loose root files with the journal-guarded - mover, then delete the stale covering item with NO query string. + swallowed subtree — `hard=1` there deletes every book beneath it off the array. + Assert the item's `path` first, and never pass `hard`. - **`Invalid data found when processing input` names the file, not the bind.** The scanner logs it per file and skips the file — the book never appears — so a missing title beside those lines is library debris, not a server fault. Census of