docs(audiobookshelf): the grouping rule, not "shallowest folder with audio"
Corrects the law pushed earlier today. The mechanism is `groupFileItemsIntoLibraryItemDirs` (utils/scandir.js), called on a live `recurseFiles` walk of the tree EVERY scan: each file attaches to the FIRST directory along its path that already has a group, so one media file at a collection root owns everything beneath it. Measured: one 949 MB `.cbr` hid 652 book folders behind a single 5,693-track item. - "Media file" is audio OR ebook (`audiobooksOnly=false` -> SupportedAudioTypes ∪ SupportedEbookTypes, 17 audio + 6 ebook extensions). An audio-only census cannot see the file that hides an Audiobooks library. - Deleting the covering item does not fix a shadow: grouping is recomputed from disk and the next scan rebuilds the item (measured: `1 Added`, same path). - Un-shadow by MOVE: one file per folder is ABS's own single-book layout. - `?force=1` is not a discovery arm (both arms walk the disk; force re-scans every existing item, 1:33 against 0:04). - Paging corrected: `offset` is ignored outright, `page` is the parameter. - `DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()` of the whole swallowed subtree. Never pass `hard`. Shape table gains the layout row and the move journal; the stale "1,103 shelves" scan count is destroyed.
This commit is contained in:
1 parent
f541566714
commit
f71c7ccce4
1 file changed
+38
-21
@@ -36,6 +36,7 @@ manager) — **Internal-only:** LAN address, no vhost, no DNS name.
|
|||||||
| Self-healing | `Restart=always`, `RestartSec=5`, `WantedBy=default.target` |
|
| Self-healing | `Restart=always`, `RestartSec=5`, `WantedBy=default.target` |
|
||||||
| Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` |
|
| Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` |
|
||||||
| Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` |
|
| Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` |
|
||||||
|
| Library layout | one book per folder — ABS groups by the first directory holding a media file (see Laws); every move is journalled at `/mnt/entropy/media-relayout-audiobooks.jsonl` |
|
||||||
|
|
||||||
## Bootstrap
|
## Bootstrap
|
||||||
|
|
||||||
@@ -59,7 +60,7 @@ Dankserver. Change it in **Settings → Users → frank → Password**; the secr
|
|||||||
then holds a stale value and only matters again if the `config` bind is ever
|
then holds a stale value and only matters again if the `config` bind is ever
|
||||||
destroyed (a fresh DB makes the script create a new owner from it).
|
destroyed (a fresh DB makes the script create a new owner from it).
|
||||||
|
|
||||||
The scan of 1,103 shelves is slow and background; watch `items:` climb from 0,
|
A full scan is slow and runs in the background; watch the `items:` total climb,
|
||||||
never `/healthcheck`. It also reads files that are genuinely broken and logs
|
never `/healthcheck`. It also reads files that are genuinely broken and logs
|
||||||
`[AudioFileScanner] Invalid data found when processing input` per file — that is
|
`[AudioFileScanner] Invalid data found when processing input` per file — that is
|
||||||
the file, not the bind: the same line count grows while the total climbs.
|
the file, not the bind: the same line count grows while the total climbs.
|
||||||
@@ -120,31 +121,47 @@ the file, not the bind: the same line count grows while the total climbs.
|
|||||||
- **The item count is not a field on the library DTO, and paging never ends
|
- **The item count is not a field on the library DTO, and paging never ends
|
||||||
itself.** The door is `GET /api/libraries/<id>/items?limit=1` → `total`; a poller
|
itself.** The door is `GET /api/libraries/<id>/items?limit=1` → `total`; a poller
|
||||||
that guesses a plausible name (`numItems`) prints `0` forever beside a scan that
|
that guesses a plausible name (`numItems`) prints `0` forever beside a scan that
|
||||||
is working. An out-of-range `offset` returns the last item rather than an empty
|
is working. Paging is on `page` (0-based) plus `limit`, and **`offset` is
|
||||||
page, so a census that loops "until empty" invents its own total: `limit=500`
|
ignored**: `limit=500&offset=500` returns the same 500 ids as `offset=0`, while
|
||||||
against a `total` of 4,628 hands back exactly 5,000 rows — the collected count
|
`page=1` shares none of them. A census that pages on `offset` therefore reads
|
||||||
is `ceil(total / limit) × limit`, which is why two runs reported "exactly 1,500"
|
`ceil(total / limit) × limit` rows containing only `limit` distinct items — the
|
||||||
and "exactly 4,000" for one library. Stop on `total`, count *unique ids*, and
|
exact origin of the "exactly 1,500" and "exactly 4,000" counts two runs
|
||||||
take the number after the pass settles (`total` itself moves during a scan).
|
reported for one library. Page on `page`, count *unique ids*, stop on `total`,
|
||||||
|
and take the number after the pass settles (`total` moves during a scan).
|
||||||
- **A fresh scan is the only thing that notices new audio.** `inotify` does not
|
- **A fresh scan is the only thing that notices new audio.** `inotify` does not
|
||||||
cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that
|
cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that
|
||||||
law — so the in-container watcher sees nothing a host process files onto the
|
law — so the in-container watcher sees nothing a host process files onto the
|
||||||
array, and the deployer's re-run of the bootstrapper is the cheap scan trigger.
|
array, and the deployer's re-run of the bootstrapper is the cheap scan trigger.
|
||||||
- **A book is the shallowest folder that holds audio — and that item owns the
|
- **One stray media file at a collection root turns the whole collection into one
|
||||||
subtree beneath it.** A folder with audio in its own file list becomes ONE book
|
book.** Every scan re-walks the tree from disk (`scanFolder` → `recurseFiles`),
|
||||||
and the scanner folds everything below it into that book's tracks instead of
|
and `utils/scandir.js:groupFileItemsIntoLibraryItemDirs` assigns each file to the
|
||||||
making items: 6 stray files at the root of `_unsorted` turned the whole
|
FIRST directory along its path that already has a group: the first file seen at a
|
||||||
collection dump into a single item of 5,693 tracks, titled after whichever file
|
root owns every deeper file, and the folders below it never become items. One
|
||||||
supplied metadata, with 542 real books inside it. So "hundreds of books are
|
949 MB `.cbr` sitting at `_unsorted`'s root collapsed 652 book folders into a
|
||||||
missing" is a layout fault, not a scan fault — and no scan, forced or not,
|
single item of 5,693 tracks, titled after whichever file supplied metadata.
|
||||||
subdivides an existing item. Census the layout the server's way: a directory is
|
"Hundreds of books are missing" is a layout fault, not a scan fault.
|
||||||
a book folder iff its *own* files include an audio extension.
|
- **"Media file" means audio OR ebook, so an audio-only census cannot see the file
|
||||||
- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** Read
|
that is hiding the library.** With `audiobooksOnly=false` (the deployed shape)
|
||||||
`LibraryItemController.delete`: `hardDelete = req.query.hard == 1` and then
|
the predicate is `SupportedAudioTypes` ∪ `SupportedEbookTypes`: 20 audio
|
||||||
|
extensions (including `mp4`, `mpg`, `webm`) plus `epub pdf mobi azw3 cbr cbz`. A
|
||||||
|
census reporting "that root holds no loose audio" can be correct and still
|
||||||
|
useless — the file doing the shadowing here was a comic archive.
|
||||||
|
- **Deleting the covering item does not un-shadow anything.** Grouping is recomputed
|
||||||
|
from disk, so the next scan rebuilds the item from the same stray file (measured:
|
||||||
|
`DELETE`, then scan → `1 Added`, the item back at the same path). The fix is the
|
||||||
|
MOVE: give every stray root file a folder of its own (one file per folder is
|
||||||
|
ABS's own single-book layout; formats of one book share a folder), through the
|
||||||
|
journal-guarded mover, then delete the stale item so the old path is not kept,
|
||||||
|
then scan.
|
||||||
|
- **`?force=1` is not a discovery arm.** Both arms of `POST /libraries/:id/scan`
|
||||||
|
walk the disk; force additionally re-scans every item that already exists
|
||||||
|
(measured 1:33 against 0:04 on ~4,600 items). Finding new folders needs neither,
|
||||||
|
so the deployer's cheap trigger stays on the plain arm.
|
||||||
|
- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** In
|
||||||
|
`LibraryItemController.delete`, `hardDelete = req.query.hard == 1` and then
|
||||||
`fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole
|
`fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole
|
||||||
swallowed subtree — `hard=1` there deletes every audiobook under it off the
|
swallowed subtree — `hard=1` there deletes every book beneath it off the array.
|
||||||
array. Un-swallowing is: file the loose root files with the journal-guarded
|
Assert the item's `path` first, and never pass `hard`.
|
||||||
mover, then delete the stale covering item with NO query string.
|
|
||||||
- **`Invalid data found when processing input` names the file, not the bind.** The
|
- **`Invalid data found when processing input` names the file, not the bind.** The
|
||||||
scanner logs it per file and skips the file — the book never appears — so a
|
scanner logs it per file and skips the file — the book never appears — so a
|
||||||
missing title beside those lines is library debris, not a server fault. Census of
|
missing title beside those lines is library debris, not a server fault. Census of
|
||||||
|
|||||||
Reference in new issue
Block a user