media-verify reads the WAL; the metadata view is live; duplicates get a page #114

2026-11-01T01:12:26-07:00

dank commented

2026-11-01 01:12:26 -07:00

Two commits out of tonight's duplicate sweep of /mnt/entropy/Movies.

`fix(media-verify)`: read the WAL the server commits through

mode=ro&immutable=1 tells sqlite the file never changes, so it skips the
write-ahead log entirely: every row the server has committed but not yet
checkpointed reads as MISSING, and a read taken beside a running scanner can
fail outright as database disk image is malformed. Both servers keep their
libraries in WAL, so plain mode=ro is now the first door and immutable=1 only
the fallback for a database whose -shm cannot be read.

The verdict stopped guessing from the database alone too: a file still at src is
REAL (this pass's own damage), a file at neither path is gone, a file sitting
at dst with no row naming it is simply not indexed. A row still naming src is
the server's stale pointer, and it costs a re-scan — never a restore.

`docs(media)`: the view is live, and duplicates get their own page

Media Metadata View.md claimed "Not deployed on Dankserver — verified
2026-10-31". That was true yesterday. The overlay is mounted and both installed
quadlets bind it (Volume=/var/lib/media-meta/view:/data rw for Jellyfin,
view:/mnt/entropy:ro for Plex), so the page now states the live shape and the
order that proves it (media-view.sh verify, then the Volume= lines). The same
false claim sat in the AGENTS.md summary table; it is corrected there too.

New page Media Duplicate Prune.md: the screen ladder (size → head+tail
blake2b → sparse-window confirmation before anything moves), which copy
survives, the guards that each encode a file that would otherwise have been
destroyed (qBittorrent .!qB state, year-required-with-digits-kept, feature
length, disc/cd/part, media containers only), and the law that a hold on
the same volume frees nothing until it is purged. It also carries the way
media-verify.py has to be read for a prune — landed: 0 is the correct end
state because no server may index the hold — and the planner bug that explains it:
re-deriving "largest wins" from the hold list alone dropped every keeper out of its
own group and pruned 17 files instead of 205.

Media Relayout.md keeps a one-line pointer; carrying the whole procedure there
pushed it past the vault's 500-line ceiling.

Fleet state this run produced (record, not a proposal)

226 files (~1.1 TiB) held in /mnt/entropy/.AGENTTRASH/20261101-duplicate-prune;
journals /mnt/entropy/dup-prune-exact.jsonl and dup-prune-similar.jsonl.
verify reports all 226 at their targets, REAL: 0.
The 8,044-row census reconciles exactly: 7,818 present + 226 held + 0
unexplained.
10 release folders the pass emptied were swept.
The purge has not been run — same filesystem, so it frees nothing yet, and
purging is Frank's word.
Two byte-identical .!qB qBittorrent leftovers (53.8 GiB, 13 months stale) were
deliberately left where they are.
Scans: Jellyfin Scan Media Library queued once and running; Plex's Movies
section (key 1) refreshes only after Jellyfin reaches Idle, per the rescan law.

Evidence

tests/run.sh: 761 tests, all pass. scripts/check-okf.py: 0 violations. Vault
index regenerated, which the index seam requires to be a no-op.

Local branch label is this session's worktree branch (wt-…) because workspace
policy blocks git checkout/switch in a shared checkout; the remote branch name
above is the conventional one, and the diff against main is exactly these two
commits.

cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb

Two commits out of tonight's duplicate sweep of `/mnt/entropy/Movies`. ## `fix(media-verify)`: read the WAL the server commits through `mode=ro&immutable=1` tells sqlite the file never changes, so it skips the write-ahead log entirely: every row the server has committed but not yet checkpointed reads as **MISSING**, and a read taken beside a running scanner can fail outright as `database disk image is malformed`. Both servers keep their libraries in WAL, so plain `mode=ro` is now the first door and `immutable=1` only the fallback for a database whose `-shm` cannot be read. The verdict stopped guessing from the database alone too: a file still at `src` is `REAL` (this pass's own damage), a file at neither path is `gone`, a file sitting at `dst` with no row naming it is simply not indexed. A row still naming `src` is the server's stale pointer, and it costs a re-scan — never a restore. ## `docs(media)`: the view is live, and duplicates get their own page `Media Metadata View.md` claimed "Not deployed on Dankserver — verified 2026-10-31". That was true yesterday. The overlay is mounted and **both installed quadlets bind it** (`Volume=/var/lib/media-meta/view:/data` rw for Jellyfin, `view:/mnt/entropy:ro` for Plex), so the page now states the live shape and the order that proves it (`media-view.sh verify`, then the `Volume=` lines). The same false claim sat in the `AGENTS.md` summary table; it is corrected there too. New page `Media Duplicate Prune.md`: the screen ladder (size → head+tail `blake2b` → sparse-window confirmation *before* anything moves), which copy survives, the guards that each encode a file that would otherwise have been destroyed (qBittorrent `.!qB` state, year-required-with-digits-kept, feature length, `disc`/`cd`/`part`, media containers only), and the law that **a hold on the same volume frees nothing** until it is purged. It also carries the way `media-verify.py` has to be *read* for a prune — `landed: 0` is the correct end state because no server may index the hold — and the planner bug that explains it: re-deriving "largest wins" from the hold list alone dropped every keeper out of its own group and pruned 17 files instead of 205. `Media Relayout.md` keeps a one-line pointer; carrying the whole procedure there pushed it past the vault's 500-line ceiling. ## Fleet state this run produced (record, not a proposal) - 226 files (~1.1 TiB) held in `/mnt/entropy/.AGENTTRASH/20261101-duplicate-prune`; journals `/mnt/entropy/dup-prune-exact.jsonl` and `dup-prune-similar.jsonl`. `verify` reports all 226 at their targets, `REAL: 0`. - The 8,044-row census reconciles exactly: 7,818 present + 226 held + **0 unexplained**. - 10 release folders the pass emptied were swept. - **The purge has not been run** — same filesystem, so it frees nothing yet, and purging is Frank's word. - Two byte-identical `.!qB` qBittorrent leftovers (53.8 GiB, 13 months stale) were deliberately left where they are. - Scans: Jellyfin `Scan Media Library` queued once and running; Plex's Movies section (key 1) refreshes only after Jellyfin reaches `Idle`, per the rescan law. ## Evidence `tests/run.sh`: 761 tests, all pass. `scripts/check-okf.py`: 0 violations. Vault index regenerated, which the index seam requires to be a no-op. Local branch label is this session's worktree branch (`wt-…`) because workspace policy blocks `git checkout`/`switch` in a shared checkout; the remote branch name above is the conventional one, and the diff against `main` is exactly these two commits. --- cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb

dank added 2 commits 2026-11-01 01:12:26 -07:00

fix(media-verify): read the WAL the server commits through 649bbafa17

`mode=ro&immutable=1` claims the file never changes, so sqlite skips the
write-ahead log entirely: every row the server has committed but not yet
checkpointed reads as MISSING, and a read beside a running scanner can fail
outright as "database disk image is malformed". Both servers keep their
libraries in WAL, so plain `mode=ro` is now the first door and `immutable=1`
only the fallback for a database whose `-shm` cannot be read.

The verdict also stopped guessing from the database alone. A file still at its
`src` is REAL (this pass's own damage); a file at neither path is `gone` (a
later pass or the server renamed it); a file sitting at `dst` with no row
naming it is something the server does not index. A row that still names `src`
is none of those — it is the server's stale pointer, and it costs a re-scan,
never a restore.

docs(media): the metadata view is live, and duplicate pruning gets a page 50e0c5a786

`Media Metadata View.md` said "Not deployed on Dankserver — verified
2026-10-31". That was true yesterday and is false now: the overlay is mounted
and both installed quadlets bind it (`view:/data` rw for Jellyfin,
`view:/mnt/entropy:ro` for Plex). The page now states the live shape and the
order that proves it — `media-view.sh verify`, then the `Volume=` lines —
because the whole point of the page is to answer "is the view really serving?".
`AGENTS.md` carried the same stale claim in its summary table.

`Media Duplicate Prune.md` is new: the screen ladder (size, then head+tail
blake2b, then a sparse-window confirmation before anything moves), which copy
survives and why, the guards that each encode a file that would otherwise have
been destroyed (qBittorrent `.!qB` state, year-required with digits kept,
feature length, disc/part exclusion, media containers only), and the law that a
hold on the same volume frees nothing until it is purged. It also carries the
different way `media-verify.py` has to be READ for a prune — `landed: 0` is the
correct end state because no server may index the hold — and the planner bug
that class explains: when the planner re-derived "largest wins" from the hold
list alone it dropped every keeper out of its own group and pruned 17 files
instead of 205.

`Media Relayout.md` keeps a one-line pointer; carrying the whole procedure there
pushed it past the vault's 500-line ceiling, which is the linter telling the
truth about progressive disclosure.

dank closed this pull request

2026-11-01 01:34:59 -07:00

dank deleted branch fix/media-verify-wal-and-prune-record

2026-11-01 01:34:59 -07:00

dank referenced this pull request

2026-11-01 01:42:20 -07:00

media-verify reads the WAL; the metadata view is live; duplicates get a page #115

dank commented

2026-11-01 01:42:20 -07:00

Superseded by #115, which carries the same commits with origin/main merged in. This PR auto-closed when its head branch was deleted (the cleanup step ran before the merge was verified); the branch was re-pushed, so nothing was lost. Landed as #115.

cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb

Superseded by #115, which carries the same commits with `origin/main` merged in. This PR auto-closed when its head branch was deleted (the cleanup step ran before the merge was verified); the branch was re-pushed, so nothing was lost. Landed as #115. --- cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dank/serverAppConfigs#114