fix(media-verify): read the WAL the server commits through

`mode=ro&immutable=1` claims the file never changes, so sqlite skips the
write-ahead log entirely: every row the server has committed but not yet
checkpointed reads as MISSING, and a read beside a running scanner can fail
outright as "database disk image is malformed". Both servers keep their
libraries in WAL, so plain `mode=ro` is now the first door and `immutable=1`
only the fallback for a database whose `-shm` cannot be read.

The verdict also stopped guessing from the database alone. A file still at its
`src` is REAL (this pass's own damage); a file at neither path is `gone` (a
later pass or the server renamed it); a file sitting at `dst` with no row
naming it is something the server does not index. A row that still names `src`
is none of those — it is the server's stale pointer, and it costs a re-scan,
never a restore.
This commit is contained in:
Frank committed 2026-11-01 01:04:22 -07:00
1 parent f430e2264d
commit 649bbafa17
3 files changed
+131 -16

No files matched your search

+47 -12
View File
@@ -9,8 +9,13 @@ its destination, and NO row may still name its source — a source row is a scan
that has not caught up, and it is the one signature of a library pointing at
files that no longer exist.
Two doors, both read-only, both without a lock while the server runs
(`mode=ro&immutable=1`; there is no `sqlite3` binary on these hosts):
Two doors, both read-only, both without a lock while the server runs (there is
no `sqlite3` binary on these hosts). Open `mode=ro` PLAIN: both servers keep
their libraries in WAL mode, and `immutable=1` — which claims the file never
changes — makes sqlite skip the write-ahead log entirely, so every row the
server has committed but not yet checkpointed reads as MISSING, and a read
beside a running scanner can even fail as "database disk image is malformed".
`immutable=1` is only the fallback, for a database whose `-shm` cannot be read.
jellyfin `BaseItems.Path`. The container binds the share at `/data`, so the
database holds `/data/<library>/…` where the journal holds
@@ -22,9 +27,14 @@ Two doors, both read-only, both without a lock while the server runs
Neither server holds a row for everything the journal renames: a `.log`, `.cue`
or cover image has no row to land in, so its absence is not a fault. The report
therefore breaks the missing rows down by extension, and `--ignore-ext` names
the extensions this server does not index, which are then counted separately and
kept out of the verdict.
therefore breaks the missing rows down by extension, and `--ignore-ext` names the
extensions this server does not index, which are then counted separately and kept
out of the verdict. What is left is split by what the filesystem says, because
the database alone cannot: a file still at its `src` is `REAL` (this pass's own
damage), a file at neither path is `gone` (a later pass or the server renamed it),
and a file sitting at its `dst` with no row naming it is simply something the
server does not index. A row that still names `src` is none of these: it is the
server's own stale pointer — a phantom — and it costs a re-scan, never a restore.
Exit codes: 0 every move landed; 1 something is stale or missing (the details are
printed); 2 the journal or the database is not usable at all.
@@ -92,10 +102,18 @@ def read_moves(path):
def server_paths(db, kind):
"""Every path the server holds, or a refusal that names what is missing."""
spec = KINDS[kind]
try:
con = sqlite3.connect(f"file:{db}?mode=ro&immutable=1", uri=True)
except sqlite3.Error as e:
die(f"cannot open {db}: {e}")
con = None
err = None
# Plain ro reads the WAL the server commits through; immutable is the last
# resort, because it cannot see a row the server has not checkpointed.
for opt in ("mode=ro", "mode=ro&immutable=1"):
try:
con = sqlite3.connect(f"file:{db}?{opt}", uri=True)
break
except sqlite3.Error as e:
err = e
if con is None:
die(f"cannot open {db}: {err}")
with con:
have = {r[1] for r in con.execute(f"pragma table_info({spec['table']})")}
if not spec["columns"] <= have:
@@ -161,20 +179,37 @@ def main():
if at_src:
stale.append(src)
if not at_dst:
missing.append(dst)
missing.append((src, dst))
# The database cannot tell a lost file from one it never indexes: both read
# as "no row at the destination". Stat both paths, so the report separates
# the classes the runbook names — `REAL` is the only one that is this pass's
# own damage, and `gone` is a later pass or the server's own rename.
on_disk, real, gone = [], [], []
for src, dst in missing:
if os.path.exists(dst):
on_disk.append(dst)
elif os.path.exists(src):
real.append(src)
else:
gone.append(dst)
print(f"{args.db_kind}: {len(moves)} moves checked")
print(f"landed: {landed}")
print(f"still at src: {len(stale)}")
print(f"missing from the DB: {len(missing)}")
print(f" on disk at dst, no row names it: {len(on_disk)}")
print(f" REAL — file still at src: {len(real)}")
print(f" gone — at neither path: {len(gone)}")
if ignored:
print(f"not indexed (extension ignored): {ignored}")
if missing:
exts = Counter(os.path.splitext(d)[1].lower() or "(no extension)"
for d in missing)
for _, d in missing)
print("missing by extension: "
+ ", ".join(f"{e} {n}" for e, n in sorted(exts.items())))
for label, items in (("stale", stale), ("missing", missing)):
for label, items in (("real", real), ("gone", gone), ("missing", on_disk),
("stale", stale)):
for path in items[:args.show]:
print(f" {label}: {path}")
if len(items) > args.show: