fix(media): verify read the library the way the servers write it

A verify run beside a running scan reported rows the scanner had just
written as MISSING, and could fail outright as "database disk image is
malformed". Both libraries live in WAL mode, and the verifier opened them
with immutable=1, which tells sqlite the file never changes: it skips the
write-ahead log, so every committed-but-uncheckpointed row is invisible.
134 MB of jellyfin.db-wal was riding past the verifier's eyes during the
2026-10-31 rescan.

Open plain mode=ro first — sqlite then reads the WAL through its -shm —
and keep immutable=1 only as the fallback for a database whose -shm
cannot be read. The bats fixture leaves a journaled move uncheckpointed
and proves the verifier still counts it landed.
This commit is contained in:
Frank committed 2026-10-31 10:00:11 -07:00
1 parent 5dbbfcb8a0
commit afad21dda5
2 files changed
+52 -4

No files matched your search

+17 -4
View File
@@ -9,8 +9,13 @@ its destination, and NO row may still name its source — a source row is a scan
that has not caught up, and it is the one signature of a library pointing at
files that no longer exist.
Two doors, both read-only, both without a lock while the server runs
(`mode=ro&immutable=1`; there is no `sqlite3` binary on these hosts):
Two doors, both read-only, both without a lock while the server runs (there is
no `sqlite3` binary on these hosts). Open `mode=ro` PLAIN: both servers keep
their libraries in WAL mode, and `immutable=1` — which claims the file never
changes — makes sqlite skip the write-ahead log entirely, so every row the
server has committed but not yet checkpointed reads as MISSING, and a read
beside a running scanner can even fail as "database disk image is malformed".
`immutable=1` is only the fallback, for a database whose `-shm` cannot be read.
jellyfin `BaseItems.Path`. The container binds the share at `/data`, so the
database holds `/data/<library>/…` where the journal holds
@@ -92,10 +97,18 @@ def read_moves(path):
def server_paths(db, kind):
"""Every path the server holds, or a refusal that names what is missing."""
spec = KINDS[kind]
con = None
err = None
# Plain ro reads the WAL the server commits through; immutable is the last
# resort, because it cannot see a row the server has not checkpointed.
for opt in ("mode=ro", "mode=ro&immutable=1"):
try:
con = sqlite3.connect(f"file:{db}?mode=ro&immutable=1", uri=True)
con = sqlite3.connect(f"file:{db}?{opt}", uri=True)
break
except sqlite3.Error as e:
die(f"cannot open {db}: {e}")
err = e
if con is None:
die(f"cannot open {db}: {err}")
with con:
have = {r[1] for r in con.execute(f"pragma table_info({spec['table']})")}
if not spec["columns"] <= have:
+35
View File
@@ -210,3 +210,38 @@ setup() {
[ "${lines[3]}" = "missing from the DB: 0" ]
[ "${lines[4]}" = "not indexed (extension ignored): 1" ]
}
# wal_case <db> — a Jellyfin database whose NEW row lives only in the
# write-ahead log: the server has committed the rename but not checkpointed it,
# which is the normal state of a library a scan is writing. The holder
# connection keeps the WAL from folding into the main file, and the verifier
# runs as a real subprocess so it faces the same files Plex and Jellyfin leave.
wal_case() {
python3 - "$JFDB" "$JOURNAL" "$SHARE" "$VERIFY" <<'PY'
import sqlite3, subprocess, sys
db, journal, share, verify = sys.argv[1:5]
w = sqlite3.connect(db)
w.execute("pragma journal_mode=wal")
w.execute("pragma wal_autocheckpoint=0")
w.execute("insert into BaseItems (Path, Type) values ('/data/Movies/Old.mkv', 'x')")
w.commit()
w.execute("pragma wal_checkpoint(truncate)")
holder = sqlite3.connect(db) # last-connection close would checkpoint
w.execute("update BaseItems set Path='/data/Movies/Old/Old.mkv'")
w.commit()
r = subprocess.run([sys.executable, verify, "--journal", journal,
"--db-kind", "jellyfin", "--db", db,
"--share", share, "--container", "/data"],
capture_output=True, text=True)
print(r.stdout, end="")
sys.exit(r.returncode)
PY
}
@test "a row the server committed but has not checkpointed is proof, not a miss" {
move_row "$SHARE/Movies/Old.mkv" "$SHARE/Movies/Old/Old.mkv"
run wal_case
[ "$status" -eq 0 ]
[ "${lines[1]}" = "landed: 1" ]
[ "${lines[3]}" = "missing from the DB: 0" ]
}