48 lines
2.9 KiB
Markdown
48 lines
2.9 KiB
Markdown
---
|
|
name: media-journal-chain-verification
|
|
description: "Verify a journal-guarded media relayout end to end: counts/bytes/size-mismatch from the JSONL journals, plus cross-journal accounting that proves \"missing\" destinations were moved onward rather than lost."
|
|
---
|
|
|
|
# Verifying a media relayout across a chain of journals
|
|
|
|
`scripts/media-apply.py` (serverAppConfigs) writes a write-ahead JSONL journal beside the library root — one line per intent, `op` ∈ `mkdir|move`, a `move` record carrying `src`, `dst`, `bytes`, `mode`. Multiple runs over the same library produce **several journals** (relayout, then TV move, then duplicate staging), and the same file legitimately appears in more than one. Verifying one journal in isolation then reads as data loss.
|
|
|
|
## Verify counts and bytes from the journals, never from scratch files
|
|
|
|
`/tmp` and shell history are not records; the journals are. Read-only pass over every journal:
|
|
|
|
```python
|
|
import json, os
|
|
def recs(j): return [r for r in (json.loads(l) for l in open(j) if l.strip()) if r.get("op")=="move"]
|
|
for j in journals:
|
|
m = recs(j)
|
|
gone = sum(1 for r in m if not os.path.exists(r["src"]))
|
|
at = sum(1 for r in m if os.path.exists(r["dst"]))
|
|
bad = sum(1 for r in m if os.path.exists(r["dst"])
|
|
and os.path.getsize(r["dst"]) != r.get("bytes"))
|
|
print(os.path.basename(j), len(m), sum(r.get("bytes",0) for r in m)/2**30, gone, at, bad)
|
|
```
|
|
|
|
Assertions that matter: `src-gone == moves` (nothing half-moved), `size-mismatch == 0` (no truncation or clobber), `dst-present == moves` **for the newest journal only**.
|
|
|
|
## The chain step: account for every missing destination
|
|
|
|
An older journal's `dst` disappears when a later journal moves the file onward (episodes shelled out to the TV library, duplicates staged into `.AGENTTRASH`). Prove it instead of asserting it:
|
|
|
|
```python
|
|
later = {}
|
|
for j in later_journals:
|
|
for r in recs(j): later.setdefault(r["src"], []).append(os.path.basename(j))
|
|
missing = [r["dst"] for r in older_recs if not os.path.exists(r["dst"])]
|
|
print(len(missing), sum(1 for d in missing if d in later),
|
|
[d for d in missing if d not in later])
|
|
```
|
|
|
|
Expect `unexplained = 0`. Measured 2026-10-30 on dankserver: 8,189 movie moves / 22,380 GiB with 174 destinations gone — exactly 124 (duplicate staging) + 6 (hand-filed strays) + 44 (mover episodes), `accounted-by-later-journal=174 unexplained=0`, zero size mismatches across all three journals.
|
|
|
|
## Post-purge consequence to state in the report
|
|
|
|
After an approved `.AGENTTRASH` purge the staging journal still parses, but `dst-present=0`: `media-apply.py rollback` on it recreates the folders **empty**. Say so — that journal is a directory-shape undo, not a byte restore; the keeper copy is the one that stayed in the library.
|
|
|
|
Do not trust a `dst-present` shortfall as loss until the cross-journal check has run, and never report counts from a `| grep -c` pipeline — mask-free exit status only.
|