live: heartbeat sync 2026-10-30T21:32:40-07:00 (Refs #11)
This commit is contained in:
1 parent
d03fe57bbd
commit
d627368f71
13 files changed
+280
-26
No files matched your search
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: media-journal-chain-verification
|
||||
description: "Verify a journal-guarded media relayout end to end: counts/bytes/size-mismatch from the JSONL journals, plus cross-journal accounting that proves \"missing\" destinations were moved onward rather than lost."
|
||||
---
|
||||
|
||||
# Verifying a media relayout across a chain of journals
|
||||
|
||||
`scripts/media-apply.py` (serverAppConfigs) writes a write-ahead JSONL journal beside the library root — one line per intent, `op` ∈ `mkdir|move`, a `move` record carrying `src`, `dst`, `bytes`, `mode`. Multiple runs over the same library produce **several journals** (relayout, then TV move, then duplicate staging), and the same file legitimately appears in more than one. Verifying one journal in isolation then reads as data loss.
|
||||
|
||||
## Verify counts and bytes from the journals, never from scratch files
|
||||
|
||||
`/tmp` and shell history are not records; the journals are. Read-only pass over every journal:
|
||||
|
||||
```python
|
||||
import json, os
|
||||
def recs(j): return [r for r in (json.loads(l) for l in open(j) if l.strip()) if r.get("op")=="move"]
|
||||
for j in journals:
|
||||
m = recs(j)
|
||||
gone = sum(1 for r in m if not os.path.exists(r["src"]))
|
||||
at = sum(1 for r in m if os.path.exists(r["dst"]))
|
||||
bad = sum(1 for r in m if os.path.exists(r["dst"])
|
||||
and os.path.getsize(r["dst"]) != r.get("bytes"))
|
||||
print(os.path.basename(j), len(m), sum(r.get("bytes",0) for r in m)/2**30, gone, at, bad)
|
||||
```
|
||||
|
||||
Assertions that matter: `src-gone == moves` (nothing half-moved), `size-mismatch == 0` (no truncation or clobber), `dst-present == moves` **for the newest journal only**.
|
||||
|
||||
## The chain step: account for every missing destination
|
||||
|
||||
An older journal's `dst` disappears when a later journal moves the file onward (episodes shelled out to the TV library, duplicates staged into `.AGENTTRASH`). Prove it instead of asserting it:
|
||||
|
||||
```python
|
||||
later = {}
|
||||
for j in later_journals:
|
||||
for r in recs(j): later.setdefault(r["src"], []).append(os.path.basename(j))
|
||||
missing = [r["dst"] for r in older_recs if not os.path.exists(r["dst"])]
|
||||
print(len(missing), sum(1 for d in missing if d in later),
|
||||
[d for d in missing if d not in later])
|
||||
```
|
||||
|
||||
Expect `unexplained = 0`. Measured 2026-10-30 on dankserver: 8,189 movie moves / 22,380 GiB with 174 destinations gone — exactly 124 (duplicate staging) + 6 (hand-filed strays) + 44 (mover episodes), `accounted-by-later-journal=174 unexplained=0`, zero size mismatches across all three journals.
|
||||
|
||||
## Post-purge consequence to state in the report
|
||||
|
||||
After an approved `.AGENTTRASH` purge the staging journal still parses, but `dst-present=0`: `media-apply.py rollback` on it recreates the folders **empty**. Say so — that journal is a directory-shape undo, not a byte restore; the keeper copy is the one that stayed in the library.
|
||||
|
||||
Do not trust a `dst-present` shortfall as loss until the cross-journal check has run, and never report counts from a `| grep -c` pipeline — mask-free exit status only.
|
||||
Reference in new issue
Block a user