---
name: script-refactor-parity-vs-previous-revision
description: "Prove a refactor of sibling data-pipeline scripts is behavior-identical: run the pre-refactor revision from git HEAD beside the new one on the host that owns the venv/data, compare parsed records rather than bytes, and bootstrap the shared module so file, runpy and container paths all import it."
---

Refactoring a pipeline of sibling scripts (corpus builders, re-chunkers, pushers) whose only
tests are smokes. The test suite cannot see a moved record boundary, so parity has to be
*measured against the previous revision*.

## 1. Keep the old revision runnable

Never trust the diff. Materialize both sides where the data lives (the host with the venv,
tokenizer, parquet, corpus):

```sh
git show HEAD:scripts/<tool>.py > /tmp/qc-old/<tool>.py          # pre-refactor
cp scripts/<tool>.py scripts/<newmodule>.py /tmp/qc-new/         # post, module beside it
```

`ssh host 'cat > /tmp/...'` with the bytes on stdin is enough; no repo on the target needed.
Run both with **identical seeds and flags**, into separate output dirs.

## 2. Compare records, not bytes — unless the escaping is unchanged

- If the refactor funnels writes through one helper, that helper's `ensure_ascii` choice moves
  every non-ASCII character. A byte diff then reports a false regression while the rows are
  identical.
- Prove it correctly: parse both sides and compare in order —

```python
a = [json.loads(l) for l in open(old) if l.strip()]
b = [json.loads(l) for l in open(new) if l.strip()]
print(a == b, hashlib.sha256(json.dumps(a, sort_keys=True).encode()).hexdigest()[:16])
```

- Where the refactor does not touch serialization (a tool that already wrote literal UTF-8),
  demand the stronger proof: `cmp` + md5 equality of the artifact.
- Order matters: shuffled output with an unchanged seed proves the RNG draw sequence survived
  the hoist of nested closures. Compare the *sorted* file only as a second check — permutation
  equality alone hides a reordered `rng.choice`.
- Hashing the record set with `sort_keys=True` also catches dict-key drift in manifests.

## 3. Sibling-module bootstrap that survives every invocation

A new shared module must import when the tool is run three ways: `python3 scripts/tool.py`,
`runpy.run_path('/work/tool.py')` from a container mount (`-v scripts:/work:ro`), and from any
cwd. Standard preamble:

```python
_HERE = os.path.dirname(os.path.abspath(__file__)) if "__file__" in globals() else os.getcwd()
if _HERE not in sys.path:
    sys.path.insert(0, _HERE)
import <module>  # noqa: E402 - sibling module, same directory
```

Piping the script through `python3 -` has no `__file__` and defeats this — run tools by path and
say so in the module docstring. Verify both forms explicitly before claiming it works.

## 4. Unit-test the refusals without the heavy dependency

Hoist validation into a function so it can be driven with a stub, in-process:

```python
spec = importlib.util.spec_from_file_location("<module>", ".../<module>.py")
m = importlib.util.module_from_spec(spec); sys.modules["<module>"] = m; spec.loader.exec_module(m)
```

Registering the module in `sys.modules` under its real name *before* loading the dependent
script is required, or the script's own `import <module>` re-imports it and your stub is invisible.
A stub object with `encode`/`decode` that mangles ids past a threshold exercises a
"token identity" check in one page of output, no tokenizer file needed.

## 5. Gate order

`py_compile` every touched file → run the tool's `--help` through both invocation forms →
old-vs-new parity on the host → suite → commit. A suite that enumerates shell seams will not
notice a corpus-tool regression; the parity run is the only evidence there.