command-center/agent/managed-skills/script-refactor-parity-vs-previous-revision/SKILL.md

name, description

name	description
script-refactor-parity-vs-previous-revision	Prove a refactor of sibling data-pipeline scripts is behavior-identical: run the pre-refactor revision from git HEAD beside the new one on the host that owns the venv/data, compare parsed records rather than bytes, and bootstrap the shared module so file, runpy and container paths all import it.

Refactoring a pipeline of sibling scripts (corpus builders, re-chunkers, pushers) whose only tests are smokes. The test suite cannot see a moved record boundary, so parity has to be measured against the previous revision.

1. Keep the old revision runnable

Never trust the diff. Materialize both sides where the data lives (the host with the venv, tokenizer, parquet, corpus):

git show HEAD:scripts/<tool>.py > /tmp/qc-old/<tool>.py          # pre-refactor
cp scripts/<tool>.py scripts/<newmodule>.py /tmp/qc-new/         # post, module beside it

ssh host 'cat > /tmp/...' with the bytes on stdin is enough; no repo on the target needed. Run both with identical seeds and flags, into separate output dirs.

2. Compare records, not bytes — unless the escaping is unchanged

• If the refactor funnels writes through one helper, that helper's ensure_ascii choice moves every non-ASCII character. A byte diff then reports a false regression while the rows are identical.
• Prove it correctly: parse both sides and compare in order —

a = [json.loads(l) for l in open(old) if l.strip()]
b = [json.loads(l) for l in open(new) if l.strip()]
print(a == b, hashlib.sha256(json.dumps(a, sort_keys=True).encode()).hexdigest()[:16])

• Where the refactor does not touch serialization (a tool that already wrote literal UTF-8), demand the stronger proof: cmp + md5 equality of the artifact.
• Order matters: shuffled output with an unchanged seed proves the RNG draw sequence survived the hoist of nested closures. Compare the sorted file only as a second check — permutation equality alone hides a reordered rng.choice.
• Hashing the record set with sort_keys=True also catches dict-key drift in manifests.

3. Sibling-module bootstrap that survives every invocation

A new shared module must import when the tool is run three ways: python3 scripts/tool.py, runpy.run_path('/work/tool.py') from a container mount (-v scripts:/work:ro), and from any cwd. Standard preamble:

_HERE = os.path.dirname(os.path.abspath(__file__)) if "__file__" in globals() else os.getcwd()
if _HERE not in sys.path:
    sys.path.insert(0, _HERE)
import <module>  # noqa: E402 - sibling module, same directory

Piping the script through python3 - has no __file__ and defeats this — run tools by path and say so in the module docstring. Verify both forms explicitly before claiming it works.

4. Unit-test the refusals without the heavy dependency

Hoist validation into a function so it can be driven with a stub, in-process:

spec = importlib.util.spec_from_file_location("<module>", ".../<module>.py")
m = importlib.util.module_from_spec(spec); sys.modules["<module>"] = m; spec.loader.exec_module(m)

Registering the module in sys.modules under its real name before loading the dependent script is required, or the script's own import <module> re-imports it and your stub is invisible. A stub object with encode/decode that mangles ids past a threshold exercises a "token identity" check in one page of output, no tokenizer file needed.

5. Gate order

py_compile every touched file → run the tool's --help through both invocation forms → old-vs-new parity on the host → suite → commit. A suite that enumerates shell seams will not notice a corpus-tool regression; the parity run is the only evidence there.