command-center/agent/managed-skills/characterization-fixture-engine-cutover/SKILL.md

name, description

name	description
characterization-fixture-engine-cutover	Swap a path→service planner/mapping engine behind a golden fixture: capture, delta-reduce, document deltas, flip grep seams to behavioral, add render/drift gates, validate Caddyfile renders, Gitea PR URLs (serverAppConfigs cutover playbook)

Playbook for replacing a hand-maintained mapping engine (deploy planner, seam mapper, allow-lists) with a data-driven one without changing observable behavior. Proven on the 2026-10-13 shared/services.yml + scripts/service-map.py cutover (921 deltas → 17 intentional → green; 408/408 bats).

1. Capture the characterization fixture FIRST

• Enumerate every path the old engine sees: all tracked files x all targets (git ls-files), plus hand-added edge paths (retired files, cross-host paths, shared networks).
• Run the OLD engine on each, record path<TAB>target<TAB>services to tests/fixtures/*-golden.tsv. Old engine = current deploy-plan.py before edits.
• This fixture is the contract; the new engine must reproduce it except documented deltas.

2. Cut over, then delta-reduce with the fixture

• Diff new-engine output vs fixture; bucket every delta into: engine bug / intentional change / fixture staleness.
• Engine bugs found this way (generic classes): in membership on dict-of-tuples, host-scoping of relative directory keys, riders/children resolved only in the changed file's own directory, stem collisions between unit files and generated artifacts (.network stem == service dir stem), .network files entering the unit branch by stem match.
• Intentional deltas get documented individually in the commit message; NEVER batch-accept a delta count.
• Regenerate the fixture LAST, from the new engine, and record the delta count + reasons in the commit.

3. Flip text-grep seams to behavioral calls

• Old seams that grep a config file for the mapping (grep -q 'dankclaw:hermes-webui') die silently on cutover. Replace with a call to the new engine: run python3 scripts/service-map.py allows --host H --service S; assert_success.
• Vhost-scoped content greps (Caddyfile awk blocks) → grep the new canonical form (import guarded), not the inlined chain.

4. Renders + drift gates

• Config artifacts that were hand-maintained mirrors (node-exporter quadlet pair, pihole addn-hosts) become service-map.py render <name> --host H artifacts; a drift test asserts tracked-file == render output.
• Renderers must emit exactly what the consumer parses: dnsmasq addn-hosts treats inline # comments as hostnames — render no trailing comments.
• Named-snippet collapse (Caddyfile guard chain): the snippet MUST be defined before the first site that imports it (put (guarded) right after the global options block), or real caddy reports File to import not found.

5. Suite + deploy gates

• ./tests/run.sh fully green before each rank commit; one atomic commit per rank.
• Deploy is service-scoped: deploy.sh <host> --service <svc>; never a fleet drain for one service. The Playwright e2e gate runs inside deploy.sh against the live fleet.

Repo-specific gotchas (serverAppConfigs)

• Validate the Caddyfile with stock caddy:latest by DELETING the whole acme_dns porkbun { ... } block (awk 'NR<53 || NR>56' dankclaw/Caddyfile). A sed s/acme_dns porkbun {/acme_dns acme {/ variant that drops the { swallows the following tokens into the global block and fakes an "import not found" on (guarded) — debug sed swaps with a minimal repro file before blaming the config. Stock-image dns.providers.porkbun unregistered + basicauth REPLACE_WITH_HASH base64 errors are the expected endpoints of a clean parse.
• Gitea: branches with / 404 on /compare/main...wt%2F... — use the push-hook URL …/pulls/new/<branch>; private repo returns 404 to anonymous curl, which proves nothing about the page.
• test-changed.sh contract: new seam selection ⊇ old selection per path; unmapped repo path must die, never pass silently.
• Retire-to-trash: retired files (compose.yaml, dead templates, dead bats) → dated .AGENTTRASH/YYYYMMDD-topic/ beside them; every repo sweep must exclude .AGENTTRASH.
• bash test -f a b c && ! test -e d is invalid syntax (test takes one operand per operator); use ls a b c + separate ls d to census artifacts.