command-center/agent/managed-skills/stale-signal-audit/SKILL.md

name, description

name	description
stale-signal-audit	Audit homelab/fleet surfaces (status boards, metrics targets, test gates, docs, rule files) for signals that are already wrong — impossible-to-pass probes, assertions that pass by absence, linters that produce false positives, and prose invalidated by your own change. Use after a change lands, when a gate looks green but you want to know what it actually proves, or when asked what is broken.

Stale-signal audit

Signals rot without failing. This is the battery that found real defects on 4 of 4 passes in serverAppConfig; run it after any change and before trusting a green gate.

The five checks

1. Probe reachability. For every monitor/probe/alert target, ask: can this checker physically reach that endpoint? A probe that cannot succeed reports a false outage forever, and an always-red signal teaches the operator to ignore the surface.

   • Homepage (shared/homepage/services.json + homepage-status on Dankclaw): the monitor sees Dankclaw's loopback, container names, and published LAN ports only. A card probing another host's container-internal port can never be up → mark wip with the reason and let the app's card carry the status.
   • Prometheus: a scrape cannot express "is this HTTP service answering?" — scraping an HTML page gives unsupported Content-Type; a /metrics that 404s stays down. Remedy: blackbox exporter in probe mode. Never "fix" a probe by publishing a database or adding a wildcard bind.

2. Assertions that pass by absence. Grep-based tests against a file that was deleted or retired pass vacuously: a missing path makes grep exit 2 and a negated ! grep succeeds. Look for tests referencing paths that no longer exist (git ls-files the target, or just [ -f ] it).

   • Fix: assert the invariant directly ("no competes with the unit"), keep a conditional live check for the file returning, and drop dead variables that only referenced it.
   • Prove teeth: recreate the file/condition and watch the test go red. A test you have never seen fail is not a test.

3. Linter validity. Before acting on any checker's output, validate the checker against the resolver/contract it models. Obsidian example: wikilinks are case-insensitive, [[folder/Note]] resolves by path suffix not stem, and %20 in markdown links is URL-encoding — a naive checker reported 45 broken links when 2 were real. Same class: a find … | head -1 selector that reads the wrong file; a truncated listing that makes you "fix" a file that exists.

4. Gates that never run. Playwright specs in tests/e2e/ are matched by number prefix in playwright.config.ts; an unregistered spec silently never runs and the suite count stays the same. Verify by counting tests before/after (tests/e2e_scaffold.bats now fails on an unregistered spec).

5. Prose and comments invalidated by your own change. Clean cutover includes docs, unit docstrings, and the repo's rule file (AGENTS.md) — which is what every future agent reads first, so a stale claim there is the most expensive kind. Check: runbooks/rows naming a retired service, "unconfigured" claims after you configured something, upgrade instructions pointing at a deleted file, and lessons that the deployment has since contradicted.

Order

1. Enumerate the surfaces (status board, metrics targets, alert rules, test gates, docs/rule files, dashboard panels).
2. For each, ask what the signal can express, not what it says.
3. Validate any checker before editing the things it flags.
4. Fix at the root; never weaken an assertion to make it pass.
5. Gate the class so it cannot return (a seam), then prove the seam fails on the defect.

Verification

• Vault links: ./scripts/check-vault-links.py (exit 1 on breaks).
• Prometheus panels/rules: query each panel expression against the API — 0 series on a panel that should show data is a defect, 0 ALERTS when nothing fires is not.
• Suite: ./tests/run.sh; declares ALL TESTS PASSED.
• A gate you changed: recreate the defect and confirm red.