---
name: stale-signal-audit
description: "Audit homelab/fleet surfaces (status boards, metrics targets, test gates, docs, rule files) for signals that are already wrong — impossible-to-pass probes, assertions that pass by absence, linters that produce false positives, and prose invalidated by your own change. Use after a change lands, when a gate looks green but you want to know what it actually proves, or when asked what is broken."
---

# Stale-signal audit

Signals rot without failing. This is the battery that found real defects on 4 of 4
passes in serverAppConfig; run it after any change and before trusting a green gate.

## The five checks

1. **Probe reachability.** For every monitor/probe/alert target, ask: *can this
   checker physically reach that endpoint?* A probe that cannot succeed reports a
   false outage forever, and an always-red signal teaches the operator to ignore
   the surface.
   - Homepage (`shared/homepage/services.json` + `homepage-status` on Dankclaw): the
     monitor sees Dankclaw's loopback, container names, and **published** LAN ports
     only. A card probing another host's container-internal port can never be up →
     mark `wip` with the reason and let the app's card carry the status.
   - Prometheus: a *scrape* cannot express "is this HTTP service answering?" —
     scraping an HTML page gives `unsupported Content-Type`; a `/metrics` that 404s
     stays down. Remedy: blackbox exporter in probe mode. Never "fix" a probe by
     publishing a database or adding a wildcard bind.

2. **Assertions that pass by absence.** Grep-based tests against a file that was
   deleted or retired pass vacuously: a missing path makes `grep` exit 2 and a
   negated `! grep` succeeds. Look for tests referencing paths that no longer exist
   (`git ls-files` the target, or just `[ -f ]` it).
   - Fix: assert the invariant directly ("no <file> competes with the unit"), keep a
     conditional live check for the file returning, and drop dead variables that
     only referenced it.
   - **Prove teeth**: recreate the file/condition and watch the test go red. A test
     you have never seen fail is not a test.

3. **Linter validity.** Before acting on any checker's output, validate the checker
   against the resolver/contract it models. Obsidian example: wikilinks are
   case-insensitive, `[[folder/Note]]` resolves by **path suffix** not stem, and
   `%20` in markdown links is URL-encoding — a naive checker reported 45 broken
   links when 2 were real. Same class: a `find … | head -1` selector that reads the
   wrong file; a truncated listing that makes you "fix" a file that exists.

4. **Gates that never run.** Playwright specs in `tests/e2e/` are matched by
   **number prefix** in `playwright.config.ts`; an unregistered spec silently never
   runs and the suite count stays the same. Verify by counting tests before/after
   (`tests/e2e_scaffold.bats` now fails on an unregistered spec).

5. **Prose and comments invalidated by your own change.** Clean cutover includes
   docs, unit docstrings, and the repo's rule file (`AGENTS.md`) — which is what
   every future agent reads first, so a stale claim there is the most expensive kind.
   Check: runbooks/rows naming a retired service, "unconfigured" claims after you
   configured something, upgrade instructions pointing at a deleted file, and
   lessons that the deployment has since contradicted.

## Order

1. Enumerate the surfaces (status board, metrics targets, alert rules, test gates,
   docs/rule files, dashboard panels).
2. For each, ask what the signal *can* express, not what it says.
3. Validate any checker before editing the things it flags.
4. Fix at the root; never weaken an assertion to make it pass.
5. Gate the class so it cannot return (a seam), then prove the seam fails on the
   defect.

## Verification

- Vault links: `./scripts/check-vault-links.py` (exit 1 on breaks).
- Prometheus panels/rules: query each panel expression against the API — 0 series on
  a panel that should show data is a defect, 0 `ALERTS` when nothing fires is not.
- Suite: `./tests/run.sh`; declares `ALL TESTS PASSED`.
- A gate you changed: recreate the defect and confirm red.