command-center/agent/managed-skills/bats-assertion-trust/SKILL.md

name, description

name	description
bats-assertion-trust	When writing or reviewing bats shell tests, catch vacuously-passing assertions: grep -vq, sed range boundaries, unanchored section slices, and fixtures too small to trigger pipefail/SIGPIPE bugs.

Prove a bats assertion can actually fail before trusting that it passes. These four traps each made a green suite lie in real work.

1. grep -vq proves nothing

some_list | grep -vq 'x' succeeds when ANY line lacks x — nearly always true. To assert absence, capture the text and test it:

s="$(report_section)"
[[ "$s" != *libfoo* ]]          # absence
[[ "$s" == *bun* ]]             # presence

Same trap in reverse: bash [[ $s == *A*B* ]] — * crosses newlines, so a substring check spanning two lines matches the wrong lines. Slice to one line first, then assert.

2. Section slicing helpers must have explicit boundary semantics

A helper used to scope assertions to one report section must define, and comment, both ends. The default you want: start inclusive (labels carry the count/names), terminator exclusive (a sed -n "/A/,/B/p" range includes B and leaks the next section's first line):

# Call sites pass a bare anchored regex; both ends documented.
section() { printf '%s\n' "$output" | awk -v s="$1" -v e="$2" '$0 ~ e {on=0} $0 ~ s {on=1} on'; }

Two bugs this shape hides:

• Helper prepends ^ while call sites already pass ^name → pattern /^^name/ → never matches → empty slice.
• An unanchored start address matches the first line and the range runs to end-of-output, so every absence assertion becomes a vacuous pass. Symptom: tests that "verify X is not in section A" pass while X is printed later in the same output.

After fixing a helper like this, re-run the whole suite: tests that were passing vacuously suddenly fail, and that is the fix working.

3. Fixtures too small hide pipefail bugs

set -euo pipefail + sort | head -1: head exits, closes the pipe, sort dies of SIGPIPE, the command substitution returns 141, set -e aborts the script. Small fixtures never overflow the 64 KiB pipe buffer, so the suite stays green and the first run against real data fails with no output.

• Take a minimum in one pass: awk '{ if (m=="" || $1<m) m=$1 } END { print m }' file
• awk 'NR==1{print; exit}' is not a fix — exiting early closes the pipe exactly like head.
• Pin it with a fixture larger than the pipe buffer: seq 1 20000 | awk '{print "[2026-01-01T10:00:00-0700] [ALPM] installed pkgfiller (1-1)"}' > big.log (use seq, not yes | head, or the fixture itself SIGPIPEs).

4. tr swallows the final newline and downstream tools preserve that

tr '\n' ' ' < file emits no trailing newline; fold, sed, and cat faithfully preserve the absence, so the next output line gets glued on. Breaks any tool that later slices output by line-start labels.

{ tr '\n' ' ' < file; echo; } | fold -s -w 88 | sed 's/^/  /'

Pin with: assert the following section's label does NOT appear inside the preceding section's slice.

Debugging order when a bats assertion fails but the script output looks right

1. Reproduce the exact command outside bats (bash -x against the script under test) — if it works there, the bug is in the test helper, not the script.
2. Dump the constructed argument: printf '%s' "$sed_script" | od -An -c inside the helper. That is how /^^ repo / (double anchor) got found.
3. Keep stubs faithful to the real interface: if the stub answers -Sqq but the script calls -Sl, the test measures nothing. Make the stub exit 64 on unhandled args so a new real query fails loudly instead of silently returning empty.