---
name: locale-c-multibyte-awk-silent-loss
description: "Debug generated output that silently loses data under LC_ALL=C — a regex quantifier on a multibyte character in awk/grep, plus the locale-dependent red test it produces."
---

# LC_ALL=C kills multibyte regex quantifiers — silently

Symptom: a generator emits a **complete, exit-0, missing-a-third-of-the-page** artifact in
cron / systemd user units / agent shells, and the same command looks perfect in the user's
terminal. Companion symptom: a repo test is red for the agent and green for the operator.

## Probe (do these in order, they are 5 seconds apart)

```bash
echo "LANG=[$LANG] LC_ALL=[$LC_ALL]"   # LC_ALL overrides LANG — en_US.UTF-8 + LC_ALL=C is byte-wise
locale -a | grep -iE 'utf-?8' | head    # which UTF-8 locale actually exists on this box
```

Then test the suspect pattern alone, under both locales, with the literal character:

```bash
printf '%s\n' '    // ─── Applications ───' | awk '/─+ / { print "QUANT MATCHED" }'
printf '%s\n' '    // ─── Applications ───' | awk '/─/    { print "LITERAL MATCHED" }'
printf '%s\n' '    // ─── Applications ───' | LC_ALL=C.utf8 awk '/─+ / { print "QUANT MATCHED under UTF-8" }'
```

Literal matches everywhere; the quantifier matches only under a UTF-8 locale. That asymmetry IS
the bug: gawk under `LC_ALL=C` matches bytes, so `─+` means "the last byte of U+2500, repeated" —
a byte sequence that never occurs. mawk/busybox/Go regexp have the same class of failure
(quantifier or bracket class spanning a multibyte char).

## Fix

Match and strip the multibyte literal with no quantifier:

```awk
/^[[:space:]]*\/\/[[:space:]]*─/ {          # literal gate
  s = $0; sub(/^[[:space:]]*\/\/[[:space:]]*/, "", s); sub(/[[:space:]]+$/, "", s)
  if (s ~ /─ .*─/) { gsub(/─/, "", s); gsub(/[[:space:]]+/, " ", s); sub(/^ /, "", s); ... }
  next
}
```

**Never pin `LC_ALL=en_US.UTF-8` to "fix" it** — that locale often does not exist (a CachyOS box
here shipped only `C.utf8`), so the script then breaks on another machine. Make the matcher
byte-safe instead.

## Pin it with a test

`run env LC_ALL=C "$GEN" "$IN"` + assert the lost content is present. The operator's UTF-8 terminal
can never catch this class of bug, so the explicit C-locale run is the only durable guard.

## Suite-verification law (how this got missed)

NEVER judge `bats tests/` from `tail` output — a failure at test 28 is invisible in a 25-line
window ending at test 109. Always:

```bash
out=$(bats tests/ 2>&1); echo "$out" | grep -cE '^ok'; echo "$out" | grep -E '^not ok' || echo GREEN
```

Run it under `LC_ALL=C` too: that is the environment every automated caller inherits. Same rule
for `| grep -c` on a multi-line JSON field — count what you claim to have counted.

## Related, do not confuse

- `bats-assertion-trust` — vacuously-passing assertions (green that proves nothing). This skill is
  the mirror case: correct assertions that are only red in one locale.
- `gitea-pr-api-cycle` — unrelated; landed here only because the PR work is where this surfaced.