Frank
40b9725433
Three things from the review (#60), one of them a finding of my own that did not survive measurement: - The reported defect — additions never subtract `pacman -Qtdq`, so an orphan can be emitted as a manifest line — is wrong. A package carries one install reason: `-Qtdq` selects installed-as-dependency, additions come from `-Qqe`, so the sets are disjoint (blankpad: 206 explicit, 3 orphans, none in common). A subtraction would be dead code that looks like a guard, so the header now says why not to add one. The packages page law was rewritten to the same effect. - What the old test should have pinned: an implicit dependency installed AFTER the baseline. Only `-Qqe` keeps it out of drift — mutate that query to `-Qq` and the new test goes red (verified), where the old assertion passed no matter what. - `--help` printed `set -euo pipefail` and the LC_ALL line as documentation: the hardcoded sed range ran past the comment block. It now stops at the first line of code, so the header can keep growing. Refs #60
167 lines
8.7 KiB
Bash
Executable File
167 lines
8.7 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# scripts/pkg-inventory.sh — what does this machine have that packages/*.pkgs does not declare?
|
|
#
|
|
# install.sh walks desired → installed. This walks installed → declared, and the pair is what
|
|
# makes a rebuild honest: a manifest the machine has quietly outgrown reproduces a machine that
|
|
# is missing the tools you installed by hand six weeks ago, and nothing notices until the new box
|
|
# has no tmux. Refs #51.
|
|
#
|
|
# The baseline is what separates "you installed this" from "the installer installed this". It is
|
|
# read from the pacman log — the install-day transactions, dated at the oldest `[ALPM] installed`
|
|
# line — because declaring Calamares furniture in a manifest is bookkeeping for something a
|
|
# reinstall provides for free. Rotated logs (pacman.log.N, .N.gz) are read too: on a machine older
|
|
# than the rotation window the install day lives only there, and skipping them would report the
|
|
# entire base system as hand-installed.
|
|
#
|
|
# Report (stdout):
|
|
# undeclared additions (declare these): N hand-installed, absent from the manifests
|
|
# repo (sync dbs): <names> → belongs in packages/<host>.pkgs
|
|
# aur (needs a helper): <names> → belongs in packages/aur.pkgs (yay)
|
|
# installer-provided, undeclared: N a reinstall supplies these; do not declare
|
|
# declared, not installed: N install.sh installs these
|
|
# orphans ignored: <names> pacman -Qtdq — never drift, never emitted
|
|
# flatpak: <app ids> reported only, install.sh installs no flatpaks
|
|
#
|
|
# --emit prints the additions as manifest lines, ready to fold into packages/. Names are bare
|
|
# tokens so install.sh's `sed 's/#.*//'` + first-field parser reads them unchanged.
|
|
|
|
# Drift is `pacman -Qqe`: explicitly installed, therefore something you asked for and a rebuild
|
|
# must declare. `pacman -Qtdq` orphans are packages installed AS A DEPENDENCY and no longer
|
|
# required, and a package carries exactly one install reason, so the two sets are disjoint by
|
|
# construction (blankpad, 2026-10-21: 206 explicit, 3 orphans, none in common). Nothing here
|
|
# subtracts the orphan list from the additions and nothing should — the query is the guard. Do not
|
|
# add such a subtraction: it cannot fire, and it would hide the real invariant (implicit installs
|
|
# are never drift, whenever they arrived) behind a line that looks like protection.
|
|
#
|
|
# Exit status: 0 clean · 1 drift (undeclared additions exist) · 2 usage · 3 cannot classify
|
|
# (pacman log unreadable or sync databases unavailable — worse to guess than to say so, because
|
|
# guessing would dump every package into aur.pkgs).
|
|
#
|
|
# Environment overrides (tests):
|
|
# PKGINV_REPO repo root holding packages/ (default: git toplevel of this script)
|
|
# PKGINV_HOST host name (default: $(hostname))
|
|
# PKGINV_LOG space-separated pacman logs (default: /var/log/pacman.log + rotated)
|
|
# PKGINV_BASELINE YYYY-MM-DD, inclusive (default: oldest install date in the logs)
|
|
|
|
set -euo pipefail
|
|
export LC_ALL=C # comm/sort agreement: pacman's order is not this order
|
|
|
|
MODE="report"
|
|
while [[ $# -gt 0 ]]; do
|
|
case "$1" in
|
|
--emit) MODE="emit"; shift ;;
|
|
-h | --help) awk '/^set /{exit} NR>1' "$0"; exit 0 ;;
|
|
*) echo "usage: pkg-inventory.sh [--emit]" >&2; exit 2 ;;
|
|
esac
|
|
done
|
|
|
|
HERE="$(dirname "$(realpath "$0")")"
|
|
REPO="${PKGINV_REPO:-}"
|
|
if [[ -z "$REPO" ]]; then
|
|
REPO="$(git -C "$HERE" rev-parse --show-toplevel 2>/dev/null || true)"
|
|
fi
|
|
[[ -n "$REPO" && -d "$REPO/packages" ]] || { echo "pkg-inventory: no packages/ under ${REPO:-<repo>}" >&2; exit 2; }
|
|
HOST="${PKGINV_HOST:-$(hostname)}"
|
|
|
|
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
|
|
|
|
source "$HERE/lib/manifest.sh" # the same parser install.sh installs with
|
|
# ---- installed state ---------------------------------------------------------
|
|
# Four queries are the tool's whole interface to pacman. -Sl is the unprivileged sync listing:
|
|
# -Sqq asks for root, and a read of the machine must not need sudo. Field 2 of -Sl is the package
|
|
# name, and one name can appear in several databases (core/extra/cachyos), hence the sort -u.
|
|
pacman -Qqe > "$TMP/explicit"
|
|
pacman -Qq > "$TMP/installed"
|
|
pacman -Qtdq > "$TMP/orphans" 2>/dev/null || true # exit 1 when the list is empty: no orphans
|
|
if ! pacman -Sl > "$TMP/sync.raw" 2>/dev/null || [[ ! -s "$TMP/sync.raw" ]]; then
|
|
echo "pkg-inventory: no sync database available (run: sudo pacman -Sy) — repo vs AUR cannot be classified" >&2
|
|
exit 3
|
|
fi
|
|
awk '{print $2}' "$TMP/sync.raw" | sort -u > "$TMP/sync"
|
|
|
|
# ---- declared desired state --------------------------------------------------
|
|
: > "$TMP/declared"
|
|
for m in common.pkgs "$HOST.pkgs" aur.pkgs; do
|
|
[[ -f "$REPO/packages/$m" ]] || continue
|
|
read_manifest "$REPO/packages/$m" >> "$TMP/declared"
|
|
done
|
|
|
|
# ---- baseline: what the installer put here -----------------------------------
|
|
logs=()
|
|
if [[ -n "${PKGINV_LOG:-}" ]]; then
|
|
read -ra logs <<< "$PKGINV_LOG"
|
|
else
|
|
logs=(/var/log/pacman.log)
|
|
for f in /var/log/pacman.log.[0-9]*; do [[ -e "$f" ]] && logs+=("$f"); done
|
|
fi
|
|
read_logs() {
|
|
local f
|
|
for f in "${logs[@]}"; do
|
|
[[ -r "$f" ]] || continue
|
|
case "$f" in *.gz) zcat "$f" ;; *) cat "$f" ;; esac
|
|
done
|
|
}
|
|
# "YYYY-MM-DD name" for every install event the log remembers.
|
|
read_logs | sed -nE 's/^\[([0-9-]{10})T[^]]*\] \[ALPM\] installed ([^ ]+) \(.*/\1 \2/p' > "$TMP/events"
|
|
[[ -s "$TMP/events" ]] || { echo "pkg-inventory: no install events readable in ${logs[*]} — baseline unknown" >&2; exit 3; }
|
|
|
|
# Minimum date in a single awk pass. Sorting and then taking the first line — whether with `head`
|
|
# or with an awk that exits early — closes the pipe, and pipefail turns sort's SIGPIPE into a dead
|
|
# script; a real install-day log is long enough for that to bite (it killed the first real run).
|
|
BASELINE="${PKGINV_BASELINE:-$(awk '{ if (m == "" || $1 < m) m = $1 } END { print m }' "$TMP/events")}"
|
|
[[ "$BASELINE" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]] || { echo "pkg-inventory: bad PKGINV_BASELINE '$BASELINE' (want YYYY-MM-DD)" >&2; exit 2; }
|
|
|
|
# Inclusive: packages installed ON the baseline date came with the install too — Calamares keeps
|
|
# installing the desktop for minutes after the base transaction.
|
|
awk -v d="$BASELINE" '$1 <= d {print $2}' "$TMP/events" | sort -u > "$TMP/base"
|
|
|
|
for f in explicit installed sync declared base orphans; do sort -u -o "$TMP/$f" "$TMP/$f"; done
|
|
|
|
# ---- the sets ----------------------------------------------------------------
|
|
# additions: explicit, not declared, and not something the installer handed us.
|
|
comm -23 "$TMP/explicit" "$TMP/declared" > "$TMP/undeclared"
|
|
comm -23 "$TMP/undeclared" "$TMP/base" > "$TMP/additions"
|
|
comm -12 "$TMP/undeclared" "$TMP/base" > "$TMP/installside"
|
|
comm -12 "$TMP/additions" "$TMP/sync" > "$TMP/add.repo"
|
|
comm -23 "$TMP/additions" "$TMP/sync" > "$TMP/add.aur"
|
|
comm -23 "$TMP/declared" "$TMP/installed" > "$TMP/missing"
|
|
|
|
n() { wc -l < "$1" | tr -d ' '; }
|
|
|
|
if [[ "$MODE" == emit ]]; then
|
|
if [[ -s "$TMP/add.repo" ]]; then
|
|
printf '# packages/%s.pkgs — additions measured %s (pkg-inventory.sh, baseline %s)\n' \
|
|
"$HOST" "$(date +%F)" "$BASELINE"
|
|
cat "$TMP/add.repo"
|
|
fi
|
|
if [[ -s "$TMP/add.aur" ]]; then
|
|
printf '# packages/aur.pkgs — additions measured %s (AUR: not in any sync db, install via yay)\n' \
|
|
"$(date +%F)"
|
|
cat "$TMP/add.aur"
|
|
fi
|
|
exit 0
|
|
fi
|
|
|
|
printf 'pkg-inventory host=%s baseline=%s repo=%s\n' "$HOST" "$BASELINE" "$REPO"
|
|
printf 'counts declared=%s explicit=%s baseline=%s installed=%s\n' \
|
|
"$(n "$TMP/declared")" "$(n "$TMP/explicit")" "$(n "$TMP/base")" "$(n "$TMP/installed")"
|
|
printf 'undeclared additions (declare these): %s\n' "$(n "$TMP/additions")"
|
|
printf ' repo (sync dbs): %s\n' "$(tr '\n' ' ' < "$TMP/add.repo")"
|
|
printf ' aur (needs a helper): %s\n' "$(tr '\n' ' ' < "$TMP/add.aur")"
|
|
printf 'installer-provided, undeclared (a reinstall provides these): %s\n' "$(n "$TMP/installside")"
|
|
# The names, folded: this is the answer to "what did the installer mark explicit that I never
|
|
# declared", and it is long — a count alone would force a second run to see it.
|
|
# The echo matters: tr eats the final newline, fold preserves that absence, and the next label
|
|
# would otherwise be glued to the last folded line.
|
|
{ tr '\n' ' ' < "$TMP/installside"; echo; } | fold -s -w 88 | sed 's/^/ /'
|
|
printf 'declared, not installed (install.sh installs these): %s\n' "$(tr '\n' ' ' < "$TMP/missing")"
|
|
printf 'orphans ignored (pacman -Qtdq): %s\n' "$(tr '\n' ' ' < "$TMP/orphans")"
|
|
if command -v flatpak > /dev/null 2>&1 && flatpak list --app --columns=application > "$TMP/flatpak" 2>/dev/null; then
|
|
printf 'flatpak (reported only; install.sh installs no flatpaks): %s\n' "$(tr '\n' ' ' < "$TMP/flatpak")"
|
|
else
|
|
printf 'flatpak: not installed on this host\n'
|
|
fi
|
|
|
|
[[ -s "$TMP/additions" ]] && exit 1
|
|
exit 0
|