Compare commits
17
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Frank
|
289f74062a | ||
|
Frank
|
d7bd05398d | ||
 dank
|
004962ffce | ||
|
Frank
|
56fc899763 | ||
|
Frank
|
35ceae6d7b | ||
|
dank
|
655a39ffb6 | ||
|
Frank
|
b4fc881164 | ||
|
Frank
|
6df45d709a | ||
|
dank
|
5ab0a49ea4 | ||
|
Frank
|
4666d0dce0 | ||
|
dank
|
d013795922 | ||
|
Frank
|
2b7b0bc3cd | ||
|
dank
|
3809e1ed22 | ||
|
Frank
|
e278e4b863 | ||
|
dank
|
c05f5ad451 | ||
|
Frank
|
ecc089c6ef | ||
|
dank
|
7523ca5c19 |
+11
-3
@@ -54,13 +54,20 @@ Suggest these tools or languages as possible solutions to problems if they are a
|
||||
|
||||
### 1. Maintain your projects source of truth.
|
||||
|
||||
The knowledge base at `docs/` is the source of truth for your project. Your training is out of date. Default to the documentation and the code before you assume anything. Maintain a work log of actions taken and their result. Refer to the work log to remember the progress of your tasks.
|
||||
Documentation exists for ONE reason: **point of use**. A page earns its bytes only if it helps you perform a task at the moment you perform it — the shape to deploy, the flag that must be passed, the failure mode to recognise, the rollback to run. It is NOT a museum, NOT a lab notebook, and NOT a place to keep track of numbers.
|
||||
|
||||
**THE PURPOSE TEST — apply it before you write, and again before you keep:**
|
||||
- Would an agent or Frank, doing a real task tomorrow, read this line and act differently because of it? If no: **delete it.**
|
||||
- Is this a measured number (t/s, tokens, ms, MiB, %, bench ladder, acceptance rate) for a model, runtime, or hardware configuration that is NOT currently deployed and NOT currently being compared? Then it is **dead weight — destroy it**, do not archive it, do not "preserve it in an Eras row", do not cite it. AI advances make such a figure wrong within weeks, and a wrong number in a runbook is worse than no number: it gets believed.
|
||||
- "It is the only copy of that fact" is NOT a reason to keep it. Sole custody of a useless fact is still useless. Losing information is an acceptable — often correct — outcome; do not hesitate to lose it.
|
||||
- What survives the test: live configuration and its exact numbers, the procedure that must be followed, the law/causal story that prevents a repeat failure, security posture, and the interface contract a consumer depends on. Qualitative laws outlive the measurement that taught them — keep the law, drop the numbers.
|
||||
- Never let a tool's "this is duplicated / this is the only record" census decide what to keep. That is a question of USEFULNESS, which only the operator answers.
|
||||
|
||||
1. Read the relevant `index.md` before you touch a topic.
|
||||
2. Follow `omp-docs/references/okf-spec.md` for every doc you create or edit.
|
||||
3. Read the target file completely before you edit it.
|
||||
4. Always update the documentation when you've completed a task.
|
||||
5. Compact the work log when necessary.
|
||||
4. Always update the documentation when you've completed a task — and when updating means deleting a page or a paragraph, delete it.
|
||||
5. Compact the work log when necessary; a work log holds laws and incidents, never bookkeeping.
|
||||
|
||||
#### USE PROGRESSIVE DISCLOSURE.
|
||||
Always ensure that all produced documentation is laid out in files following WIKI conventions: start with a top level README.md index file that provides a brief, concise introduction to the item being documented, and then SPLIT OFF individual aspects, features, or arguments into their own interlinked pages so that future agents can read the documentation in a piecewise fashion, only taking in the details they need. Where relevant, crosslink any created documentation with other relevant documentation. This structure may also be nested, if necessary.
|
||||
@@ -90,6 +97,7 @@ Never assume the user reports correct information. He is human, and humans make
|
||||
- Before starting new work, `git pull` from the remote, and start a new branch in a separate worktree using `master` or `main` as the base branch.
|
||||
- Commit atomically: after every change that forms a complete functional unit, commit. Do so early and often.
|
||||
- When the work is complete, ask the user if they would like to open a pull request.
|
||||
- **Report a pull request as link AND title, every time.** A bare `#54` is not a reference: numbers are per-repository, so a number alone does not say which work, in which repo, is being reported. Give the PR's web URL on that forge — on the Dankclaw Gitea that is `https://gitea.franksdank.cloud/<owner>/<repo>/pulls/N` — together with the PR's title. Applies everywhere a PR is mentioned: chat replies, status summaries, issue and commit bodies, alerts. Another forge (GitHub, GitLab, …): the equivalent URL its web UI serves, same pairing.
|
||||
|
||||
### 5. Test Driven Development
|
||||
|
||||
|
||||
@@ -9,8 +9,6 @@ window-size=(360, -1)
|
||||
|
||||
[org/gnome/desktop/background]
|
||||
picture-options='zoom'
|
||||
picture-uri='file:///home/dank/.config/background'
|
||||
picture-uri-dark='file:///home/dank/.config/background'
|
||||
|
||||
[org/gnome/desktop/interface]
|
||||
color-scheme='prefer-dark'
|
||||
@@ -26,14 +24,6 @@ default-visible-columns=['name', 'size', 'type', 'date_modified']
|
||||
default-folder-viewer='list-view'
|
||||
migrated-gtk-settings=true
|
||||
|
||||
[org/gnome/nautilus/window-state]
|
||||
initial-size=(890, 550)
|
||||
initial-size-file-chooser=(890, 550)
|
||||
maximized=false
|
||||
|
||||
[org/gnome/portal/filechooser/firefox]
|
||||
last-folder-path='/home/dank/Downloads'
|
||||
|
||||
[org/gtk/gtk4/settings/file-chooser]
|
||||
show-hidden=false
|
||||
sort-directories-first=false
|
||||
|
||||
@@ -2,6 +2,13 @@
|
||||
Description=swayidle idle lock (locks via noctalia-shell IPC, suspends on lid)
|
||||
Documentation=https://github.com/swaywm/swayidle
|
||||
After=graphical-session.target
|
||||
# Restart together with the session, not merely at boot: this unit is wanted by
|
||||
# default.target, so a compositor restart used to leave it orphaned while the target came
|
||||
# back without it. Refs #47.
|
||||
PartOf=graphical-session.target
|
||||
# With Restart=always below, a fast failure loop must never wedge the watcher into `failed`:
|
||||
# a lock that keeps trying beats a lock that stopped trying. Refs #47.
|
||||
StartLimitIntervalSec=0
|
||||
|
||||
[Service]
|
||||
# Locks via the noctalia v5 shell; `noctalia msg` resolves the running instance
|
||||
@@ -11,7 +18,10 @@ ExecStart=/usr/bin/swayidle -w \
|
||||
timeout 300 '/usr/bin/noctalia msg session lock' \
|
||||
before-sleep '/usr/bin/noctalia msg session lock' \
|
||||
after-resume '/usr/bin/noctalia msg session lock'
|
||||
Restart=on-failure
|
||||
# `swayidle -w` exits 0 on SIGTERM, and that is how this watcher died on 2026-10-12: a clean
|
||||
# exit that `Restart=on-failure` cannot see, leaving the box unlocked for a week. A supervisor
|
||||
# restarts always. Refs #47.
|
||||
Restart=always
|
||||
RestartSec=5s
|
||||
|
||||
[Install]
|
||||
|
||||
@@ -135,7 +135,7 @@ configs).
|
||||
| Reason | Contents |
|
||||
|---|---|
|
||||
| **Generated palette expansion** | Every `themes/noctalia*`, `*/noctalia.*`, `*/colors/noctalia*` / `Noctalia*` (24 files outside app-profile dirs). Machine-written by matugen/noctalia, not hand-authored: the `stat -c '%y %n'` cluster `2026-10-14 19:05:40.330…40.813 -0700` writes 24 files in one second (`alacritty/themes/noctalia.toml` at `.330212397`, `btop/themes/noctalia.theme` at `.334212456`, `tmux/themes/noctalia.conf` at `.813219586`, …); later re-runs re-stamp singles — `bat/themes/noctalia.tmTheme` `2026-10-18 17:11:22`, `niri/noctalia.kdl` `2026-10-18 17:36:27`, `mango/noctalia.conf` `2026-10-17 18:23:37`. A same-second mass-write is generation, never an edit. |
|
||||
| **App state / profiles** | `mozilla` (5471), `unity3d` RimWorld state (235), `nirimod` (116), `obsidian` (99), `FreeTube` (56), `google-chrome-for-testing` (5); plus per-program state inside tracked dirs: `dconf/user`, `fish/fish_variables`, `micro/syntax/*`+`micro/buffers/history`, `gtk-4.0/servers`, `cachyos/cachyos-hello.log`. |
|
||||
| **App state / profiles** | `mozilla` (5471), `unity3d` RimWorld state (235), `nirimod` (116), `obsidian` (99), `FreeTube` (56), `google-chrome-for-testing` (5); plus per-program state inside tracked dirs: `dconf/user`, `fish/fish_variables`, `micro/syntax/*`+`micro/buffers/history`, `gtk-4.0/servers`, `cachyos/cachyos-hello.log`. State hides inside a tracked file too: `dconf/dump.txt` carried `last-folder-path`, a nautilus `window-state` section and `picture-uri`/`picture-uri-dark` — all `/home/dank` values, and the wallpaper URIs were live on blankpad, i.e. already deployed here — dropped by #44, see [tools/dotfiles-deploy](/tools/dotfiles-deploy.md). |
|
||||
| **Skills-CLI installs** | `crush` (37), `devin` (37), `goose` (37), `kimchi/harness` (37) skill trees — provisioned by the skills CLI, owned by the store, not this repo. See [skills/store-wiring](/skills/store-wiring.md). |
|
||||
| **Secret tier** | `~/.config/secrets/` (`gitea`, `hf_token`, `hf_token_write`, `kagi_api_key` — names only), top-level `frank-job-hunt.env`, `containers/systemd/postgres.env`. Never tracked, values never printed. |
|
||||
| **Trash committed inside the tree** | `dotfiles/niri/.config/niri/cfg/.AGENTTRASH/display.kdl.superseded` — a retired output config committed 2026-10-09, five days before `.gitignore:54` gained `.AGENTTRASH/` (2026-10-14). Tracked-then-ignored: an ignore rule never reaches an already-tracked path, so the only repair is `git rm --cached`, done by #41. The rule does match at depth — `git check-ignore --no-index -v dotfiles/niri/.config/niri/cfg/.AGENTTRASH/x` resolves to `.gitignore:54` — so a fresh in-tree trash dir is ignored from birth and this does not recur on its own. |
|
||||
|
||||
@@ -16,6 +16,29 @@ lesson, a few lines. Full incident narratives are recoverable from git
|
||||
(`git log --follow omp-docs/log.md`); knowledge that outlives an entry lives
|
||||
on its concept page, linked here. Never re-expand an entry in place.
|
||||
|
||||
## 2026-10-20 — two instruments rescued from `$HOME` script sediment
|
||||
|
||||
`$HOME` had accumulated 123 loose scripts at its root (archived to
|
||||
`.AGENTTRASH/home-scripts-2026-10-20`, ~1.8 GiB of stray audio consolidated into
|
||||
`~/audiosamples`). Two were durable instruments and now live in the repo with
|
||||
black-box tests and pages: [tools/remote-job](tools/remote-job.md) and
|
||||
[tools/pcie-probe](tools/pcie-probe.md). Refs #53. Laws learned:
|
||||
|
||||
- **A readiness pattern in a log is not evidence of completion.** `ssh host 'cmd |
|
||||
tail -f log | sed -n /ready/q'` fails three independent ways — the broker aborts
|
||||
the long connection, the pattern can match a header printed before the work
|
||||
starts, and `sed -n /x/q` exits the moment it matches, tearing down the job it was
|
||||
watching. The only honest signal is a sentinel the job writes last, read over a
|
||||
short-lived connection.
|
||||
- **Never benchmark beside a live service, and never let a tool decide to stop one.**
|
||||
The guard refuses before anything is built; unloading a production endpoint is a
|
||||
human's call. A number taken as noise beside serving traffic is both disruptive and
|
||||
wrong.
|
||||
- **An instrument that segfaults is not an instrument.** Two of the three shipped
|
||||
`.hip` probes called `hipGetDeviceProperties`, which crashes on this ROCm build
|
||||
while `hipGetDeviceCount` works fine; device identity now comes from sysfs, and the
|
||||
trap is recorded in the sources so it is not reintroduced.
|
||||
|
||||
## 2026-10-20 — the secret gate was reading the wrong files, and one credential had already escaped
|
||||
|
||||
Found while deciding what else belongs in `dotfiles/`. (1) The gate whitelisted
|
||||
|
||||
@@ -25,14 +25,16 @@ is stowed last and its files replace symlinks coming from base packages.
|
||||
| `deploy.sh` (sync) | adopt identicals, link new, quarantine divergents into `.quarantine/` and replace them |
|
||||
| `deploy.sh --adopt` | like sync, but divergents are reported and skipped rather than quarantined |
|
||||
|
||||
Two package shapes are not symlinks: `dconf/dump.txt` is loaded with `dconf load /`, and
|
||||
`systemd-user` units get `daemon-reload` plus `enable --now` only when their `ExecStart`
|
||||
binary resolves on the current machine, so a unit belonging to another box stays inert
|
||||
instead of crash-looping.
|
||||
Two package shapes are not symlinks. `dconf/dump.txt` is loaded with `dconf load /`, and
|
||||
`systemd-user` units get `daemon-reload` plus `enable`; a unit is **started** only when
|
||||
systemd reports it as nothing-yet (`""`/`disabled`), and never when its `ExecStart` binary
|
||||
is absent here, so a unit belonging to another box stays inert instead of crash-looping.
|
||||
Starting something already installed is what makes `deploy.sh` unsafe to run beside a live
|
||||
session — see *Never start a unit a session may already be running*.
|
||||
|
||||
**The tracked unit is the file, not the directory.** A live directory can be a real
|
||||
directory and still be fully tracked, because every file inside it is a symlink into the
|
||||
repo (`niri` is 9 tracked / 1 box-only; `gtk-3.0` is 3/0). Any audit that classifies by
|
||||
repo (`niri` is 8 tracked / 2 box-only; `gtk-3.0` is 3/0). Any audit that classifies by
|
||||
directory mislabels the tree — classify per file with `readlink -f`. The live census and
|
||||
its classifier command are in [home-config-census](/inventory/home-config-census.md).
|
||||
|
||||
@@ -123,5 +125,131 @@ run and one unattended `deploy.sh` would have reverted the amber focus ring with
|
||||
quarantine and no warning. The duplicate is deleted in #43.
|
||||
|
||||
Rule: a `zz` file exists only to differ. When base absorbs the change, delete the `zz` copy
|
||||
in the same commit — and never read `RELINK` lines from `--check` as cosmetic noise; they
|
||||
are the sound of a stale override waiting for someone to run the tool for real.
|
||||
in the same commit.
|
||||
|
||||
How to read `--check`, measured on blankpad: it prints `RELINK` for **every** path the `zz`
|
||||
tier carries, even when the live link already points at the override — `rules.kdl`,
|
||||
`keybinds.kdl`, `misc.kdl`, `autostart.kdl` and `.zshrc` all did — because the walk
|
||||
processes the base package first and reports the intermediate owner change. A `RELINK` line
|
||||
therefore proves a pending change only when `readlink` shows the live target is *not*
|
||||
already the `zz-$HOST` file. `actions=0` is the only honest summary.
|
||||
## Law: never start a unit a session may already be running
|
||||
|
||||
`sync` used to end with `systemctl --user enable --now` for every unit in `systemd-user/`
|
||||
whose `ExecStart` resolves. Measured on blankpad before one such run:
|
||||
|
||||
|unit|`UnitFileState`|`ActiveState`|
|
||||
|---|---|---|
|
||||
|`live-sync.service`|`linked`|`inactive`|
|
||||
|`shelly-notifications.service`|`enabled`|`active`|
|
||||
|`swayidle-lock.service`|`enabled`|`inactive`|
|
||||
|
||||
`swayidle-lock.service` was installed and **not running** — nothing started it, no swayidle
|
||||
process existed (`pgrep -c swayidle` = 0), and `autostart.kdl` does not launch one. A
|
||||
`deploy.sh` run would have armed it: monitors blanking and locking on the idle timer while
|
||||
the operator was mid-sentence. `live-sync.service` is `linked`, driven by its timer;
|
||||
`--now` forces a heartbeat commit at an arbitrary moment.
|
||||
|
||||
Two rules follow. Install state must be read from the **printed state**, not the exit
|
||||
status — `is-enabled` exits 1 for `linked` exactly as for `disabled`, so exit status cannot
|
||||
tell "installed, driven by a timer" apart from "never installed". And a unit is started only
|
||||
in the latter case: a machine with a session on it has already decided what runs, while a
|
||||
fresh box — where everything reports `disabled` — still gets its units up. Both directions
|
||||
are pinned in `tests/deploy.bats` over a stub `systemctl`, because the real one on this box
|
||||
is exactly what must not be invoked from a test. Refs #44.
|
||||
|
||||
## Law: expand systemd specifiers before calling a unit foreign
|
||||
|
||||
`ExecStart=%h/.omp/scripts/live-sync.sh` is valid systemd and meaningless to a shell. The
|
||||
resolution check handed that literal string to `command -v`, which fails everywhere, so the
|
||||
deployment concluded the unit belonged to another machine and skipped it — silently, through
|
||||
the same code path that is supposed to keep a *foreign* unit inert. The unit it kept skipping
|
||||
was this repo's own heartbeat, and `%h` is precisely where this checkout lives, so the false
|
||||
verdict was the same on every box that deployed it. Measured on blankpad: the first real
|
||||
`deploy.sh` run printed `SKIP live-sync.service (ExecStart '%h/.omp/scripts/live-sync.sh'
|
||||
absent on this host)` while `/home/blank/.omp/scripts/live-sync.sh` sits right there.
|
||||
|
||||
Expand the specifiers a user unit can carry — `%h`, `%u`, `%U` — before resolving, and let an
|
||||
unknown specifier fail rather than guess at it. The probe needs no privileges and writes
|
||||
nothing:
|
||||
|
||||
```bash
|
||||
bin=$(sed -n 's/^ExecStart=\([^ ]*\).*/\1/p' \
|
||||
~/.omp/dotfiles/systemd-user/.config/systemd/user/live-sync.service | head -1)
|
||||
command -v "${bin//%h/$HOME}" # resolves → the unit is this box's; the SKIP was the lie
|
||||
```
|
||||
|
||||
## Law: the dconf dump holds preferences, not per-box state
|
||||
|
||||
`dconf load /` writes every key in `dotfiles/dconf/dump.txt`, so the dump carries whatever
|
||||
box it was taken from. Comparing key by key against live state on blankpad found **six of
|
||||
24** keys that were not this box's, and not one of them was a preference:
|
||||
|
||||
[org/gnome/portal/filechooser/firefox] last-folder-path
|
||||
repo: '/home/dank/Downloads' live: '/home/blank/Downloads'
|
||||
[org/gnome/nautilus/window-state] initial-size, initial-size-file-chooser, maximized
|
||||
[org/gnome/desktop/background] picture-uri, picture-uri-dark
|
||||
both: 'file:///home/dank/.config/background'
|
||||
|
||||
A file-chooser memory, a window geometry, a wallpaper URI: application state that arrived
|
||||
with the dotfiles import. The last five did not appear as conflicts at all, because
|
||||
blankpad's **live** tree already held the same `/home/dank` values — an earlier deploy had
|
||||
written them here. A foreign value that agrees with live is invisible to any diff and still
|
||||
wrong. All six come out in #44; the two wallpaper URIs are also cleared on this box
|
||||
(`gsettings set org.gnome.desktop.background picture-uri ''`, and its `-dark` twin), because
|
||||
niri draws the wallpaper, noctalia owns it, and nothing reads a dead path.
|
||||
|
||||
Compare **key by key**, not with a plain `diff`: `dconf load` writes only the keys present,
|
||||
so live-only keys survive and a diff overstates the damage in one direction and understates
|
||||
it in the other. Re-dump on the box whose settings are canonical (`dconf dump / >
|
||||
dotfiles/dconf/dump.txt`), then delete the state sections before committing — the census's
|
||||
*Never track: app state* class applies inside a tracked file, not just to whole directories.
|
||||
Refs #44.
|
||||
|
||||
## Law: a watcher must restart on a clean exit too
|
||||
|
||||
`swayidle-lock.service` was `enabled`, present in `default.target.wants/`, and **dead for eight
|
||||
days**: `Result=success`, `NRestarts=0`, gone since 2026-10-12 16:22 after 2d 21h of uptime.
|
||||
`Restart=on-failure` cannot see a clean exit, and `swayidle -w` exits 0 on SIGTERM. The trigger is
|
||||
attributable to eight seconds: **`niri` and `noctalia` both started `Mon Oct 12 16:22:31 2026`**
|
||||
(`ps -o lstart=`) while swayidle's last state change was `16:22:23` — the compositor session
|
||||
restarted, the watcher went with it, and because it was wanted by `default.target` rather than tied
|
||||
to the session target, nothing brought it back. The user journal keeps no entries for that window,
|
||||
so the attribution rests on process ages rather than logs, which is its own lesson: diagnose a
|
||||
lapse within the retention you actually have.
|
||||
|
||||
**The severity claim in this page's first draft was wrong, and is corrected here.** It said closing
|
||||
the lid suspended the machine unlocked, because the `before-sleep` hook lived in the dead unit. But
|
||||
`systemd-inhibit --list` shows `noctalia … sleep "Lock before sleep" delay` — the shell holds its
|
||||
own sleep inhibitor and locks on suspend independently of swayidle, so the suspend path was covered
|
||||
the whole time. What was actually dead for eight days is the **idle** lock: five minutes of
|
||||
inactivity locked nothing, and `after-resume` never fired. `pgrep -c -u blankpad
|
||||
'swayidle|swaylock|hyprlock|gtklock'` = 0 remains true; it simply did not mean *no lock path at
|
||||
all*, because a locker can be a hook inside a running shell rather than a name on the process list.
|
||||
Read the inhibitor table, not only `pgrep`.
|
||||
|
||||
Three settings, each with a job: `Restart=always`, because a supervisor must not assume it can
|
||||
only die badly; `StartLimitIntervalSec=0`, because a fast loop must never wedge it into `failed` —
|
||||
a lock that keeps trying beats one that stopped trying; and `PartOf=graphical-session.target`,
|
||||
because the unit is wanted by `default.target`, so the session coming back did not bring *it*
|
||||
back.
|
||||
|
||||
Verify by behaviour rather than by reading the file — the point is surviving the event that killed
|
||||
it in October:
|
||||
|
||||
```bash
|
||||
systemctl --user start swayidle-lock.service
|
||||
kill -TERM "$(systemctl --user show swayidle-lock.service -p MainPID --value)"
|
||||
sleep 8 && systemctl --user show swayidle-lock.service -p NRestarts -p MainPID -p ActiveState
|
||||
```
|
||||
|
||||
An idle inhibitor also disarms the timeout outright, which surprises people: while an `omp` agent
|
||||
session is running, swayidle logs `Not enabling timeouts: idle inhibitor found` and arms no idle
|
||||
timeout at all — `systemd-inhibit --list` shows `omp … idle "omp agent session" block`. A box with
|
||||
an agent working therefore does not lock on inactivity, by harness design, while `before-sleep` and
|
||||
`after-resume` keep working. Anyone asking "why did the screen never lock" should read the inhibitor
|
||||
table before suspecting the unit.
|
||||
|
||||
The general shape: `deploy.sh` only ever *enables*. Nothing on any box re-checks that a unit the
|
||||
system depends on is still running, so a security posture can lapse silently and stay lapsed for a
|
||||
week. A liveness check for this tier belongs in the heartbeat, not in a human's memory. Refs #47.
|
||||
|
||||
@@ -19,6 +19,9 @@ Configuration guides and reference material for installed software. Each tool ha
|
||||
* [neovim](neovim.md) — Two editors in one binary: vanilla `nvim` for quick edits and the `lazyvim` NVIM_APPNAME profile, with the base16 wallpaper-palette hook and fresh-box install.
|
||||
* [niri](niri/index.md) — Scrollable-tiling Wayland compositor — configuration, keybindings, and architecture for the workstation.
|
||||
* [noctalia-shell](noctalia-shell.md) — Native desktop environment for Niri — launcher, control center, settings, lock screen, session menu, hotkey overlay, media controls, and system management.
|
||||
* [packages](packages.md) — `packages/*.pkgs` desired state and the two tools that keep a machine and its manifest agreeing: `install.sh` installs the delta, `pkg-inventory.sh` reports what the machine has outgrown.
|
||||
* [pcie-probe](pcie-probe.md) — HIP probes that separate PCIe link bandwidth from the engine's read path at the expert-slab size, and the refuse-by-default guard against benchmarking beside a live service.
|
||||
* [remote-job](remote-job.md) — Detached job runner for `ssh` work: completion sentinels on the host replace the `tail -f | sed -n /ready/q` pattern that reported success on jobs that never finished.
|
||||
* [tmux](tmux.md) — Terminal multiplexer configuration, extended-keys setup, keyboard input best practices, and reload/restart procedures.
|
||||
|
||||
## Adding a New Tool
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
---
|
||||
type: reference
|
||||
title: package desired state
|
||||
description: packages/*.pkgs plus the two tools that keep a machine and its manifest agreeing — install.sh installs desired-minus-installed, pkg-inventory.sh reports installed-minus-declared.
|
||||
tags:
|
||||
- packages
|
||||
- pacman
|
||||
- provisioning
|
||||
- fleet
|
||||
- blankpad
|
||||
verified: 2026-10-20
|
||||
timestamp: 2026-10-20
|
||||
---
|
||||
|
||||
# Package desired state
|
||||
|
||||
Two tools, one invariant: **the manifests must describe the machine.** A rebuild is only faithful
|
||||
if both directions are checked.
|
||||
|
||||
|tool|direction|writes|
|
||||
|---|---|---|
|
||||
|`scripts/install.sh`|desired → installed|installs `desired − installed` with `--apply`; never removes|
|
||||
|`scripts/pkg-inventory.sh`|installed → declared|nothing — read-only report|
|
||||
|
||||
## Manifests
|
||||
|
||||
`packages/common.pkgs` (every host) + `packages/<host>.pkgs` (this host) + `packages/aur.pkgs`
|
||||
(AUR-only, installed through `yay`). The parser is `sed 's/#.*//'` then the first field, so `#`
|
||||
comments trail freely and one token per line is the whole grammar. An unknown `--host` gets the
|
||||
common manifest only: a wrong-host run is a no-op, not a hazard.
|
||||
|
||||
## Rebuild a workstation
|
||||
|
||||
1. Install the OS (CachyOS, niri/noctalia profile). The installer's own package set is *not* in the
|
||||
manifests and must not be — see the baseline law below.
|
||||
2. Clone this repo to `~/.omp`, then `scripts/install.sh --apply` — packages first, because the
|
||||
enablement step needs `sshd` and `firewalld` present ([dotfiles-deploy](dotfiles-deploy.md)).
|
||||
3. `scripts/deploy.sh sync` for the config tier ([dotfiles-deploy](dotfiles-deploy.md)).
|
||||
4. `scripts/pkg-inventory.sh --emit` on a *known-good* machine and fold the output into
|
||||
`packages/<host>.pkgs` / `packages/aur.pkgs` before step 2 — that is how a hand install stops
|
||||
being invisible to the next rebuild.
|
||||
|
||||
## pkg-inventory report
|
||||
|
||||
```
|
||||
undeclared additions (declare these): N ← exit 1 when N > 0
|
||||
repo (sync dbs): <names> → belongs in packages/<host>.pkgs
|
||||
aur (needs a helper): <names> → belongs in packages/aur.pkgs
|
||||
installer-provided, undeclared: N + names → a reinstall supplies these; do not declare
|
||||
declared, not installed: <names> → install.sh installs these
|
||||
orphans ignored: <names> → pacman -Qtdq
|
||||
flatpak: <app ids> → reported only, install.sh installs no flatpaks
|
||||
```
|
||||
|
||||
Exit codes: `0` clean · `1` drift · `2` usage · `3` classification impossible. Overrides
|
||||
`PKGINV_REPO` / `PKGINV_HOST` / `PKGINV_LOG` / `PKGINV_BASELINE` exist for tests.
|
||||
|
||||
## Law: the installer's own day is the baseline
|
||||
|
||||
Whether a package is *yours* is decided by the pacman log, not by opinion: the baseline date is the
|
||||
oldest `[ALPM] installed` entry — Calamares' pacstrap — and everything installed **on or before** it
|
||||
is installer furniture. Without that cut, a base install reports several hundred "additions" and the
|
||||
list nobody can act on is worse than no list.
|
||||
|
||||
- **Read the rotated logs too** (`pacman.log.N`, `.N.gz`). On a machine older than the rotation
|
||||
window the install day exists only there; skip them and the whole base system is reported as
|
||||
hand-installed.
|
||||
- Calamares keeps installing the desktop for minutes after the base transaction, which is why the
|
||||
date cut is inclusive.
|
||||
|
||||
## Law: refuse rather than guess the AUR split
|
||||
|
||||
`aur.pkgs` vs `<host>.pkgs` is decided by whether any sync database can serve the name — Chaotic-AUR
|
||||
counts as a sync database, so only true AUR belongs in `aur.pkgs`. If the sync databases are
|
||||
unavailable the tool exits `3` instead of assuming: guessing sends every package to `yay`, which is
|
||||
how a manifest becomes a pile of source builds.
|
||||
|
||||
Classify with `pacman -Sl` (world-readable). **`pacman -Sqq` requires root** — a tool that only
|
||||
reads the machine must not need sudo to answer a question about it.
|
||||
|
||||
**Known gap:** `aur.pkgs` is read for *every* host, so a desktop-only AUR package (a niri module, a
|
||||
GUI client) cannot be declared there without asking the headless servers to build it. Such a name is
|
||||
still reported as an AUR addition; leave it undeclared until `install.sh` grows a per-host AUR
|
||||
manifest.
|
||||
|
||||
## Law: orphans never reach a manifest
|
||||
|
||||
`pacman -Qtdq` names are packages nothing depends on; emitting them would teach a fresh machine to
|
||||
install the same dead weight. They are printed as `orphans ignored` and excluded from `--emit`.
|
||||
|
||||
## Two traps that produced this tool
|
||||
|
||||
- **`set -o pipefail` plus `head -1`**: `head` closes the pipe, `sort` dies of `SIGPIPE`, the
|
||||
substitution returns `141`, and `set -e` aborts the script. It survives a small test fixture and
|
||||
fails on a real 1000-line install-day log — take a minimum in one pass (`awk 'END'`-style), never
|
||||
sort-then-first. The same applies to `awk 'NR==1{exit}'`, which closes the pipe exactly as `head`
|
||||
does.
|
||||
- **`tr '\n' ' '` swallows the final newline** and `fold` faithfully preserves that absence, so the
|
||||
next report label lands on the same line as the last name. `tr '\n' ' '; echo` instead.
|
||||
|
||||
## Related
|
||||
|
||||
- [tools/index](index.md) · [dotfiles-deploy](dotfiles-deploy.md) (the config tier that follows) ·
|
||||
[live-sync](live-sync.md) (the heartbeat that commits machine state)
|
||||
- Tests: `tests/pkg-inventory.bats` (fixtures + stubbed `pacman`, no real package database touched).
|
||||
- Refs #51.
|
||||
@@ -0,0 +1,92 @@
|
||||
---
|
||||
type: reference
|
||||
title: pcie-probe — link or read path
|
||||
description: HIP instruments that separate PCIe link bandwidth from the engine's read path at the real expert-slab size, plus the refuse-by-default guard that stops a benchmark from running beside a live service.
|
||||
tags:
|
||||
- gpu
|
||||
- rocm
|
||||
- rdna4
|
||||
- benchmarking
|
||||
- dankserver
|
||||
verified: 2026-10-20
|
||||
timestamp: 2026-10-20
|
||||
---
|
||||
|
||||
# pcie-probe
|
||||
|
||||
`scripts/pcie-probe.sh` answers one question, which comes up every time MoE
|
||||
expert offload is discussed on the R9700 box
|
||||
([dankserver](../system/dankserver.md)): **is the stall the PCIe link, or is it
|
||||
the engine's read path?** The two look identical from the outside — decode is
|
||||
slow while weights stream in — and they have opposite fixes.
|
||||
|
||||
## Instruments
|
||||
|
||||
Sources in `scripts/probes/`, built with the host `hipcc`:
|
||||
|
||||
| Probe | Question it answers |
|
||||
|-------|---------------------|
|
||||
| `hip_smoke.hip` | is host ROCm usable from a plain process at all (count, alloc, copy) |
|
||||
| `pcie_bw.hip` | host→device bandwidth at the engine's exact slab size, pinned vs pageable |
|
||||
| `pcie_probe.hip` | the size sweep — where throughput collapses as transfers shrink |
|
||||
|
||||
```bash
|
||||
scripts/pcie-probe.sh # guard, build, topology, measure
|
||||
scripts/pcie-probe.sh --device 0 --device 1
|
||||
scripts/pcie-probe.sh --build-only # compile, never touch the cards
|
||||
```
|
||||
|
||||
## The law: never benchmark beside a live service
|
||||
|
||||
The guard runs **before anything is built or run** and refuses when the GPU is
|
||||
occupied — `gpu_busy_percent` above `PCIE_PROBE_MAX_BUSY` (default 5 %) or any
|
||||
process holding `/dev/kfd`. Refusal is exit `3`, and it stops nothing.
|
||||
|
||||
Auto-stopping a production endpoint is not this tool's decision to make, and a
|
||||
measurement taken as noise beside serving traffic is worthless anyway: it is
|
||||
both disruptive and wrong. Take the service down deliberately, or pass `--force`
|
||||
and mean it. `tests/pcie-probe.bats` pins that the refusal happens before the
|
||||
build step, so a contended box cannot be measured by accident.
|
||||
|
||||
## Reading the numbers
|
||||
|
||||
The reference is the **link**, printed as a topology block from sysfs
|
||||
(`current_link_width` / `current_link_speed` against `max_*`): a card trained to
|
||||
fewer lanes or a lower generation than it is wired for caps everything
|
||||
downstream, and no engine change will recover it.
|
||||
|
||||
Then the sweep table, whose first row is the engine's actual expert-slab size —
|
||||
throughput at *that* size is what matters, not the big-transfer figure:
|
||||
|
||||
- Small transfers reach the trained-link ceiling → the link is healthy, the
|
||||
engine's read path (fetch granularity, pipelining, pinning) is the limit.
|
||||
- Small transfers stall far below a large-transfer figure → the wall is the
|
||||
link or the IOMMU path, not the engine.
|
||||
- Pinned versus pageable columns apart means pinning is load-bearing; together
|
||||
means the copies are already pinned by someone else.
|
||||
|
||||
Measure one device at a time (`--device`). A run that lets the runtime pick
|
||||
several devices cannot attribute bandwidth to a card at all.
|
||||
|
||||
## The trap carried in the sources
|
||||
|
||||
`hipGetDeviceProperties` **segfaults** on the ROCm build these probes were
|
||||
written against, while `hipGetDeviceCount` works fine and reports every device.
|
||||
None of the three sources calls it; device identity comes from sysfs instead.
|
||||
The setvbuf call in `hip_smoke.hip` is a fossil of discovering that — an
|
||||
unflushed stdout hid the crash point for an afternoon. Keep both facts intact
|
||||
when touching these files.
|
||||
|
||||
## Exit status and seams
|
||||
|
||||
`0` measured · `2` usage or no sources · `3` refused, GPU busy · `4` no HIP
|
||||
toolchain or build failed. A client box without the SDK exits `4` loudly rather
|
||||
than reporting an empty success; the probes belong on the box with the cards.
|
||||
|
||||
Test seams (all documented in the script header): `PCIE_PROBE_SRC`,
|
||||
`PCIE_PROBE_BUILD`, `PCIE_PROBE_SYSFS`, `PCIE_PROBE_KFD`, `PCIE_PROBE_HIPCC`,
|
||||
`PCIE_PROBE_MAX_BUSY` — the sysfs/kfd paths are what make a refusal testable on
|
||||
a machine with no cards, which is the only honest way to test a refusal.
|
||||
|
||||
Measured campaign numbers are not kept here; they belong with the endpoint they
|
||||
were taken on, in the serverAppConfigs LLM runbook vault.
|
||||
@@ -0,0 +1,83 @@
|
||||
---
|
||||
type: reference
|
||||
title: remote-job — detached jobs that outlive the session
|
||||
description: Run a command on a remote box so its result survives the connection, the restart, and the agent that started it — the replacement for ssh piped through tail and sed, which reported success on jobs that never finished.
|
||||
tags:
|
||||
- ssh
|
||||
- fleet
|
||||
- automation
|
||||
- verification
|
||||
verified: 2026-10-20
|
||||
timestamp: 2026-10-20
|
||||
---
|
||||
|
||||
# remote-job
|
||||
|
||||
`scripts/remote-job.sh` starts a command **detached on the host**, and reports
|
||||
what actually happened from files on disk. It exists because
|
||||
`ssh host 'cmd | tail -f log | sed -n /ready/q'` lied repeatedly during the
|
||||
worktree campaign.
|
||||
|
||||
## Why the pipe lies
|
||||
|
||||
Three independent failure modes, all of them structural:
|
||||
|
||||
- **The broker kills long-lived ssh.** A job longer than the connection's
|
||||
patience dies mid-run; the caller sees a closed pipe, not a failure.
|
||||
- **A readiness pattern can match a log header.** `sed -n /building/q` fires on
|
||||
a banner printed before the work starts — the job is at line 1 and the caller
|
||||
already moved on.
|
||||
- **`sed -n /x/q` exits the instant it matches.** Even when the pattern is
|
||||
genuinely ready-shaped, the pipeline tears down the remote command it was
|
||||
watching. Result: green report, dead job.
|
||||
|
||||
The only reliable signal is a **completion sentinel written by the job itself**,
|
||||
read afterwards by a short-lived connection. That is what this is.
|
||||
|
||||
## Contract
|
||||
|
||||
```bash
|
||||
remote-job.sh start NAME [--] CMD... # launch detached, return at once
|
||||
remote-job.sh status [NAME] # one line per job; exit 0 only if all DONE
|
||||
remote-job.sh wait NAME [SEC] [N] # block until DONE or timeout, then tail N lines
|
||||
remote-job.sh out NAME [N] # tail a job's output
|
||||
remote-job.sh tail [NAME] # follow one job, or every job at once
|
||||
remote-job.sh watch # refreshing status, for a spare pane
|
||||
```
|
||||
|
||||
Job dir `$REMOTE_JOBS_DIR` (default `$HOME/jobs`). Per job: `NAME.out` (stdout+
|
||||
stderr), `NAME.rc` (exit code), `NAME.done` (sentinel, **written last**).
|
||||
|
||||
Exit status of the *report*: `0` all reported jobs finished · `1` unfinished or
|
||||
timed out · `2` usage. The `rc=` column is the job's own code — a job that
|
||||
failed is still `DONE rc=7` and `status` exits `0`, because "is it finished" and
|
||||
"did it work" are different questions a supervisor must be able to ask
|
||||
separately.
|
||||
|
||||
Written for the case where the job **restarts the endpoint the agent itself is
|
||||
running on**: the ssh connection dies, the session may die, and the answer is
|
||||
still sitting in `$JOBS` when a session comes back.
|
||||
|
||||
## Use over ssh
|
||||
|
||||
```bash
|
||||
ssh dankserver ~/.omp/scripts/remote-job.sh start swap -- \
|
||||
bash -c 'systemctl --user restart frank-archon && <the health probe that endpoint answers>'
|
||||
ssh dankserver ~/.omp/scripts/remote-job.sh wait swap 300 20 # short-lived reconnect
|
||||
```
|
||||
|
||||
Each ssh call is short, so there is nothing there to abort, and the second call
|
||||
can be made from a fresh session on a machine that rebooted underneath it.
|
||||
|
||||
## Failure modes
|
||||
|
||||
| Event | Result |
|
||||
|-------|--------|
|
||||
| Session/connection dies mid-job | job keeps running; `status` from anywhere reports it |
|
||||
| Command name is wrong | `PENDING (no output yet)`, exit 1 — never a false DONE |
|
||||
| Job exits non-zero | `DONE rc=N`; the code is data, not the report's status |
|
||||
| Two jobs, same NAME | the second replaces the first's state files |
|
||||
| Job dir missing | created on first use |
|
||||
|
||||
Pinned by `tests/remote-job.bats` (black-box: output, exit codes, sentinel
|
||||
ordering — never internals).
|
||||
@@ -3,3 +3,35 @@
|
||||
# pkgbase amd-ucode (split pkgname amd-ucode). No module options needed.
|
||||
|
||||
amd-ucode
|
||||
|
||||
# Hand-installed additions, measured by scripts/pkg-inventory.sh on 2026-10-20 against the
|
||||
# Calamares install day (baseline 2026-09-07). These stay out of common.pkgs: none of them belong
|
||||
# on the headless servers. install.sh installs them only where they are missing, so this list
|
||||
# changes nothing on the running box — it is what makes the next rebuild faithful.
|
||||
bun
|
||||
cachyos-gaming-applications
|
||||
curlftpfs
|
||||
flatpak
|
||||
fuse2
|
||||
fuseiso
|
||||
ghostty-nautilus
|
||||
glow
|
||||
gnome-calculator
|
||||
gnome-disk-utility
|
||||
nodejs
|
||||
npm
|
||||
obsidian
|
||||
partitionmanager
|
||||
plezy
|
||||
proton-mail-bin
|
||||
python-yaml
|
||||
quickshell
|
||||
shelly-flatpak-backend
|
||||
sshfs
|
||||
tmux
|
||||
ttf-proggyclean-nerd
|
||||
ttf-terminus-nerd
|
||||
wine-cachyos
|
||||
|
||||
# nirimod-git is an AUR addition deliberately left undeclared: aur.pkgs is read by every host, and
|
||||
# a niri module must not be built on the servers. See the known gap in omp-docs/tools/packages.md.
|
||||
|
||||
+22
-2
@@ -123,9 +123,29 @@ if [ "$MODE" != "check" ] && [ -d "$PKGS_DIR/systemd-user/.config/systemd/user"
|
||||
for unit in "$PKGS_DIR"/systemd-user/.config/systemd/user/*.service; do
|
||||
[ -e "$unit" ] || continue
|
||||
bin="$(sed -n 's/^ExecStart=\([^ ]*\).*/\1/p' "$unit" | head -1)"
|
||||
# systemd expands specifiers inside ExecStart; this shell does not. Matching the literal
|
||||
# `%h/…` makes a unit of THIS box look foreign and skips it in silence — which is exactly
|
||||
# how live-sync.service, the repo's own heartbeat, has been reported "absent on this
|
||||
# host" on every machine that deployed it. Expand what a user unit can carry; leave any
|
||||
# other specifier unmatched rather than guess at it. Refs #44.
|
||||
bin="${bin//%h/$HOME}"
|
||||
bin="${bin//%u/$(id -un)}"
|
||||
bin="${bin//%U/$(id -u)}"
|
||||
if [ -n "$bin" ] && command -v "$bin" >/dev/null 2>&1; then
|
||||
systemctl --user enable --now "$(basename "$unit")" || true
|
||||
echo "ENABLE $(basename "$unit") (ExecStart resolves here)"
|
||||
name="$(basename "$unit")"
|
||||
# Read the state, not the exit status: `is-enabled` exits 1 for a `linked` unit exactly
|
||||
# as for a `disabled` one. Starting something already installed is how a deploy locks a
|
||||
# live session — swayidle-lock sits enabled/inactive on blankpad and `--now` arms it
|
||||
# mid-conversation. Only a unit nothing has installed yet gets started. Refs #44.
|
||||
state="$(systemctl --user is-enabled "$name" 2>/dev/null || true)"
|
||||
case "$state" in
|
||||
"" | disabled)
|
||||
systemctl --user enable --now "$name" || true
|
||||
echo "ENABLE $name (ExecStart resolves here; nothing running it yet)" ;;
|
||||
*)
|
||||
systemctl --user enable "$name" || true
|
||||
echo "ENABLE $name (already $state; not started — a live session decides what runs)" ;;
|
||||
esac
|
||||
else
|
||||
echo "SKIP $(basename "$unit") (ExecStart '$bin' absent on this host)"
|
||||
fi
|
||||
|
||||
Executable
+126
@@ -0,0 +1,126 @@
|
||||
#!/usr/bin/env bash
|
||||
# scripts/pcie-probe.sh — is the decode bottleneck the LINK, or the engine's read path?
|
||||
#
|
||||
# Instruments live in scripts/probes/: hip_smoke.hip answers "is the host ROCm usable from a plain
|
||||
# process", pcie_bw.hip measures host→device bandwidth at the engine's real expert-slab size, and
|
||||
# pcie_probe.hip sweeps sizes so a stall can be attributed to the link rather than to the engine.
|
||||
# Written during the RDNA4 expert-offload campaign, kept because the same cards are in Dankserver.
|
||||
#
|
||||
# LAW — never benchmark beside a live service. The guard below runs BEFORE anything is built or run,
|
||||
# and it stops nothing: putting a production endpoint down is not this tool's decision to make. When
|
||||
# you have deliberately decided that contention is acceptable, say so with --force.
|
||||
#
|
||||
# Trap carried in the sources: hipGetDeviceProperties SEGFAULTS on this ROCm build even though
|
||||
# hipGetDeviceCount works fine and reports every device. Nothing here calls it. Keep it that way.
|
||||
#
|
||||
# usage: pcie-probe.sh [--device N]... [--build-only] [--force]
|
||||
#
|
||||
# --device N measure this device; repeat for each card. Without it the probes run once with the
|
||||
# environment's own device selection — one shared run cannot attribute bandwidth.
|
||||
# --build-only compile and stop; never touches the cards.
|
||||
# --force measure even though the guard says the GPU is busy.
|
||||
#
|
||||
# Environment (defaults in parentheses) — the sysfs and kfd paths are overridable so the guard can
|
||||
# be tested on a machine with no cards, which is the only honest way to test a refusal:
|
||||
# PCIE_PROBE_SRC probe sources (<repo>/scripts/probes)
|
||||
# PCIE_PROBE_BUILD build output ($HOME/.cache/pcie-probe)
|
||||
# PCIE_PROBE_SYSFS drm sysfs root (/sys/class/drm)
|
||||
# PCIE_PROBE_KFD KFD device node (/dev/kfd)
|
||||
# PCIE_PROBE_HIPCC compiler (hipcc)
|
||||
# PCIE_PROBE_MAX_BUSY refusal threshold, percent (5)
|
||||
#
|
||||
# Exit status: 0 measured · 2 usage or missing sources · 3 REFUSED, GPU busy · 4 no toolchain or
|
||||
# build failed.
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
SRC="${PCIE_PROBE_SRC:-$(dirname "$(realpath "$0")")/probes}"
|
||||
BUILD="${PCIE_PROBE_BUILD:-$HOME/.cache/pcie-probe}"
|
||||
SYSFS="${PCIE_PROBE_SYSFS:-/sys/class/drm}"
|
||||
KFD="${PCIE_PROBE_KFD:-/dev/kfd}"
|
||||
HIPCC="${PCIE_PROBE_HIPCC:-hipcc}"
|
||||
MAX_BUSY="${PCIE_PROBE_MAX_BUSY:-5}"
|
||||
|
||||
force=0
|
||||
build_only=0
|
||||
devices=()
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--device) [[ $# -ge 2 ]] || { echo "pcie-probe: --device needs an argument" >&2; exit 2; }
|
||||
devices+=("$2"); shift 2 ;;
|
||||
--build-only) build_only=1; shift ;;
|
||||
--force) force=1; shift ;;
|
||||
*) echo "usage: pcie-probe.sh [--device N]... [--build-only] [--force]" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
mapfile -t sources < <(find "$SRC" -maxdepth 1 -name '*.hip' 2>/dev/null | sort)
|
||||
if (( ${#sources[@]} == 0 )); then
|
||||
echo "pcie-probe: no .hip probe sources in $SRC" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# ---- the guard, before anything touches the card --------------------------------
|
||||
if (( ! force )); then
|
||||
busy=""
|
||||
if [[ -e "$KFD" ]] && command -v fuser > /dev/null 2>&1; then
|
||||
holders="$(fuser "$KFD" 2>/dev/null | tr -s '[:space:]' ' ')"
|
||||
[[ -n "${holders// /}" ]] && busy="KFD in use by:${holders}"
|
||||
fi
|
||||
for card in "$SYSFS"/card*; do
|
||||
[[ -f "$card/device/gpu_busy_percent" ]] || continue
|
||||
pct="$(cat "$card/device/gpu_busy_percent" 2>/dev/null)"
|
||||
[[ "$pct" =~ ^[0-9]+$ ]] || continue
|
||||
(( pct > MAX_BUSY )) && busy="${busy:+$busy; }$(basename "$card") at ${pct}%"
|
||||
done
|
||||
if [[ -n "$busy" ]]; then
|
||||
printf 'pcie-probe: REFUSED — GPU busy (%s)\n' "$busy" >&2
|
||||
printf ' never benchmark beside a live service. Take the service down deliberately,\n' >&2
|
||||
printf ' or pass --force if you have decided the contention is acceptable.\n' >&2
|
||||
exit 3
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---- build ----------------------------------------------------------------------
|
||||
if ! command -v "$HIPCC" > /dev/null 2>&1; then
|
||||
printf 'pcie-probe: no HIP toolchain (%s) — run this on the box with the cards, not here\n' "$HIPCC" >&2
|
||||
exit 4
|
||||
fi
|
||||
mkdir -p "$BUILD"
|
||||
for src in "${sources[@]}"; do
|
||||
bin="$BUILD/$(basename "${src%.hip}")"
|
||||
if ! "$HIPCC" -O2 -o "$bin" "$src" 2>&1 | tail -3; then
|
||||
printf 'pcie-probe: build failed for %s\n' "$(basename "$src")" >&2
|
||||
exit 4
|
||||
fi
|
||||
done
|
||||
if (( build_only )); then
|
||||
printf 'pcie-probe: built %s probe(s) into %s\n' "${#sources[@]}" "$BUILD"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# ---- topology: the link the numbers must be read against ------------------------
|
||||
printf 'pcie-probe host=%s hipcc=%s probes=%s\n' "$(hostname)" "$HIPCC" "${#sources[@]}"
|
||||
for card in "$SYSFS"/card*; do
|
||||
[[ -f "$card/device/current_link_width" ]] || continue
|
||||
printf ' link %-9s current x%s @ %s max x%s @ %s\n' "$(basename "$card")" \
|
||||
"$(cat "$card/device/current_link_width")" \
|
||||
"$(cat "$card/device/current_link_speed" 2>/dev/null || echo '?')" \
|
||||
"$(cat "$card/device/max_link_width" 2>/dev/null || echo '?')" \
|
||||
"$(cat "$card/device/max_link_speed" 2>/dev/null || echo '?')"
|
||||
done
|
||||
|
||||
# ---- measure, per device --------------------------------------------------------
|
||||
(( ${#devices[@]} > 0 )) || devices=(default)
|
||||
for d in "${devices[@]}"; do
|
||||
if [[ $d == default ]]; then
|
||||
printf '\n########## engine device selection ##########\n'
|
||||
else
|
||||
printf '\n########## device %s ##########\n' "$d"
|
||||
fi
|
||||
for name in hip_smoke pcie_bw pcie_probe; do
|
||||
bin="$BUILD/$name"
|
||||
[[ -x "$bin" ]] || continue
|
||||
if [[ $d == default ]]; then "$bin"; else HIP_VISIBLE_DEVICES="$d" "$bin" "$d"; fi
|
||||
done
|
||||
done
|
||||
Executable
+159
@@ -0,0 +1,159 @@
|
||||
#!/usr/bin/env bash
|
||||
# scripts/pkg-inventory.sh — what does this machine have that packages/*.pkgs does not declare?
|
||||
#
|
||||
# install.sh walks desired → installed. This walks installed → declared, and the pair is what
|
||||
# makes a rebuild honest: a manifest the machine has quietly outgrown reproduces a machine that
|
||||
# is missing the tools you installed by hand six weeks ago, and nothing notices until the new box
|
||||
# has no tmux. Refs #51.
|
||||
#
|
||||
# The baseline is what separates "you installed this" from "the installer installed this". It is
|
||||
# read from the pacman log — the install-day transactions, dated at the oldest `[ALPM] installed`
|
||||
# line — because declaring Calamares furniture in a manifest is bookkeeping for something a
|
||||
# reinstall provides for free. Rotated logs (pacman.log.N, .N.gz) are read too: on a machine older
|
||||
# than the rotation window the install day lives only there, and skipping them would report the
|
||||
# entire base system as hand-installed.
|
||||
#
|
||||
# Report (stdout):
|
||||
# undeclared additions (declare these): N hand-installed, absent from the manifests
|
||||
# repo (sync dbs): <names> → belongs in packages/<host>.pkgs
|
||||
# aur (needs a helper): <names> → belongs in packages/aur.pkgs (yay)
|
||||
# installer-provided, undeclared: N a reinstall supplies these; do not declare
|
||||
# declared, not installed: N install.sh installs these
|
||||
# orphans ignored: <names> pacman -Qtdq — never emitted
|
||||
# flatpak: <app ids> reported only, install.sh installs no flatpaks
|
||||
#
|
||||
# --emit prints the additions as manifest lines, ready to fold into packages/. Names are bare
|
||||
# tokens so install.sh's `sed 's/#.*//'` + first-field parser reads them unchanged.
|
||||
#
|
||||
# Exit status: 0 clean · 1 drift (undeclared additions exist) · 2 usage · 3 cannot classify
|
||||
# (pacman log unreadable or sync databases unavailable — worse to guess than to say so, because
|
||||
# guessing would dump every package into aur.pkgs).
|
||||
#
|
||||
# Environment overrides (tests):
|
||||
# PKGINV_REPO repo root holding packages/ (default: git toplevel of this script)
|
||||
# PKGINV_HOST host name (default: $(hostname))
|
||||
# PKGINV_LOG space-separated pacman logs (default: /var/log/pacman.log + rotated)
|
||||
# PKGINV_BASELINE YYYY-MM-DD, inclusive (default: oldest install date in the logs)
|
||||
|
||||
set -euo pipefail
|
||||
export LC_ALL=C # comm/sort agreement: pacman's order is not this order
|
||||
|
||||
MODE="report"
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--emit) MODE="emit"; shift ;;
|
||||
-h | --help) sed -n '2,40p' "$0"; exit 0 ;;
|
||||
*) echo "usage: pkg-inventory.sh [--emit]" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
REPO="${PKGINV_REPO:-}"
|
||||
if [[ -z "$REPO" ]]; then
|
||||
REPO="$(git -C "$(dirname "$(realpath "$0")")" rev-parse --show-toplevel 2>/dev/null || true)"
|
||||
fi
|
||||
[[ -n "$REPO" && -d "$REPO/packages" ]] || { echo "pkg-inventory: no packages/ under ${REPO:-<repo>}" >&2; exit 2; }
|
||||
HOST="${PKGINV_HOST:-$(hostname)}"
|
||||
|
||||
TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
read_manifest() { # same parser install.sh uses, so a name means the same thing to both
|
||||
sed 's/#.*//' "$1" | tr -d '\r' | awk '{if ($1) print $1}'
|
||||
}
|
||||
# ---- installed state ---------------------------------------------------------
|
||||
# Four queries are the tool's whole interface to pacman. -Sl is the unprivileged sync listing:
|
||||
# -Sqq asks for root, and a read of the machine must not need sudo. Field 2 of -Sl is the package
|
||||
# name, and one name can appear in several databases (core/extra/cachyos), hence the sort -u.
|
||||
pacman -Qqe > "$TMP/explicit"
|
||||
pacman -Qq > "$TMP/installed"
|
||||
pacman -Qtdq > "$TMP/orphans" 2>/dev/null || true # exit 1 when the list is empty: no orphans
|
||||
if ! pacman -Sl > "$TMP/sync.raw" 2>/dev/null || [[ ! -s "$TMP/sync.raw" ]]; then
|
||||
echo "pkg-inventory: no sync database available (run: sudo pacman -Sy) — repo vs AUR cannot be classified" >&2
|
||||
exit 3
|
||||
fi
|
||||
awk '{print $2}' "$TMP/sync.raw" | sort -u > "$TMP/sync"
|
||||
|
||||
# ---- declared desired state --------------------------------------------------
|
||||
: > "$TMP/declared"
|
||||
for m in common.pkgs "$HOST.pkgs" aur.pkgs; do
|
||||
[[ -f "$REPO/packages/$m" ]] || continue
|
||||
read_manifest "$REPO/packages/$m" >> "$TMP/declared"
|
||||
done
|
||||
|
||||
# ---- baseline: what the installer put here -----------------------------------
|
||||
logs=()
|
||||
if [[ -n "${PKGINV_LOG:-}" ]]; then
|
||||
read -ra logs <<< "$PKGINV_LOG"
|
||||
else
|
||||
logs=(/var/log/pacman.log)
|
||||
for f in /var/log/pacman.log.[0-9]*; do [[ -e "$f" ]] && logs+=("$f"); done
|
||||
fi
|
||||
read_logs() {
|
||||
local f
|
||||
for f in "${logs[@]}"; do
|
||||
[[ -r "$f" ]] || continue
|
||||
case "$f" in *.gz) zcat "$f" ;; *) cat "$f" ;; esac
|
||||
done
|
||||
}
|
||||
# "YYYY-MM-DD name" for every install event the log remembers.
|
||||
read_logs | sed -nE 's/^\[([0-9-]{10})T[^]]*\] \[ALPM\] installed ([^ ]+) \(.*/\1 \2/p' > "$TMP/events"
|
||||
[[ -s "$TMP/events" ]] || { echo "pkg-inventory: no install events readable in ${logs[*]} — baseline unknown" >&2; exit 3; }
|
||||
|
||||
# Minimum date in a single awk pass. Sorting and then taking the first line — whether with `head`
|
||||
# or with an awk that exits early — closes the pipe, and pipefail turns sort's SIGPIPE into a dead
|
||||
# script; a real install-day log is long enough for that to bite (it killed the first real run).
|
||||
BASELINE="${PKGINV_BASELINE:-$(awk '{ if (m == "" || $1 < m) m = $1 } END { print m }' "$TMP/events")}"
|
||||
[[ "$BASELINE" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]] || { echo "pkg-inventory: bad PKGINV_BASELINE '$BASELINE' (want YYYY-MM-DD)" >&2; exit 2; }
|
||||
|
||||
# Inclusive: packages installed ON the baseline date came with the install too — Calamares keeps
|
||||
# installing the desktop for minutes after the base transaction.
|
||||
awk -v d="$BASELINE" '$1 <= d {print $2}' "$TMP/events" | sort -u > "$TMP/base"
|
||||
|
||||
for f in explicit installed sync declared base orphans; do sort -u -o "$TMP/$f" "$TMP/$f"; done
|
||||
|
||||
# ---- the sets ----------------------------------------------------------------
|
||||
# additions: explicit, not declared, and not something the installer handed us.
|
||||
comm -23 "$TMP/explicit" "$TMP/declared" > "$TMP/undeclared"
|
||||
comm -23 "$TMP/undeclared" "$TMP/base" > "$TMP/additions"
|
||||
comm -12 "$TMP/undeclared" "$TMP/base" > "$TMP/installside"
|
||||
comm -12 "$TMP/additions" "$TMP/sync" > "$TMP/add.repo"
|
||||
comm -23 "$TMP/additions" "$TMP/sync" > "$TMP/add.aur"
|
||||
comm -23 "$TMP/declared" "$TMP/installed" > "$TMP/missing"
|
||||
|
||||
n() { wc -l < "$1" | tr -d ' '; }
|
||||
|
||||
if [[ "$MODE" == emit ]]; then
|
||||
if [[ -s "$TMP/add.repo" ]]; then
|
||||
printf '# packages/%s.pkgs — additions measured %s (pkg-inventory.sh, baseline %s)\n' \
|
||||
"$HOST" "$(date +%F)" "$BASELINE"
|
||||
cat "$TMP/add.repo"
|
||||
fi
|
||||
if [[ -s "$TMP/add.aur" ]]; then
|
||||
printf '# packages/aur.pkgs — additions measured %s (AUR: not in any sync db, install via yay)\n' \
|
||||
"$(date +%F)"
|
||||
cat "$TMP/add.aur"
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
|
||||
printf 'pkg-inventory host=%s baseline=%s repo=%s\n' "$HOST" "$BASELINE" "$REPO"
|
||||
printf 'counts declared=%s explicit=%s baseline=%s installed=%s\n' \
|
||||
"$(n "$TMP/declared")" "$(n "$TMP/explicit")" "$(n "$TMP/base")" "$(n "$TMP/installed")"
|
||||
printf 'undeclared additions (declare these): %s\n' "$(n "$TMP/additions")"
|
||||
printf ' repo (sync dbs): %s\n' "$(tr '\n' ' ' < "$TMP/add.repo")"
|
||||
printf ' aur (needs a helper): %s\n' "$(tr '\n' ' ' < "$TMP/add.aur")"
|
||||
printf 'installer-provided, undeclared (a reinstall provides these): %s\n' "$(n "$TMP/installside")"
|
||||
# The names, folded: this is the answer to "what did the installer mark explicit that I never
|
||||
# declared", and it is long — a count alone would force a second run to see it.
|
||||
# The echo matters: tr eats the final newline, fold preserves that absence, and the next label
|
||||
# would otherwise be glued to the last folded line.
|
||||
{ tr '\n' ' ' < "$TMP/installside"; echo; } | fold -s -w 88 | sed 's/^/ /'
|
||||
printf 'declared, not installed (install.sh installs these): %s\n' "$(tr '\n' ' ' < "$TMP/missing")"
|
||||
printf 'orphans ignored (pacman -Qtdq): %s\n' "$(tr '\n' ' ' < "$TMP/orphans")"
|
||||
if command -v flatpak > /dev/null 2>&1 && flatpak list --app --columns=application > "$TMP/flatpak" 2>/dev/null; then
|
||||
printf 'flatpak (reported only; install.sh installs no flatpaks): %s\n' "$(tr '\n' ' ' < "$TMP/flatpak")"
|
||||
else
|
||||
printf 'flatpak: not installed on this host\n'
|
||||
fi
|
||||
|
||||
[[ -s "$TMP/additions" ]] && exit 1
|
||||
exit 0
|
||||
@@ -0,0 +1,36 @@
|
||||
// Minimal HIP smoke test: is the host ROCm usable from a plain process?
|
||||
#include <cstdio>
|
||||
#include <cstring>
|
||||
#include <hip/hip_runtime.h>
|
||||
|
||||
int main(void) {
|
||||
int n = -1;
|
||||
hipError_t e = hipGetDeviceCount(&n);
|
||||
printf("hipGetDeviceCount -> %d (%s), count=%d\n",
|
||||
(int)e, hipGetErrorString(e), n);
|
||||
if (e != hipSuccess || n <= 0) return 1;
|
||||
setvbuf(stdout, NULL, _IONBF, 0); /* an earlier crash lost the buffer, hiding the crash point */
|
||||
// Device properties are deliberately NOT queried here: hipGetDeviceProperties segfaults on the
|
||||
// ROCm build this probe was written for, even though hipGetDeviceCount above works fine. Card
|
||||
// identity comes from sysfs anyway — scripts/pcie-probe.sh prints the link topology.
|
||||
|
||||
const size_t bytes = 2459136; /* 2.344 MiB: the engine's expert slab */
|
||||
void *dev = NULL, *host = NULL;
|
||||
if (hipMalloc(&dev, bytes) != hipSuccess) { printf("hipMalloc failed\n"); return 1; }
|
||||
if (hipHostMalloc(&host, bytes, hipHostMallocDefault) != hipSuccess) {
|
||||
printf("hipHostMalloc failed\n"); return 1;
|
||||
}
|
||||
memset(host, 0x5a, bytes);
|
||||
|
||||
hipEvent_t a, b; hipEventCreate(&a); hipEventCreate(&b);
|
||||
hipDeviceSynchronize();
|
||||
hipEventRecord(a, 0);
|
||||
const int iters = 200;
|
||||
for (int i = 0; i < iters; i++) hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice);
|
||||
hipEventRecord(b, 0); hipEventSynchronize(b);
|
||||
float ms = 0; hipEventElapsedTime(&ms, a, b);
|
||||
printf("\n %.3f MiB x %d iters in %.2f ms -> %.2f GB/s H2D (pinned)\n",
|
||||
bytes/1048576.0, iters, ms, (double)bytes*iters/(ms/1000.0)/1e9);
|
||||
printf(" Gen5 x8 ceiling = 31.5 GB/s\n");
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
// H2D bandwidth at the engine's expert-slab size.
|
||||
// Deliberately avoids hipGetDeviceProperties: it segfaults on this ROCm build
|
||||
// (hipGetDeviceCount works fine and reports 3 devices).
|
||||
#include <cstdio>
|
||||
#include <cstring>
|
||||
#include <cstdlib>
|
||||
#include <hip/hip_runtime.h>
|
||||
|
||||
static double one(size_t bytes, bool pinned, int iters) {
|
||||
void *dev = NULL, *host = NULL;
|
||||
if (hipMalloc(&dev, bytes) != hipSuccess) return -1;
|
||||
if (pinned) { if (hipHostMalloc(&host, bytes, hipHostMallocDefault) != hipSuccess) return -1; }
|
||||
else { host = malloc(bytes); }
|
||||
memset(host, 0x5a, bytes);
|
||||
|
||||
hipEvent_t a, b; hipEventCreate(&a); hipEventCreate(&b);
|
||||
for (int i = 0; i < 3; i++) hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice);
|
||||
hipDeviceSynchronize();
|
||||
|
||||
hipEventRecord(a, 0);
|
||||
for (int i = 0; i < iters; i++) hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice);
|
||||
hipEventRecord(b, 0); hipEventSynchronize(b);
|
||||
float ms = 0; hipEventElapsedTime(&ms, a, b);
|
||||
double gb = (double)bytes * iters / (ms / 1000.0) / 1e9;
|
||||
|
||||
if (!pinned) free(host); else hipHostFree(host);
|
||||
hipFree(dev); hipEventDestroy(a); hipEventDestroy(b);
|
||||
return gb;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
setvbuf(stdout, NULL, _IONBF, 0);
|
||||
int dev = (argc > 1) ? atoi(argv[1]) : 0;
|
||||
if (hipSetDevice(dev) != hipSuccess) { printf("setDevice(%d) failed\n", dev); return 1; }
|
||||
printf("device %d selected\n", dev);
|
||||
|
||||
size_t sz[] = {2459136UL, 3UL<<20, 16UL<<20, 64UL<<20, 256UL<<20, 1UL<<30};
|
||||
printf(" %10s | %7s | %7s | %7s\n", "MiB", "pinned", "pagin.", "xfer us");
|
||||
printf(" %s\n", "-------------------------------------------");
|
||||
for (int i = 0; i < 6; i++) {
|
||||
size_t b = sz[i];
|
||||
int iters = (b >= (1UL<<30)) ? 10 : (b >= (256UL<<20)) ? 20 : 100;
|
||||
double p = one(b, true, iters);
|
||||
double g = one(b, false, iters);
|
||||
// single-transfer latency
|
||||
void *dv, *hv;
|
||||
if (hipMalloc(&dv, b) == hipSuccess && hipHostMalloc(&hv, b, hipHostMallocDefault) == hipSuccess) {
|
||||
memset(hv, 0x5a, b);
|
||||
hipEvent_t a, c; hipEventCreate(&a); hipEventCreate(&c);
|
||||
hipMemcpy(dv, hv, b, hipMemcpyHostToDevice); hipDeviceSynchronize();
|
||||
hipEventRecord(a, 0);
|
||||
hipMemcpy(dv, hv, b, hipMemcpyHostToDevice);
|
||||
hipEventRecord(c, 0); hipEventSynchronize(c);
|
||||
float ms = 0; hipEventElapsedTime(&ms, a, c);
|
||||
printf(" %10.3f | %7.2f | %7.2f | %7.1f\n", b/1048576.0, p, g, ms*1000.0);
|
||||
hipHostFree(hv); hipFree(dv);
|
||||
} else {
|
||||
printf(" %10.3f | %7.2f | %7.2f | (alloc failed)\n", b/1048576.0, p, g);
|
||||
}
|
||||
}
|
||||
printf("\n 2.344 MiB is the engine's exact expert slab size.\n");
|
||||
printf(" Gen5 x8 ceiling = 31.5 GB/s per direction.\n");
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
// HIP adaptation of the pcie-link-capability-probe.
|
||||
// Question: is the decode bottleneck the LINK, or the engine's read PATH?
|
||||
// Measure H2D at the engine's actual expert slab size (2.344 MiB) and across
|
||||
// a size sweep. If 2.344 MiB copies reach ~31 GB/s, the link is healthy and
|
||||
// the engine is leaving bandwidth on the table. If they stall near 10 GB/s,
|
||||
// the link is the wall.
|
||||
//
|
||||
// build: hipcc -O2 -o pcie_probe pcie_probe.hip (scripts/pcie-probe.sh does this and guards the run)
|
||||
#include <cstdio>
|
||||
#include <cstring>
|
||||
#include <cstdlib>
|
||||
#include <hip/hip_runtime.h>
|
||||
|
||||
#define CK(x) do { hipError_t e = (x); if (e != hipSuccess) { \
|
||||
fprintf(stderr, "HIP %s (%s:%d)\n", hipGetErrorString(e), __FILE__, __LINE__); \
|
||||
return 1; } } while (0)
|
||||
|
||||
static double ms_between(hipEvent_t a, hipEvent_t b) {
|
||||
float ms = 0; hipEventElapsedTime(&ms, a, b); return (double)ms;
|
||||
}
|
||||
|
||||
static int bench_size(size_t bytes) {
|
||||
const int iters = bytes >= (1UL<<30) ? 20
|
||||
: bytes >= (256UL<<20) ? 30
|
||||
: bytes >= (16UL<<20) ? 60 : 150;
|
||||
void *dev = NULL, *host = NULL, *pageable = NULL;
|
||||
hipStream_t s = NULL;
|
||||
CK(hipMalloc(&dev, bytes));
|
||||
CK(hipHostMalloc(&host, bytes, hipHostMallocDefault)); // PINNED, like the engine's slabs
|
||||
memset(host, 0x5a, bytes);
|
||||
CK(hipStreamCreate(&s));
|
||||
hipEvent_t a, b; CK(hipEventCreate(&a)); CK(hipEventCreate(&b));
|
||||
|
||||
for (int i = 0; i < 3; i++) CK(hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice));
|
||||
CK(hipDeviceSynchronize());
|
||||
|
||||
// sync copy
|
||||
CK(hipEventRecord(a, 0));
|
||||
for (int i = 0; i < iters; i++) CK(hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice));
|
||||
CK(hipEventRecord(b, 0)); CK(hipEventSynchronize(b));
|
||||
double h2d = (double)bytes * iters / (ms_between(a,b)/1000.0) / 1e9;
|
||||
|
||||
// async copy on a stream (the engine's fill pipeline shape)
|
||||
CK(hipEventRecord(a, 0));
|
||||
for (int i = 0; i < iters; i++) CK(hipMemcpyAsync(dev, host, bytes, hipMemcpyHostToDevice, s));
|
||||
CK(hipEventRecord(b, 0)); CK(hipEventSynchronize(b));
|
||||
double h2da = (double)bytes * iters / (ms_between(a,b)/1000.0) / 1e9;
|
||||
|
||||
// single-transfer latency at this size
|
||||
CK(hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice));
|
||||
CK(hipDeviceSynchronize());
|
||||
CK(hipEventRecord(a, 0));
|
||||
CK(hipMemcpy(dev, host, bytes, hipMemcpyHostToDevice));
|
||||
CK(hipEventRecord(b, 0)); CK(hipEventSynchronize(b));
|
||||
double one_us = ms_between(a,b) * 1000.0;
|
||||
|
||||
// pageable, to see whether pinning is load-bearing here
|
||||
pageable = malloc(bytes); memset(pageable, 0x5a, bytes);
|
||||
CK(hipMemcpy(dev, pageable, bytes, hipMemcpyHostToDevice));
|
||||
CK(hipEventRecord(a, 0));
|
||||
for (int i = 0; i < iters; i++) CK(hipMemcpy(dev, pageable, bytes, hipMemcpyHostToDevice));
|
||||
CK(hipEventRecord(b, 0)); CK(hipEventSynchronize(b));
|
||||
double pag = (double)bytes * iters / (ms_between(a,b)/1000.0) / 1e9;
|
||||
|
||||
printf(" %8.3f MiB | %6.2f | %6.2f | %8.1f us | %6.2f\n",
|
||||
bytes/1048576.0, h2d, h2da, one_us, pag);
|
||||
|
||||
free(pageable); hipHostFree(host); hipFree(dev);
|
||||
hipEventDestroy(a); hipEventDestroy(b); hipStreamDestroy(s);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
int dev_id = 0;
|
||||
if (argc > 1) dev_id = atoi(argv[1]);
|
||||
// No hipGetDeviceProperties here: it segfaults on this ROCm build while hipGetDeviceCount is
|
||||
// fine, and the link topology these numbers must be read against comes from sysfs, which
|
||||
// scripts/pcie-probe.sh prints. Only the device index is needed to pick what to measure.
|
||||
printf("device %d (PCI bus from sysfs, see the topology block above)\n", dev_id);
|
||||
printf(" size | H2D | async | 1x xfer | pageable [GB/s]\n");
|
||||
printf(" %s\n", "-----------------------------------------------------------");
|
||||
size_t sz[] = {2459136UL, /* 2.344 MiB: the engine's exact expert slab */
|
||||
3UL<<20, 16UL<<20, 64UL<<20, 256UL<<20, 1UL<<30, 2UL<<30};
|
||||
for (int i = 0; i < 7; i++) if (bench_size(sz[i])) return 1;
|
||||
printf("\nGen5 x8 ceiling = 31.5 GB/s per direction\n");
|
||||
return 0;
|
||||
}
|
||||
Executable
+115
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env bash
|
||||
# scripts/remote-job.sh — a detached job whose result outlives the session that started it
|
||||
#
|
||||
# Why this exists rather than `ssh host 'cmd | tail -f | sed -n /ready/q'`, which lied about success
|
||||
# repeatedly, for three structural reasons — all of them the pipe's fault: the broker aborts
|
||||
# long-lived ssh; a readiness pattern can match a log *header* before the work has begun; and
|
||||
# `sed -n /x/q` exits the instant it matches, so the supervisor reports green on a job that never
|
||||
# finished.
|
||||
#
|
||||
# Here the job runs detached on the box it is invoked on (over ssh, or locally). It writes NAME.out,
|
||||
# then NAME.rc, then touches NAME.done LAST. Status is then a cheap read of a directory: no
|
||||
# long-lived connection to lose, and the answer survives the session, the terminal, and the agent —
|
||||
# including the case where the job restarts the very service the agent is running on.
|
||||
#
|
||||
# remote-job.sh start NAME [--] CMD... launch detached, return at once
|
||||
# remote-job.sh status [NAME] one line per job
|
||||
# remote-job.sh wait NAME [SECONDS] [N] block until DONE or timeout, then tail N lines
|
||||
# remote-job.sh out NAME [LINES] print the tail of a job's output
|
||||
# remote-job.sh tail [NAME] follow one job, or every job at once
|
||||
# remote-job.sh watch refreshing status, for a spare pane
|
||||
#
|
||||
# Job directory: $REMOTE_JOBS_DIR, default $HOME/jobs. Ship this file to the host and run it there;
|
||||
# nothing about it is box-specific, which is the point.
|
||||
#
|
||||
# Exit status: 0 when every job reported is finished · 1 when something is unfinished or a wait
|
||||
# timed out · 2 usage. A job that FAILED still counts as finished for `status`: "is it done" and
|
||||
# "did it work" are different questions, and the second one is the rc= column.
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
JOBS="${REMOTE_JOBS_DIR:-$HOME/jobs}"
|
||||
mkdir -p "$JOBS"
|
||||
|
||||
case "${1:-}" in
|
||||
start)
|
||||
[[ $# -ge 2 ]] || { echo "usage: remote-job.sh start NAME [--] CMD..." >&2; exit 2; }
|
||||
name="$2"; shift 2
|
||||
[[ "${1:-}" == "--" ]] && shift
|
||||
[[ $# -ge 1 ]] || { echo "remote-job: start needs a command for '$name'" >&2; exit 2; }
|
||||
# Clear the previous run entirely: a stale .out would make the new run's line count a lie.
|
||||
rm -f "$JOBS/$name.out" "$JOBS/$name.rc" "$JOBS/$name.done"
|
||||
# setsid leaves the invoking process group, so a dropped ssh or a killed agent takes nothing
|
||||
# down with it. rc lands BEFORE the sentinel: a waiter must never see DONE with no exit code.
|
||||
JOBS="$JOBS" setsid nohup bash -c '
|
||||
name="$1"; shift
|
||||
"$@" > "$JOBS/$name.out" 2>&1
|
||||
echo $? > "$JOBS/$name.rc"
|
||||
touch "$JOBS/$name.done"
|
||||
' _ "$name" "$@" < /dev/null > /dev/null 2>&1 &
|
||||
printf 'started %s\n' "$name"
|
||||
;;
|
||||
|
||||
status)
|
||||
if [[ -n "${2:-}" ]]; then
|
||||
names="$2"
|
||||
else
|
||||
names=$(cd "$JOBS" && ls *.out 2>/dev/null | sed 's/\.out$//' | sort)
|
||||
fi
|
||||
pending=0
|
||||
for n in $names; do
|
||||
if [[ -f "$JOBS/$n.done" ]]; then
|
||||
printf ' %-20s DONE rc=%s %s lines\n' "$n" \
|
||||
"$(cat "$JOBS/$n.rc" 2>/dev/null)" "$(wc -l < "$JOBS/$n.out" 2>/dev/null | tr -d ' ')"
|
||||
elif [[ -f "$JOBS/$n.out" ]]; then
|
||||
printf ' %-20s RUN %s lines so far\n' "$n" "$(wc -l < "$JOBS/$n.out" 2>/dev/null | tr -d ' ')"
|
||||
pending=1
|
||||
else
|
||||
printf ' %-20s PENDING (no output yet)\n' "$n"
|
||||
pending=1
|
||||
fi
|
||||
done
|
||||
if [[ -z "$names" ]]; then echo " (no jobs)"; exit 0; fi
|
||||
exit "$pending"
|
||||
;;
|
||||
|
||||
wait)
|
||||
[[ $# -ge 2 ]] || { echo "usage: remote-job.sh wait NAME [SECONDS] [TAIL]" >&2; exit 2; }
|
||||
name="$2"; limit="${3:-3600}"; lines="${4:-25}"; waited=0
|
||||
while [[ ! -e "$JOBS/$name.done" ]] && (( waited < limit )); do
|
||||
sleep 1; waited=$((waited + 1))
|
||||
done
|
||||
if [[ -e "$JOBS/$name.done" ]]; then
|
||||
printf 'DONE %s rc=%s\n' "$name" "$(cat "$JOBS/$name.rc" 2>/dev/null)"; rc=0
|
||||
else
|
||||
printf 'TIMEOUT %s after %ss\n' "$name" "$waited"; rc=1
|
||||
fi
|
||||
tail -n "$lines" "$JOBS/$name.out" 2>/dev/null
|
||||
exit "$rc"
|
||||
;;
|
||||
|
||||
out)
|
||||
[[ $# -ge 2 ]] || { echo "usage: remote-job.sh out NAME [LINES]" >&2; exit 2; }
|
||||
tail -n "${3:-40}" "$JOBS/$2.out" 2>/dev/null
|
||||
;;
|
||||
|
||||
tail)
|
||||
# Multiplexed follow: no NAME means every job at once, so one pane shows the whole harness.
|
||||
if [[ -n "${2:-}" ]]; then exec tail -f "$JOBS/$2.out"; fi
|
||||
exec tail -f "$JOBS"/*.out
|
||||
;;
|
||||
|
||||
watch)
|
||||
while true; do
|
||||
clear 2>/dev/null || true
|
||||
date '+%H:%M:%S'
|
||||
"$0" status
|
||||
sleep 5
|
||||
done
|
||||
;;
|
||||
|
||||
*)
|
||||
echo "usage: remote-job.sh {start|status|wait|out|tail|watch} ..." >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
@@ -23,6 +23,24 @@ setup() {
|
||||
export HOME="$TEST_TMP/home"
|
||||
mkdir -p "$HOME"
|
||||
export DEPLOY_REPO="$REPO" DEPLOY_HOST=box
|
||||
|
||||
# systemctl stub: records every call and answers is-enabled from a file. The real
|
||||
# systemctl must never run here — on this box it would start a screen lock.
|
||||
mkdir -p "$TEST_TMP/bin"
|
||||
export STUB_LOG="$TEST_TMP/systemctl.log" UNIT_STATE="$TEST_TMP/unit-state"
|
||||
: > "$STUB_LOG"
|
||||
cat > "$TEST_TMP/bin/systemctl" <<'STUB'
|
||||
#!/usr/bin/env bash
|
||||
echo "systemctl $*" >> "$STUB_LOG"
|
||||
if [ "$2" = is-enabled ]; then
|
||||
state="$(cat "$UNIT_STATE" 2>/dev/null || echo disabled)"
|
||||
printf '%s\n' "$state"
|
||||
case "$state" in disabled | "") exit 1 ;; *) exit 0 ;; esac
|
||||
fi
|
||||
exit 0
|
||||
STUB
|
||||
chmod +x "$TEST_TMP/bin/systemctl"
|
||||
export PATH="$TEST_TMP/bin:$PATH"
|
||||
}
|
||||
|
||||
teardown() { rm -rf "$TEST_TMP"; }
|
||||
@@ -32,6 +50,13 @@ repo_file() { mkdir -p "$(dirname "$REPO/dotfiles/$1")"; printf '%s\n' "$2" > "$
|
||||
# live_file <absolute path under $HOME> <content>
|
||||
live_file() { mkdir -p "$(dirname "$1")"; printf '%s\n' "$2" > "$1"; }
|
||||
|
||||
# systemd_unit <name> <ExecStart> — a unit in the systemd-user package
|
||||
systemd_unit() {
|
||||
mkdir -p "$REPO/dotfiles/systemd-user/.config/systemd/user"
|
||||
printf '[Unit]\nDescription=%s\n[Service]\nExecStart=%s\n[Install]\nWantedBy=default.target\n' "$1" "$2" \
|
||||
> "$REPO/dotfiles/systemd-user/.config/systemd/user/$1"
|
||||
}
|
||||
|
||||
@test "generated theme output is never adopted into a symlink" {
|
||||
repo_file "ghostty/.config/ghostty/themes/noctalia" 'background = "#f43841"'
|
||||
live_file "$HOME/.config/ghostty/themes/noctalia" 'background = "#f43841"'
|
||||
@@ -74,3 +99,56 @@ live_file() { mkdir -p "$(dirname "$1")"; printf '%s\n' "$2" > "$1"; }
|
||||
grep -q 'theme = "tokyo-night"' "$HOME/.config/ghostty/config.ghostty"
|
||||
[ ! -d "$REPO/.quarantine" ]
|
||||
}
|
||||
|
||||
@test "a unit already enabled is enabled but not started" {
|
||||
systemd_unit lock.service /bin/sh
|
||||
printf 'enabled\n' > "$UNIT_STATE"
|
||||
run "$REPO/scripts/deploy.sh"
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"not started"* ]]
|
||||
grep -q 'systemctl --user enable lock.service' "$STUB_LOG"
|
||||
! grep -q -- '--now' "$STUB_LOG"
|
||||
}
|
||||
|
||||
@test "a linked unit is not started either" {
|
||||
# `is-enabled` exits 1 for a linked unit, exactly as it does for a disabled one, so the
|
||||
# printed state is the signal — exit status cannot tell them apart.
|
||||
systemd_unit timer-driven.service /bin/sh
|
||||
printf 'linked\n' > "$UNIT_STATE"
|
||||
run "$REPO/scripts/deploy.sh"
|
||||
[ "$status" -eq 0 ]
|
||||
! grep -q -- '--now' "$STUB_LOG"
|
||||
}
|
||||
|
||||
@test "a disabled unit is enabled and started, so a fresh box still comes up" {
|
||||
systemd_unit fresh.service /bin/sh
|
||||
printf 'disabled\n' > "$UNIT_STATE"
|
||||
run "$REPO/scripts/deploy.sh"
|
||||
[ "$status" -eq 0 ]
|
||||
grep -q 'systemctl --user enable --now fresh.service' "$STUB_LOG"
|
||||
}
|
||||
|
||||
@test "a unit whose ExecStart is absent here stays inert" {
|
||||
systemd_unit otherbox.service /nonexistent/daemon
|
||||
printf 'disabled\n' > "$UNIT_STATE"
|
||||
run "$REPO/scripts/deploy.sh"
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"SKIP"*otherbox.service* ]]
|
||||
! grep -q enable "$STUB_LOG"
|
||||
}
|
||||
|
||||
@test "a unit whose ExecStart uses a systemd specifier still resolves here" {
|
||||
# `%h` is expanded by systemd, not by the shell, so matching the literal string makes a
|
||||
# unit belonging to THIS box look foreign and skip it in silence. The victim is this repo's
|
||||
# own heartbeat: `ExecStart=%h/.omp/scripts/live-sync.sh`, skipped on every box. Refs #44.
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
printf '#!/bin/sh\ntrue\n' > "$HOME/.local/bin/helper"
|
||||
chmod +x "$HOME/.local/bin/helper"
|
||||
systemd_unit heartbeat.service '%h/.local/bin/helper'
|
||||
printf 'linked\n' > "$UNIT_STATE"
|
||||
run "$REPO/scripts/deploy.sh"
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"heartbeat.service (already linked"* ]]
|
||||
grep -q 'systemctl --user enable heartbeat.service' "$STUB_LOG"
|
||||
! grep -q -- '--now' "$STUB_LOG"
|
||||
}
|
||||
|
||||
Executable
+144
@@ -0,0 +1,144 @@
|
||||
#!/usr/bin/env bats
|
||||
# tests/pcie-probe.bats — black-box tests for scripts/pcie-probe.sh (Refs #53)
|
||||
#
|
||||
# Contract: measure host→device bandwidth on an RDNA4 box, and refuse to measure while the GPU is
|
||||
# in use. Fleet law: never benchmark beside a live service — so the guard runs BEFORE anything is
|
||||
# built or run, and the tool stops nothing and starts nothing (auto-stopping a production service is
|
||||
# the thing it must not do unasked).
|
||||
#
|
||||
# Exit status: 0 measured · 2 usage · 3 refused (GPU busy) · 4 no toolchain / build failed.
|
||||
#
|
||||
# hipcc and the compiled probes are stubbed: this box has no HIP SDK (it is a client laptop) and a
|
||||
# real measurement belongs on a machine with the cards, beside no running model.
|
||||
|
||||
setup() {
|
||||
TEST_TMP="$(mktemp -d)"
|
||||
SCRIPT="$BATS_TEST_DIRNAME/../scripts/pcie-probe.sh"
|
||||
SRC="$TEST_TMP/src"
|
||||
BUILD="$TEST_TMP/build"
|
||||
SYSFS="$TEST_TMP/drm"
|
||||
BIN="$TEST_TMP/bin"
|
||||
mkdir -p "$SRC" "$BUILD" "$BIN" "$SYSFS/card0/device" "$SYSFS/card1/device"
|
||||
|
||||
# Fixtures stand in for the real sysfs files the driver reads.
|
||||
printf '8\n' > "$SYSFS/card0/device/current_link_width"
|
||||
printf '32.0 GT/s PCIe\n' > "$SYSFS/card0/device/current_link_speed"
|
||||
printf '16\n' > "$SYSFS/card0/device/max_link_width"
|
||||
printf '32.0 GT/s PCIe\n' > "$SYSFS/card0/device/max_link_speed"
|
||||
printf '0\n' > "$SYSFS/card0/device/gpu_busy_percent"
|
||||
printf '8\n' > "$SYSFS/card1/device/current_link_width"
|
||||
printf '32.0 GT/s PCIe\n' > "$SYSFS/card1/device/current_link_speed"
|
||||
printf '16\n' > "$SYSFS/card1/device/max_link_width"
|
||||
printf '32.0 GT/s PCIe\n' > "$SYSFS/card1/device/max_link_speed"
|
||||
printf '0\n' > "$SYSFS/card1/device/gpu_busy_percent"
|
||||
|
||||
# Probe sources: the real .hip files are shipped by the script's own directory, so the fixtures
|
||||
# only need to exist for the build step to be invoked.
|
||||
: > "$SRC/pcie_probe.hip"; : > "$SRC/pcie_bw.hip"; : > "$SRC/hip_smoke.hip"
|
||||
|
||||
# Stub toolchain: records that it was called, and emits a probe binary that records that IT ran.
|
||||
cat > "$BIN/hipcc" <<'STUB'
|
||||
#!/usr/bin/env bash
|
||||
[ "${STUB_HIPCC_FAILS:-0}" = 1 ] && { echo "stub: no matching GPU target" >&2; exit 1; }
|
||||
out=""
|
||||
while [ $# -gt 0 ]; do [ "$1" = "-o" ] && out="$2"; shift; done
|
||||
[ -n "$out" ] || exit 2
|
||||
: > "$PCIE_PROBE_BUILD/.built"
|
||||
{ echo '#!/usr/bin/env bash'
|
||||
echo ': > "$PCIE_PROBE_BUILD/.ran"'
|
||||
echo 'echo "PROBE RAN dev=${HIP_VISIBLE_DEVICES:-none} bin=$(basename "$0")"'
|
||||
echo 'echo " slab 2.344 MiB: 31.4 GB/s"'
|
||||
} > "$out"
|
||||
chmod +x "$out"
|
||||
STUB
|
||||
cat > "$BIN/fuser" <<'STUB'
|
||||
#!/usr/bin/env bash
|
||||
[ -n "${STUB_KFD_PIDS:-}" ] && printf '%s\n' $STUB_KFD_PIDS
|
||||
exit 0
|
||||
STUB
|
||||
chmod +x "$BIN/hipcc" "$BIN/fuser"
|
||||
|
||||
export PATH="$BIN:$PATH" PCIE_PROBE_SRC="$SRC" PCIE_PROBE_BUILD="$BUILD" \
|
||||
PCIE_PROBE_SYSFS="$SYSFS" PCIE_PROBE_KFD="$TEST_TMP/kfd" PCIE_PROBE_HIPCC="$BIN/hipcc"
|
||||
: > "$TEST_TMP/kfd"
|
||||
}
|
||||
|
||||
teardown() { rm -rf "$TEST_TMP"; }
|
||||
|
||||
@test "a missing HIP toolchain is a loud failure, not a silent skip" {
|
||||
run env PCIE_PROBE_HIPCC="$TEST_TMP/nope-hipcc" "$SCRIPT"
|
||||
[ "$status" -eq 4 ]
|
||||
[[ "$output" == *"hip"* ]]
|
||||
[ ! -e "$BUILD/.built" ]
|
||||
}
|
||||
|
||||
@test "a failed build is reported as a failure" {
|
||||
run env STUB_HIPCC_FAILS=1 "$SCRIPT"
|
||||
[ "$status" -eq 4 ]
|
||||
[[ "$output" == *"build"* ]]
|
||||
}
|
||||
|
||||
@test "a GPU held by another process refuses the measurement before anything is built" {
|
||||
# Order is the contract: build-and-then-ask would already have loaded the card beside a live
|
||||
# service. The .built marker proves the guard ran first.
|
||||
run env STUB_KFD_PIDS="4242" "$SCRIPT"
|
||||
[ "$status" -eq 3 ]
|
||||
[[ "$output" == *"busy"* ]]
|
||||
[ ! -e "$BUILD/.built" ]
|
||||
}
|
||||
|
||||
@test "a busy GPU percent refuses too" {
|
||||
printf '91\n' > "$SYSFS/card0/device/gpu_busy_percent"
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 3 ]
|
||||
[[ "$output" == *"card0"* ]]
|
||||
[ ! -e "$BUILD/.built" ]
|
||||
}
|
||||
|
||||
@test "--force overrides the guard for a deliberately contended measurement" {
|
||||
run env STUB_KFD_PIDS="4242" "$SCRIPT" --force
|
||||
[ "$status" -eq 0 ]
|
||||
[ -e "$BUILD/.ran" ]
|
||||
}
|
||||
|
||||
@test "an idle GPU builds and runs the probe" {
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 0 ]
|
||||
[ -e "$BUILD/.built" ]
|
||||
[ -e "$BUILD/.ran" ]
|
||||
[[ "$output" == *"PROBE RAN"* ]]
|
||||
[[ "$output" == *"31.4 GB/s"* ]] # probe output passes through untouched
|
||||
}
|
||||
|
||||
@test "--build-only compiles without touching the card" {
|
||||
run "$SCRIPT" --build-only
|
||||
[ "$status" -eq 0 ]
|
||||
[ -e "$BUILD/.built" ]
|
||||
[ ! -e "$BUILD/.ran" ]
|
||||
}
|
||||
|
||||
@test "the link topology is reported from sysfs, current against max" {
|
||||
run "$SCRIPT"
|
||||
[[ "$output" == *"card0"*"x8"*"32.0 GT/s"* ]]
|
||||
[[ "$output" == *"max x16"* ]]
|
||||
}
|
||||
|
||||
@test "each requested device is measured separately" {
|
||||
# One shared run cannot attribute bandwidth to a card; the whole point is per-card numbers.
|
||||
run "$SCRIPT" --device 0 --device 1
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"dev=0"* ]]
|
||||
[[ "$output" == *"dev=1"* ]]
|
||||
}
|
||||
|
||||
@test "no probe sources is a usage-level error, not an empty success" {
|
||||
rm -f "$SRC"/*.hip
|
||||
run "$SCRIPT"
|
||||
[ "$status" -ne 0 ]
|
||||
[[ "$output" == *"probe"* ]]
|
||||
}
|
||||
|
||||
@test "an unknown flag is a usage error" {
|
||||
run "$SCRIPT" --frobnicate
|
||||
[ "$status" -eq 2 ]
|
||||
}
|
||||
Executable
+223
@@ -0,0 +1,223 @@
|
||||
#!/usr/bin/env bats
|
||||
# tests/pkg-inventory.bats — black-box tests for scripts/pkg-inventory.sh (Refs #51)
|
||||
#
|
||||
# Contract: `install.sh` installs desired-minus-installed from packages/*.pkgs. This tool walks
|
||||
# the other direction — installed-minus-declared — because a manifest the machine has quietly
|
||||
# outgrown cannot reproduce the machine on a rebuild. Only stdout, stderr and the exit status are
|
||||
# observed: 0 no drift, 1 drift (hand-installed packages undeclared), 2 usage, 3 sync databases
|
||||
# unavailable so repo/AUR classification is impossible.
|
||||
#
|
||||
# Every pacman/flatpak call hits a stub: nothing here may query the real package database, and a
|
||||
# real pacman on PATH would make the fixture answers depend on the workstation.
|
||||
|
||||
setup() {
|
||||
TEST_TMP="$(mktemp -d)"
|
||||
SCRIPT="$BATS_TEST_DIRNAME/../scripts/pkg-inventory.sh"
|
||||
REPO="$TEST_TMP/repo"
|
||||
mkdir -p "$REPO/packages" "$TEST_TMP/bin"
|
||||
|
||||
# --- manifests: common + host(box) + aur ------------------------------------
|
||||
cat > "$REPO/packages/common.pkgs" <<'EOF'
|
||||
# common desired state
|
||||
ghostty
|
||||
zoxide
|
||||
vlc # declared, deliberately NOT installed below
|
||||
EOF
|
||||
cat > "$REPO/packages/box.pkgs" <<'EOF'
|
||||
neovim
|
||||
EOF
|
||||
cat > "$REPO/packages/aur.pkgs" <<'EOF'
|
||||
typora # AUR-only desired state
|
||||
EOF
|
||||
|
||||
# --- pacman log: install day 2026-01-01, hand installs from 2026-03 ----------
|
||||
cat > "$TEST_TMP/pacman.log" <<'EOF'
|
||||
[2026-01-01T10:00:00-0800] [PACMAN] Running 'pacman --sysroot /tmp/calamares-root-x -S base'
|
||||
[2026-01-01T10:00:01-0800] [ALPM] installed filesystem (2026.01.01-1)
|
||||
[2026-01-01T10:00:02-0800] [ALPM] installed base (3-1)
|
||||
[2026-01-01T10:00:03-0800] [ALPM] installed libfoo (1.0-1)
|
||||
[2026-03-04T09:00:00-0800] [ALPM] installed bun (1.2-1)
|
||||
[2026-03-04T09:30:00-0800] [ALPM] upgraded filesystem (2026.01.01-1 -> 2026.03.01-1)
|
||||
[2026-03-05T09:00:00-0800] [ALPM] installed nirimod-git (r7-1)
|
||||
[2026-03-06T09:00:00-0800] [ALPM] removed base (3-1)
|
||||
EOF
|
||||
|
||||
# --- installed state --------------------------------------------------------
|
||||
# ghostty/zoxide/neovim/typora declared. libfoo: explicit, arrived with the installer.
|
||||
# bun/nirimod-git: installed by hand afterwards, undeclared. python-hatchling: orphan.
|
||||
printf '%s\n' base bun filesystem ghostty libfoo neovim nirimod-git typora zoxide > "$TEST_TMP/local-all"
|
||||
printf '%s\n' bun ghostty libfoo neovim nirimod-git typora zoxide > "$TEST_TMP/local-explicit"
|
||||
printf '%s\n' python-hatchling > "$TEST_TMP/orphans"
|
||||
# nirimod-git is deliberately ABSENT here: no sync db can serve it, which is the fact that
|
||||
# routes it to aur.pkgs in the test below.
|
||||
printf '%s\n' bun ghostty neovim typora zoxide libfoo vlc python-hatchling base filesystem > "$TEST_TMP/sync-all"
|
||||
|
||||
# Four flag combinations are the script's whole interface to the package manager; the stub
|
||||
# answers those and refuses anything else, so an accidental new query fails loudly.
|
||||
cat > "$TEST_TMP/bin/pacman" <<'STUB'
|
||||
#!/usr/bin/env bash
|
||||
case "$1" in
|
||||
-Sl) [ "${STUB_SYNC_EMPTY:-0}" = 1 ] && exit 1
|
||||
sed 's/^/syncdb /' "$SYNC_ALL" ;;
|
||||
-Qqe) cat "$LOCAL_EXPLICIT" ;;
|
||||
-Qtdq) cat "$ORPHANS"; [ -s "$ORPHANS" ] ;;
|
||||
-Qq) cat "$LOCAL_ALL" ;;
|
||||
*) echo "stub pacman: unhandled args: $*" >&2; exit 64 ;;
|
||||
esac
|
||||
STUB
|
||||
cat > "$TEST_TMP/bin/flatpak" <<'STUB'
|
||||
#!/usr/bin/env bash
|
||||
[ "${STUB_NO_FLATPAK:-0}" = 1 ] && exit 127
|
||||
printf '%s\n' com.discordapp.Discord
|
||||
STUB
|
||||
chmod +x "$TEST_TMP/bin/pacman" "$TEST_TMP/bin/flatpak"
|
||||
|
||||
export PATH="$TEST_TMP/bin:$PATH"
|
||||
export PKGINV_REPO="$REPO" PKGINV_HOST=box PKGINV_LOG="$TEST_TMP/pacman.log"
|
||||
export LOCAL_ALL="$TEST_TMP/local-all" LOCAL_EXPLICIT="$TEST_TMP/local-explicit"
|
||||
export ORPHANS="$TEST_TMP/orphans" SYNC_ALL="$TEST_TMP/sync-all"
|
||||
}
|
||||
|
||||
teardown() { rm -rf "$TEST_TMP"; }
|
||||
|
||||
# section RE_START RE_END — the report from RE_START's line up to (not including) RE_END, so an
|
||||
# assertion proves a name landed in a specific section instead of merely appearing somewhere in the
|
||||
# report. Start inclusive (the count and the names sit on the label line), terminator exclusive (a
|
||||
# plain sed range would leak the next section's first line). Anchoring matters: an unanchored start
|
||||
# runs the range to end-of-output, silently turning every absence assertion into a vacuous pass —
|
||||
# exactly how four of these tests were passing before this helper was fixed.
|
||||
section() { printf '%s\n' "$output" | awk -v s="$1" -v e="$2" '$0 ~ e {on=0} $0 ~ s {on=1} on'; }
|
||||
ADDITIONS() { section "^undeclared additions" "^installer-provided"; }
|
||||
|
||||
@test "header names the host and takes the baseline from the oldest install line" {
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 1 ] # the fixture has drift by design
|
||||
[[ "$output" == *"pkg-inventory host=box"* ]]
|
||||
[[ "$output" == *"baseline=2026-01-01"* ]]
|
||||
}
|
||||
|
||||
@test "hand-installed packages are reported as additions" {
|
||||
run "$SCRIPT"
|
||||
[[ "$output" == *"undeclared additions (declare these): 2"* ]]
|
||||
s="$(ADDITIONS)"
|
||||
[[ "$s" == *bun* ]]
|
||||
[[ "$s" == *nirimod-git* ]]
|
||||
}
|
||||
|
||||
@test "a package the installer provided is not reported as an addition" {
|
||||
run "$SCRIPT"
|
||||
additions="$(ADDITIONS)"
|
||||
[[ "$additions" != *libfoo* ]]
|
||||
[[ "$(section "^installer-provided" "^declared,")" == *libfoo* ]]
|
||||
# The folded block must end in a newline. Without one, the following label is glued onto the
|
||||
# last folded line and every section after it vanishes from a sliced report.
|
||||
[[ "$(section "^installer-provided" "^declared,")" != *declared,* ]]
|
||||
}
|
||||
|
||||
@test "AUR-only packages are separated from sync-repo packages" {
|
||||
run "$SCRIPT"
|
||||
# nirimod-git is missing from the sync-DB fixture, so only a helper can install it: it must be
|
||||
# routed to aur.pkgs, not to a pacman line that would fail on a fresh box. Bash `*` crosses
|
||||
# newlines, so the absence check runs on one line rather than on the whole section.
|
||||
repo_line="$(section '^ repo ' '^ aur ')"
|
||||
[[ "$repo_line" == *"repo (sync dbs): bun"* ]]
|
||||
[[ "$repo_line" != *nirimod-git* ]]
|
||||
[[ "$(section '^ aur ' '^installer-provided')" == *"aur (needs a helper): nirimod-git"* ]]
|
||||
}
|
||||
|
||||
@test "declared-but-missing packages are listed for install.sh" {
|
||||
run "$SCRIPT"
|
||||
[[ "$output" == *"declared, not installed"* ]]
|
||||
[[ "$(section "^declared," "^orphans")" == *vlc* ]]
|
||||
}
|
||||
|
||||
@test "orphans are reported as ignored and never emitted" {
|
||||
run "$SCRIPT"
|
||||
[[ "$output" == *orphans*python-hatchling* ]]
|
||||
run "$SCRIPT" --emit
|
||||
[[ "$output" != *python-hatchling* ]]
|
||||
}
|
||||
|
||||
@test "--baseline moves packages installed on that date into the installer set" {
|
||||
# Inclusive date semantics: pushing the baseline to bun's install date makes bun installer
|
||||
# furniture, while nirimod-git (the next day) is still a hand install.
|
||||
run env PKGINV_BASELINE=2026-03-04 "$SCRIPT"
|
||||
[[ "$output" == *"baseline=2026-03-04"* ]]
|
||||
s="$(ADDITIONS)"
|
||||
[[ "$s" != *bun* ]]
|
||||
[[ "$s" == *nirimod-git* ]]
|
||||
}
|
||||
|
||||
@test "--emit prints lines that survive the manifest parser" {
|
||||
run "$SCRIPT" --emit
|
||||
[ "$status" -eq 0 ]
|
||||
# install.sh reads manifests as `sed 's/#.*//'` then the first field, so an emitted name must
|
||||
# be a bare token: a stray path or trailing note would be handed to pacman as a package name.
|
||||
bad="$(printf '%s\n' "$output" | grep -v '^#' | grep -vE '^[A-Za-z0-9@._+-]+$' || true)"
|
||||
[ -z "$bad" ]
|
||||
printf '%s\n' "$output" > "$REPO/packages/emitted.pkgs"
|
||||
names="$(sed 's/#.*//' "$REPO/packages/emitted.pkgs" | awk '{if ($1) print $1}' | sort | tr '\n' ' ')"
|
||||
[[ "$names" == *bun* ]]
|
||||
[[ "$names" == *nirimod-git* ]]
|
||||
[[ "$names" != *ghostty* ]]
|
||||
[[ "$names" != *libfoo* ]]
|
||||
[[ "$names" != *vlc* ]]
|
||||
}
|
||||
|
||||
@test "an unknown host gets the common manifest only, so host packages look undeclared" {
|
||||
# Mirrors install.sh: wrong-host runs are a no-op, not a hazard. Here that means neovim,
|
||||
# declared only in box.pkgs, counts as an addition when the host is not box.
|
||||
run env PKGINV_HOST=otherbox "$SCRIPT"
|
||||
[[ "$(ADDITIONS)" == *neovim* ]]
|
||||
}
|
||||
|
||||
@test "no drift exits 0" {
|
||||
printf '%s\n' ghostty libfoo zoxide > "$TEST_TMP/local-explicit"
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"undeclared additions (declare these): 0"* ]]
|
||||
}
|
||||
|
||||
@test "missing sync databases fail loudly rather than misclassifying everything as AUR" {
|
||||
run env STUB_SYNC_EMPTY=1 "$SCRIPT"
|
||||
[ "$status" -eq 3 ]
|
||||
[[ "$output" == *"sync"* ]]
|
||||
}
|
||||
|
||||
@test "flatpak apps are reported separately from pacman packages" {
|
||||
run "$SCRIPT"
|
||||
[[ "$output" == *"flatpak"*com.discordapp.Discord* ]]
|
||||
}
|
||||
|
||||
@test "a box without flatpak still reports the pacman inventory" {
|
||||
run env STUB_NO_FLATPAK=1 "$SCRIPT"
|
||||
[ "$status" -eq 1 ]
|
||||
[[ "$output" == *"flatpak"* ]]
|
||||
[[ "$(ADDITIONS)" == *bun* ]]
|
||||
}
|
||||
|
||||
@test "the baseline is found in a rotated log when the install day has aged out of the live one" {
|
||||
# pacman rotates its log; without reading the rotated file the whole base system would be
|
||||
# reported as hand-installed additions on any machine older than the rotation window.
|
||||
mv "$TEST_TMP/pacman.log" "$TEST_TMP/pacman.log.1"
|
||||
printf '[2026-10-01T09:00:00-0700] [ALPM] upgraded zoxide (0.9-1 -> 0.10-1)\n' > "$TEST_TMP/pacman.log"
|
||||
run env PKGINV_LOG="$TEST_TMP/pacman.log $TEST_TMP/pacman.log.1" "$SCRIPT"
|
||||
[[ "$output" == *"baseline=2026-01-01"* ]]
|
||||
[[ "$(ADDITIONS)" != *libfoo* ]]
|
||||
}
|
||||
|
||||
@test "a log larger than the pipe buffer still yields the baseline" {
|
||||
# A real install day writes thousands of install lines. `sort | head -1` under pipefail dies of
|
||||
# SIGPIPE once that output exceeds the pipe buffer, and no small fixture can show it — the first
|
||||
# run against a real log is where it surfaced. seq, not yes: yes would SIGPIPE in the fixture.
|
||||
seq 1 20000 | awk '{print "[2025-05-05T10:00:00-0700] [ALPM] installed pkgfiller (1-1)"}' > "$TEST_TMP/big.log"
|
||||
cat "$TEST_TMP/big.log" "$TEST_TMP/pacman.log" > "$TEST_TMP/combined.log"
|
||||
run env PKGINV_LOG="$TEST_TMP/combined.log" "$SCRIPT"
|
||||
[ "$status" -eq 1 ]
|
||||
[[ "$output" == *"baseline=2025-05-05"* ]]
|
||||
}
|
||||
|
||||
@test "usage is a distinct exit status from drift" {
|
||||
run "$SCRIPT" --nope
|
||||
[ "$status" -eq 2 ]
|
||||
}
|
||||
Executable
+135
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env bats
|
||||
# tests/remote-job.bats — black-box tests for scripts/remote-job.sh (Refs #53)
|
||||
#
|
||||
# Contract: a job runs detached, writes its output, then its exit code, then touches .done LAST.
|
||||
# Status is then a cheap independent read that survives the session that started the job — the
|
||||
# whole point is replacing `ssh host 'tail -f | sed'`, which reported success on jobs that had not
|
||||
# finished. Exit status answers "is everything finished"; the rc= field answers "did it work".
|
||||
#
|
||||
# Jobs really run here, against a temp job dir. No fake filesystem state is pre-seeded, because a
|
||||
# fixture that pre-creates .done proves nothing about the code that is supposed to create it.
|
||||
|
||||
setup() {
|
||||
TEST_TMP="$(mktemp -d)"
|
||||
SCRIPT="$BATS_TEST_DIRNAME/../scripts/remote-job.sh"
|
||||
export REMOTE_JOBS_DIR="$TEST_TMP/jobs"
|
||||
}
|
||||
|
||||
teardown() {
|
||||
# Jobs are detached on purpose; leave nothing behind that could outlive the test run.
|
||||
pkill -P 1 -f "$TEST_TMP" 2>/dev/null || true
|
||||
rm -rf "$TEST_TMP"
|
||||
}
|
||||
|
||||
# until_done NAME SECONDS — poll instead of sleeping a fixed amount, so the suite is neither slow
|
||||
# nor flaky on a loaded machine.
|
||||
until_done() {
|
||||
local name="$1" limit="${2:-10}" waited=0
|
||||
while [[ ! -e "$REMOTE_JOBS_DIR/$name.done" && $waited -lt $((limit * 10)) ]]; do
|
||||
sleep 0.1; waited=$((waited + 1))
|
||||
done
|
||||
[[ -e "$REMOTE_JOBS_DIR/$name.done" ]]
|
||||
}
|
||||
|
||||
@test "start returns immediately while the job is still running" {
|
||||
run timeout 10 "$SCRIPT" start slow -- sleep 2
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == "started slow" ]]
|
||||
# A 2-second job must not have blocked a start that is supposed to return at once, and while it
|
||||
# is still in flight the report must say unfinished — never DONE.
|
||||
run "$SCRIPT" status slow
|
||||
[[ "$output" == *RUN* || "$output" == *PENDING* ]]
|
||||
[ "$status" -eq 1 ]
|
||||
}
|
||||
|
||||
@test "a successful job is reported DONE with its output" {
|
||||
"$SCRIPT" start ok -- bash -c 'echo marker-4711'
|
||||
until_done ok
|
||||
run "$SCRIPT" status ok
|
||||
[[ "$output" == *"ok"*DONE*rc=0* ]]
|
||||
[ "$status" -eq 0 ]
|
||||
run "$SCRIPT" out ok
|
||||
[[ "$output" == *marker-4711* ]]
|
||||
}
|
||||
|
||||
@test "a failing job's exit code survives into the report" {
|
||||
# This is the reason the .rc file exists: a supervisor that cannot see the failure code is the
|
||||
# bug the runner was written to fix.
|
||||
"$SCRIPT" start boom -- bash -c 'echo it failed here >&2; exit 7'
|
||||
until_done boom
|
||||
run "$SCRIPT" status boom
|
||||
[[ "$output" == *"boom"*DONE*rc=7* ]]
|
||||
run "$SCRIPT" out boom
|
||||
[[ "$output" == *"it failed here"* ]]
|
||||
}
|
||||
|
||||
@test "the .done sentinel is never written before the exit code" {
|
||||
# If .done landed first, a waiter could read an empty rc and call the job successful.
|
||||
"$SCRIPT" start order -- bash -c 'echo x'
|
||||
until_done order
|
||||
[ -f "$REMOTE_JOBS_DIR/order.rc" ]
|
||||
[ -s "$REMOTE_JOBS_DIR/order.rc" ]
|
||||
run cat "$REMOTE_JOBS_DIR/order.rc"
|
||||
[[ "$output" =~ ^[0-9]+$ ]]
|
||||
}
|
||||
|
||||
@test "wait prints DONE with the exit code and a tail of the output" {
|
||||
"$SCRIPT" start fin -- bash -c 'echo one; echo two'
|
||||
run "$SCRIPT" wait fin 15
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"DONE fin rc=0"* ]]
|
||||
[[ "$output" == *two* ]]
|
||||
}
|
||||
|
||||
@test "wait gives up on its own and says so" {
|
||||
"$SCRIPT" start never -- sleep 30
|
||||
run "$SCRIPT" wait never 1
|
||||
[ "$status" -eq 1 ]
|
||||
[[ "$output" == *TIMEOUT*never* ]]
|
||||
"$SCRIPT" status never | grep -q . # the job is still there, unfinished
|
||||
}
|
||||
|
||||
@test "status distinguishes running from pending from done" {
|
||||
run "$SCRIPT" status
|
||||
[[ "$output" == *"(no jobs)"* ]]
|
||||
[ "$status" -eq 0 ]
|
||||
|
||||
"$SCRIPT" start later -- sleep 3
|
||||
run "$SCRIPT" status later
|
||||
[[ "$output" == *later* ]]
|
||||
[[ "$output" == *RUN* || "$output" == *PENDING* ]]
|
||||
[ "$status" -eq 1 ] # unfinished: a supervisor must be able to branch on this
|
||||
}
|
||||
|
||||
@test "two jobs in flight keep separate state" {
|
||||
"$SCRIPT" start a -- bash -c 'echo A; exit 0'
|
||||
"$SCRIPT" start b -- bash -c 'echo B; exit 2'
|
||||
until_done a && until_done b
|
||||
run "$SCRIPT" status
|
||||
[[ "$output" == *"a"*DONE*rc=0* ]]
|
||||
[[ "$output" == *"b"*DONE*rc=2* ]]
|
||||
run "$SCRIPT" out a
|
||||
[[ "$output" == A ]]
|
||||
run "$SCRIPT" out b
|
||||
[[ "$output" == B ]]
|
||||
}
|
||||
|
||||
@test "out honours a line limit" {
|
||||
"$SCRIPT" start many -- bash -c 'seq 1 40'
|
||||
until_done many
|
||||
run "$SCRIPT" out many 3
|
||||
[ "$(printf '%s\n' "$output" | grep -c .)" -eq 3 ]
|
||||
[[ "$output" == *38* ]] # the tail, not the head
|
||||
}
|
||||
|
||||
@test "an unknown verb is a usage error" {
|
||||
run "$SCRIPT" frobnicate
|
||||
[ "$status" -eq 2 ]
|
||||
[[ "$output" == *usage* ]]
|
||||
}
|
||||
|
||||
@test "the job directory is created on first use" {
|
||||
[ ! -d "$REMOTE_JOBS_DIR" ]
|
||||
"$SCRIPT" status
|
||||
[ -d "$REMOTE_JOBS_DIR" ]
|
||||
}
|
||||
Reference in New Issue
Block a user