Ship the PCIe probe instruments with a guard that refuses to contend with a service

The RDNA4 offload campaign left three HIP probes and a driver sitting loose in
$HOME, where the next agent cannot find them and a stray `rm -rf` ends them. They
answer a recurring question on the R9700 box — is decode limited by the link or by
the engine's read path — so they move into scripts/probes/ with a driver, tests,
and a page.

Two things changed in the code, not just the address:

- The driver measures only after a guard that runs BEFORE anything is built:
  gpu_busy_percent over PCIE_PROBE_MAX_BUSY, or any process holding /dev/kfd, is a
  refusal (exit 3) rather than a measurement. A benchmark taken beside a live
  endpoint is both disruptive and worthless, and silently unloading a production
  service is not a tool's decision — --force is the human saying otherwise. The
  refusal-before-build ordering is pinned by a test, because the sysfs and kfd
  paths are overridable and a refusal is otherwise untestable off the card box.
- hip_smoke.hip and pcie_probe.hip called hipGetDeviceProperties, which SEGFAULTS
  on this ROCm build even though hipGetDeviceCount works; only pcie_bw.hip avoided
  it, and that is the one that produced the campaign's numbers. Device identity now
  comes from the sysfs topology block the driver prints, so all three probes are
  runnable on the build we ship. Verified here: refusal on a real busy APU (exit 3),
  missing toolchain (exit 4), missing sources (exit 2). Compile-and-run still needs
  the box with hipcc.

Refs #53
This commit is contained in:
Frank committed 2026-10-20 19:24:58 -07:00
1 parent 35ceae6d7b
commit 56fc899763
8 files changed
+573

No files matched your search

+23
View File
@@ -16,6 +16,29 @@ lesson, a few lines. Full incident narratives are recoverable from git
(`git log --follow omp-docs/log.md`); knowledge that outlives an entry lives
on its concept page, linked here. Never re-expand an entry in place.
## 2026-10-20 — two instruments rescued from `$HOME` script sediment
`$HOME` had accumulated 123 loose scripts at its root (archived to
`.AGENTTRASH/home-scripts-2026-10-20`, ~1.8 GiB of stray audio consolidated into
`~/audiosamples`). Two were durable instruments and now live in the repo with
black-box tests and pages: [tools/remote-job](tools/remote-job.md) and
[tools/pcie-probe](tools/pcie-probe.md). Refs #53. Laws learned:
- **A readiness pattern in a log is not evidence of completion.** `ssh host 'cmd |
tail -f log | sed -n /ready/q'` fails three independent ways — the broker aborts
the long connection, the pattern can match a header printed before the work
starts, and `sed -n /x/q` exits the moment it matches, tearing down the job it was
watching. The only honest signal is a sentinel the job writes last, read over a
short-lived connection.
- **Never benchmark beside a live service, and never let a tool decide to stop one.**
The guard refuses before anything is built; unloading a production endpoint is a
human's call. A number taken as noise beside serving traffic is both disruptive and
wrong.
- **An instrument that segfaults is not an instrument.** Two of the three shipped
`.hip` probes called `hipGetDeviceProperties`, which crashes on this ROCm build
while `hipGetDeviceCount` works fine; device identity now comes from sysfs, and the
trap is recorded in the sources so it is not reintroduced.
## 2026-10-20 — the secret gate was reading the wrong files, and one credential had already escaped
Found while deciding what else belongs in `dotfiles/`. (1) The gate whitelisted