fix(qwen38): one identity preflight owns every fact a calibration window needs #91

Merged
dank merged 1 commits from fix/quant-pipeline-hardening into main 2026-10-30 16:26:41 -07:00
Owner

Every gate this window used was a proxy, and each one lied at least once in a single day:

  • /health answered 200 from the dying production process while the rig container had bound nothing — the window died at second zero with health ok waited=0s.
  • podman inspect named a rig container that was merely created.
  • the free-space gate asked a directory that only looked related to the one the bytes go to.
  • the identity check written to catch all of the above compared different number spaces: podman top <ctr> pid prints container-internal pids (1, 7), never host pids, and a rootless cgroup carries the container ID rather than the service name, so it could only ever fail.

scripts/qwen38-preflight.sh is now the single owner of those facts — cwd, the digest a tag actually resolves to, free space on the tap's own filesystem, stale grams, every /dev/kfd holder and the :7770 listener resolved to host pids via podman top <ctr> hpid, plus the corpus and the reference recording. It prints PASS/FAIL per fact and exits 2. qwen38-calib-window.sh calls it before a card is touched and its duplicated checks are gone.

7 new tests pin the failures: wrong digest, a listener the service does not own, a foreign GPU holder, stale grams (and the documented --allow-existing-tap escape), and a launcher that lost its cwd.


cwd: /home/blank/.omp/wt/wt-20261027-142335-6d1fc1f · session: 01a112cd-f9a8-724e-8ec6-285213a85a68

Every gate this window used was a proxy, and each one lied at least once in a single day: - `/health` answered **200 from the dying production process** while the rig container had bound nothing — the window died at second zero with `health ok waited=0s`. - `podman inspect` named a rig container that was merely *created*. - the free-space gate asked a directory that only looked related to the one the bytes go to. - the identity check written to catch all of the above compared **different number spaces**: `podman top <ctr> pid` prints container-internal pids (`1`, `7`), never host pids, and a rootless cgroup carries the container ID rather than the service name, so it could only ever fail. `scripts/qwen38-preflight.sh` is now the single owner of those facts — cwd, the digest a tag actually resolves to, free space on the tap's own filesystem, stale grams, every `/dev/kfd` holder and the `:7770` listener resolved to **host** pids via `podman top <ctr> hpid`, plus the corpus and the reference recording. It prints `PASS`/`FAIL` per fact and exits 2. `qwen38-calib-window.sh` calls it before a card is touched and its duplicated checks are gone. 7 new tests pin the failures: wrong digest, a listener the service does not own, a foreign GPU holder, stale grams (and the documented `--allow-existing-tap` escape), and a launcher that lost its cwd. --- cwd: /home/blank/.omp/wt/wt-20261027-142335-6d1fc1f · session: 01a112cd-f9a8-724e-8ec6-285213a85a68
dank added 1 commit 2026-10-30 16:26:41 -07:00
Each gate the window used was a proxy, and each lied at least once:
- /health answered 200 from the dying production process (window dead at second zero)
- podman inspect named a rig container that had bound nothing
- the space check asked a directory that merely looked related
- the identity check compared podman's CONTAINER-internal pids (1, 7) to an ss host pid

scripts/qwen38-preflight.sh now checks cwd, the digest a tag actually resolves to,
free space on the tap's own filesystem, stale grams, every /dev/kfd holder's host pid
(podman top <ctr> hpid) and the :7770 listener's host pid against the service's, the
corpus and the reference recording; it prints PASS/FAIL per fact and exits 2. The
window calls it before a card is touched, and its duplicated checks are gone.
dank merged commit 8073d0aa9d into main 2026-10-30 16:26:41 -07:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dank/serverAppConfigs#91