refactor(skills): consolidate managed skill library 94 -> 25
The managed-skill library was regenerated: 94 narrow per-incident skills
replaced by 25 broader ones (radiance-*, llamacpp-*, homepage-*, fleet/
endpoint ops). Git renders this as mass delete + mass add because whole
directories changed name; old content stays in history at 12c5e88.
Surface check: the 25 on disk are exactly the skills the harness advertises
this session.
This commit is contained in:
1 parent
94c71a016f
commit
a34322aeb0
119 files changed
+740
-5816
No files matched your search
@@ -0,0 +1,46 @@
|
||||
---
|
||||
name: detached-fleet-job-launch
|
||||
description: Launch a long-running remote job on Dank fleet hosts so it survives ssh/logind teardown
|
||||
---
|
||||
|
||||
# Detached long-running jobs on Dank fleet hosts
|
||||
|
||||
## Law
|
||||
`ssh host 'setsid nohup CMD & '` is NOT detached. systemd-logind deactivates
|
||||
`session-N.scope` when the ssh command returns and SIGKILLs the whole cgroup:
|
||||
no journal entry, no OOM trace, python block-buffers stdout so the log stays
|
||||
0 bytes. setsid/nohup do not escape the cgroup.
|
||||
|
||||
## Canonical launch
|
||||
```
|
||||
ssh host 'export XDG_RUNTIME_DIR=/run/user/$(id -u); \
|
||||
systemd-run --user --collect --unit=NAME \
|
||||
-p SendSIGKILL=no \
|
||||
-p StandardOutput=append:/home/dank/logs/NAME.log \
|
||||
-p StandardError=append:/home/dank/logs/NAME.log \
|
||||
sh -c "CMD -u; echo \$? > /home/dank/logs/NAME.rc"'
|
||||
```
|
||||
- transient user unit = outside every session scope; survives ssh teardown
|
||||
(requires `Linger=yes`, true for `dank` on both hosts)
|
||||
- `echo $? > …rc` inside the unit = completion sentinel the unit itself writes
|
||||
- `-u` / unbuffered output so the log is live
|
||||
- verify launch in the same ssh call: `systemctl --user is-active NAME.service`
|
||||
|
||||
## Watcher (one async job, never poll)
|
||||
```
|
||||
ssh host 'while [ ! -f /home/dank/logs/NAME.rc ]; do sleep 60; done; \
|
||||
echo RC=$(cat /home/dank/logs/NAME.rc); tail -25 /home/dank/logs/NAME.log'
|
||||
```
|
||||
Run it async with timeout 0; the result is injected on completion.
|
||||
|
||||
## Forensics when a detached job died
|
||||
1. `ps -p PID` — gone.
|
||||
2. Log 0 bytes → buffered stdout lost at SIGKILL.
|
||||
3. `journalctl --since … | grep -E "session-[0-9]+.*(Deactivated|Removed session)"` —
|
||||
timestamps matching the death window confirm the scope kill (vs OOM, which
|
||||
leaves kernel lines).
|
||||
|
||||
## Notes
|
||||
- zsh remote shells: `echo ===X===` fails (`=cmd` expansion); quote sentinels.
|
||||
- The HF upload of 2026-10-08 (113.7 GiB, hf_xet) died this way ~2 min in;
|
||||
relaunch as above succeeded.
|
||||
Reference in new issue
Block a user