chore(skills): promote runbook skills to managed-skills; drop superseded SKILL.md

- 31 new managed skills (anubis/arr/authentik/hermes/freetoken/podman/llm-swap
  runbooks etc.) now tracked.
- Removed superseded skills: omp_harness_config, peon_ping_integration
  (replaced by user-authored peon-ping-* managed skills),
  rocmfp4-strix-benchmark-orchestration.
- New memory-root skills: niri_compositor_ops, omp_git_workflow,
  omp_model_config, quadlet-media-migration, homelab-service-triage,
  hermes-dankclaw-ops, frank-default-large-ops, caddy-auth-dns-playbook.
This commit is contained in:
Frank committed 2026-09-09 16:20:28 -07:00
1 parent 7be39d60dd
commit 9bfa0be85d
48 files changed
+1361 -42

No files matched your search

@@ -0,0 +1,46 @@
---
name: gui-hang-remote-diagnosis
description: "Diagnose a frozen GUI/VT-switch failure over SSH by reading kernel hung-task evidence, decoding the lock-holder stack, and deciding recoverability"
---
# GUI Hang Remote Diagnosis
Use when the desktop is frozen, VT switches do nothing, but SSH still works.
## Sequence
1. **Find the smoking gun** — kernel hung-task reporter already logged it:
```bash
journalctl -b -p err --since "-2 hours" --no-pager | tail -80
```
Look for `INFO: task X blocked for more than N seconds` and `blocked on a semaphore likely last held by task Y:PID` (modern kernels name the lock holder outright).
2. **Get full stacks** for victim and holder:
```bash
journalctl -b -k --since "<hang minute -5>" --no-pager | tail -120
cat /proc/<holder_pid>/status | head -5 # State: R (spinning) vs D (blocked)
cat /proc/<victim_pid>/status | head -5
```
- Victim `D` + holder `R` = livelock/deadlock with lock held.
- `voluntary_ctxt_switches` in the millions on an `R` kthread = spinning, not progressing.
3. **Prove whether hardware or software** — GPU/CPU still alive? No Xid errors?
```bash
nvidia-smi --query-gpu=driver_version,pstate,temperature.gpu,clocks_event_reasons.active --format=csv
journalctl -b -k --no-pager | grep -i xid
```
If those answer instantly with no Xid, it's a driver lock issue, not hardware.
4. **Find the trigger window**: block start = (first hung message timestamp − reported seconds). Then inspect user-session events:
```bash
journalctl -b --since "<start-2m>" --until "<start+1m>" --no-pager | grep -v audit
```
5. **Enumerate collateral**: any other `D` tasks (`ps -eo pid,stat,comm | awk '$2 ~ /D/'`), USB workers, etc.
## Recoverability verdict
- Compositor in `D` state → unkillable; spinning kernel thread holding the lock → cannot be poked.
**Only exit is reboot**: `sync && systemctl reboot --force --force` (double-force skips stop-jobs that wait forever on D-state PIDs).
- If holder is `D` (sleeping on something that can complete, e.g. storage), recovery may be possible by clearing that dependency first.
## After diagnosing
- Map stack symbols to driver internals to name the exact subsystem (e.g. `DifrPrefetchEventDeferredWork` → NVKMS DIFR prefetch).
- Check upstream: search the exact symbol names + driver version; propose targeted off-switches (`NVreg_RegistryDwords`, module params from `/proc/driver/nvidia/params`) rather than blanket driver swaps.