chore(skills): promote runbook skills to managed-skills; drop superseded SKILL.md

- 31 new managed skills (anubis/arr/authentik/hermes/freetoken/podman/llm-swap
  runbooks etc.) now tracked.
- Removed superseded skills: omp_harness_config, peon_ping_integration
  (replaced by user-authored peon-ping-* managed skills),
  rocmfp4-strix-benchmark-orchestration.
- New memory-root skills: niri_compositor_ops, omp_git_workflow,
  omp_model_config, quadlet-media-migration, homelab-service-triage,
  hermes-dankclaw-ops, frank-default-large-ops, caddy-auth-dns-playbook.
This commit is contained in:
Frank committed 2026-09-09 16:20:28 -07:00
1 parent 7be39d60dd
commit 9bfa0be85d
48 files changed
+1361 -42

No files matched your search

@@ -0,0 +1,50 @@
---
name: llm-model-swap-fit-preflight
description: "Preflight an LLM model swap for hardware fit before deploying; catch the silent swap-thrash outage mode (large RAM OOM, self-restarting + boot-enabled units) and recover."
---
# LLM Model-Swap Fit Preflight
Never deploy a model swap without confirming the checkpoint FITS, and know the
failure mode when it doesn't: it is a silent, self-sustaining thrash, not a crash.
## Preflight (before deploy)
1. Get the checkpoint's on-disk/weight size AND precision from the HF card
(`Precision: BF16` = 2 bytes/param; FP8/Q8 ≈ 1 byte; Q4 ≈ 0.5 byte).
35B params: BF16 ≈ 66–70 GB, FP8/Q8 ≈ 33–35 GB, Q4 ≈ 17–20 GB.
2. Compare against the HOST'S USABLE RAM (not just VRAM) for runtimes that
stream weights from RAM (FreeToken host-RAM experts, llama.cpp CPU offload,
PP/worker splits).
3. Budget RAM for weights + KV pool + OS + page cache. If weights alone exceed
RAM, ABORT — pool tuning shrinks KV, it does not shrink weights.
## The self-sustaining thrash (why 'it'll just OOM and recover' is wrong)
- Large swap (tens of GB) → kernel PAGES instead of OOM-killing → sshd/user
space starved indefinitely; no recovery window opens by itself.
- systemd `Restart=on-failure|always` → crash-reload-rethrash loop.
- Unit `enabled` + `WantedBy=multi-user.target` → a reboot re-thrashes on boot
until the bad config is masked/reverted. A plain power-cycle does NOT fix it.
## Recovery (needs a shell; may need console / SysRq-to-GRUB)
```
sudo systemctl mask --now <svc> # stop + break the reboot trap FIRST
cd <checkout> && git fetch origin && git reset --hard origin/main # pick up revert
grep MODEL_DIR <env> # confirm fixed target
sudo systemctl unmask <svc> && sudo systemctl restart <svc>
curl -s <health> # then verify with a REAL completion
```
## Deploy-script caveat
`deploy.sh`/health-gate 'healthy after Ns' = systemd ACTIVE, not model LOADED.
Always confirm load with a real completion (curl /v1/chat/completions), not
status alone.
## GPU quant note
RTX 3090 (Ampere) has no FP8/FP4 tensor cores: FP8 weights run via upcast but
still must fit in RAM/VRAM. Choose quant by SIZE fit, then accept the compute
path the card supports.