llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot

The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.

Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.
This commit is contained in:
Frank
2026-10-10 17:03:14 -07:00
parent 5ced474eb9
commit 55f87fcdce
195 changed files with 2085 additions and 2064 deletions
@@ -12,7 +12,7 @@ timestamp: 2026-10-05T00:00:00-07:00
Use when `:7770`/`:7771` gives **connection refused** (not a timeout) and the
service was not crashed. Applies to `frank-medium` (dankserver, root unit),
`frank-secondary` (dankclaw, root unit), `frank-large` (dankclaw, rootless
`frank-zealot` (dankclaw, root unit), `frank-dragoon` (dankclaw, rootless
Quadlet).
**1. Classify the stop before touching anything**
@@ -95,22 +95,22 @@ the local Gitea on Dankclaw — push allowed).
unquoted word with `=` in an ssh one-liner (`echo ===` aborts the chain). A
freshly restarted endpoint under live agent-queue backlog may return empty
bodies/timeouts for minutes while `/health` is 200 — re-probe with a long
timeout. First request after a halogen (`frank-large`) restart can go silent
45–60 s and self-recover; do not restart over it. Rootless `frank-large`:
timeout. First request after a halogen (`frank-dragoon`) restart can go silent
45–60 s and self-recover; do not restart over it. Rootless `frank-dragoon`:
read logs **without** sudo (per-user storage). If the restore never happened,
the sudo journal names who and when: `journalctl -b | grep sudo`.
## Scratch vLLM boot on a fleet GPU host — stop-first law
Any scratch/experimental vLLM container on a host that also runs a production
LLM service (`frank-primary`, `frank-large`, `frank-secondary`) MUST stop the
LLM service (`frank-archon`, `frank-dragoon`, `frank-zealot`) MUST stop the
production unit first, and the window script must verify it. Two independent
failure modes if skipped:
1. **Expert-offload planner dies on host RAM.**
`vllm/model_executor/offloader/expert_plan.py` reads `MemAvailable` at
plan time. A production rank worker holds tens of GiB of unevictable
pinned expert rows (frank-primary: ~76 GiB × 2 ranks), so a beside-prod
pinned expert rows (frank-archon: ~76 GiB × 2 ranks), so a beside-prod
boot sees ~86 GiB instead of ~239 and dies at plan time:
`ExpertPlanError: host holds 5113 of 12096 swappable experts per rank`.
Looks like a planner bug; is pure RAM contention. Signature: container
@@ -124,15 +124,15 @@ failure modes if skipped:
Window skeleton (dankserver):
```bash
sudo systemctl stop frank-primary.service
sudo systemctl mask --runtime frank-primary.service # relinked unit: plain mask refuses; runtime mask self-clears at reboot
sudo systemctl stop frank-archon.service
sudo systemctl mask --runtime frank-archon.service # relinked unit: plain mask refuses; runtime mask self-clears at reboot
sleep 3
podman ps --format "{{.Names}}" | grep -q frank-primary-glm && { echo "prod still running - abort"; exit 1; }
podman ps --format "{{.Names}}" | grep -q frank-archon-glm && { echo "prod still running - abort"; exit 1; }
# ... launch scratch container, watch logs ...
podman rm -f <scratch> # stuck vLLM ignores SIGTERM
sudo systemctl unmask --runtime frank-primary.service
sudo systemctl reset-failed frank-primary.service
sudo systemctl start frank-primary.service
sudo systemctl unmask --runtime frank-archon.service
sudo systemctl reset-failed frank-archon.service
sudo systemctl start frank-archon.service
# restore oracle is a CONJUNCTION: health 200 AND is-active active AND is-enabled enabled AND no /run/systemd/system mask symlink
```