Files
command-center/agent/managed-skills/self-severing-model-swap-cutover/SKILL.md
T

3.4 KiB
Raw Blame History

name, description
name description
self-severing-model-swap-cutover Swap the model on frank-archon (the endpoint the operator session runs on): full pre-cutover repo unit, detached deploy that survives session death, post-return boot-ledger harvest, e2e attribution.

Swapping the model you are running on (frank-archon)

The harness agent shares the endpoint being swapped: the deploy restarts frank-archon, the session dies mid-tool-call, the harness catches the return. Plan for the severing; never keep critical state only in-session.

Pre-cutover (all must be committed/pushed BEFORE the deploy)

  1. Read scripts/frank-archon.env live state and record the shape; the swap changes ONLY MODEL_FILE (fleet model-swap law) unless asked otherwise.
  2. Fetch script per payload (hfget + exact bytes + Hub LFS sha256 + --verify-only, no token on the command line); run detached via systemd-run --unit=<name>-fetch on dankserver.
  3. Re-key in one commit: frank-archon.env (+ trio headers), tests/frank_archon_unit.bats payload id/sha256 (keep the outgoing payload as the documented rollback line), homepage card (shared/homepage/services.json), runbook (new era page + index flips + old page retired), AGENTS.md topology row/section/runbook index.
  4. Full ./tests/run.sh green, push origin main, git pull --ff-only on dankserver. Verify the engine pin's flag set matches the payload card (radiance pins have drifted on flag renames).
  5. Disk: check df -h /home/dank before launching a >100 GB download; the outgoing payload stays on disk as rollback — deleting it requires Frank's one-time rm authorization.

Cutover, detached from the session

Launch from the workstation as a transient unit so ssh/session teardown cannot kill it:

setsid systemd-run --unit=archon-<tag>-cutover --remain-after-exit \
  bash -lc 'cd <repo> && ./scripts/deploy.sh dankserver --service frank-archon \
    > ~/archon-<tag>-cutover.log 2>&1; echo $? > ~/archon-<tag>.rc'

The session dies when frank-archon restarts (~1–2 min load). On return: read the .rc sentinel and log tail; then the journal ledger:

journalctl -u frank-archon --no-pager --since "<deploy>" | grep -iE "kv cache|slab|sampler|MTP|prefix"

Post-boot harvest (fill the runbook placeholders)

  • KV: kv_attn KiB/4-token block, pool token count vs seqs × max-model-len.
  • Expert slab slots per rank (compare to the outgoing payload).
  • Sampler rows = seqs × (1 + spec) (wht6 exactness bound 25).
  • Prefix-cache RECOVERED line (survives swaps when arch+tokenizer unchanged).
  • Vision probe: solid-color PNG ≥256 px, read reasoning_content (small max_tokens → thinking eats the budget, content empty with finish_reason: stop = healthy, not broken).
  • Decode ladder needs a stop-and-mask window — separate bench task.

e2e gate redness on dankclaw surfaces

deploy.sh dankserver runs the Playwright gate against the LIVE fleet, so dankclaw faults fail your deploy. Attribute before believing: crawl 502 / SSO fail / Marreta tile → run the gluetun-dependent-boot-race-recovery census first (reboot-triggered, unrelated to the swap), cascade-restart, rerun tests/e2e/run.sh directly — a frank-archon health gate that passed means the swap itself is fine.

Rollback

Revert the swap commit, redeploy --service frank-archon — the outgoing .rad remains on disk (until deleted by authorized rm).