---
name: llm-fleet-model-swap
description: "Swap the model/runtime behind frank-default-large/small on Dankclaw/Dankserver via the git flow, with pre-deploy live validation and the port-holder health-check trap check."
---

# LLM fleet model swap (frank-default-large / frank-default-small)

Use when switching model, quant, or llama.cpp/FreeToken runtime for a `frank-default-*` service in `~/Projects/serverAppConfigs`.

## 0. Ground truth first
- Read the HF model card (runtime pin, hard constraints, recommended args) + repo files `scripts/frank-default-<size>.env`, `scripts/serve-frank-<size>.sh`, `scripts/frank-default-<size>.service`, and the runbook under `serverAppConfigs-docs/LLM/`.
- Check the host before downloading anything: `ls /home/dank/models/`, `df -h /home`, `du -sh` — bench-era or prior-deployment files often already exist.

## 1. Artifact integrity
- List repo files: `curl -s https://huggingface.co/api/models/<repo>/tree/main?recursive=true` → `.lfs.oid` holds the sha256.
- On-host file: compare size AND `sha256sum` against the LFS oid before trusting any existing copy (hash runs ~few min for >100 GiB; run it in background).
- New download on Dankserver: `/home/dank/freetoken/.venv/bin/hf download <repo> --local-dir ...`.

## 2. Runtime (if the card pins a fork/build)
- Clone + build with the card's exact cmake line, **plus `-DLLAMA_BUILD_WEBUI=OFF`** — WebUI asset provisioning (npm / HF llama-ui bucket) aborts llama.cpp link at 98% on these hosts; API server unaffected.
- Verify every flag our serve script passes exists in the new binary before touching the service:
  `llama-server --help | grep -q -- "<flag>"` for each; list MISSes.

## 3. Config via git only
- Edit `scripts/frank-default-<size>.env` (LLAMA_BIN, MODEL_DIR, MODEL_FILE, MMPROJ_PATH) + serve script header/defaults; `bash -n` the script; `./tests/run.sh` must pass; commit; push; `git pull --ff-only` on the target.
- Do NOT edit configs on the remote. Renaming a model dir (`mv`) is host state, allowed.

## 4. Live validation (no scratch ports — GPU/RAM can't host two big models)
- Ensure the old unit is truly gone: `systemctl is-active` may hang in `stop-sigterm` because llama.cpp never preempts an in-flight generation; poll until the PID exits before launching.
- Stop service, run the real serve script manually (`setsid nohup bash scripts/serve-frank-<size>.sh > /tmp/x.log 2>&1 &` — plain `ssh 'nohup ... &'` dies with its ssh when foregrounded; keep ssh under timeout or it gets killed mid-stop).
- Verify: health 200; a real `/v1/chat/completions` (FreeToken REQUIRES explicit `"model"` field; llama.cpp servers don't care); reasoning flows (`reasoning_content` non-empty; content may be empty with finish=length if the thinking eats the budget).

## 5. systemd handoff + THE TRAP
- `./scripts/deploy.sh <host> --service <svc>` — but a leftover manual server holding :7770 makes the health check pass while the UNIT crash-loops on bind.
- Always confirm ownership: `systemctl show <svc> -p MainPID` equals the single `pgrep -x llama-server` (or the ft serve PID); only then trust "healthy after Ns".
- If a real long-context task is queued/running, do the handoff anyway (client aborts are recoverable; drift is not) — or defer cosmetic-only redeploys.

## 6. Close-out (mandatory)
- Docs: active runbook updated (build recipe, profile, validation numbers incl. real-workload prefill/decode), superseded runbook gets RETIRED banner kept as rollback record, `LLM/index.md` mermaid + active list, AGENTS.md fleet table + Runtime section + history chain, work log entry in `serverAppConfigs-docs/log.md`.
- `~/.pi/agent/models.json`: contextWindow must equal the REAL per-slot context — llama.cpp logs `n_ctx_slot` per prompt; parallel N clamps `n_ctx_train` (e.g. 262144/2 = 131072/slot). Overselling causes silent client-side overflow.
- Rollback = one env commit (paths + LLAMA_BIN); keep prior binary + payload on disk.