command-center/agent/managed-skills/llamacpp-endpoint-model-swap/SKILL.md

name, description

name	description
llamacpp-endpoint-model-swap	Swap the model on a dank-fleet llama.cpp endpoint: fetch, runtime-door census, env edit, deploy through git, rewrite the bats seam, sweep every live-path straggler, verify VRAM with the engine's own numbers.

llama.cpp endpoint model swap (dank fleet)

Use when replacing the model/shape on a llama.cpp endpoint (frank-zealot on the RTX 3090). The engine's boot numbers are the verdict — do NOT commit to a feasibility verdict from head math; boot it and read nvidia-smi.

Sequence

1. Runtime door BEFORE anything else. Histogram the GGUF's tensor types and metadata (one python range-read of the KV section): a custom type (e.g. PQ2_0 = GGML type 142 + prism.hadamard.*) means the vendor/fork build is FORCED, not chosen — stock llama.cpp refuses it at load. Record the build identity (path, build number, commit, sha256) in the env comment; a --help census on that binary is what settles flag questions (this fleet's builds have NO fp8 KV; q8_0 is the "FP8 class"). NEVER grep -c <archname> on the packed binary to check arch support — it returns 0 for every arch.
2. Geometry first, from the loaded model's own log (--log-verbosity 4, or the HF config.json under text_config for multimodal): layers, n_head_kv (an ARRAY — per-layer), head_dim, sliding_window_pattern, n_swa, attention_k_eq_v. KV scales only with full-attention layers; sliding layers cost a fixed ring; recurrent/linear layers (Gated DeltaNet) bill a fixed state per SLOT (~150 MiB measured), invisible to pool arithmetic. Single-value HF fields lie about per-layer heterogeneity.
3. Fit the KV rate law to the engine, not to arithmetic. One boot at -lv 4 prints llama_kv_cache / sched_reserve MiB lines. Master 5fc4f3c law: bytes/elem = (K + V) x2 when the file declares attention_k_eq_v. Full cache = n_full x n_head_kv x head_dim x rate x ctx. If your formula disagrees with the printed MiB, your law is wrong.
4. Price the KV dtype by BOOTING the arms, at the target window. f16 may simply OOM (cudaMalloc failed after the engine's own fit line names the MiB it wanted); q8_0 may be the only pair that funds 2 x 200,704; q4_0 is the fallback rung. Record every arm's used/free. The mmproj lands AFTER the engine's fit projection (projected fit < settled VRAM by ~1.5x the projector file size) — quote both numbers.
5. Fetch with -f + byte-count + sha256 check into /home/dank/models/<name>/; --verify-only mode for re-census.
6. Edit the repo trio (scripts/serve-<svc>.sh, scripts/<svc>.env, ~/.omp/agent/models.yml). Use the serve script's OWN variable names (CACHE_TYPE_K/CACHE_TYPE_V, BIND_ADDR, NGL) — an invented name is silently swallowed by : "${VAR:=default}" and the arm boots the DEFAULT (cost a whole trial window: a KV_CACHE_TYPE scratch env booted f16 and OOMed while reading as a model fault). Sweep the script's DEFAULTS to the new payload too: a defaulted run must land on the shipped shape, never the retired one. Spec flags move as a SET (SPEC_TYPE=none strips --spec-type AND --spec-draft-*); MTP is a property of the FILE (no nextn tensors = no drafter, ever). models.yml contextWindow = per-slot window (CTX_SIZE/PARALLEL); recompute maxTokens for the compaction clamp min(maxTokens, ctx - estimate - 4096). Check the payload's own template: if it reads reasoning_effort, the reasoningEffortMap fold becomes LOAD-BEARING (an unmapped level is a 500 naming Jinja, not the model) and quote the file's general.sampling.* trio in extraBody.
7. Rewrite the bats seam in the same commit (tests/<svc>_unit.bats): payload trio paths, slot geometry, spec state, batch pair, verbosity, mmproj opt-out, reasoning flags — argv-asserted against the stub engine; where the contract is a repo value (batch pair, verbosity, KV dtype) assert argv == env value so a retune moves one file and a wrong-name env edit cannot pass silently.
8. Trial window protocol (when the shape is unproven on the card): stop the unit (trap-guarded systemctl start restore in the scratch script), run the arm census + behavior probes as a DETACHED script writing to /home/dank/logs/<name>.out, then before touching production CENSUS the live state: systemctl is-active, ss -ltnp | grep <port>, pgrep -x llama-server, nvidia-smi. A scratch server orphaned by a client-side timeout holds the whole card and will OOM the real boot — kill by PID (never pkill -f llama-server). A trap restore can race your manual kill; read the script's own RESTORE log line before assuming who did what.
9. Deploy through git (no direct remote edits): commit atomically on the worktree branch, git fetch && git rebase origin/main (log.md conflicts at EOF are two waves appending — keep both), push branch (--force-with-lease after a rebase), then in the MAIN checkout git pull --ff-only + git merge --ff-only <branch> + git push origin main, then scripts/deploy.sh <host> --service <svc> (--service is REPEATABLE — name the homepage too when cards changed, or the target ships the model without its docs). A deploy.sh with no --service is a fleet-wide drain: never.
10. Verify the engine, not your math: systemctl is-active/is-enabled + the new Description, the journal fit/KV/buffer ledger (projected to use N, KV buffer size, n_ctx_seq, n_slots), /v1/models served alias (radiance-style echo does not apply to llama.cpp, but the alias is still the honest check), nvidia-smi settled, then live probes through the REAL request shape the harness sends (exact-instruction chat + reasoning_content, tool_calls JSON args, solid-color image_url >=256 px expecting the exact word, timings.predicted_per_second). Never report the FIRST request after a boot (warmup artifact). Push, and record numbers in the env comment + runbook.
11. Cutover every consumer, same branch: AGENTS.md (topology row, ### <svc> section, Model Config note, runbook index ACTIVE + RETIRED rows); the new docs page MUST be linked from the TOP-LEVEL serverAppConfigs-docs/index.md (the docs_completeness seam fails otherwise, not from LLM/index.md); shared/homepage/services.json + docs.html (model link, spec line, rate, and the WARNING — a censored base breaks the "common uncensored text" equality in homepage_config.bats, so re-key the test's premise too); serverAppConfigs-docs/Dankclaw/OpenWebUI.md + the live model row meta.description (live DB state: snapshot webui.db.bak-<date><letter>, UPDATE via host python3 over ssh; do NOT restart open-webui for a cosmetic string — it logs every family session out).

Gotchas

• A "we can rebuild the fork ourselves for provenance" claim needs the source tree AT the vendor build's commit — check git -C <src> log --oneline -1 first. A fork checkout at branch HEAD (prism at 0082ac2) is NOT the vendor build (2459f68b5), so the rebuild produces a different binary, not the verified one. Also: if the vendor binary already demonstrates the requirement (KV dtype buffer math in the ledger + live probes), a rebuild buys no behavior change — offer it explicitly as provenance hardening, never do it silently.
• A systemd restart reporting inactive in the first seconds is usually model-load time; check journalctl -u <svc> for model done/server is listening before declaring failure.
• Rollback = the previous payload stays on disk; one MODEL_FILE + shape edit back. Record a refusal-freedom fallback the same way (an official censored base ↔ a CRACK re-export with byte-identical header geometry is a one-line swap; prove the identity with the tensor histogram, not the card).
• A thinking model can consume the whole token budget in reasoning_content (empty content, finish_reason: length) — probe with a budget that clears thinking; a short-budget vision probe reads as "the model is blind".
• /completion at -lv 4 prints failed to parse reasoning content ... on raw prompts closing with an end-of-turn marker: cosmetic (chat path parses correctly). At ~26k+ a bare /completion can return 0 tokens with no error — raw context has no role frame; chat is unaffected.
• Bats vacuity trap: serve_argv REWRITES $STAGE/test.env from the shipped env, so a test that mutates the file then calls serve_argv asserts on regenerated content — a tautology. Tests that mutate must exec via a run_stage helper running the file as-is.
• env NAME=value cmd needs valid identifiers; a flag-like token is only legal as an env ARGUMENT, not as a shell assignment prefix.
• Gitea git push on a rebased branch is rejected non-fast-forward: --force-with-lease (your own branch), never on main.