command-center/agent/managed-skills/llamacpp-candidate-arm-census/SKILL.md

name, description

name	description
llamacpp-candidate-arm-census	Measure a candidate GGUF/KV-shape on a live dank-fleet llama.cpp endpoint (frank-zealot pattern) in ONE service-stop window: pre-flight flag check, per-arm env overrides, boot ledger + nvidia-smi census, behavior probes, trap-guarded restore.

Measure candidate shapes on a production llama.cpp endpoint without cutting anything over. One outage window, many arms, guaranteed restore.

0. Pre-flight (no GPU needed — do all of it before stopping anything)

• ssh <host> <candidate-bin> --list-devices — the candidate must see the card (backend .so releases resolve via $ORIGIN, so run from the extracted asset dir).
• Flag-compat gate: loop the exact long options the serve script emits and grep the candidate's --help. A single unknown flag = a dead boot in the middle of the window. Remote shells may be zsh: $flag[ ,<] is parsed as an array subscript ("bad math expression") — pipe the check through ssh <host> 'bash -s' <<'CHK' … CHK and use grep -qE "(^|[[:space:]])$f([[:space:],<]|$)".
• Pause any concurrent compile (pkill -f "[r]un-<build>" — the bracket stops pkill -f matching your own command line; pkill -x ninja). Ninja is incremental: the build resumes where it stopped, and CPU contention otherwise skews every rate.

1. Runner shape

Write one script to /tmp on the target via ssh <host> 'cat >/tmp/….sh' <<'EOF'. Never edit the repo checkout on the target; drive the shipped serve script with a per-arm env file and FRANK_<SVC>_ENV_FILE (the serve scripts already honor it).

SRV=""
used(){ nvidia-smi --query-gpu=memory.used --format=csv,noheader,nounits; }
restore(){ [[ -n "$SRV" ]] && kill "$SRV" 2>/dev/null; pkill -x llama-server
  for i in $(seq 1 90); do [[ "$(used)" -lt 1500 ]] && break; sleep 2; done
  sudo systemctl start <svc>
  for i in $(seq 1 150); do [[ "$(curl -s -o /dev/null -w %{http_code} http://127.0.0.1:<port>/health)" == 200 ]] && break; sleep 2; done
  echo "RESTORE health=$(curl -s -o /dev/null -w %{http_code} …/health) used=$(used) state=$(systemctl is-active <svc>)"; }
trap restore EXIT

• trap restore EXIT is mandatory, not polite: it also fires on the failure paths, so a script bug cannot leave the endpoint down.
• Preserve the production log first (cp -f /home/dank/logs/<svc>.log …prod-<ts>.log) — the serve script truncates it.
• Health-poll cap ~120×2 s. 300 polls turns an expected-allocation-failure arm (the "wall" arms) into 10 wasted minutes.
• Arm = sed the base env (CTX_SIZE / PARALLEL / KV_CACHE_TYPE / LLAMA_BIN / LOG_FILE), boot with setsid nohup env FRANK_<SVC>_ENV_FILE=… bash <serve-script>, then capture: nvidia-smi used+free, ledger grep (type |model size|n_ctx|flash|KV buffer|buffer size|slots|error|exceed), /slots (per-slot n_ctx is the harness contract), /v1/models.
• Point LLAMA_BIN at a binary that exists. Take 1 of this procedure died in 8 s because the trial env named a still-compiling source build; the trap saved the endpoint.

2. Probes that actually discriminate

• chat: content vs reasoning_content (channel differs per runtime) + finish_reason.
• tools: real tools schema, assert tool_calls[0].function.arguments parses as JSON — empty/null arguments is a known fork bug.
• vision: 512×512 solid color PNG built with zlib+struct (no PIL), sent as an OpenAI image_url content part; expect the exact color word. The legacy top-level images field hallucinates a plausible answer.
• rate: /completion (not /v1/chat/completions) — it returns timings (prefill/decode). ≥128 tokens.
• long-context fill: ~190k tokens through /completion proves the per-slot window AND tests the "quantized KV near 200K" stability reports.

Skip llama-bench for shape work: it has no -c/-np, and its -mmp is --mmap (deprecated), not mmproj. Booting the server per arm measures the real shape and prints the ledger, which is the referee.

3. Report

Per arm: ledger KV-buffer MiB, used/free at settle, FITS / tight / NO FIT vs the card's used+free ceiling from settled boots (not the spec sheet). Then restore, and state the downtime.