command-center/agent/managed-skills/llamacpp-trial-boot-census/SKILL.md

name, description

name	description
llamacpp-trial-boot-census	Measure whether a candidate GGUF fits a live llama.cpp endpoint's card: verify serve-script env names first, run a dtype/slot arm matrix with trap-guarded production restore, read the engine fit ledger.

llama.cpp trial-boot census (does candidate X fit this card?)

Paper arithmetic on KV rates is a hypothesis. The engine's own fit ledger plus a real boot are the proof. This protocol ran on frank-zealot (RTX 3090, Qwen3.5-9B Q6_K → prism-ml Ternary-Bonsai-2-27B PQ2_0, 2026-10-17).

0. Gate: env variable names, not invented ones

A trial env written from memory silently falls back to the serve script's defaults. KV_CACHE_TYPE=q8_0 when the script reads CACHE_TYPE_K/CACHE_TYPE_V boots f16 and every later number is wrong (this wasted a whole outage; two arms printed identical allocations, which is the tell).

grep -nE ': "\$\{[A-Z_]+[:=-]' scripts/serve-<svc>.sh   # the real names + defaults
grep -nE '\-\-[a-z-]+ "\$' scripts/serve-<svc>.sh       # what actually reaches argv

Never invent: KV dtype, bind addr, n-gpu-layers, log-file (units redirect it).

1. Runner skeleton

One script on the host, set -u, trap restore EXIT:

restore(){ pkill -x llama-server; for i in $(seq 1 90); do [[ "$(used)" -lt 1500 ]] && break; sleep 2; done
  sudo systemctl start <svc>; for i in $(seq 1 150); do [[ "$(curl -s -o /dev/null -w %{http_code} $BASE/health)" == 200 ]] && break; sleep 2; done; }
trap restore EXIT
sudo systemctl stop <svc>            # benching law: stop first, mask fails on relinked units
setsid nohup env FRANK_<SVC>_ENV_FILE=/tmp/armA.env bash scripts/serve-<svc>.sh >$LOG 2>&1 &

Launch with setsid nohup … &, wait with ONE async watcher (grep -q DONE), never poll. Per-arm drop-in envs via sed on a base env — one variable per arm.

2. Arm matrix (cheap, each ≈ boot + probes)

Arm	Purpose
target shape × target KV dtype	the verdict + behavior probes
target shape × f16 KV	proves the wall (or funds the shape)
1 × native-window	answers "can we serve the full context"
target shape × q4_0	the roomy fallback rung

3. Read the ledger, not nvidia-smi alone

grep -a -A8 "memory breakdown" $LOG     # total = free + (model + context + compute)
grep -aE "KV buffer size|model buffer size|compute buffer size|estimated worst-case memory usage of mmproj|projected to use|cannot meet free memory target|cudaMalloc failed" $LOG

Laws learned the hard way:

• projected to use N MiB vs M MiB of free + cannot meet free memory target of R ⇒ requirement is projected + R ≤ free. With --n-gpu-layers pinned by the user the fitter aborts instead of shrinking — the boot then dies at the KV cudaMalloc.
• context bills more than the KV pool: hybrid models add a fixed recurrent state per slot (~150 MiB/slot for qwen35's 48 DeltaNet layers). Pool math alone under-prices.
• The projector's estimated worst-case memory usage of mmproj is X MiB is NOT inside the fit projection. It lands after the fit check, so real headroom = paper headroom − X. This is the "boots fine, then dies / dark" class.
• q8_0 K+V bills exactly 2 × (layers_kv × heads × dim / 32 blocks × 34 B)/token — check the reported KV MiB against that to prove the dtype actually took.

4. Probe traps

• First request after boot is warm-up garbage: it paid 2.6 t/s prefill / 37.7 t/s decode where the warm run gave 1,258 t/s. Never bench the first request; discard it or warm with a throwaway.
• Reasoning-on chat probes need ≥512 max_tokens: at 64 the answer was empty (finish=length mid-reasoning_content) while vision actually worked. Judge content only at a budget the thinking can fit.
• Raw /completion (no chat template) returns content only — thinking is invisible; timings.prompt_n / predicted_per_second are the real keys, not prompt_tokens.
• Vision probe: solid-color PNG ≥256 px, expect an exact color word.
• Concurrency honesty: fire two identical requests in parallel; per-slot rates from each response's timings.

5. Restore proof

health=200 and systemctl is-active — an inactive unit behind a 200 means a stray trial process holds the port. Kill by pid; never pkill -f llama-server.