live: heartbeat sync 2026-10-18T16:47:59-07:00 (Refs #11)

agent/managed-skills/bench-arm-provenance-recovery/SKILL.md

@@ -0,0 +1,42 @@
+---
+name: bench-arm-provenance-recovery
+description: "Before calling a leftover LLM bench drop-in a rogue/hijack, reconstruct whether Frank ordered and measured that arm: omp archive prompt timestamps + journald print_timing for the exact pid, then land the verdict in the repo."
+---
+
+Use when a live service boots a payload you did not configure (stale `systemd` drop-in / `*_ENV_FILE` arm env) and the docs — or your own compaction-inherited context — call it "unfinished", "armed but never measured", or "hijack". Do NOT report that until proven: the arm may be an ordered, measured, approved configuration whose record lived only in `/tmp` + journald.
+
+## Why this exists
+2026-10-18 `frank-zealot`: a `frank-zealot.service.d/ptq1.conf` drop-in made the cutover deploy boot a different model at a different shape. I reported it as an unfinished experiment. It was actually the 27B PTQ1_0 arm Frank ordered at 02:36, that served production 03:14:39→03:38:57, and whose numbers existed only in tool output that context compaction later dropped.
+
+## Step 1 — recover the ORDER (the harness archive, not memory)
+`eval` with the `archive` prelude:
+
+```python
+ps = await archive.prompts()          # searchable prompt history, survives compaction
+# filter for the payload name / repo URL keywords, print p['at'] + p['text'][:90]
+r = await archive.session("<sessionId>")   # full session record
+recaps = await archive.recaps()             # recap stamps — check they postdate the window
+```
+
+`archive.prompts()` keeps **user prompts with ISO timestamps** even when the middle of the transcript was dropped. Timestamp every instruction: ordered → measured → superseded. Convert to local (`UTC−7` here) before quoting. A recap older than the instruction proves the recap layer never covered the window.
+
+## Step 2 — recover the MEASUREMENT (journal, per pid)
+The engine's own per-request timings are the record; they outlive `/tmp` env files.
+
+```bash
+ssh dank@<host> 'journalctl -u <svc> --no-pager' | grep -F 'serve-<svc>.sh[<PID>]' \
+  | grep -E 'prompt eval time|eval time' | awk '... median per task ...'
+```
+
+- Pin the **pid** (`sh[NNNN]`): `--since` windows alone pull the previous process's numbers too.
+- llama.cpp format: `prompt eval time = 1234.56 ms / 100 tokens (…, 81.03 tokens per second)`; decode rows are `eval time = … ms / N tokens (…)`. Multiple spaces between `task`/`eval` break naive regexes — match `task\s+(\d+)\s+prompt eval time` exactly, or parse both labels separately.
+- Filter pathological rows: generations running concurrently on other slots, or ending in a 4-byte EOS, read ~4× low. Report clean single-stream probes (≥200 generated tokens) and label the rest.
+- Cross-check the boot banner for the payload identity (`type ptq1_0: 402 tensors`, KV ledger, slot/ctx lines) — that proves *which* shape that pid served.
+
+## Step 3 — adjudicate, then land it in the repo
+Write the verdict in the same session you derive it: runbook deploy-ledger section, `log.md` dated section, the arm script's header comment, and the AGENTS.md law. `/tmp` and journald are not records — that absence is what makes the next agent misread an approved arm as a rogue one.
+
+## Traps
+- `mask` fails on a relinked unit; never `systemctl disable`; retire drop-ins to `~/.AGENTTRASH/<date>-<topic>/` + `daemon-reload` (recoverable, not deleted).
+- `/health` 200, `/v1/models` echoing the alias, and a green e2e gate all fail to read the payload — check `systemctl show <svc> -p DropInPaths` (must be empty), then `tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot ledger.
+- Verification sequence that is honest: correction commit must quote commands you ran; state `[INFERENCE]` for anything the journal does not show.