governance: bench is removed from the model-swap procedure

Model changes verify by driving the live service — boot ledger,
per-request timings, one real request per harness role. No bench window
is ever scheduled for a swap. The stop-and-mask policy stays for
deliberate standalone bench windows (it is host protection, not a swap
step). Codified in AGENTS.md (policy lead + summary row), Model Swap and
Rollback (No bench step law, §8 close-out source, Related rescope).

AGENTS.md

@@ -226,6 +226,13 @@ archive.
 
 ## Benchmarking Policy (MANDATORY)
 
+**Benchmarking is NOT part of the model-swap procedure** (2026-10-15). A
+model change is verified by driving the live service — boot ledger,
+per-request `timings`/`Server-Timing`, one real request per harness role
+(`serverAppConfigs-docs/LLM/Model Swap and Rollback.md` §6/§8). No bench
+window is scheduled for a swap, ever. The rest of this policy governs
+*deliberate* bench windows (flag A/Bs, shape sweeps), which are separate
+operations on a live host:
 **STOP the production service BEFORE benchmarking** — these servers serve
 real users, and a bench beside a live service starves the GPU and corrupts
 results. Use the runtime's own harness: `llama-bench` for the llama.cpp
@@ -1157,7 +1164,7 @@ history.
 | Unstaged local changes | **Commit or discard** — check `git status` before starting work; ask if unsure. |
 | Deploying a committed change | **`scripts/deploy.sh <target>`** — pushes, pulls ff-only, plans, dispatches, health-checks. |
 | Container without a Quadlet unit | **Give it one** — boot-persistent, self-healing `.container` via git. |
-| Benchmarking | **Stop and mask the service first**, bench, unmask, restart, verify health. |
+| Benchmarking | **Not part of a model swap** — swaps verify on the live service. A deliberate bench window: **stop and mask the service first**, bench, unmask, restart, verify health. |
 | Deleting a file | **Move it to trash** — `.AGENTTRASH/` (in-repo) or `~/.AGENTTRASH/YYYYMMDD-topic`. Never `rm`. |
 | Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
 | Real secrets | **Never commit** — host `.env` files; `.env.example` holds placeholders only. |

serverAppConfigs-docs/LLM/Model Swap and Rollback.md

@@ -3,7 +3,7 @@ type: reference
 title: Model Swap and Rollback
 description: The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag checks, git-only config, live validation, systemd handoff, close-out, the three-pin revert law, and research-verification discipline.
 tags: [models, swap, gguf, huggingface, rollback, deployment, verification]
-timestamp: 2026-10-05T00:00:00-07:00
+timestamp: 2026-10-15T00:00:00-07:00
 ---
 
 # Model Swap and Rollback
@@ -12,6 +12,15 @@ Applies to every endpoint swap on the fleet (frank-zealot / frank-dragoon /
 frank-archon). Repo law: all config flows through git; the live models are
 host state under `/home/dank/models/<name>/`.
 
+**No bench step (law, 2026-10-15).** A model swap NEVER schedules a bench
+window. Performance evidence comes from the deployed service itself: the
+engine's boot ledger, per-request timings (`/completion` `timings`,
+`Server-Timing`), and one real request per harness role (§6, §8). A
+deliberate standalone bench (flag A/B, shape sweep) is OUT OF PROCEDURE —
+a separate operation that follows the AGENTS.md Benchmarking Policy
+(stop & mask). The env-file hook (§5) stays so those out-of-procedure
+arms remain possible.
+
 ## 1. Fit preflight — never deploy a swap without confirming it FITS
 
 1. Get the checkpoint's on-disk/weight size AND precision from the HF card
@@ -144,7 +153,8 @@ recoverable, drift is not.
 ## 8. Close-out (mandatory)
 
 1. Docs: active runbook updated (build recipe, profile, validation numbers
-   incl. real-workload prefill/decode); superseded runbook gets a RETIRED
+   incl. real-workload prefill/decode measured on the LIVE service — no
+   bench window); superseded runbook gets a RETIRED
    banner and stays as the rollback record; `LLM/index.md` mermaid + active
    list; AGENTS.md fleet table + runtime section + history chain; work-log
    entry in `serverAppConfigs-docs/log.md`.
@@ -277,5 +287,6 @@ predecessor "stopped" (the port race above).
 - [[GGUF Compatibility]] — which runtime can load a payload at all
 - [[Llama Cpp Serving Sizing]] — pricing the shape once it loads
 - [[RDNA4 MoE Expert Offload VRAM Budget]] — vLLM-side fit arithmetic
-- [[LLM Bench Windows and Measurement]] — measuring the new deployment
+- [[LLM Bench Windows and Measurement]] — out-of-procedure bench windows
+  only; a new deployment is measured by §6 live validation, not by a bench
 - [[dankserver-qwen38-fp8-expert-offload-tuning]] — the current endpoint

serverAppConfigs-docs/log.md

@@ -1745,3 +1745,22 @@ every restart since the cutover.
   [[frank-archon-radiance-swift15-w4dn8]], and the LLM index entry corrected
   (the w4dn8 entry said STAGED, the fp8-native one said ACTIVE).
 
+
+## 2026-10-15 — governance: bench removed from the model-change procedure
+
+- Frank's call: a model swap never carries a bench window. Codified as law:
+  - `AGENTS.md` Benchmarking Policy opens with "Benchmarking is NOT part of
+    the model-swap procedure"; swap verification is the live service (boot
+    ledger, `timings`/`Server-Timing`, one real request per harness role).
+    The stop-and-mask rule stays for DELIBERATE bench windows (flag A/Bs,
+    shape sweeps) — it is host protection (2026-09-16 hard-down precedent),
+    not a swap step. Summary-table row updated to match.
+  - [[Model Swap and Rollback]]: "No bench step (law, 2026-10-15)" up top;
+    §8 close-out numbers must be measured on the live service; the Related
+    link to [[LLM Bench Windows and Measurement]] rescoped to
+    out-of-procedure windows only.
+- Untouched on purpose: the `FRANK_*_ENV_FILE` hook (staged/out-of-procedure
+  arms), the Fleet Ops Playbook bench-window mechanics (deliberate windows
+  still need the procedure), `deployment.md` (already says bench work is a
+  SEPARATE procedure). Managed skill `llamacpp-endpoint-model-swap` verifies
+  on the live engine — it never had a bench step.