Clarify llama-bench harness scope

docs/.obsidian/app.json

@@ -5,4 +5,4 @@
   "attachmentFolderPath": "_attachments",
   "useMarkdownLinks": false,
   "showUnsupportedFiles": true
-}
+}

docs/LLM/index.md

@@ -50,7 +50,7 @@ MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
 - [kingjones Qwen3.8 Uncensored ROCmFP4 STRIX MTP](https://huggingface.co/kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF) — active frank-default-large model on Dankclaw; Q6 AGENT main model with standalone MTP
 - [[kingjones-strix-mtp]] — the active Q6 AGENT runbook: serving profile, standalone MTP, reasoning + preserve_thinking, 114 GiB TTM/GTT
-- [[ubatch-benchmark-large]] — active-model ubatch sweep plan and results
+- [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results
 - [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
 - [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
   on Dankclaw (retired)

docs/LLM/ubatch-benchmark-large.md

@@ -12,13 +12,13 @@ timestamp: 2026-09-01
 
 # frank-default-large ubatch sweep
 
-This benchmark sweeps the active `frank-default-large` model on Dankclaw using
-scratch ports and a 32K-context workload. The production service is stopped
-before the sweep and restarted by the benchmark's exit handler.
+This benchmark uses the active `frank-default-large` model on Dankclaw with
+`llama-bench`. It does not use `llama-server`, HTTP requests, or server-log
+scraping. Stop the production service before running it and start the service
+again afterward.
 
-The 2026-09-01 run used `ctx=32768` and `batch=32768`. It completed through
-`ubatch=3072`; `ubatch=4096` failed during initialization with a ROCm
-allocation error, so 4096, 5120, and 6144 were not run.
+The old custom harnesses are retained in `.AGENTTRASH/bench-scripts-20260901/`
+for auditability only. Their results are invalid and must not be used.
 
 ## Sweep
 
@@ -30,32 +30,25 @@ requests by default.
 Run on Dankclaw from the repository checkout:
 
 ```bash
-STOPSVC=1 ./scripts/bench-ubatch-rocmi4.sh
+sudo systemctl stop frank-default-large
+./scripts/bench-ubatch.sh
+sudo systemctl start frank-default-large
 ```
 
-The script uses the active Q6 AGENT model, standalone MTP draft, a fixed
-32K batch, and the tracked Sharp
-template, and `--reasoning-format deepseek`. It reports server timing lines and
-per-run generation throughput. Run only when the production service can be
-interrupted; the GPU is shared with live workloads.
+The script uses the active Q6 AGENT model, a fixed 16K prompt workload, and
+`llama-bench` JSONL output. The initial ubatch-only sweep is:
+```text
+256, 512, 768, 1024, 1280, 1536, 1792, 2048, 3072, 4096, 5120, 6144
+```
+
+The GPU is shared with live workloads; do not run with the production service
+up.
 
 ## Results
 
-Record results below after the sweep, including failures or OOM points.
-
-| ubatch | cold prefill | generation runs 1/2/3 | notes |
-| ---: | ---: | ---: | --- |
-| 256 | 214.73 tok/s | 17.68 / 17.15 / 17.12 tok/s | completed |
-| 512 | 213.47 tok/s | 17.91 / 17.29 / 17.72 tok/s | completed |
-| 768 | 215.60 tok/s | 17.96 / 18.17 / 17.98 tok/s | completed |
-| 1024 | 222.20 tok/s | 18.07 / 18.10 / 18.12 tok/s | completed |
-| 1280 | 219.13 tok/s | 17.47 / 17.69 / 17.49 tok/s | completed |
-| 1536 | 220.52 tok/s | 16.85 / 16.86 / 16.86 tok/s | completed |
-| 2048 | 60.97 tok/s | 6.24 / 4.94 / 11.19 tok/s | severe slowdown |
-| 3072 | 28.50 tok/s | 1.45 / 5.03 / 4.44 tok/s | severe slowdown |
-| 4096 | — | — | initialization OOM: rs-cache allocation |
-| 5120 | — | — | not reached |
-| 6144 | — | — | not reached |
+No valid results are recorded here. All results from the retired custom
+harnesses are explicitly discarded. Record the raw `llama-bench` JSONL output
+from the new harness here after a clean run.
 
 ## See also
 

docs/index.md

@@ -119,7 +119,7 @@ The LLM inference fleet on the two hosts. Full index:
 ### Benchmarks and Sizing
 
 - [[frank-default-large-benchmark]] — backend/spec benchmark behind the old head
-- [[ubatch-benchmark-large]] — active-model ubatch sweep plan and results
+- [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results
 - [[frank-default-small-benchmark]] — the small-model benchmark suite
 - [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
 - [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU