frank-default-large: threads-batch=16 confirmed optimal + decode benchmark results

Decode sweep (TB=8,16,32, 8K+ prompt, 256 gen tokens):
  TB=8:  3.3 tok/s (76.6s)
  TB=16: 3.4 tok/s (74.8s)
  TB=32: 3.4 tok/s (75.0s)

Decode is flat; TB=16 wins on prefill (5.1% faster) with zero decode tradeoff.

Also:
- Fix :: syntax error in serve-frank-large.sh
- Document systemctl disable destroys unit file; use mask/unmask instead

AGENTS.md

@@ -227,14 +227,23 @@ ALWAYS bench using llama-bench for llama.cpp or vllm bench for vllm runtimes.
 DO NOT ROLL YOUR OWN BENCH UNLESS YOU CANNOT TEST THE DESIRED PARAMETER WITH THE OFFICIAL BENCHES. ASK THE USER BEFORE DEVIATING FROM THIS DIRECTIVE.
 
 1. Stop the service before any benchmark:
-   `sudo systemctl stop frank-default-small` (Dankserver) or
-   `sudo systemctl stop frank-default-large` (Dankclaw).
+   `sudo systemctl stop frank-default-large` (Dankclaw) or
+   `sudo systemctl stop frank-default-small` (Dankserver).
+   Then mask it to prevent the self-healing unit from restarting:
+   `sudo systemctl mask frank-default-large` (Dankclaw).
+   **CRITICAL WARNING:** Do NOT use `systemctl disable` on these service units.
+   On these hosts, `systemctl disable` doesn't just remove symlinks — it **deletes
+   the unit file itself** from `/etc/systemd/system/`. You'd have to manually
+   copy the unit file back from this repo, run `daemon-reload`, then enable and
+   start. Mask (`systemctl mask`) creates a symlink to `/dev/null` instead, which
+   is safe and reversible.
 2. Run the bench (`llama-bench` for llama.cpp / vllm bench for vLLM).
    Scratch ports are not used for tests or benches on the LLM hosts — a
    test process cannot share the single GPU with the running service anyway
    (the model + KV fill the card), so changes are validated directly against
    the service itself.
-3. Restart the service when the bench completes: `sudo systemctl start <svc>`.
+3. Restart the service when the bench completes: unmask, then start:
+   `sudo systemctl unmask frank-default-large && sudo systemctl start frank-default-large` (Dankclaw).
 4. Verify health before reporting:
    `curl http://192.168.0.235:7770/health` or
    `curl http://192.168.0.143:7770/health`.

docs/LLM/kingjones-strix-rocmfp4.md

@@ -109,6 +109,41 @@ sized to 85% of context window:
 
 Short prompts (~3,300 tokens): 381 tok/s prompt processing, 22.7 tok/s
 generation (median of 3 runs).
+## Benchmark: threads-batch tuning
+
+Sweeped `--threads-batch` (16, 32, 48, 64, 80, 96, 128) with a 64K-token
+prompt via HTTP API (`max_tokens=1`). Fresh server start per value.
+
+| threads-batch | prefill tok/s | wall time |
+|---------------|--------------|-----------|
+| **16** | **373.5** | 62.9s |
+| 32 | 354.4 | 66.3s |
+| 48 | 344.1 | 68.3s |
+| 64 | 344.7 | 68.2s |
+| 80 | 344.4 | 68.2s |
+| 96 | 357.1 | 65.8s |
+| 128 | 321.1 | 73.2s |
+
+**Optimal: `--threads-batch 16`** (373.5 tok/s). Lower thread batching wins
+for prefill on this model -- fewer threads compete for GPU resources during
+the prompt processing phase. The default of 32 is 5.1% slower. Above 64,
+performance plateaus then declines (128 is 14% worse than optimal).
+
+## Benchmark: decode throughput
+
+Sweeped `--threads-batch` (8, 16, 32) with a ~23.5K-token code prompt and
+256 generated tokens via HTTP API. Fresh server start per value.
+
+| threads-batch | gen tokens | wall time | gen tok/s |
+|---------------|-----------|-----------|-----------|
+| 16 | 256 | 74.8s | 3.4 |
+| 32 | 256 | 75.0s | 3.4 |
+| 8 | 256 | 76.6s | 3.3 |
+
+**Decode throughput is essentially flat across all tested values.** TB=16 and
+TB=32 are identical at 3.4 tok/s; TB=8 is marginally slower at 3.3 tok/s.
+Combined with the 5.1% prefill advantage at lower thread counts, `--threads-batch
+16` is the optimal setting for both phases.
 
 ## Serving Profile
 
@@ -126,7 +161,7 @@ llama-server \
   --ctx-size 524288 --parallel 2 \
   --batch-size 4096 --ubatch-size 4096 \
   --cache-type-k f16 --cache-type-v f16 \
-  --threads 16 --threads-batch 32 \
+  --threads 16 --threads-batch 16 \
   --ctx-checkpoints 16 --cache-ram 8192 \
   --cache-prompt --cache-idle-slots --metrics \
   --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.0 \

scripts/frank-default-large.env

@@ -59,7 +59,7 @@ PARALLEL=2
 BATCH_SIZE=4096
 UBATCH_SIZE=4096
 NUM_THREADS=16
-THREAD_BATCH=32
+THREAD_BATCH=16
 CACHE_TYPE_K=f16
 CACHE_TYPE_V=f16
 CTX_CHECKPOINTS=16

scripts/serve-frank-large.sh

@@ -49,7 +49,7 @@ fi
 : "${BATCH_SIZE:=4096}"
 : "${UBATCH_SIZE:=4096}"
 : "${NUM_THREADS:=16}"
-: "${THREAD_BATCH:=32}"
+: "${THREAD_BATCH:=16}"
 : "${CACHE_TYPE_K:=f16}"
 : "${CACHE_TYPE_V:=f16}"
 : "${CTX_CHECKPOINTS:=16}"