2.3 KiB
type, title, description, tags, timestamp
| type | title | description | tags | timestamp | ||||
|---|---|---|---|---|---|---|---|---|
| benchmark plan | frank-default-large ubatch sweep | Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. |
|
2026-09-01 |
frank-default-large ubatch sweep
This benchmark sweeps the active frank-default-large model on Dankclaw using
scratch ports and a 32K-context workload. The production service is stopped
before the sweep and restarted by the benchmark's exit handler.
The 2026-09-01 run used ctx=32768 and batch=32768. It completed through
ubatch=3072; ubatch=4096 failed during initialization with a ROCm
allocation error, so 4096, 5120, and 6144 were not run.
Sweep
The default points are 256, 512, 768, 1024, 1280, 1536, 2048, 3072, 4096, 5120, 6144. This implements the requested 256-step range through 1536 and
1024-step range from 2048 through 6144. Each point runs three 256-token decode
requests by default.
Run on Dankclaw from the repository checkout:
STOPSVC=1 ./scripts/bench-ubatch-rocmi4.sh
The script uses the active Q6 AGENT model, standalone MTP draft, a fixed
32K batch, and the tracked Sharp
template, and --reasoning-format deepseek. It reports server timing lines and
per-run generation throughput. Run only when the production service can be
interrupted; the GPU is shared with live workloads.
Results
Record results below after the sweep, including failures or OOM points.
| ubatch | cold prefill | generation runs 1/2/3 | notes |
|---|---|---|---|
| 256 | 214.73 tok/s | 17.68 / 17.15 / 17.12 tok/s | completed |
| 512 | 213.47 tok/s | 17.91 / 17.29 / 17.72 tok/s | completed |
| 768 | 215.60 tok/s | 17.96 / 18.17 / 17.98 tok/s | completed |
| 1024 | 222.20 tok/s | 18.07 / 18.10 / 18.12 tok/s | completed |
| 1280 | 219.13 tok/s | 17.47 / 17.69 / 17.49 tok/s | completed |
| 1536 | 220.52 tok/s | 16.85 / 16.86 / 16.86 tok/s | completed |
| 2048 | 60.97 tok/s | 6.24 / 4.94 / 11.19 tok/s | severe slowdown |
| 3072 | 28.50 tok/s | 1.45 / 5.03 / 4.44 tok/s | severe slowdown |
| 4096 | — | — | initialization OOM: rs-cache allocation |
| 5120 | — | — | not reached |
| 6144 | — | — | not reached |