dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

Set ubatch sweep context and batch to 16K

Browse Source
This commit is contained in:
Frank
2026-08-25 17:32:12 -07:00
parent babc823f11
commit 4cb67d7b33
3 changed files with 23 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/LLM/ubatch-benchmark-large.md
+3 -2
View File
@@ -13,7 +13,7 @@ timestamp: 2026-09-01
# frank-default-large ubatch sweep
This benchmark sweeps the active `frank-default-large` model on Dankclaw using
scratch ports and an 8K-context workload. The production service is stopped
scratch ports and a 16K-context workload. The production service is stopped
before the sweep and restarted by the benchmark's exit handler.
## Sweep
@@ -29,7 +29,8 @@ Run on Dankclaw from the repository checkout:
STOPSVC=1 ./scripts/bench-ubatch-rocmi4.sh
```
The script uses the active Q6 AGENT model, standalone MTP draft, tracked Sharp
The script uses the active Q6 AGENT model, standalone MTP draft, a fixed
16K batch, and the tracked Sharp
template, and `--reasoning-format deepseek`. It reports server timing lines and
per-run generation throughput. Run only when the production service can be
interrupted; the GPU is shared with live workloads.