dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Set ubatch sweep to requested 32k matrix

Browse Source
This commit is contained in:
Frank
2026-08-25 19:12:42 -07:00
parent c02d67513d
commit 5766ff7dab
2 changed files with 11 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/LLM/ubatch-benchmark-large.md
+6 -7
View File
@@ -22,10 +22,9 @@ for auditability only. Their results are invalid and must not be used.
## Sweep
The default points are `256, 512, 768, 1024, 1280, 1536, 2048, 3072, 4096,
5120, 6144`. This implements the requested 256-step range through 1536 and
1024-step range from 2048 through 6144. Each point runs three 256-token decode
requests by default.
The requested points are `256, 512, 768, 1024, 1280, 1536, 1792, 2048,
3072, 4096` at 32K context. Each point runs three 128-token repetitions by
default.
Run on Dankclaw from the repository checkout:
@@ -35,10 +34,10 @@ sudo systemctl stop frank-default-large
sudo systemctl start frank-default-large
```
The script uses the active Q6 AGENT model, a fixed 16K prompt workload, and
`llama-bench` JSONL output. The initial ubatch-only sweep is:
The script uses the active Q6 AGENT model, a fixed 32K prompt workload, and
`llama-bench` JSONL output. The ubatch-only sweep is:
```text
256, 512, 768, 1024, 1280, 1536, 1792, 2048, 3072, 4096, 5120, 6144
256, 512, 768, 1024, 1280, 1536, 1792, 2048, 3072, 4096
```
The GPU is shared with live workloads; do not run with the production service