Set ubatch sweep to requested 32k matrix
This commit is contained in:
1 parent
c02d67513d
commit
5766ff7dab
2 files changed
+11
-11
No files matched your search
@@ -22,10 +22,9 @@ for auditability only. Their results are invalid and must not be used.
|
|||||||
|
|
||||||
## Sweep
|
## Sweep
|
||||||
|
|
||||||
The default points are `256, 512, 768, 1024, 1280, 1536, 2048, 3072, 4096,
|
The requested points are `256, 512, 768, 1024, 1280, 1536, 1792, 2048,
|
||||||
5120, 6144`. This implements the requested 256-step range through 1536 and
|
3072, 4096` at 32K context. Each point runs three 128-token repetitions by
|
||||||
1024-step range from 2048 through 6144. Each point runs three 256-token decode
|
default.
|
||||||
requests by default.
|
|
||||||
|
|
||||||
Run on Dankclaw from the repository checkout:
|
Run on Dankclaw from the repository checkout:
|
||||||
|
|
||||||
@@ -35,10 +34,10 @@ sudo systemctl stop frank-default-large
|
|||||||
sudo systemctl start frank-default-large
|
sudo systemctl start frank-default-large
|
||||||
```
|
```
|
||||||
|
|
||||||
The script uses the active Q6 AGENT model, a fixed 16K prompt workload, and
|
The script uses the active Q6 AGENT model, a fixed 32K prompt workload, and
|
||||||
`llama-bench` JSONL output. The initial ubatch-only sweep is:
|
`llama-bench` JSONL output. The ubatch-only sweep is:
|
||||||
```text
|
```text
|
||||||
256, 512, 768, 1024, 1280, 1536, 1792, 2048, 3072, 4096, 5120, 6144
|
256, 512, 768, 1024, 1280, 1536, 1792, 2048, 3072, 4096
|
||||||
```
|
```
|
||||||
|
|
||||||
The GPU is shared with live workloads; do not run with the production service
|
The GPU is shared with live workloads; do not run with the production service
|
||||||
|
|||||||
@@ -2,8 +2,9 @@
|
|||||||
# Benchmark ubatch with llama-bench for frank-default-large.
|
# Benchmark ubatch with llama-bench for frank-default-large.
|
||||||
#
|
#
|
||||||
# This is intentionally a ubatch-only benchmark. It uses one llama-bench
|
# This is intentionally a ubatch-only benchmark. It uses one llama-bench
|
||||||
# process, a fixed 16K prompt workload, and a fixed batch size large enough to
|
# process, a fixed 32K prompt workload, and a fixed scheduler size large
|
||||||
# hold that workload. No server process, HTTP requests, or log scraping are used.
|
# enough to hold that workload. No server process, HTTP requests, or log
|
||||||
|
# scraping are used.
|
||||||
#
|
#
|
||||||
# Run on Dankclaw after stopping frank-default-large:
|
# Run on Dankclaw after stopping frank-default-large:
|
||||||
# sudo systemctl stop frank-default-large
|
# sudo systemctl stop frank-default-large
|
||||||
@@ -13,8 +14,8 @@ set -euo pipefail
|
|||||||
|
|
||||||
: "${LLAMA_BENCH:=/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-bench}"
|
: "${LLAMA_BENCH:=/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-bench}"
|
||||||
: "${MODEL:=/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf}"
|
: "${MODEL:=/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf}"
|
||||||
: "${UBATCH_VALUES:=256,512,768,1024,1280,1536,1792,2048,3072,4096,5120,6144}"
|
: "${UBATCH_VALUES:=256,512,768,1024,1280,1536,1792,2048,3072,4096}"
|
||||||
: "${CONTEXT:=16384}"
|
: "${CONTEXT:=32768}"
|
||||||
: "${GEN_TOKENS:=128}"
|
: "${GEN_TOKENS:=128}"
|
||||||
: "${REPETITIONS:=3}"
|
: "${REPETITIONS:=3}"
|
||||||
: "${DEVICE:=ROCm0}"
|
: "${DEVICE:=ROCm0}"
|
||||||
|
|||||||
Reference in new issue
Block a user