large: batch profile flipped to the runner's qualified 2048/512 (fastest measured prefill, no decode change)
This commit is contained in:
1 parent
3eb29de6ca
commit
8b3cd2ff79
4 files changed
+35
-34
No files matched your search
@@ -138,10 +138,11 @@ RDNA 3.5 native IU4 matrix instruction. The build uses the release script
|
||||
against the private TheRock ROCm 7.15 SDK kept in the retired ling3-ciru
|
||||
venv. The model files (GGUF + FFN/GDN/GDN-Output .pfs, 26.05 GiB) live in
|
||||
`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`. The production profile
|
||||
adds batch 4096 / ubatch 4096, `--no-mmap`, and reasoning on (deepseek
|
||||
format, preserve_thinking) on top of the pinned runner. Ubatch 4096 slows
|
||||
prefill ~25% versus ubatch 2048 with no decode change; see
|
||||
docs/kairic-edge.md before changing the batch profile.
|
||||
adds `--no-mmap` and reasoning on (deepseek format, preserve_thinking) on
|
||||
top of the pinned runner; batch 2048 / ubatch 512 are the runner's
|
||||
qualified sizes. Benchmarks: ubatch 512 prefill is 60-90% faster than
|
||||
ubatch 4096 with no decode change; see docs/kairic-edge.md before
|
||||
changing the batch profile.
|
||||
|
||||
**Release gap — CK `config.h`:** the pinned CK commit ships only
|
||||
`config.h.in`; the release script and guide never generate
|
||||
|
||||
Reference in new issue
Block a user