large: batch profile flipped to the runner's qualified 2048/512 (fastest measured prefill, no decode change)

This commit is contained in:
Frank committed 2026-08-22 03:30:58 -07:00
1 parent 3eb29de6ca
commit 8b3cd2ff79
4 files changed
+35 -34

No files matched your search

+5 -4
View File
@@ -138,10 +138,11 @@ RDNA 3.5 native IU4 matrix instruction. The build uses the release script
against the private TheRock ROCm 7.15 SDK kept in the retired ling3-ciru
venv. The model files (GGUF + FFN/GDN/GDN-Output .pfs, 26.05 GiB) live in
`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`. The production profile
adds batch 4096 / ubatch 4096, `--no-mmap`, and reasoning on (deepseek
format, preserve_thinking) on top of the pinned runner. Ubatch 4096 slows
prefill ~25% versus ubatch 2048 with no decode change; see
docs/kairic-edge.md before changing the batch profile.
adds `--no-mmap` and reasoning on (deepseek format, preserve_thinking) on
top of the pinned runner; batch 2048 / ubatch 512 are the runner's
qualified sizes. Benchmarks: ubatch 512 prefill is 60-90% faster than
ubatch 4096 with no decode change; see docs/kairic-edge.md before
changing the batch profile.
**Release gap — CK `config.h`:** the pinned CK commit ships only
`config.h.in`; the release script and guide never generate