Reduce large model context to 64K

This commit is contained in:
Frank committed 2026-08-25 04:13:00 -07:00
1 parent b0eb571321
commit 3cf6353050
4 files changed
+5 -5

No files matched your search

+1 -1
View File
@@ -145,7 +145,7 @@ The active service on Dankclaw is the charlie12345/ROCmFPX llama.cpp runtime
against system ROCm 7.2.4 / amdclang 22) with a **strict Qwen MTP** draft
(`--spec-type draft-mtp --spec-mtp-strict-qwen`) running on ROCm0 with all
layers (`--spec-draft-ngl all`) — no separate draft file, because the MTP head
is embedded in the main GGUF. K/V are f16, 131072 context (128K below the
is embedded in the main GGUF. K/V are f16, 65536 context (192K below the
native 262144 maximum; 1 slot, concurrent requests queue, the server never
preempts a running generation),
`-t 16 -tb 32`, batch 512 / ubatch 256 (ubatch is swappable; sweep it on 8k