diff --git a/serverAppConfigs-docs/LLM/frank-primary-llamacpp-qwen38-flash-next-q8.md b/serverAppConfigs-docs/LLM/frank-primary-llamacpp-qwen38-flash-next-q8.md index ffc112e..f4d50aa 100644 --- a/serverAppConfigs-docs/LLM/frank-primary-llamacpp-qwen38-flash-next-q8.md +++ b/serverAppConfigs-docs/LLM/frank-primary-llamacpp-qwen38-flash-next-q8.md @@ -142,6 +142,14 @@ answers; after any aborted deploy, confirm the responder is the unit's own MainPID), and llama-server's SIGTERM cleanup hung after spec-decode service (logged `cleaning up before exit` and never exited — `kill -9` was needed; `systemctl stop` relies on TimeoutStopSec)._ +**KV law (from the -lv 4 load log, cross-checked by the GGUF header):** +29,696 B/token total ctx-scaled VRAM — 24,576 B main KV (only 12 of 48 +layers carry attention, `full_attention_interval=4`; 2 KV heads x 256, +f16) + 3,072 B indexer + 2,048 B draft KV. Everything else (weights +~6.7 GiB, recurrent state 0.33 GiB, compute buffers ~4.1 GiB) is +ctx-independent. At 262,144: 7.25 GiB scaled, 18.75 GiB total, 9.37 +GiB/card. At 100,000: 2.77 GiB scaled, 14.3 GiB total, 7.1 GiB/card. +`--parallel N` splits the same pool — slots do not multiply KV. ## Rollback