llama.cpp splits -c across -np slots; to give each of the 2 slots the full 262144 native context, set CTX_SIZE=524288. KV ~32 GiB; fits the 96 GiB unified pool. Update env default, serve-script fallback, and the deployment doc line.
llama.cpp splits -c across -np slots; to give each of the 2 slots the full 262144 native context, set CTX_SIZE=524288. KV ~32 GiB; fits the 96 GiB unified pool. Update env default, serve-script fallback, and the deployment doc line.