frank-default-small: use split-mode none on single-GPU dankserver
--split-mode graph requires >= 2 devices in ik_llama.cpp and errors hard
at model load on a single RTX 3090 ("It is not possible to use split mode
'graph' with less than 2 devices"), looping the service. Set SPLIT_MODE
=none (the default), which works with the --override-tensor CPU expert
offload. KB updated with the two-device requirement so we do not assume
graph degrades gracefully on one GPU.
This commit is contained in:
1 parent
b8de25d70b
commit
e2085f1f7d
5 files changed
+25
-23
No files matched your search
@@ -76,9 +76,11 @@ CACHE_TYPE_V=q8_KV
|
||||
K_CACHE_HADAMARD=1
|
||||
V_CACHE_HADAMARD=1
|
||||
|
||||
# Split mode: graph (multi-GPU graph split; on a single 3090 this is a no-op
|
||||
# for correctness but kept per operator preference).
|
||||
SPLIT_MODE=graph
|
||||
# Split mode: none. Dankserver has a single RTX 3090 and ik_llama requires
|
||||
# >= 2 devices for split-mode graph ('It is not possible to use split mode
|
||||
# graph with less than 2 devices' is a hard load error). none is correct
|
||||
# here and still works with OVERRIDE_EXPERT CPU expert offload.
|
||||
SPLIT_MODE=none
|
||||
|
||||
VERBOSITY=4
|
||||
# Store context checkpoints in RAM to hold peak GPU memory down.
|
||||
|
||||
Reference in new issue
Block a user