dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-default-small: ncpu-moe=38 (2 experts on GPU)

Browse Source
This commit is contained in:
Frank
2026-08-20 18:02:47 -07:00
parent 99c7ec531f
commit e09733ee62
2 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-default-small.env
+2 -2
View File
@@ -31,8 +31,8 @@ NGRAM_MOD_N_MIN=48
NGRAM_MOD_N_MAX=64
NGRAM_MOD_N_MATCH=24
# Expert offload: 32 of 40 layers keep MoE weights on CPU; 8 on GPU.
N_CPU_MOE=32
# Expert offload: 38 of 40 layers keep MoE weights on CPU; 2 on GPU.
N_CPU_MOE=38
N_GPU_LAYERS=all
# KV cache on GPU, both K and V at f16.