dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-default-small: ncpu-moe=35(5GPU), ubatch=2048, ctx-checkpoints=16

Browse Source
This commit is contained in:
Frank committed 2026-08-20 16:57:03 -07:00
1 parent f8deaa798a
commit 90a8b1c562
2 files changed
+8 -8
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

scripts/serve-frank-small.sh
+3 -3
View File
@@ -36,11 +36,11 @@ fi
: "${CACHE_TYPE_K:=f16}"
: "${CACHE_TYPE_V:=f16}"
: "${VERBOSITY:=4}"
: "${N_CPU_MOE:=36}"
: "${N_CPU_MOE:=35}"
: "${N_GPU_LAYERS:=all}"
: "${BATCH_SIZE:=4096}"
: "${UBATCH_SIZE:=1024}"
: "${CTX_CHECKPOINTS:=2}"
: "${UBATCH_SIZE:=2048}"
: "${CTX_CHECKPOINTS:=16}"
: "${NO_MMAP:=1}"
: "${TEMPERATURE:=1.0}"
: "${TOP_K:=20}"