frank-default-large: cap CTX_SIZE at 32768 (author quick-start) — 87 GiB fully-resident model + context buffer OOMs HIP heap (50.8 GiB at 131072, 103 GiB at 262144)

This commit is contained in:
Frank committed 2026-08-29 23:14:50 -07:00
1 parent 7ac172c4f4
commit 101b1f8be2
4 files changed
+25 -20

No files matched your search

+1 -1
View File
@@ -45,7 +45,7 @@ fi
: "${ALIAS:=frank-default-large}"
: "${BIND_ADDR:=0.0.0.0}"
: "${LISTEN_PORT:=7770}"
: "${CTX_SIZE:=131072}"
: "${CTX_SIZE:=32768}"
: "${PARALLEL:=1}"
: "${BATCH_SIZE:=4096}"
: "${UBATCH_SIZE:=4096}"