frank-default-large: threads-batch=16 confirmed optimal + decode benchmark results

Decode sweep (TB=8,16,32, 8K+ prompt, 256 gen tokens):
  TB=8:  3.3 tok/s (76.6s)
  TB=16: 3.4 tok/s (74.8s)
  TB=32: 3.4 tok/s (75.0s)

Decode is flat; TB=16 wins on prefill (5.1% faster) with zero decode tradeoff.

Also:
- Fix :: syntax error in serve-frank-large.sh
- Document systemctl disable destroys unit file; use mask/unmask instead
This commit is contained in:
Frank committed 2026-08-27 13:20:52 -07:00
1 parent 26173df1fd
commit cadc7f2768
4 files changed
+50 -6

No files matched your search

+1 -1
View File
@@ -49,7 +49,7 @@ fi
: "${BATCH_SIZE:=4096}"
: "${UBATCH_SIZE:=4096}"
: "${NUM_THREADS:=16}"
: "${THREAD_BATCH:=32}"
: "${THREAD_BATCH:=16}"
: "${CACHE_TYPE_K:=f16}"
: "${CACHE_TYPE_V:=f16}"
: "${CTX_CHECKPOINTS:=16}"