 Frank
|
cadc7f2768
|
frank-default-large: threads-batch=16 confirmed optimal + decode benchmark results
Decode sweep (TB=8,16,32, 8K+ prompt, 256 gen tokens):
TB=8: 3.3 tok/s (76.6s)
TB=16: 3.4 tok/s (74.8s)
TB=32: 3.4 tok/s (75.0s)
Decode is flat; TB=16 wins on prefill (5.1% faster) with zero decode tradeoff.
Also:
- Fix :: syntax error in serve-frank-large.sh
- Document systemctl disable destroys unit file; use mask/unmask instead
|
2026-08-27 13:20:52 -07:00 |
|