frank-default-small: pmode turbo, ASR (Whisper-V3-Turbo) on, ctx-len 131072 (max verified)

This commit is contained in:
Frank committed 2026-08-16 14:37:32 -07:00
1 parent 02a1447602
commit fd4746422d
2 files changed
+23 -2

No files matched your search

+10 -1
View File
@@ -15,4 +15,13 @@
FLM_MODEL=gemma4-it:e4b
FLM_HOST=0.0.0.0
FLM_PORT=52625
FLM_PMODE=performance
# Power mode: turbo. Measured decode is bandwidth-bound (turbo ~= performance
# for this model), but the user requested turbo; it may help prefill.
FLM_PMODE=turbo
# Load the ASR model (Whisper-V3-Turbo-NPU2) alongside Gemma4. Exposes
# /v1/audio/transcriptions. Adds ~1 GiB to the resident footprint.
FLM_ASR=1
# Maximum context length. Gemma4 natively supports 128K; the NPU budget
# (~15 GiB minus the 9.1 GiB footprint) fits 131072. Verified 2026-08-16:
# loads, KV cache allocates to ~115k, decode ~12 t/s, 13 GiB headroom.
FLM_CTX_LEN=131072