dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs: frank-default-small live config — turbo, ASR, 131072 ctx verified

Browse Source
This commit is contained in:
Frank
2026-08-16 14:38:37 -07:00
parent fd4746422d
commit 2f1999b2e9
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/FastFlowLM NPU Inference.md
+2
View File
@@ -199,6 +199,8 @@ Notes:
`frank-default-small` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-default-small.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-default-small.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
Live config (2026-08-16): `FLM_PMODE=turbo`, `FLM_ASR=1` (loads Whisper-V3-Turbo-NPU2; exposes `/v1/audio/transcriptions`), `FLM_CTX_LEN=131072` (verified max — Gemma4 natively supports 128K and the NPU budget fits it; KV occupancy measured at 0.11% for a 123-token exchange, decode 12.0 t/s, 13 GiB headroom).
The PP split was renamed `frank-pp-head` → `frank-default-large` (2026-08-16, commit `48142f5`).
## Parallel Streams (measured 2026-08-16)