Frank
55f87fcdce
The 2026-10-09 wire aliases become the service names everywhere: systemd units, serve scripts, env files and FRANK_* variables, deploy-plan/deploy-remote service keys, container names, Prometheus job names + alert rules, Grafana dashboard queries, bats seams, runbook filenames, homepage/AGENTS lore, and historical prose. Rollback tags renamed in lockstep. Residue in the tree: zero. Retired frank-medium/small/npu names are history and stay. Known casualty: scripts/patches/*.patch context lines were rewritten; those retired-era vLLM-fork patches no longer apply byte-exact.
24 lines
1.3 KiB
Bash
24 lines
1.3 KiB
Bash
# Tunables for frank-npu (FastFlowLM NPU endpoint, Dankclaw)
|
|
# Runtime: FLM 1.0.1 (/usr/bin/flm) on the XDNA2 NPU (50 TOPS). Boot tag is
|
|
# nuextract3:4b (NuExtract3-4B-NPU2, Qwen3.5-4B skeleton, structured-extraction
|
|
# model); serve routes every request by the exact FLM registry tag in the
|
|
# request's model field, so any registered tag (gemma4-it:e4b, qwythos:9b, ...)
|
|
# is reachable on this port regardless of the boot tag. The NPU is a
|
|
# single-lane device: generations serialize FIFO, and the first request after a
|
|
# model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
|
|
# pool shared with frank-dragoon (iGPU).
|
|
FLM_MODEL=nuextract3:4b
|
|
FLM_HOST=0.0.0.0
|
|
FLM_PORT=7771
|
|
# turbo = max NPU power mode (xrt-smi configure --pmode turbo at serve start).
|
|
# No other tenant uses the NPU on this host, so the top mode costs nothing to
|
|
# neighbours; drop to `performance` if iGPU power-budget contention ever shows.
|
|
FLM_PMODE=turbo
|
|
# FLM_ASR=1 additionally loads Whisper-V3-Turbo-NPU2 (+0.62 GB) and exposes
|
|
# /v1/audio/transcriptions. NuExtract3 answers text-only; audio stays off.
|
|
FLM_ASR=0
|
|
# -1 = no --ctx-len flag: every pack loads with its own trained context
|
|
# (nuextract3:4b -> 32768, gemma4-it:e4b -> 131072). A global override would
|
|
# over-allocate KV for the small packs.
|
|
FLM_CTX_LEN=-1
|