Merge vllmScripts under llm/ (history-preserving import)
The vllmScripts repository (LLM inference fleet: serve scripts, systemd units, env tunables, deployment runbook, research docs) is merged into this repository. Its full history is grafted under llm/ with a two-parent merge commit so no commit is lost. A follow-up commit reorganizes llm/ into the repo layout: llm/docs -> docs/LLM, llm/scripts -> scripts, llm/old -> old.
This commit is contained in:
commit
918fe4eaec
82 files changed
+10855
No files matched your search
@@ -0,0 +1,17 @@
|
||||
# Tunables for frank-default-npu (FastFlowLM NPU endpoint, Dankclaw)
|
||||
# Runtime: FLM 1.0.1 on the XDNA2 NPU (50 TOPS). Model: Gemma4-E4B-IT
|
||||
# (8B, Q4_1, vision + audio + reasoning). The NPU allocates from the
|
||||
# unified pool shared with frank-default-large (the CIRU llama.cpp
|
||||
# service on the iGPU). See docs/FastFlowLM NPU Inference.md.
|
||||
#
|
||||
# 2026-08-20: gemma4-it:e4b deployed. Footprint ~9.1 GB. 64k context.
|
||||
|
||||
FLM_MODEL=gemma4-it:e4b
|
||||
FLM_HOST=0.0.0.0
|
||||
FLM_PORT=52626
|
||||
FLM_PMODE=performance
|
||||
FLM_ASR=0
|
||||
# 64K context (native max for gemma4-it is 32768, but NPU supports up to 256k).
|
||||
# The model's max_prefill_len is 4096, so prompts longer than that get
|
||||
# truncated at prefill time.
|
||||
FLM_CTX_LEN=65536
|
||||
Reference in new issue
Block a user