frank-small: --attention-backend flag (env var removed in this build)

The VLLM_ATTENTION_BACKEND env var does not exist in this vLLM dev
build (warns 'Unknown vLLM environment variable' and is ignored), so
the pin landed nowhere and Gemma4's hook force-selected TRITON_ATTN
again. Pass --attention-backend TURBOQUANT as a CLI flag instead.
This commit is contained in:
Frank committed 2026-09-19 00:17:41 -07:00
1 parent 966b70db1a
commit 7e6ef1ac99
3 files changed
+18 -12

No files matched your search

+2
View File
@@ -44,6 +44,7 @@ fi
: "${MAX_NUM_SEQS:=16}"
: "${MAX_NUM_BATCHED_TOKENS:=16384}"
: "${SPEC_NUM_TOKENS:=4}"
: "${ATTENTION_BACKEND:=TURBOQUANT}"
: "${ENFORCE_EAGER:=0}"
# Fail loudly rather than half-loading a model into VRAM: a missing file here
@@ -66,6 +67,7 @@ VLLM_FLAGS=(
--kv-cache-dtype "$KV_CACHE_DTYPE"
--max-num-seqs "$MAX_NUM_SEQS"
--max-num-batched-tokens "$MAX_NUM_BATCHED_TOKENS"
--attention-backend "$ATTENTION_BACKEND"
--speculative-config "$SPEC_CONFIG"
--reasoning-parser gemma4
--enable-auto-tool-choice