frank-small: revert to bf16 KV — the only dtype that boots Gemma4

int8_per_token_head: page-size unification fails (scale tensor breaks
the 256/512 head-dim ratio). turboquant_k8v4: backend validation fails
(no mm-prefix attention, which Gemma4's vision/audio path requires).
fp8: hardware-dead on sm_86. bf16 it is; history documented in
frank-small.env, seam pins auto + no backend pin.

scripts/frank-small.env

@@ -84,17 +84,18 @@ LISTEN_PORT=8000
 # 16k prefill chunk, 4-token MTP draft (the assistant carries 4 layers).
 MAX_MODEL_LEN=262144
 GPU_MEM_UTIL=0.95
-KV_CACHE_DTYPE=turboquant_k8v4
-# ATTENTION_BACKEND: Gemma4's config hook force-selects TRITON_ATTN for the
-# mixed 256/512 head dims whenever no backend is selected, and TRITON_ATTN
-# rejects the turboquant KV dtype (measured: ValueError "kv_cache_dtype not
-# supported", crash loop). Pin the TURBOQUANT backend explicitly via the
-# --attention-backend flag (this vLLM build removed the VLLM_ATTENTION_
-# BACKEND env var — it warns "Unknown vLLM environment variable" and
-# ignores it). The TURBOQUANT backend accepts any head size and the
-# turboquant_* dtypes, and the Gemma4 MTP speculator carries the target's
-# selected backend through to the draft layers, so the drafter follows.
-ATTENTION_BACKEND=TURBOQUANT
+KV_CACHE_DTYPE=auto
+# KV quant history (measured 2026-09-19, all on this image): bf16 (auto)
+# is the ONLY dtype that boots Gemma4 here. int8_per_token_head dies in
+# unify_kv_cache_spec_page_size — its separate fp32 scale tensor makes the
+# per-token page bytes 4160 (SWA, head_dim 256) vs 8256 (full, head_dim
+# 512), not an integer ratio. turboquant_k8v4 dies in backend validation —
+# the TURBOQUANT backend rejects "partial multimodal token full attention"
+# (the mm-prefix path Gemma4's vision/audio tokens require), and the
+# VLLM_ATTENTION_BACKEND env var does not exist in this build (the pin
+# goes through --attention-backend). fp8 is hardware-dead on sm_86
+# (cvt.e4m3x2 is sm_89+). Revisit on a vLLM release that documents TurboQuant
+# mm-prefix support.
 MAX_NUM_SEQS=16
 MAX_NUM_BATCHED_TOKENS=16384
 SPEC_NUM_TOKENS=4

scripts/serve-frank-small.sh

@@ -44,7 +44,6 @@ fi
 : "${MAX_NUM_SEQS:=16}"
 : "${MAX_NUM_BATCHED_TOKENS:=16384}"
 : "${SPEC_NUM_TOKENS:=4}"
-: "${ATTENTION_BACKEND:=TURBOQUANT}"
 : "${ENFORCE_EAGER:=0}"
 
 # Fail loudly rather than half-loading a model into VRAM: a missing file here
@@ -67,7 +66,6 @@ VLLM_FLAGS=(
 --kv-cache-dtype "$KV_CACHE_DTYPE"
 --max-num-seqs "$MAX_NUM_SEQS"
 --max-num-batched-tokens "$MAX_NUM_BATCHED_TOKENS"
---attention-backend "$ATTENTION_BACKEND"
 --speculative-config "$SPEC_CONFIG"
 --reasoning-parser gemma4
 --enable-auto-tool-choice