frank-medium: GPU_UTIL 0.98 -> 0.97 — the 0.98 startup threshold sits within ~15 MiB of the free memory measured at engine init (worker HIP context + amdgpu per-boot baseline ~690 MiB), so 2026-09-26's boot crash-looped on every restart; 0.97 buys ~160 MiB headroom at ~5k KV tokens

2026-09-26 15:02:20 -07:00
parent cd07a66939
commit c5b55e1e3b
1 changed files with 8 additions and 1 deletions
@@ -116,7 +116,14 @@ Environment=MAXLEN=262144
 # through the Gemma4 merge path and it is not MXFP4-quantized, so vLLM serves
 # image input on either checkpoint. harness models.yml keeps frank-medium
 # input: [text, image].
-Environment=GPU_UTIL=0.98
+# 0.97, down from 0.98: the startup free-memory check (free >= util x 31.86
+# GiB) sat ~15 MiB inside the threshold on the 2026-09-26 boot (free 31.21
+# GiB vs required 31.22 GiB) — the worker's own HIP context plus the
+# amdgpu per-boot baseline is ~690 MiB at check time and that drifts a few
+# tens of MiB between boots, so 0.98 crash-looped on every restart. 0.97
+# buys ~160 MiB of headroom at the cost of ~5k KV tokens on the
+# auto-sized pool.
+Environment=GPU_UTIL=0.97
 # Speculative decoding: DFlash2 external drafter at depth 7. A/B tested
 # against MTP self-speculation (depth 8) on this FP8 checkpoint: DFlash2
 # wins on decode (100.0 vs 85.9 t/s geomean, 287.1 vs 179.7 t/s burst-4)