diff --git a/dankclaw/frank-large/quadlet/frank-large.container b/dankclaw/frank-large/quadlet/frank-large.container index 17ddbfe..199091e 100644 --- a/dankclaw/frank-large/quadlet/frank-large.container +++ b/dankclaw/frank-large/quadlet/frank-large.container @@ -100,7 +100,15 @@ Environment=HALOGEN_CTX=262144 # Doubled 2026-09-12 from the 262144 this had run since 09-11: two full-length # conversations resident at once, or four at 131k, instead of one. Environment=HALOGEN_KV_POOL_POSITIONS=524288 -Environment=HALOGEN_KV_SLOTS=4 +# Concurrency cap. Each slot is 111.0 MiB of O(1) state; the KV is ONE +# 524288-position pool carved into per-request regions, so slots cap how many +# requests run at once while the pool binds how many positions exist. These +# slots OVERSUBSCRIBE the pool (4 x 262144 > 524288), so admission was always +# pool-bound -- at most two full-length conversations, or four at 131k. +# LOWERED 4 -> 3 on 2026-09-12: frees ~111 MiB on a host with 7.1 GiB left, +# and three users is the real fleet size. A fourth concurrent request now +# queues behind the three rather than contending. +Environment=HALOGEN_KV_SLOTS=3 # Making the doubling stick is budget arithmetic, all of it. ONE budget covers # the KV pool, the prefill arena and the vision tower together: #