From 616927891119396b8f77a044bf2aff4ebed62d39 Mon Sep 17 00:00:00 2001 From: Frank Date: Sat, 12 Sep 2026 23:20:16 -0700 Subject: [PATCH] frank-large: lower KV slots 4 -> 3 Concurrency cap trimmed to the real fleet size. Each slot is 111.0 MiB of O(1) state, so this hands ~111 MiB back on a host sitting at 7.1 GiB left for everything else. Note the slots already oversubscribed the pool (4 x 262144 > 524288), so admission was pool-bound regardless: at most two full-length conversations, or four at 131k. This does not change how much context fits -- it caps simultaneous requests at three, and a fourth now queues. --- dankclaw/frank-large/quadlet/frank-large.container | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/dankclaw/frank-large/quadlet/frank-large.container b/dankclaw/frank-large/quadlet/frank-large.container index 17ddbfe..199091e 100644 --- a/dankclaw/frank-large/quadlet/frank-large.container +++ b/dankclaw/frank-large/quadlet/frank-large.container @@ -100,7 +100,15 @@ Environment=HALOGEN_CTX=262144 # Doubled 2026-09-12 from the 262144 this had run since 09-11: two full-length # conversations resident at once, or four at 131k, instead of one. Environment=HALOGEN_KV_POOL_POSITIONS=524288 -Environment=HALOGEN_KV_SLOTS=4 +# Concurrency cap. Each slot is 111.0 MiB of O(1) state; the KV is ONE +# 524288-position pool carved into per-request regions, so slots cap how many +# requests run at once while the pool binds how many positions exist. These +# slots OVERSUBSCRIBE the pool (4 x 262144 > 524288), so admission was always +# pool-bound -- at most two full-length conversations, or four at 131k. +# LOWERED 4 -> 3 on 2026-09-12: frees ~111 MiB on a host with 7.1 GiB left, +# and three users is the real fleet size. A fourth concurrent request now +# queues behind the three rather than contending. +Environment=HALOGEN_KV_SLOTS=3 # Making the doubling stick is budget arithmetic, all of it. ONE budget covers # the KV pool, the prefill arena and the vision tower together: #