frank-large: lower KV slots 4 -> 3

Concurrency cap trimmed to the real fleet size. Each slot is 111.0 MiB of
O(1) state, so this hands ~111 MiB back on a host sitting at 7.1 GiB left
for everything else.

Note the slots already oversubscribed the pool (4 x 262144 > 524288), so
admission was pool-bound regardless: at most two full-length conversations,
or four at 131k. This does not change how much context fits -- it caps
simultaneous requests at three, and a fourth now queues.
This commit is contained in:
Frank
2026-09-12 23:20:16 -07:00
parent 5dfcde6c8b
commit 6169278911
@@ -100,7 +100,15 @@ Environment=HALOGEN_CTX=262144
# Doubled 2026-09-12 from the 262144 this had run since 09-11: two full-length
# conversations resident at once, or four at 131k, instead of one.
Environment=HALOGEN_KV_POOL_POSITIONS=524288
Environment=HALOGEN_KV_SLOTS=4
# Concurrency cap. Each slot is 111.0 MiB of O(1) state; the KV is ONE
# 524288-position pool carved into per-request regions, so slots cap how many
# requests run at once while the pool binds how many positions exist. These
# slots OVERSUBSCRIBE the pool (4 x 262144 > 524288), so admission was always
# pool-bound -- at most two full-length conversations, or four at 131k.
# LOWERED 4 -> 3 on 2026-09-12: frees ~111 MiB on a host with 7.1 GiB left,
# and three users is the real fleet size. A fourth concurrent request now
# queues behind the three rather than contending.
Environment=HALOGEN_KV_SLOTS=3
# Making the doubling stick is budget arithmetic, all of it. ONE budget covers
# the KV pool, the prefill arena and the vision tower together:
#