frank-small: four FULL 262144-token slots — the max that fits the card

CTX_SIZE=1048576 / PARALLEL=4: every slot gets the model's full trained
window. Measured fit arithmetic on the 3090: 1 full-context slot idles
at 11,996 MiB, 4 at 22,346 -> 3,450 MiB per slot, ~8,546 MiB fixed.
A 5th slot projects 25.8 GiB past the 24,576 MiB card, and the KV pool
allocates upfront, so 4 is the ceiling. Explicit CTX_SIZE bypasses --fit
on purpose: fit maximises the TOTAL pool and would divide a smaller
window across slots at parallel>1. The slot-grid seam now pins both the
full-window division and the budget inequality so a future capacity
bump fails pre-deploy, not in the boot allocation.

Audio verified while measuring: the mmproj's audio encoder transcribes
input_audio parts verbatim (JFK WAV sample). Header comments updated.

scripts/frank-small.env

@@ -27,15 +27,20 @@
 # context-backed drafters cannot share one process, so draft-mtp,ngram-mod
 # is the only legal shape that carries both.
 #
-# SIZING (RTX 3090, 24576 MiB): not modelled by hand -- CTX_SIZE is left
-# unset and --fit sizes the KV pool (q8_0 K and V) against what the 6.87
-# GiB of weights, the 0.16 GiB projector, the MTP sidecar's draft context
-# and the compute buffers leave on the card. The engine prints the chosen
-# context at startup; read it from the log, do not hard-code it back here.
+# SIZING (RTX 3090, 24576 MiB): CTX_SIZE = PARALLEL x 262144 -- four
+# FULL-window slots, measured to fit: the 4-slot boot idles at 22,346
+# MiB and each extra full-context slot costs 3,450 MiB (measured 1-slot
+# 11,996 -> 4-slot 22,346). A fifth slot projects to ~25.8 GiB, past the
+# card, and the KV pool allocates upfront -- so 4 is the max. Explicit
+# CTX_SIZE bypasses --fit on purpose: --fit maximises the TOTAL pool,
+# which would shrink each slot below the full 262144 window once
+# parallel > 1.
 #
-# Vision: --mmproj loads the BF16 projector (11 tensors, 52M params; it
-# also carries an audio encoder, unused by the fleet). The fleet's image
-# path lives here, which is why halogen no longer sets HALOGEN_VISION_TOWER.
+# Vision + audio: --mmproj loads the BF16 projector (11 tensors, 52M
+# params; it carries an audio encoder too). The fleet's image path lives
+# here, which is why halogen no longer sets HALOGEN_VISION_TOWER. Audio
+# WORKS: input_audio content parts transcribe verbatim (verified
+# 2026-09-17 with a 16k mono WAV).
 #
 # Sampling: greedy by default (the fleet contract); a client that sends a
 # temperature gets its own, and the harness applies per-model overrides
@@ -62,9 +67,11 @@ ALIAS=frank-small
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7771
 
-# Context: intentionally unset -- --fit decides it (see the sizing note).
-# One slot; a second concurrent request queues behind the first.
-PARALLEL=1
+# Four FULL 262144-token contexts, one per slot (CTX_SIZE is the total
+# pool; --parallel divides it). See the SIZING note above for the fit
+# arithmetic; a fifth slot does not fit.
+CTX_SIZE=1048576
+PARALLEL=4
 CACHE_TYPE_K=q8_0
 CACHE_TYPE_V=q8_0
 

scripts/serve-frank-small.sh

@@ -15,13 +15,16 @@
 # context-backed drafters cannot share one process, so this chain is the
 # only shape that carries both.
 #
-# Context: CTX_SIZE is left unset and --fit sizes the KV pool (q8_0 K and
-# V) against what the weights, projector, draft context and compute
-# buffers leave on the card; the chosen value lands in the startup log.
+# Context: CTX_SIZE is set explicitly to PARALLEL x 262144 so every slot
+# gets the model's FULL window; --fit (the CTX_SIZE-unset branch below)
+# maximises the TOTAL pool instead, which would divide a smaller window
+# across slots. The 4-slot layout is measured to fit the card (see the
+# SIZING note in frank-small.env); more slots would fail the KV allocation.
 #
-# Vision: --mmproj loads the BF16 projector (it also carries an audio
-# encoder; vision is the fleet contract). The fleet's image path lives
-# here, which is why halogen no longer sets HALOGEN_VISION_TOWER.
+# Vision + audio: --mmproj loads the BF16 projector, which also carries
+# an audio encoder — input_audio content parts transcribe verbatim
+# (verified 2026-09-17). The fleet's image path lives here, which is why
+# halogen no longer sets HALOGEN_VISION_TOWER.
 #
 # Sampling: greedy by default (the fleet contract). Clients that send their own
 # temperature get their own values, and the harness overrides per model from