CTX_SIZE=1048576 / PARALLEL=4: every slot gets the model's full trained
window. Measured fit arithmetic on the 3090: 1 full-context slot idles
at 11,996 MiB, 4 at 22,346 -> 3,450 MiB per slot, ~8,546 MiB fixed.
A 5th slot projects 25.8 GiB past the 24,576 MiB card, and the KV pool
allocates upfront, so 4 is the ceiling. Explicit CTX_SIZE bypasses --fit
on purpose: fit maximises the TOTAL pool and would divide a smaller
window across slots at parallel>1. The slot-grid seam now pins both the
full-window division and the budget inequality so a future capacity
bump fails pre-deploy, not in the boot allocation.
Audio verified while measuring: the mmproj's audio encoder transcribes
input_audio parts verbatim (JFK WAV sample). Header comments updated.
The Qwen3.8-27B IQ3_S file carried its own MTP head; Gemma4's ships as a
separate assistant file (mtp-gemma-4-12B-it.gguf, arch gemma4-assistant,
4 nextn layers) that loads via --model-draft and shares the target's last
layers' KV. The draftless ngram-mod matcher chains behind it in the same
--spec-type value — two context-backed drafters cannot share one process,
so this chain is the only shape that carries both.
KV is q8_0 (16 KiB/token/layer vs 32 at f16) and CTX_SIZE is unset so
--fit sizes the pool against what the 7.4 GiB of payloads leave on the
24 GiB card. build-cuda was rebuilt with -DGGML_CUDA_FA_ALL_QUANTS=ON,
which quantized KV on CUDA requires. The three seams that pinned the Qwen
contract (MTP-in-file, the Qwen mmproj name, the f16 KV arithmetic) now
pin the Gemma4 contract instead, and the deploy handler's presence check
points at the new payload.
The 2026-09-16 halogen restore failed exactly here. The ik root unit stopped,
systemd logged "Deactivated successfully", pasta tried to bind 7770 about a
second later, reported "Address already in use", and the quadlet exited 126.
Only Restart=on-failure recovered it -- minutes later, which is not a deploy.
The handler now waits for every socket on the port to clear (listener, accepted
and TIME_WAIT) before installing and starting the quadlet, and fails loudly with
the socket table attached if it never does. The retirement also reports when the
unit file was already gone, because this host's `disable` deletes it.
Dankclaw's endpoints are re-split. frank-large is the halogen-flash-server
quadlet again -- the 2026-09-13 unit file was never touched by the two-day
ik_llama.cpp interlude, so the runtime was still in the repo and its image is
still in local storage -- retuned to the shape asked for: HALOGEN_CTX 200192
with a 400,384-position pool and two slots, exactly two 200k conversations.
frank-small is new: mainline llama.cpp on the RTX 3090 serving
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF at IQ3_S, using the -mtp build (it carries
blk.64.nextn.*, which is what lets --spec-type draft-mtp run on the main
model's own weights instead of a second 12 GiB copy) plus the BF16 vision
projector, as one 131,072-token slot with f16 KV on port 7771.
The host is shared now, and that is what the halogen retune pays for:
HALOGEN_MAX_TOK 24576 (arena 20.6 -> 16.4 GiB, ~4-5% prefill) and the vision
tower dropped (1.97 GiB) hand ~6 GiB back to the host, and the reserve rises to
20 to keep the fit check honest. Vision moved wholesale to frank-small.
Retired to scripts/.AGENTTRASH/: the ik trio (env, service, serve script), the
Qwen-Sharp template only that runtime read, and four benchmark harnesses bound
to retired runtimes. Because retiring a file makes any test that greps it pass
by absence, the seams assert the archive exists AND the live path does not.
Frank