frank-small: two 128k slots, no spec, ubatch 512 (Frank's tight-fit order, 2026-10-06)

CTX 262144 / PARALLEL 2 = two 131,072 conversations; SPEC_TYPE=none strips both speculative flags in the serve script (ngram-mod skipped on purpose — on a qwen35 hybrid it disables recurrent-state rollback); ubatch 1024->512 claws back ~0.3 GiB. Budget: non-KV ~15.5 GiB + two q8_0 slots 8,704 MiB lands AT the card — the 2-3 GiB floor is knowingly violated by explicit order; nvidia-smi free at boot is the measured truth. Seam retargeted: shape (2 slots, 131072/slot), budget (two fit, three don't), spec-off (absence of both flags), unit Description.

scripts/serve-frank-small.sh

@@ -3,11 +3,13 @@
 # frank-small on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA
 # sm_86).
 #
-# 2026-10-05 evening (Frank): a 27B on the 24 GiB card — ONE 131,072-token
-# conversation (Frank's cap), KV at FP8 class (q8_0 both halves), the MTP
-# head that lives IN the -mtp file drafting 3, vision on. The q8_0 KV
-# contract supersedes the same-day f16/f16 rule, which was written for the
-# Ornith 9B fit; the per-tensor flags stay explicit either way.
+# 2026-10-06 (Frank): the 128k cap keeps the per-conversation window but
+# the pool doubles to TWO 131,072 conversations (262144 / 2), speculative
+# decoding is OFF (no draft-mtp; ngram skipped — on a qwen35 hybrid it
+# disables recurrent-state rollback), and ubatch drops to 512. Frank
+# accepted the tight fit explicitly (the 2-3 GiB floor is knowingly
+# violated; frank-small.env carries the arithmetic). The q8_0 KV contract
+# (FP8 class) is unchanged; the per-tensor flags stay explicit either way.
 #
 # Architecture: qwen35 hybrid — 64 real layers, 16 full-attention + 48
 # Gated DeltaNet (full_attention_interval 4), native 262144 window, plus
@@ -40,29 +42,30 @@ fi
 : "${ALIAS:=frank-small}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7771}"
-# ctx-size is the POOL llama.cpp divides across slots: 131072 / 1 gives
-# 131,072 per conversation — Frank's 128k cap. A second 131,072 slot does
-# not fit the 2-3 GiB floor on the 24 GiB card with the 27B payload
-# (frank-small.env carries the arithmetic); "as many 128k contexts as fit"
-# = 1.
-: "${CTX_SIZE:=131072}"
-: "${PARALLEL:=1}"
-# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves),
-# and Frank's standing law for a resize is LOWER UBATCH FIRST; the shipped
-# pair is 4096/1024. If boot lands well above the 2-3 GiB floor, ubatch
-# 2048 is the first A/B for prefill (a miss fails loud at graph capture).
-# The logical batch is VRAM- and throughput-neutral at a fixed ubatch.
+# ctx-size is the POOL llama.cpp divides across slots: 262144 / 2 gives
+# TWO 131,072 conversations — Frank's 128k cap each. 2026-10-06 (Frank):
+# drop MTP, drop ubatch to 512, take two slots; the fit is tight by his
+# explicit order (frank-small.env carries the arithmetic and the floor
+# violation).
+: "${CTX_SIZE:=262144}"
+: "${PARALLEL:=2}"
+# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
+# Frank's 2026-10-06 tight-fit order ships 4096/512 (claws back ~0.3 GiB
+# toward the second slot); ubatch 1024 is the first A/B back if a future
+# boot lands with headroom (a miss fails loud at graph capture). The
+# logical batch is VRAM- and throughput-neutral at a fixed ubatch.
 : "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=1024}"
+: "${UBATCH_SIZE:=512}"
 # KV at FP8 class = q8_0 on llama.cpp (1.0625 B/elem => 8.5 effective bits;
 # there is no true fp8 KV dtype). Explicit either way so the contract is
 # visible in the argv.
 : "${CACHE_TYPE_K:=q8_0}"
 : "${CACHE_TYPE_V:=q8_0}"
-# Speculative decoding: draft-mtp loads the MTP head that is IN the -mtp
-# GGUF; the flags move as a set.
-: "${SPEC_TYPE:=draft-mtp}"
-: "${SPEC_DRAFT_N_MAX:=3}"
+# Speculative decoding is OFF by default (Frank, 2026-10-06): SPEC_TYPE
+# none strips BOTH speculative flags (flags move as a set). Setting a
+# drafter here re-enables the pair — draft-mtp needs the head that is IN
+# the -mtp GGUF.
+: "${SPEC_TYPE:=none}"
 : "${NGL:=99}"
 # Vision projector. No colon on purpose: an explicitly empty value means
 # "text-only, on purpose" and must NOT be replaced by the default path.
@@ -80,11 +83,12 @@ if [[ -n "$MMPROJ_FILE" ]]; then
     EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
 fi
 
-# The MTP head ships IN the -mtp GGUF (blk.64.nextn.*,
-# nextn_predict_layers=1): draft-mtp loads it against the main model's own
-# weights — no --model-draft and no second KV pool. Speculative flags move
-# as a set: a no-spec baseline must strip BOTH of these, not just one.
-EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
+# Speculative flags move as a set: they reach the argv ONLY when a drafter
+# is named. SPEC_TYPE=none (the shipped contract) strips both — a no-spec
+# baseline must not carry half a spec pair.
+if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
+    EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
+fi
 
 # The UVM device node is created lazily by nvidia-modprobe on the first CUDA
 # open after boot; make sure it exists before the server needs it.