frank-small: reslice to 2 x 262,144 native slots (was 4 x 151,552)

The KV pool stays pinned at 606,208 tokens (card is full: 531 MiB free at
the old shape; autofit's 687,388 would leave 0.53 GB and OOM on the first
long request). 2 x 262,144 = 524,288 leaves ~82k tokens of pool slack.

Motivation: harness compaction. frank-primary's contextWindow 262,144
compacts at 70% = 183,501 tokens, and the compaction request carries a
32,768-token summary budget -> 216,269 must fit ONE frank-small request.
The 151,552 slot window could not hold it; the native window does.

No spec-decode interaction: checkpoint ships zero mtp.* tensors (config
lies), seam keeps every --speculative-* flag away from the engine.
This commit is contained in:
Frank committed 2026-10-01 14:03:25 -07:00
1 parent 389b605baf
commit 35e24db7e6
4 files changed
+35 -21

No files matched your search

+1 -1
View File
@@ -14,7 +14,7 @@
# WITHOUT speculative decoding — a NEXTN draft would be an uninitialised
# 4.24 GiB ghost model drafting at 0% acceptance.
#
# Context: CONTEXT_LENGTH x MAX_RUNNING=4 slots must fit the fp8 KV pool,
# Context: CONTEXT_LENGTH x MAX_RUNNING slots must fit the fp8 KV pool,
# which the engine autofits around the weights AND the DeltaNet ssm state
# pool (see frank-small.env). The mamba pool defaults to ~5.1 GB, so it is
# pinned (bf16, 32 slots). With the grid at its measured fit the card has