frank-small: reslice to 2 x 262,144 native slots (was 4 x 151,552)
The KV pool stays pinned at 606,208 tokens (card is full: 531 MiB free at the old shape; autofit's 687,388 would leave 0.53 GB and OOM on the first long request). 2 x 262,144 = 524,288 leaves ~82k tokens of pool slack. Motivation: harness compaction. frank-primary's contextWindow 262,144 compacts at 70% = 183,501 tokens, and the compaction request carries a 32,768-token summary budget -> 216,269 must fit ONE frank-small request. The 151,552 slot window could not hold it; the native window does. No spec-decode interaction: checkpoint ships zero mtp.* tensors (config lies), seam keeps every --speculative-* flag away from the engine.
This commit is contained in:
1 parent
389b605baf
commit
35e24db7e6
4 files changed
+35
-21
No files matched your search
@@ -14,7 +14,7 @@
|
||||
# WITHOUT speculative decoding — a NEXTN draft would be an uninitialised
|
||||
# 4.24 GiB ghost model drafting at 0% acceptance.
|
||||
#
|
||||
# Context: CONTEXT_LENGTH x MAX_RUNNING=4 slots must fit the fp8 KV pool,
|
||||
# Context: CONTEXT_LENGTH x MAX_RUNNING slots must fit the fp8 KV pool,
|
||||
# which the engine autofits around the weights AND the DeltaNet ssm state
|
||||
# pool (see frank-small.env). The mamba pool defaults to ~5.1 GB, so it is
|
||||
# pinned (bf16, 32 slots). With the grid at its measured fit the card has
|
||||
|
||||
Reference in new issue
Block a user