dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-primary: 3 full windows, pool at the contract floor, MNBT 2048

Browse Source 
Frank's call 2026-10-01: MAXLEN 262144 x SEQS 3 = 786,432 tokens of contract.
Measured fp8 KV costs 7,698 B/token (4-slot boot 8,311,013,376 B -> 1,079,582
tokens; 5-slot boot 10,388,766,720 B -> 1,350,182 tokens), so the pool floor
for the contract is 6.054e9 B. Pinned at 6,100,000,000 (~792k tokens, ~5.9k
margin) so the remaining ~2.2 GiB versus the 4-slot shape goes to the expert
ARENA - the fork sizes its decode-critical expert cache from the remainder
after weights, pool and workspace.

MNBT 2048 (Frank): the arena sweep measured 6,499 slots at every rung <= 4096,
so the larger batch costs the arena nothing.

Seam retargeted (3 seqs / 6.1e9 / MNBT 2048); docs + AGENTS row + log updated,
including the closed DCP investigation.
This commit is contained in:
Frank
2026-10-01 13:25:27 -07:00
parent 6bbcaf3999
commit a1b650aabc
5 changed files with 51 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-primary.env
+20 -14
View File
@@ -59,22 +59,28 @@ FRANK_PRIMARY_PUBLISH=7770:7770
# hoc experiment identity (both hold the GPUs / name registrations).
FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-r9v qwen-fp8"
# --- qualified engine shape (2026-10-01 night ladder, Frank's method) ---------
# SEQS 4 x MAXLEN 262144 at fp8 KV is the DEPLOYED shape: four contexts of
# the model's FULL native window, with VRAM headroom the five-slot shape did
# not have. Measured 2026-10-01: 5 x 262144 booted (KV 1,350,182 tokens /
# 5.15x) but left ~0 bytes free at runtime, and a real client request
# (995-token prompt, max_tokens=261149) killed the worker with
# `torch.OutOfMemoryError: Tried to allocate 50.00 MiB ... 0 bytes free`.
# Dropping to four slots returns ~1.93 GiB per rank for activations/graphs.
# At SEQS=3 bf16, 262144/245760 also OOMed at boot and 241664 was max
# possible; at SEQS=6 fp8, six full-length contexts do NOT fit at all
# (6 x 241664 asked 10.70 GiB against 9.18 GiB free).
# --- qualified engine shape (2026-10-01, final: 3 windows + max arena) --------
# SEQS 3 x MAXLEN 262144 at fp8 KV (Frank's call, 2026-10-01): three contexts
# of the model's FULL native window, with the KV pool pinned to the FLOOR the
# contract needs so that everything else goes to the expert ARENA — the
# remainder is what the fork's planner gives the decode-critical expert cache.
# 3 x 262144 = 786,432 tokens; measured fp8 rate 7,698 B/token (from the
# 4-slot boot: 8,311,013,376 B -> 1,079,582 tokens, and the 5-slot boot:
# 10,388,766,720 B -> 1,350,182 tokens) => 6.054e9 B is the floor. Pinned at
# 6,100,000,000 => ~792k tokens, a ~5.9k-token margin, and ~2.2 GiB returned
# to the arena versus the 4-slot shape.
# History in this shape: 5 x 262144 booted (KV 1,350,182 / 5.15x) but left 0
# bytes free and a real request (995-token prompt, max_tokens=261149) killed
# the worker; 4 x 262144 (KV 1,079,582 / 4.12x) was the runtime-survivable
# stopgap. Six full-length contexts never fit (6 x 241664 asked 10.70 GiB
# against 9.18 GiB free).
# MNBT 2048 (Frank's call): the arena sweep measured it at the same 6,499-slot
# ceiling as <=4096, so the bigger batch costs the arena nothing.
FRANK_PRIMARY_MAXLEN=262144
FRANK_PRIMARY_SEQS=4
FRANK_PRIMARY_KV_BYTES=8311013376
FRANK_PRIMARY_SEQS=3
FRANK_PRIMARY_KV_BYTES=6100000000
FRANK_PRIMARY_KV_DTYPE=fp8
FRANK_PRIMARY_MNBT=1024
FRANK_PRIMARY_MNBT=2048
FRANK_PRIMARY_GATHER_CHUNKS=32
FRANK_PRIMARY_GATHER_LANES=64
FRANK_PRIMARY_UTIL=0.98