frank-primary: drop --expert-precise-memory (boot crash loop with bf16)

NRestarts climbed to 3, each attempt ~6.7 min then 'Deactivated successfully'
(API server drains after an EngineCore fatal error; Restart=always turned the
silent death into a visible loop). The arena is device-resident, so the budget
decides it: weights 6.28 + pool 6.80 (bf16 2 x 262144) + graphs 0.55 + arena
16.65 (precise overhead 5.25) + non-torch 1.45 = 31.73 of 31.86 GiB -> ~0.13
GiB free at boot, and graph capture is what allocates transiently.

Dropping the flag restores the 7.60 GiB default overhead: arena 14.31 GiB,
total ~29.4 GiB, ~2.4 GiB free. bf16 2 x 262144 (Frank's KV change) stays.
The serve script keeps the opt-in flag; the seam now asserts it is ABSENT.
This commit is contained in:
Frank committed 2026-10-01 14:16:47 -07:00
1 parent db54fa3af1
commit d3d180fd3f
3 files changed
+48 -9

No files matched your search

+12 -7
View File
@@ -70,12 +70,18 @@ FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-r9v qwen-fp8"
# Budget (per rank): 31.22 GiB total; non-expert weights 6.28, graphs 0.55,
# non-torch ~1.45, pool 6.76 -> ~29.35 used, ~2.5 GiB free (measured basis:
# the fp8 3-window shape used 28.27 and left 3.59).
# PRECISE MEMORY: the registry's default overhead for this architecture is
# 7.60 GiB = the fitted 5.25 GiB (gfx1201 TP2 nbt4096/seqs16/len262144, heavier
# rank 5.41) PLUS 2.35 GiB of compile/graph-capture headroom. The flag selects
# the fitted figure, which only moves the plan's `room` and therefore the arena:
# fixed 16.91 -> 14.56 GiB, arena 6,248 -> ~7,278 slots (~2.35 GiB = ~1,025
# slots at 2.344 MiB/slot). The risk it buys back is boot-time capture slack.
# PRECISE MEMORY: DROPPED 2026-10-01 after it crashed the boot in combination
# with bf16. The flag selects the fitted 5.25 GiB overhead instead of the
# 7.60 GiB default, moving the plan's `room` and therefore the arena
# (6,248 -> ~7,278 slots). But the arena is DEVICE-resident, and the budget
# for this shape does not have 2.35 GiB to spare:
# weights 6.28 + pool 6.80 (bf16 2 x 262144) + graphs 0.55 + arena 16.65
# + non-torch ~1.45 = 31.73 of 31.86 GiB => ~0.13 GiB free at boot
# measured as a CrashLoop: every attempt reached graph capture, the engine
# exited cleanly, systemd restarted it (NRestarts=3 at ~6.7 min each).
# With the default overhead the arena is 14.31 GiB and the same shape totals
# ~29.4 GiB, leaving ~2.4 GiB for capture. Re-enable the flag only with a
# pool/context reduction that pays for it.
# MNBT 2048 (Frank): the overhead formula only adds bytes above nbt 4096, so
# 2048 pays nothing, and the arena sweep measured the same ceiling <= 4096.
FRANK_PRIMARY_MAXLEN=262144
@@ -83,7 +89,6 @@ FRANK_PRIMARY_SEQS=2
FRANK_PRIMARY_KV_BYTES=7300000000
FRANK_PRIMARY_KV_DTYPE=auto
FRANK_PRIMARY_MNBT=2048
FRANK_PRIMARY_PRECISE_MEMORY=1
FRANK_PRIMARY_GATHER_CHUNKS=32
FRANK_PRIMARY_GATHER_LANES=64
FRANK_PRIMARY_UTIL=0.98