frank-primary: drop --expert-precise-memory (boot crash loop with bf16)
NRestarts climbed to 3, each attempt ~6.7 min then 'Deactivated successfully' (API server drains after an EngineCore fatal error; Restart=always turned the silent death into a visible loop). The arena is device-resident, so the budget decides it: weights 6.28 + pool 6.80 (bf16 2 x 262144) + graphs 0.55 + arena 16.65 (precise overhead 5.25) + non-torch 1.45 = 31.73 of 31.86 GiB -> ~0.13 GiB free at boot, and graph capture is what allocates transiently. Dropping the flag restores the 7.60 GiB default overhead: arena 14.31 GiB, total ~29.4 GiB, ~2.4 GiB free. bf16 2 x 262144 (Frank's KV change) stays. The serve script keeps the opt-in flag; the seam now asserts it is ABSENT.
This commit is contained in:
1 parent
db54fa3af1
commit
d3d180fd3f
3 files changed
+48
-9
No files matched your search
@@ -70,12 +70,18 @@ FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-r9v qwen-fp8"
|
||||
# Budget (per rank): 31.22 GiB total; non-expert weights 6.28, graphs 0.55,
|
||||
# non-torch ~1.45, pool 6.76 -> ~29.35 used, ~2.5 GiB free (measured basis:
|
||||
# the fp8 3-window shape used 28.27 and left 3.59).
|
||||
# PRECISE MEMORY: the registry's default overhead for this architecture is
|
||||
# 7.60 GiB = the fitted 5.25 GiB (gfx1201 TP2 nbt4096/seqs16/len262144, heavier
|
||||
# rank 5.41) PLUS 2.35 GiB of compile/graph-capture headroom. The flag selects
|
||||
# the fitted figure, which only moves the plan's `room` and therefore the arena:
|
||||
# fixed 16.91 -> 14.56 GiB, arena 6,248 -> ~7,278 slots (~2.35 GiB = ~1,025
|
||||
# slots at 2.344 MiB/slot). The risk it buys back is boot-time capture slack.
|
||||
# PRECISE MEMORY: DROPPED 2026-10-01 after it crashed the boot in combination
|
||||
# with bf16. The flag selects the fitted 5.25 GiB overhead instead of the
|
||||
# 7.60 GiB default, moving the plan's `room` and therefore the arena
|
||||
# (6,248 -> ~7,278 slots). But the arena is DEVICE-resident, and the budget
|
||||
# for this shape does not have 2.35 GiB to spare:
|
||||
# weights 6.28 + pool 6.80 (bf16 2 x 262144) + graphs 0.55 + arena 16.65
|
||||
# + non-torch ~1.45 = 31.73 of 31.86 GiB => ~0.13 GiB free at boot
|
||||
# measured as a CrashLoop: every attempt reached graph capture, the engine
|
||||
# exited cleanly, systemd restarted it (NRestarts=3 at ~6.7 min each).
|
||||
# With the default overhead the arena is 14.31 GiB and the same shape totals
|
||||
# ~29.4 GiB, leaving ~2.4 GiB for capture. Re-enable the flag only with a
|
||||
# pool/context reduction that pays for it.
|
||||
# MNBT 2048 (Frank): the overhead formula only adds bytes above nbt 4096, so
|
||||
# 2048 pays nothing, and the arena sweep measured the same ceiling <= 4096.
|
||||
FRANK_PRIMARY_MAXLEN=262144
|
||||
@@ -83,7 +89,6 @@ FRANK_PRIMARY_SEQS=2
|
||||
FRANK_PRIMARY_KV_BYTES=7300000000
|
||||
FRANK_PRIMARY_KV_DTYPE=auto
|
||||
FRANK_PRIMARY_MNBT=2048
|
||||
FRANK_PRIMARY_PRECISE_MEMORY=1
|
||||
FRANK_PRIMARY_GATHER_CHUNKS=32
|
||||
FRANK_PRIMARY_GATHER_LANES=64
|
||||
FRANK_PRIMARY_UTIL=0.98
|
||||
|
||||
Reference in new issue
Block a user