frank-primary: pin PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
The deployed unit failed its first boot with the engine core OOMing during profile/graph capture: "Tried to allocate 662.00 MiB. GPU 0 has a total capacity of 31.86 GiB of which 240.00 MiB is free ... try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True". The expert-offload plan fills VRAM to its modelled headroom by design, so the default allocator's fragmentation leaves the capture phase nothing — every successful bring-up probe (both scratch-port and 7770) carried this env, and the repo recipe dropped it. Also pins the seam: the setting is part of the qualified shape, not an optional tuning.
This commit is contained in:
@@ -106,6 +106,15 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
grep -q '^FRANK_PRIMARY_REASONINGPARSER=glm47$' "$ENV_FILE"
|
||||
}
|
||||
|
||||
@test "expandable segments is ON — the cache arena leaves no slack" {
|
||||
# The expert-offload plan fills VRAM to the modelled headroom, so the
|
||||
# profile/graph-capture phase has ~0.2 GiB to work with and the default
|
||||
# allocator's fragmentation fails it: measured 2026-09-30 on the deployed
|
||||
# unit, "Tried to allocate 662.00 MiB ... 240.00 MiB is free" from the
|
||||
# engine core while the identical shape booted with this env set.
|
||||
grep -q 'PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True' "$SERVE"
|
||||
}
|
||||
|
||||
@test "expert offload is ON — the payload cannot fit VRAM without it" {
|
||||
# 170 GiB of int4 experts against 64 GiB of VRAM. Without the offload flag
|
||||
# the engine has no home for the expert tensors at all; the arena
|
||||
|
||||
Reference in New Issue
Block a user