frank-primary: shape G v2 — UTIL back to the 0.982 law; the 1.00 force was unbootable
Worker init refuses budget >= device total on BOTH ranks: ValueError: Free memory on device cuda:0 (31.29/31.86 GiB ...) NRestarts=2 crash loop, zero weights loaded. Experts stay at the v1 shape (graph term 0.07 GiB, KV 3.1665e9 = the 400,700-token ask); only the budget returns under the law. Arena 17.78 GiB / ~7,912 slots predicted.
This commit is contained in:
@@ -117,7 +117,7 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
# runtime headroom, not arena.
|
||||
grep -q '^FRANK_PRIMARY_MAXLEN=200704$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_SEQS=3$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_UTIL=1.0$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_UTIL=0.982$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_KV_BYTES=3166500000$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_MNBT=2048$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_KV_DTYPE=fp8$' "$ENV_FILE"
|
||||
@@ -265,15 +265,20 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
grep -q -- '-e R4D_LRU_GATHER_CHUNKS="$FRANK_PRIMARY_GATHER_CHUNKS"' "$SERVE"
|
||||
}
|
||||
|
||||
@test "the arena is forced to the flag ceiling and the KV ask is honoured" {
|
||||
@test "the arena sits just under the ceiling and the KV ask is honoured" {
|
||||
# The planner sizes expert cache as budget - non-expert - KV reserve -
|
||||
# graphs - overhead, and two of those five terms are reachable from a
|
||||
# deploy: budget (= device_total x util, field-capped at le=1) and graphs
|
||||
# (VLLM_CUDAGRAPH_MEM_*). So the shipped shape must sit at the ceiling of
|
||||
# BOTH: util at its field maximum, and a graph charge strictly below the
|
||||
# fork's analytic 563 MiB default while still paying per graph (the measured
|
||||
# post-capture delta is -0.06 GiB, i.e. the base pool is the over-charge, but
|
||||
# a future spec-decode shape captures 18 graphs and must still be charged).
|
||||
# deploy: budget (= device_total x util) and graphs (VLLM_CUDAGRAPH_MEM_*).
|
||||
# UTIL has a HARD LAW, not a field cap: 0.982, never 1.00. The 1.00 attempt
|
||||
# is UNBOOTABLE on this fork — worker init raises
|
||||
# `ValueError: Free memory on device cuda:N (31.29/31.86 GiB ...)`
|
||||
# on both ranks because the worker demands free > budget and budget ==
|
||||
# total — and even if it booted, 0.13-GiB-margin shapes have died under
|
||||
# real load. So the arena tops out at 0.982 budget (31.29 GiB).
|
||||
# Graphs: charge strictly below the fork's analytic 563 MiB default while
|
||||
# still paying per graph (the measured post-capture delta is -0.06 GiB, i.e.
|
||||
# the base pool is the over-charge, but a future spec-decode shape captures
|
||||
# 18 graphs and must still be charged).
|
||||
# MNBT stays 2048: 512 doubles TTFT at 4064 ctx, 1024-4096 are decode-
|
||||
# interchangeable AND arena-identical (the registry prices overhead flat up
|
||||
# to its nbt 4096 anchor), 12288 costs 2.5% plus arena.
|
||||
@@ -282,7 +287,7 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
# must clear 400,700 tokens to keep two full-length resident.
|
||||
grep -q '^FRANK_PRIMARY_MNBT=2048$' "$ENV_FILE"
|
||||
grep -q -- '--max-num-batched-tokens "$FRANK_PRIMARY_MNBT"' "$SERVE"
|
||||
awk -F= '/^FRANK_PRIMARY_UTIL=/{exit !($2+0 >= 1.0)}' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_UTIL=0.982$' "$ENV_FILE"
|
||||
base=$(awk -F= '/^FRANK_PRIMARY_CUDAGRAPH_BASE_MB=/{print $2}' "$ENV_FILE")
|
||||
per_graph=$(awk -F= '/^FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB=/{print $2}' "$ENV_FILE")
|
||||
[ "$base" -eq 0 ] && [ "$per_graph" -gt 0 ]
|
||||
|
||||
Reference in New Issue
Block a user