docs(frank-primary): record VLLM_CUDAGRAPH_MEM_ESTIMATE_MB, the graph-term outright pin
Verified in the shipped image (v1/worker/gpu/cudagraph_mem.py): estimate_cudagraph_memory() checks VLLM_CUDAGRAPH_MEM_ESTIMATE_MB FIRST and returns that MiB total outright, ignoring the BASE/PER_GRAPH pair — the only way to state the CUDA-graph plan term directly instead of deriving it. We keep shipping the pair (0/12) so a future spec-decode shape re-prices automatically; documented the switch condition. Also corrected two stale shape-G claims in the runbook that predated the 1.00 revert: the deployed shape block and the lever table both still said util 1.0 / 'le=1 ceiling', while shape G ships 0.982.
This commit is contained in:
1 parent
50459e4e31
commit
9a144f779e
2 files changed
+10
-3
No files matched your search
@@ -203,6 +203,13 @@ FRANK_PRIMARY_UTIL=0.982
|
||||
# Verify every deploy against the "Estimated CUDA graph memory" vs "Graph
|
||||
# capturing finished ... took X GiB" lines; if capture ever goes positive and
|
||||
# exceeds the charge, raise FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB back toward 52.
|
||||
# A THIRD env var exists and is the only way to state the graph term outright:
|
||||
# VLLM_CUDAGRAPH_MEM_ESTIMATE_MB pins the WHOLE term to an exact MiB total in
|
||||
# one knob — estimate_cudagraph_memory() checks it FIRST and returns base/
|
||||
# per-graph untouched when set (verified in the shipped image,
|
||||
# v1/worker/gpu/cudagraph_mem.py). We ship the BASE/PER_GRAPH pair instead so
|
||||
# a future spec-decode shape re-prices automatically; switch to ESTIMATE_MB
|
||||
# only if the drift check below ever shows the pair cannot track capture.
|
||||
FRANK_PRIMARY_CUDAGRAPH_BASE_MB=0
|
||||
FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB=12
|
||||
|
||||
|
||||
Reference in new issue
Block a user