frank-primary: PLE ready timeout 1800s + bench matrix runbook; index/doc-link fixes
- frank-primary.env: R9V_PLE_READY_TIMEOUT=1800 (PLE registration measured 740s under page-cache churn; engine default 600s killed a prod boot 2026-09-28; launch.sh 2de0357 forwards it) - runbook: prefill/decode bench matrix section (chunk ceiling 4096, 80-slot policy max, MTP6 3.7x worse, prefix-caching evidence + recommendation), fork commit list, residency wording - index: q8 runbook ACTIVE, uncensored RETIRED (rollback); log entry incl. the 9h downtime incident
This commit is contained in:
@@ -19,6 +19,12 @@ R9V_HOST_PORT=7770
|
||||
R9V_SERVED_MODEL_NAME=frank-primary
|
||||
R9V_CONTAINER_NAME=frank-primary-r9v
|
||||
R9V_EXPECTED_GPU_BDFS=0000:03:00.0,0000:07:00.0
|
||||
# PLE registration (54.4 GB mmap -> HIP) can exceed the engine's 600 s
|
||||
# VLLM_PLE_OFFLOAD_READY_TIMEOUT default under page-cache churn (measured
|
||||
# 740 s on 2026-09-28, right after a bench teardown; a production boot
|
||||
# failed on it the same morning). launch.sh 2de0357 forwards this to the
|
||||
# engine; 1800 s covers the slowest observed registration with margin.
|
||||
R9V_PLE_READY_TIMEOUT=1800
|
||||
# The Q8_0 payload quantizes BOTH fused hyper-connection shards (the IQ4_XS
|
||||
# package ships inject as F32), so the shard-uniformity check rejects the
|
||||
# fused projection at load. This dev-overlay adapter dequantizes a quantized
|
||||
|
||||
Reference in New Issue
Block a user