frank-primary + frank-large: export /metrics, raise primary log verbosity to 4
Both b11030 serve scripts never passed --metrics, so /metrics answered 501 and Prometheus could not scrape frank-large (silently down since its deploy) or frank-primary. frank-primary also gets -lv 4 (VERBOSITY env tunable, default 4) for the full per-buffer load breakdown.
This commit is contained in:
@@ -81,6 +81,7 @@ ARGS=(
|
||||
--parallel "$PARALLEL_SLOTS"
|
||||
--n-gpu-layers 999
|
||||
--jinja
|
||||
--metrics
|
||||
--chat-template-file "$TEMPLATE_FILE"
|
||||
--chat-template-kwargs '{"preserve_thinking":true}'
|
||||
--cache-type-k q8_0
|
||||
|
||||
@@ -56,6 +56,7 @@ fi
|
||||
: "${BATCH_SIZE:=8192}"
|
||||
: "${UBATCH_SIZE:=512}"
|
||||
: "${CACHE_RAM:=16384}"
|
||||
: "${VERBOSITY:=4}"
|
||||
|
||||
SERVER_BIN="$RUNTIME_DIR/llama-server"
|
||||
MODEL_PARTS=(
|
||||
@@ -99,6 +100,8 @@ ARGS=(
|
||||
--alias frank-primary
|
||||
--ctx-size "$CONTEXT_SIZE"
|
||||
--parallel "$PARALLEL_SLOTS"
|
||||
-lv "$VERBOSITY"
|
||||
--metrics
|
||||
--n-gpu-layers "$NGL"
|
||||
--tensor-split "$TENSOR_SPLIT"
|
||||
--jinja
|
||||
|
||||
Reference in New Issue
Block a user