frank-primary + frank-large: export /metrics, raise primary log verbosity to 4
Both b11030 serve scripts never passed --metrics, so /metrics answered 501 and Prometheus could not scrape frank-large (silently down since its deploy) or frank-primary. frank-primary also gets -lv 4 (VERBOSITY env tunable, default 4) for the full per-buffer load breakdown.
This commit is contained in:
1 parent
bacabdcd49
commit
82ff9dda20
2 files changed
+4
No files matched your search
@@ -81,6 +81,7 @@ ARGS=(
|
|||||||
--parallel "$PARALLEL_SLOTS"
|
--parallel "$PARALLEL_SLOTS"
|
||||||
--n-gpu-layers 999
|
--n-gpu-layers 999
|
||||||
--jinja
|
--jinja
|
||||||
|
--metrics
|
||||||
--chat-template-file "$TEMPLATE_FILE"
|
--chat-template-file "$TEMPLATE_FILE"
|
||||||
--chat-template-kwargs '{"preserve_thinking":true}'
|
--chat-template-kwargs '{"preserve_thinking":true}'
|
||||||
--cache-type-k q8_0
|
--cache-type-k q8_0
|
||||||
|
|||||||
@@ -56,6 +56,7 @@ fi
|
|||||||
: "${BATCH_SIZE:=8192}"
|
: "${BATCH_SIZE:=8192}"
|
||||||
: "${UBATCH_SIZE:=512}"
|
: "${UBATCH_SIZE:=512}"
|
||||||
: "${CACHE_RAM:=16384}"
|
: "${CACHE_RAM:=16384}"
|
||||||
|
: "${VERBOSITY:=4}"
|
||||||
|
|
||||||
SERVER_BIN="$RUNTIME_DIR/llama-server"
|
SERVER_BIN="$RUNTIME_DIR/llama-server"
|
||||||
MODEL_PARTS=(
|
MODEL_PARTS=(
|
||||||
@@ -99,6 +100,8 @@ ARGS=(
|
|||||||
--alias frank-primary
|
--alias frank-primary
|
||||||
--ctx-size "$CONTEXT_SIZE"
|
--ctx-size "$CONTEXT_SIZE"
|
||||||
--parallel "$PARALLEL_SLOTS"
|
--parallel "$PARALLEL_SLOTS"
|
||||||
|
-lv "$VERBOSITY"
|
||||||
|
--metrics
|
||||||
--n-gpu-layers "$NGL"
|
--n-gpu-layers "$NGL"
|
||||||
--tensor-split "$TENSOR_SPLIT"
|
--tensor-split "$TENSOR_SPLIT"
|
||||||
--jinja
|
--jinja
|
||||||
|
|||||||
Reference in new issue
Block a user