frank-primary + frank-large: export /metrics, raise primary log verbosity to 4
Both b11030 serve scripts never passed --metrics, so /metrics answered 501 and Prometheus could not scrape frank-large (silently down since its deploy) or frank-primary. frank-primary also gets -lv 4 (VERBOSITY env tunable, default 4) for the full per-buffer load breakdown.
This commit is contained in:
1 parent
bacabdcd49
commit
82ff9dda20
2 files changed
+4
No files matched your search
@@ -56,6 +56,7 @@ fi
|
||||
: "${BATCH_SIZE:=8192}"
|
||||
: "${UBATCH_SIZE:=512}"
|
||||
: "${CACHE_RAM:=16384}"
|
||||
: "${VERBOSITY:=4}"
|
||||
|
||||
SERVER_BIN="$RUNTIME_DIR/llama-server"
|
||||
MODEL_PARTS=(
|
||||
@@ -99,6 +100,8 @@ ARGS=(
|
||||
--alias frank-primary
|
||||
--ctx-size "$CONTEXT_SIZE"
|
||||
--parallel "$PARALLEL_SLOTS"
|
||||
-lv "$VERBOSITY"
|
||||
--metrics
|
||||
--n-gpu-layers "$NGL"
|
||||
--tensor-split "$TENSOR_SPLIT"
|
||||
--jinja
|
||||
|
||||
Reference in new issue
Block a user