frank-medium: pass --mmproj explicitly; this build has no autoload
The 36:29/327680/vision deploy came up healthy and text-only: image requests answered 500 'image input is not supported - hint: ... provide the mmproj' with no mmproj line in the load log, so llama-server fb27a525 does not autoload mmproj*.gguf from the model directory. Name the file, the way frank-small's serve script already does, and pin the flag in the seam.
This commit is contained in:
@@ -161,8 +161,13 @@ TEMPLATE_ARGS+=(--chat-template-kwargs "$CHAT_TEMPLATE_KWARGS")
|
||||
PLACEMENT_ARGS=(--device "$DEVICES" --split-mode "$SPLIT_MODE" --tensor-split "$TENSOR_SPLIT")
|
||||
|
||||
MMPROJ_ARGS=()
|
||||
[[ "$NO_MMPROJ" == "1" ]] && MMPROJ_ARGS=(--no-mmproj)
|
||||
|
||||
if [[ "$NO_MMPROJ" == "1" ]]; then
|
||||
MMPROJ_ARGS=(--no-mmproj)
|
||||
else
|
||||
# No autoload in this build: a server without the flag starts text-only
|
||||
# and answers image requests with a 500, so name the projector.
|
||||
MMPROJ_ARGS=(--mmproj "$MM_PROJ")
|
||||
fi
|
||||
exec "$LLAMA_BIN" \
|
||||
--model "$MODEL" \
|
||||
--alias "$ALIAS" \
|
||||
|
||||
Reference in New Issue
Block a user