frank-small: revert vLLM cutover — back to llama.cpp Gemma4-12B Q4_K_M
The vLLM detour (a2b39b1, same day) is retired: the Q4_K_M llama.cpp setup
is the requested shape and the vLLM KV-quant options on sm_86 all measured
dead (fp8 hardware-impossible, int8 page-size unification, turboquant
mm-prefix). Root unit restored, vLLM quadlet/Dockerfile/fetch script moved
to .AGENTTRASH, runbook listed as same-day-reverted. Rollback target:
frank-small-vllm-gemma4.
This commit is contained in:
1 parent
bab8f7f839
commit
374041cd7a
15 files changed
+495
-637
No files matched your search
@@ -72,11 +72,7 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
||||
|
||||
@test "frank-large loads no vision tower: the fleet's image path is frank-small" {
|
||||
! grep -q 'HALOGEN_VISION_TOWER' "$QUADLET"
|
||||
# frank-small's vLLM checkpoint (2026-09-18) is natively multimodal —
|
||||
# image content is ingested by the model itself, so the seam that used
|
||||
# to pin --mmproj now pins the projector's absence.
|
||||
! grep -q 'mmproj' "$REPO_ROOT/scripts/serve-frank-small.sh"
|
||||
grep -q '^MODEL_DIR=/models/main$' "$REPO_ROOT/scripts/frank-small.env"
|
||||
grep -q -- '--mmproj' "$REPO_ROOT/scripts/serve-frank-small.sh"
|
||||
}
|
||||
|
||||
@test "the deploy handler retires the previous ROOT unit without deleting it" {
|
||||
|
||||
Reference in new issue
Block a user