frank-small: revert vLLM cutover — back to llama.cpp Gemma4-12B Q4_K_M

The vLLM detour (a2b39b1, same day) is retired: the Q4_K_M llama.cpp setup
is the requested shape and the vLLM KV-quant options on sm_86 all measured
dead (fp8 hardware-impossible, int8 page-size unification, turboquant
mm-prefix). Root unit restored, vLLM quadlet/Dockerfile/fetch script moved
to .AGENTTRASH, runbook listed as same-day-reverted. Rollback target:
frank-small-vllm-gemma4.
This commit is contained in:
Frank committed 2026-09-19 01:42:37 -07:00
1 parent bab8f7f839
commit 374041cd7a
15 files changed
+495 -637

No files matched your search

+1 -5
View File
@@ -72,11 +72,7 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
@test "frank-large loads no vision tower: the fleet's image path is frank-small" {
! grep -q 'HALOGEN_VISION_TOWER' "$QUADLET"
# frank-small's vLLM checkpoint (2026-09-18) is natively multimodal —
# image content is ingested by the model itself, so the seam that used
# to pin --mmproj now pins the projector's absence.
! grep -q 'mmproj' "$REPO_ROOT/scripts/serve-frank-small.sh"
grep -q '^MODEL_DIR=/models/main$' "$REPO_ROOT/scripts/frank-small.env"
grep -q -- '--mmproj' "$REPO_ROOT/scripts/serve-frank-small.sh"
}
@test "the deploy handler retires the previous ROOT unit without deleting it" {