dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-small: revert vLLM cutover — back to llama.cpp Gemma4-12B Q4_K_M

Browse Source 
The vLLM detour (a2b39b1, same day) is retired: the Q4_K_M llama.cpp setup
is the requested shape and the vLLM KV-quant options on sm_86 all measured
dead (fp8 hardware-impossible, int8 page-size unification, turboquant
mm-prefix). Root unit restored, vLLM quadlet/Dockerfile/fetch script moved
to .AGENTTRASH, runbook listed as same-day-reverted. Rollback target:
frank-small-vllm-gemma4.
This commit is contained in:
Frank
2026-09-19 01:42:37 -07:00
parent bab8f7f839
commit 374041cd7a
15 changed files with 495 additions and 637 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/deploy-plan.py
+5 -13
View File
@@ -319,12 +319,6 @@ def services_for_path(path: str, target: str) -> set[str]:
# its unit and fleet settings live in one .container file.
_add(services, "frank-large")
return services
if len(parts) >= 2 and parts[1] == "frank-small":
# The service is a rootless Quadlet container since 2026-09-18
# (vLLM); its unit lives in one .container file. The old root
# unit's trio is in scripts/ and maps there.
_add(services, "frank-small")
return services
if len(parts) >= 2 and parts[1] == "observability":
# The five observability containers became Quadlet units on
# 2026-09-11. A unit change selects exactly its own service;
@@ -408,13 +402,11 @@ def services_for_path(path: str, target: str) -> set[str]:
# change to it redeploying alertmanager is the only path that
# reaches it on the host.
_add(services, "alertmanager")
if name in {"frank-small.env", "serve-frank-small.sh",
"download-frank-small-vllm.sh"}:
# vLLM frank-small (2026-09-18) keeps its env, entrypoint and
# fetch script here; each one selects frank-small only. The
# unit is the quadlet under dankclaw/frank-small/, and the old
# root unit file is in scripts/.AGENTTRASH/, which is archive
# material and plans nothing.
if name in {"frank-small.env", "frank-small.service", "serve-frank-small.sh"}:
# llama.cpp frank-small (2026-09-16) keeps its unit, env and serve
# files here; each one selects frank-small only. The ik_llama.cpp
# trio that lived here for two days is in scripts/.AGENTTRASH/,
# which is archive material and plans nothing.
_add(services, "frank-small")
if name in {"frank-medium.env", "frank-medium.service", "serve-frank-medium.sh"}:
# llama.cpp frank-medium (2026-09-16) keeps its unit, env and serve