inference: frank-large returns to halogen, frank-small takes the 3090
Dankclaw's endpoints are re-split. frank-large is the halogen-flash-server quadlet again -- the 2026-09-13 unit file was never touched by the two-day ik_llama.cpp interlude, so the runtime was still in the repo and its image is still in local storage -- retuned to the shape asked for: HALOGEN_CTX 200192 with a 400,384-position pool and two slots, exactly two 200k conversations. frank-small is new: mainline llama.cpp on the RTX 3090 serving ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF at IQ3_S, using the -mtp build (it carries blk.64.nextn.*, which is what lets --spec-type draft-mtp run on the main model's own weights instead of a second 12 GiB copy) plus the BF16 vision projector, as one 131,072-token slot with f16 KV on port 7771. The host is shared now, and that is what the halogen retune pays for: HALOGEN_MAX_TOK 24576 (arena 20.6 -> 16.4 GiB, ~4-5% prefill) and the vision tower dropped (1.97 GiB) hand ~6 GiB back to the host, and the reserve rises to 20 to keep the fit check honest. Vision moved wholesale to frank-small. Retired to scripts/.AGENTTRASH/: the ik trio (env, service, serve script), the Qwen-Sharp template only that runtime read, and four benchmark harnesses bound to retired runtimes. Because retiring a file makes any test that greps it pass by absence, the seams assert the archive exists AND the live path does not.
This commit is contained in:
@@ -60,6 +60,7 @@ SERVICE_ORDER = (
|
||||
"lidarr",
|
||||
"readarr",
|
||||
"frank-large",
|
||||
"frank-small",
|
||||
"frank-medium",
|
||||
)
|
||||
|
||||
@@ -90,6 +91,7 @@ DANKCLAW_SERVICES = {
|
||||
"caddy",
|
||||
"homepage",
|
||||
"frank-large",
|
||||
"frank-small",
|
||||
}
|
||||
|
||||
DANKSERVER_SERVICES = {
|
||||
@@ -390,14 +392,12 @@ def services_for_path(path: str, target: str) -> set[str]:
|
||||
name = p.name
|
||||
if name in {"homepage-status.py", "homepage-status.service", "homepage-status.timer"}:
|
||||
_add(services, "homepage")
|
||||
if name in {"frank-large.env", "frank-large.service", "serve-frank-large.sh"}:
|
||||
# The ik_llama.cpp runtime (2026-09-15 cutover) keeps its unit,
|
||||
# env and serve files here; each one selects frank-large only.
|
||||
_add(services, "frank-large")
|
||||
if name == "chat-template.jinja":
|
||||
# The shared chat template is read by frank-large's serve script
|
||||
# again; the vllm-radiance runtime on Dankserver ships its own.
|
||||
_add(services, "frank-large")
|
||||
if name in {"frank-small.env", "frank-small.service", "serve-frank-small.sh"}:
|
||||
# llama.cpp frank-small (2026-09-16) keeps its unit, env and serve
|
||||
# files here; each one selects frank-small only. The ik_llama.cpp
|
||||
# trio that lived here for two days is in scripts/.AGENTTRASH/,
|
||||
# which is archive material and plans nothing.
|
||||
_add(services, "frank-small")
|
||||
# Deployment/test/benchmark scripts and other repo tooling do not
|
||||
# restart a runtime service.
|
||||
return services
|
||||
|
||||
Reference in New Issue
Block a user