inference: frank-large returns to halogen, frank-small takes the 3090

Dankclaw's endpoints are re-split. frank-large is the halogen-flash-server
quadlet again -- the 2026-09-13 unit file was never touched by the two-day
ik_llama.cpp interlude, so the runtime was still in the repo and its image is
still in local storage -- retuned to the shape asked for: HALOGEN_CTX 200192
with a 400,384-position pool and two slots, exactly two 200k conversations.

frank-small is new: mainline llama.cpp on the RTX 3090 serving
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF at IQ3_S, using the -mtp build (it carries
blk.64.nextn.*, which is what lets --spec-type draft-mtp run on the main
model's own weights instead of a second 12 GiB copy) plus the BF16 vision
projector, as one 131,072-token slot with f16 KV on port 7771.

The host is shared now, and that is what the halogen retune pays for:
HALOGEN_MAX_TOK 24576 (arena 20.6 -> 16.4 GiB, ~4-5% prefill) and the vision
tower dropped (1.97 GiB) hand ~6 GiB back to the host, and the reserve rises to
20 to keep the fit check honest. Vision moved wholesale to frank-small.

Retired to scripts/.AGENTTRASH/: the ik trio (env, service, serve script), the
Qwen-Sharp template only that runtime read, and four benchmark harnesses bound
to retired runtimes. Because retiring a file makes any test that greps it pass
by absence, the seams assert the archive exists AND the live path does not.
This commit is contained in:
Frank
2026-09-16 13:58:00 -07:00
parent cc29875842
commit 43ec77fee6
19 changed files with 648 additions and 198 deletions
+8 -8
View File
@@ -60,6 +60,7 @@ SERVICE_ORDER = (
"lidarr",
"readarr",
"frank-large",
"frank-small",
"frank-medium",
)
@@ -90,6 +91,7 @@ DANKCLAW_SERVICES = {
"caddy",
"homepage",
"frank-large",
"frank-small",
}
DANKSERVER_SERVICES = {
@@ -390,14 +392,12 @@ def services_for_path(path: str, target: str) -> set[str]:
name = p.name
if name in {"homepage-status.py", "homepage-status.service", "homepage-status.timer"}:
_add(services, "homepage")
if name in {"frank-large.env", "frank-large.service", "serve-frank-large.sh"}:
# The ik_llama.cpp runtime (2026-09-15 cutover) keeps its unit,
# env and serve files here; each one selects frank-large only.
_add(services, "frank-large")
if name == "chat-template.jinja":
# The shared chat template is read by frank-large's serve script
# again; the vllm-radiance runtime on Dankserver ships its own.
_add(services, "frank-large")
if name in {"frank-small.env", "frank-small.service", "serve-frank-small.sh"}:
# llama.cpp frank-small (2026-09-16) keeps its unit, env and serve
# files here; each one selects frank-small only. The ik_llama.cpp
# trio that lived here for two days is in scripts/.AGENTTRASH/,
# which is archive material and plans nothing.
_add(services, "frank-small")
# Deployment/test/benchmark scripts and other repo tooling do not
# restart a runtime service.
return services