inference: frank-large returns to halogen, frank-small takes the 3090

Dankclaw's endpoints are re-split. frank-large is the halogen-flash-server
quadlet again -- the 2026-09-13 unit file was never touched by the two-day
ik_llama.cpp interlude, so the runtime was still in the repo and its image is
still in local storage -- retuned to the shape asked for: HALOGEN_CTX 200192
with a 400,384-position pool and two slots, exactly two 200k conversations.

frank-small is new: mainline llama.cpp on the RTX 3090 serving
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF at IQ3_S, using the -mtp build (it carries
blk.64.nextn.*, which is what lets --spec-type draft-mtp run on the main
model's own weights instead of a second 12 GiB copy) plus the BF16 vision
projector, as one 131,072-token slot with f16 KV on port 7771.

The host is shared now, and that is what the halogen retune pays for:
HALOGEN_MAX_TOK 24576 (arena 20.6 -> 16.4 GiB, ~4-5% prefill) and the vision
tower dropped (1.97 GiB) hand ~6 GiB back to the host, and the reserve rises to
20 to keep the fit check honest. Vision moved wholesale to frank-small.

Retired to scripts/.AGENTTRASH/: the ik trio (env, service, serve script), the
Qwen-Sharp template only that runtime read, and four benchmark harnesses bound
to retired runtimes. Because retiring a file makes any test that greps it pass
by absence, the seams assert the archive exists AND the live path does not.
This commit is contained in:
Frank committed 2026-09-16 13:58:00 -07:00
1 parent cc29875842
commit 43ec77fee6
19 files changed
+648 -198

No files matched your search

+15 -4
View File
@@ -145,11 +145,12 @@ for path in "${CHANGED[@]}"; do
add_test tests/homepage_caddy.bats
add_test tests/deployment_scoping.bats
;;
# frank-large moved to a rootless quadlet (halogen-flash-server) on
# 2026-09-11; its unit carries the fleet's GPU and podman posture, so
# it gets its own seam rather than the generic fallback below.
# frank-large is a rootless quadlet again since 2026-09-16 (the
# halogen-flash-server restore). Its unit carries the fleet's GPU and
# podman posture, so it gets its own seam rather than the generic
# fallback below.
dankclaw/frank-large/*)
add_test tests/frank_large_quadlet.bats
add_test tests/frank_large_unit.bats
add_test tests/deployment_scoping.bats
add_test tests/docs_completeness.bats
;;
@@ -187,6 +188,16 @@ for path in "${CHANGED[@]}"; do
add_test tests/deployment_scoping.bats
add_test tests/docs_completeness.bats
;;
# frank-small is llama.cpp on the 3090 as a ROOT unit: a native CUDA
# binary, so no generated quadlet is involved. Its env, unit and serve
# script all live in scripts/, which is why this case has to precede
# the generic scripts/*.service and scripts/* fallbacks below.
scripts/frank-small.*|scripts/serve-frank-small.sh)
add_test tests/frank_small_unit.bats
add_test tests/llm_units.bats
add_test tests/deployment_scoping.bats
add_test tests/scripts_syntax.bats
;;
scripts/*.service)
add_test tests/llm_units.bats
add_test tests/scripts_syntax.bats