inference: frank-large returns to halogen, frank-small takes the 3090

Dankclaw's endpoints are re-split. frank-large is the halogen-flash-server
quadlet again -- the 2026-09-13 unit file was never touched by the two-day
ik_llama.cpp interlude, so the runtime was still in the repo and its image is
still in local storage -- retuned to the shape asked for: HALOGEN_CTX 200192
with a 400,384-position pool and two slots, exactly two 200k conversations.

frank-small is new: mainline llama.cpp on the RTX 3090 serving
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF at IQ3_S, using the -mtp build (it carries
blk.64.nextn.*, which is what lets --spec-type draft-mtp run on the main
model's own weights instead of a second 12 GiB copy) plus the BF16 vision
projector, as one 131,072-token slot with f16 KV on port 7771.

The host is shared now, and that is what the halogen retune pays for:
HALOGEN_MAX_TOK 24576 (arena 20.6 -> 16.4 GiB, ~4-5% prefill) and the vision
tower dropped (1.97 GiB) hand ~6 GiB back to the host, and the reserve rises to
20 to keep the fit check honest. Vision moved wholesale to frank-small.

Retired to scripts/.AGENTTRASH/: the ik trio (env, service, serve script), the
Qwen-Sharp template only that runtime read, and four benchmark harnesses bound
to retired runtimes. Because retiring a file makes any test that greps it pass
by absence, the seams assert the archive exists AND the live path does not.
This commit is contained in:
Frank committed 2026-09-16 13:58:00 -07:00
1 parent cc29875842
commit 43ec77fee6
19 files changed
+648 -198

No files matched your search

+24 -10
View File
@@ -18,7 +18,7 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
@test "deployment planner maps large-model changes to the large model only" {
run python3 "$PLANNER" --target dankclaw --paths \
scripts/frank-large.env scripts/frank-large.service scripts/serve-frank-large.sh
dankclaw/frank-large/quadlet/frank-large.container
[ "$status" -eq 0 ]
[ "$output" = "frank-large" ]
}
@@ -28,14 +28,18 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
! grep -q 'frank_npu\|frank-npu' "$REMOTE_DEPLOY"
}
@test "chat-template changes plan frank-large: the ik runtime reads the tracked template" {
# ik_llama.cpp (2026-09-15 cutover) serves the Qwen-Sharp template from
# the tracked scripts/chat-template.jinja again. The vllm-radiance
# runtime on Dankserver ships its own qwen-fixed-v22.3.jinja out of the
# pinned clone, so a template edit deploys nothing there.
@test "the retired ik chat template plans nothing, and its archive is still here" {
# scripts/chat-template.jinja existed only for the ik_llama.cpp runtime
# (retired 2026-09-16 in favour of halogen-flash-server again). The
# llama.cpp frank-small runtime serves the template the GGUF itself
# carries, which is the one with reasoning_effort, preserve_thinking and
# the tool-call grammar. Editing the archive must deploy nothing -- and the
# archive must still be present, so this cannot pass by the file having
# vanished.
[ -f "$REPO_ROOT/scripts/.AGENTTRASH/chat-template.jinja.ik-llama-qwen-sharp" ]
run python3 "$PLANNER" --target dankclaw --paths scripts/chat-template.jinja
[ "$status" -eq 0 ]
[ "$output" = "frank-large" ]
[ -z "$output" ]
run python3 "$PLANNER" --target dankserver --paths scripts/chat-template.jinja
[ "$status" -eq 0 ]
@@ -51,9 +55,19 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
[ "$output" = "frank-medium" ]
}
@test "the retired small endpoint is absent from the deployment inventory" {
! grep -q '"frank-small"' "$PLANNER"
! grep -qE 'frank-small\) deploy|dankserver:frank-small' "$REMOTE_DEPLOY" "$REPO_ROOT/scripts/deploy.sh"
@test "the small endpoint is Dankclaw's again, and Dankserver's copy stays retired" {
run python3 "$PLANNER" --target dankclaw --paths \
scripts/frank-small.env scripts/frank-small.service scripts/serve-frank-small.sh
[ "$status" -eq 0 ]
[ "$output" = "frank-small" ]
# frank-small runs on Dankclaw on the RTX 3090; the Dankserver endpoint of
# that name was retired with the FreeToken runtime on 2026-09-15 and must
# not come back through a planner mapping.
run python3 "$PLANNER" --target dankserver --paths \
scripts/frank-small.env scripts/frank-small.service scripts/serve-frank-small.sh
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "documentation-only changes plan no runtime deployment" {