Commit Graph
3 Commits
Author SHA1 Message Date
Frank 7403bd3d50 deploy: never start the replacement while the predecessor holds the port
The 2026-09-16 halogen restore failed exactly here. The ik root unit stopped,
systemd logged "Deactivated successfully", pasta tried to bind 7770 about a
second later, reported "Address already in use", and the quadlet exited 126.
Only Restart=on-failure recovered it -- minutes later, which is not a deploy.

The handler now waits for every socket on the port to clear (listener, accepted
and TIME_WAIT) before installing and starting the quadlet, and fails loudly with
the socket table attached if it never does. The retirement also reports when the
unit file was already gone, because this host's `disable` deletes it.
2026-09-16 14:06:03 -07:00
Frank 43ec77fee6 inference: frank-large returns to halogen, frank-small takes the 3090
Dankclaw's endpoints are re-split. frank-large is the halogen-flash-server
quadlet again -- the 2026-09-13 unit file was never touched by the two-day
ik_llama.cpp interlude, so the runtime was still in the repo and its image is
still in local storage -- retuned to the shape asked for: HALOGEN_CTX 200192
with a 400,384-position pool and two slots, exactly two 200k conversations.

frank-small is new: mainline llama.cpp on the RTX 3090 serving
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF at IQ3_S, using the -mtp build (it carries
blk.64.nextn.*, which is what lets --spec-type draft-mtp run on the main
model's own weights instead of a second 12 GiB copy) plus the BF16 vision
projector, as one 131,072-token slot with f16 KV on port 7771.

The host is shared now, and that is what the halogen retune pays for:
HALOGEN_MAX_TOK 24576 (arena 20.6 -> 16.4 GiB, ~4-5% prefill) and the vision
tower dropped (1.97 GiB) hand ~6 GiB back to the host, and the reserve rises to
20 to keep the fit check honest. Vision moved wholesale to frank-small.

Retired to scripts/.AGENTTRASH/: the ik trio (env, service, serve script), the
Qwen-Sharp template only that runtime read, and four benchmark harnesses bound
to retired runtimes. Because retiring a file makes any test that greps it pass
by absence, the seams assert the archive exists AND the live path does not.
2026-09-16 13:58:00 -07:00
Frank 0022ba96ad frank-large: mmap the CPU-overridden weights (no CUDA_Host pinning) 2026-09-15 23:58:34 -07:00