Files

T

Frank 49e8dfff66 docs: the vault and AGENTS.md describe two Dankclaw endpoints

halogen-flash-server is ACTIVE again with its 2026-09-16 measured ledger (budget
33.5 GiB against a need of 27.2 — no clamp — and 14.7 GiB left for everything
else), ik_llama.cpp is retired with its measurements kept as the record of why,
and the new frank-small runbook covers the -mtp file, the 64 KiB/token KV
arithmetic, the MTP contract, and the reasoning_effort trap that cost me every
subagent on the endpoint.

AGENTS.md: topology, Active Services, runbook index, and three new llama.cpp
lessons (MTP is a property of the file; --jinja is what `tools` needs; a model's
template can reject the harness's thinking level). Three pre-existing defects
found while editing it: a truncated sentence mid-file, a harness path that does
not exist on either server, and the benchmarking stop/mask recipe still calling
frank-large a root unit.

Truthfulness notes worth keeping: the co-tenant stall NOTE prints again, so the
claim that it "no longer prints" is gone and the three observed block counts are
recorded instead of a guessed threshold; and the Observability page says out
loud why frank-large has no scrape job rather than leaving a silent absence.

2026-09-16 14:52:51 -07:00

5.2 KiB

Raw Blame History

type, title, description, tags, timestamp

type

title

description

Deployment and Update Process

This page describes how to deploy and update the LLM servers on Dankserver and Dankclaw. The process uses Git as the single source of truth. A change to SearXNG, Caddy, or another container service does not restart inference.

Fleet

Target	Host	Service	Port
Dankserver	`dank@dankserver`	`frank-medium`	7770
Dankclaw	`dank@dankclaw`	`frank-large`	7770
Dankclaw	`dank@dankclaw`	`frank-npu` (FLM on the XDNA2 NPU)	7771

Scoped workflow

graph LR
    A[Edit LLM files] --> B[Run tests]
    B --> C[Commit main]
    C --> D[deploy.sh selects matching service]
    D --> E[Target pulls ff-only]
    E --> F[Relink one unit]
    F --> G[Restart one inference service]
    G --> H[Wait for that service health]

Run the unified deployer from the repository root:

./scripts/deploy.sh dankserver --service frank-medium
./scripts/deploy.sh dankclaw --service frank-large
./scripts/deploy.sh dankclaw --dry-run

Without --service, scripts/deploy-plan.py maps changed files to the service on the selected host. The following files select the corresponding service:

scripts/frank-medium.env, frank-medium.service, and serve-frank-medium.sh select frank-medium on Dankserver.
scripts/frank-large.env, frank-large.service, and serve-frank-large.sh select frank-large on Dankclaw.
scripts/frank-npu.env, frank-npu.service, and serve-frank-npu.sh select frank-npu on Dankclaw.
scripts/chat-template.jinja selects the LLM on the selected host only.
Documentation, tests, deployment tooling, and unrelated container changes do not select either inference service.

The target-side scripts/deploy-remote.sh installs the selected service: for a root unit it relinks the repo file into /etc/systemd/system/, reloads systemd, enables it and restarts it (frank-small); for a Quadlet it retires the predecessor, stops the service's own unit, waits for the published port to clear, installs the .container and restarts the generated unit (frank-large). Each handler then waits on that service's health endpoint — http://127.0.0.1:7770/health for frank-large, http://127.0.0.1:7771/health for frank-small. The wait budget is 20 minutes because cold model loads are slow: halogen pins ~68 GiB of weights before it serves anything. A failed handler does not advance its per-service state.

Unit and artifact paths

The units point into the stable target checkout:

/home/dank/serverAppConfigs/scripts/frank-medium.service
/home/dank/serverAppConfigs/scripts/frank-small.service

The unit files and serve scripts validate their model/runtime artifacts before starting. frank-large is a Quadlet under dankclaw/frank-large/quadlet/ rather than a unit in scripts/; frank-small keeps the unit/env/serve trio above. Keep REMOTE_DIR stable; moving the checkout requires updating both the unit absolute paths and the deployment tests in the same change.

NPU unit

frank-npu is an inactive, historical Dankclaw endpoint. It is relinked only when explicitly selected with --service frank-npu; the large-model handler never restarts it. This is intentional: the NPU is not part of the active inference path and should not be disturbed by a large-model change.

Isolation guarantee

The remote runner has one explicit service dispatch. It does not run a host-wide LLM loop, podman-compose up for unrelated services, or a blanket systemd restart. Each successful service is recorded under the target user's state directory outside Git:

~/.local/state/serverAppConfigs/deploy-state/

On a later ordinary deployment, a service remains pending only while its recorded revision differs from the checkout and the planner still finds a change in that service's inputs.

Manual recovery

If the LLM handler fails after the target pull:

Inspect journalctl -u frank-small -n 200 (root unit) or journalctl --user -u frank-large -n 200 (rootless Quadlet).
Confirm the model files, the runtime binary and (for frank-large) the container image are present and readable by dank.

Retry only the affected service:

./scripts/deploy.sh dankserver --service frank-medium
./scripts/deploy.sh dankclaw --service frank-large
./scripts/deploy.sh dankclaw --service frank-small

Verify the matching health endpoint before declaring recovery.

Do not stop the other host's inference service during recovery. The two Dankclaw endpoints are independent processes on different devices: restarting frank-large does not disturb frank-small, and the reverse holds too — with one exception worth knowing, that the deployer waits for port 7770 to be released before it starts the replacement container. For benchmark work, follow the separate stop/benchmark/restart procedure in Testing Policy and the repository rules.

5.2 KiB Raw Blame History