Deployment and Update Process

This page describes how to deploy and update the LLM servers on Dankserver and Dankclaw. The process uses Git as the single source of truth. A change to SearXNG, Caddy, or another container service does not restart inference.

Fleet

Target	Host	Service	Port
Dankserver	`dank@dankserver`	`frank-default-small`	7770
Dankclaw	`dank@dankclaw`	`frank-default-large`	7770

Scoped workflow

graph LR
    A[Edit LLM files] --> B[Run tests]
    B --> C[Commit main]
    C --> D[deploy.sh selects matching service]
    D --> E[Target pulls ff-only]
    E --> F[Relink one unit]
    F --> G[Restart one inference service]
    G --> H[Wait for that service health]

Run the unified deployer from the repository root:

./scripts/deploy.sh dankserver --service frank-default-small
./scripts/deploy.sh dankclaw --service frank-default-large
./scripts/deploy.sh dankclaw --dry-run

Without --service, scripts/deploy-plan.py maps changed files to the service on the selected host. The following files select the corresponding service:

scripts/frank-default-small.env, frank-default-small.service, and serve-frank-small.sh select frank-default-small on Dankserver.
scripts/frank-default-large.env, frank-default-large.service, and serve-frank-large.sh select frank-default-large on Dankclaw.
scripts/chat-template.jinja selects the LLM on the selected host only.
Documentation, tests, deployment tooling, and unrelated container changes do not select either inference service.

The target-side scripts/deploy-remote.sh relinks the selected unit into /etc/systemd/system/, reloads systemd, enables it, restarts it, and waits for http://127.0.0.1:7770/health. The wait budget is 20 minutes because cold model loads can be slow. A failed handler does not advance its per-service state.

Unit and artifact paths

Both units point into the stable target checkout:

/home/dank/serverAppConfigs/scripts/frank-default-small.service
/home/dank/serverAppConfigs/scripts/frank-default-large.service

The unit files and serve scripts validate their model/runtime artifacts before starting. Keep REMOTE_DIR stable; moving the checkout requires updating both the unit absolute paths and the deployment tests in the same change.

NPU unit

frank-default-npu is an inactive, historical Dankclaw endpoint. It is relinked only when explicitly selected with --service frank-default-npu; the large-model handler never restarts it. This is intentional: the NPU is not part of the active inference path and should not be disturbed by a large-model change.

Isolation guarantee

The remote runner has one explicit service dispatch. It does not run a host-wide LLM loop, podman-compose up for unrelated services, or a blanket systemd restart. Each successful service is recorded under the target user's state directory outside Git:

~/.local/state/serverAppConfigs/deploy-state/

On a later ordinary deployment, a service remains pending only while its recorded revision differs from the checkout and the planner still finds a change in that service's inputs.

Manual recovery

If the LLM handler fails after the target pull:

Inspect journalctl -u frank-default-small -n 200 or journalctl -u frank-default-large -n 200.
Confirm the model files, custom runtime binary, and chat template are readable by dank.

Retry only the affected service:

./scripts/deploy.sh dankserver --service frank-default-small
# or
./scripts/deploy.sh dankclaw --service frank-default-large

Verify the matching health endpoint before declaring recovery.

Do not stop the other host's inference service during recovery. For benchmark work, follow the separate stop/benchmark/restart procedure in Testing Policy and the repository rules.

4.2 KiB Raw Blame History