serverAppConfigs/serverAppConfigs-docs/LLM/deployment.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
guide	LLM Deployment	Scoped deployment and update process for the two inference services.	llm deployment systemd	2026-09-03

Deployment and Update Process

This page describes how to deploy and update the LLM servers on Dankserver and Dankclaw. The process uses Git as the single source of truth. A change to SearXNG, Caddy, or another container service does not restart inference.

Fleet

Target	Host	Service	Port
Dankserver	dank@dankserver	frank-medium	7770
Dankclaw	dank@dankclaw	frank-large	7770
Dankclaw	dank@dankclaw	frank-small	7771

Scoped workflow

graph LR
    A[Edit LLM files] --> B[Run tests]
    B --> C[Commit main]
    C --> D[deploy.sh selects matching service]
    D --> E[Target pulls ff-only]
    E --> F[Relink one unit]
    F --> G[Restart one inference service]
    G --> H[Wait for that service health]

Run the unified deployer from the repository root:

./scripts/deploy.sh dankserver --service frank-medium
./scripts/deploy.sh dankclaw --service frank-large
./scripts/deploy.sh dankclaw --dry-run

Without --service, scripts/deploy-plan.py maps changed files to the service on the selected host. The following files select the corresponding service:

• Any file under dankserver/frank-medium/ (the Quadlet unit, its entrypoint and its healthcheck) selects frank-medium on Dankserver — the container mounts the entrypoint and healthcheck straight out of the checkout, so any file there changes deployed state.
• scripts/frank-large.env, frank-large.service, and serve-frank-large.sh select frank-large on Dankclaw.
• scripts/frank-small.env, frank-small.service, and serve-frank-small.sh select frank-small on Dankclaw.
• Documentation, tests, deployment tooling, and unrelated container changes do not select any inference service.

The target-side scripts/deploy-remote.sh installs the selected service. frank-large and frank-small are root units: the handler relinks the repo file into /etc/systemd/system/ (relink_unit), reloads systemd, enables it and restarts it. frank-medium is a rootless Quadlet: the handler retires its predecessors (the replica Quadlets and the retired llama root unit), stops the service's own unit, waits for the published port to clear, installs the .container and restarts the generated unit. Each handler then waits on that service's health endpoint — http://127.0.0.1:7770/health for frank-large and frank-medium, http://127.0.0.1:7771/health for frank-small. The wait budget is 20 minutes because cold model loads are slow. A failed handler does not advance its per-service state.

Unit and artifact paths

The root units point into the stable target checkout:

/home/dank/serverAppConfigs/scripts/frank-large.service
/home/dank/serverAppConfigs/scripts/frank-small.service

frank-medium is a Quadlet under dankserver/frank-medium/quadlet/ (frank-medium.container) rather than a unit in scripts/.

The unit files and serve scripts validate their model/runtime artifacts before starting. Keep REMOTE_DIR stable; moving the checkout requires updating both the unit absolute paths and the deployment tests in the same change.

Retired endpoints

frank-npu (the FastFlowLM XDNA2 NPU endpoint) was retired on 2026-09-11 with the amd_iommu=off decision and is no longer a deployable service — the planner has no frank-npu entry. Its runbook is frank-npu.

Isolation guarantee

The remote runner has one explicit service dispatch. It does not run a host-wide LLM loop, podman-compose up for unrelated services, or a blanket systemd restart. Each successful service is recorded under the target user's state directory outside Git:

~/.local/state/serverAppConfigs/deploy-state/

On a later ordinary deployment, a service remains pending only while its recorded revision differs from the checkout and the planner still finds a change in that service's inputs.

Manual recovery

If the LLM handler fails after the target pull:

1. Inspect journalctl -u frank-large -n 200 or journalctl -u frank-small -n 200 (root units), or journalctl --user -u frank-medium -n 200 (rootless Quadlet).

2. Confirm the model files, the runtime binary and (for frank-medium) the container image are present and readable by dank.

3. Retry only the affected service:

   ./scripts/deploy.sh dankserver --service frank-medium
   ./scripts/deploy.sh dankclaw --service frank-large
   ./scripts/deploy.sh dankclaw --service frank-small
   

4. Verify the matching health endpoint before declaring recovery.

Do not stop the other host's inference service during recovery. The two Dankclaw endpoints are independent processes on different devices: restarting frank-large does not disturb frank-small, and the reverse holds too — with one exception worth knowing, that the deployer waits for port 7770 to be released before it starts the replacement container. For benchmark work, follow the separate stop/benchmark/restart procedure in Testing Policy and the repository rules.

See Also

• Deployment Flow
• index
• Dankclaw deployment architecture