serverAppConfigs/serverAppConfigs-docs/LLM/deployment.md

---
type: guide
title: LLM Deployment
description: Scoped deployment and update process for the two inference services.
tags:
  - llm
  - deployment
  - systemd
timestamp: 2026-09-03
---

# Deployment and Update Process

This page describes how to deploy and update the LLM servers on Dankserver and
Dankclaw. The process uses Git as the single source of truth. A change to
SearXNG, Caddy, or another container service does **not** restart inference.

## Fleet

| Target | Host | Service | Port |
|---|---|---|---:|
| Dankserver | `dank@dankserver` | `frank-medium` | 7770 |
| Dankclaw | `dank@dankclaw` | `frank-large` | 7770 |
| Dankclaw | `dank@dankclaw` | `frank-small` | 7771 |

## Scoped workflow

```mermaid
graph LR
    A[Edit LLM files] --> B[Run tests]
    B --> C[Commit main]
    C --> D[deploy.sh selects matching service]
    D --> E[Target pulls ff-only]
    E --> F[Relink one unit]
    F --> G[Restart one inference service]
    G --> H[Wait for that service health]
```

Run the unified deployer from the repository root:

```bash
./scripts/deploy.sh dankserver --service frank-medium
./scripts/deploy.sh dankclaw --service frank-large
./scripts/deploy.sh dankclaw --dry-run
```

Without `--service`, `scripts/deploy-plan.py` maps changed files to the service
on the selected host. The following files select the corresponding service:

- Any file under `dankserver/frank-medium/` (the Quadlet unit, its
  entrypoint and its healthcheck) selects `frank-medium` on Dankserver —
  the container mounts the entrypoint and healthcheck straight out of the
  checkout, so any file there changes deployed state.
- `scripts/frank-large.env`, `frank-large.service`, and
  `serve-frank-large.sh` select `frank-large` on Dankclaw.
- `scripts/frank-small.env`, `frank-small.service`, and
  `serve-frank-small.sh` select `frank-small` on Dankclaw.
- Documentation, tests, deployment tooling, and unrelated container changes
  do not select any inference service.

The target-side `scripts/deploy-remote.sh` installs the selected service.
`frank-large` and `frank-small` are **root units**: the handler relinks the
repo file into `/etc/systemd/system/` (`relink_unit`), reloads systemd,
enables it and restarts it. `frank-medium` is a **rootless Quadlet**: the
handler retires its predecessors (the replica Quadlets and the retired llama
root unit), stops the service's own unit, waits for the published port to
clear, installs the `.container` and restarts the generated unit. Each
handler then waits on that service's health endpoint —
`http://127.0.0.1:7770/health` for frank-large and frank-medium,
`http://127.0.0.1:7771/health` for frank-small. The wait budget is 20 minutes
because cold model loads are slow. A failed handler does not advance its
per-service state.

## Unit and artifact paths

The root units point into the stable target checkout:
```text
/home/dank/serverAppConfigs/scripts/frank-large.service
/home/dank/serverAppConfigs/scripts/frank-small.service
```
`frank-medium` is a Quadlet under `dankserver/frank-medium/quadlet/`
(`frank-medium.container`) rather than a unit in `scripts/`.

The unit files and serve scripts validate their model/runtime artifacts before
starting. Keep `REMOTE_DIR` stable; moving the checkout requires updating both
the unit absolute paths and the deployment tests in the same change.

## Retired endpoints

`frank-npu` (the FastFlowLM XDNA2 NPU endpoint) was retired on 2026-09-11
with the `amd_iommu=off` decision and is no longer a deployable service — the
planner has no `frank-npu` entry. Its runbook is [[frank-npu]].

## Isolation guarantee

The remote runner has one explicit service dispatch. It does not run a
host-wide LLM loop, `podman-compose up` for unrelated services, or a blanket
systemd restart. Each successful service is recorded under the target user's
state directory outside Git:

```text
~/.local/state/serverAppConfigs/deploy-state/
```

On a later ordinary deployment, a service remains pending only while its
recorded revision differs from the checkout and the planner still finds a
change in that service's inputs.

## Manual recovery

If the LLM handler fails after the target pull:

1. Inspect `journalctl -u frank-large -n 200` or `journalctl -u frank-small
   -n 200` (root units), or `journalctl --user -u frank-medium -n 200`
   (rootless Quadlet).
2. Confirm the model files, the runtime binary and (for frank-medium) the
   container image are present and readable by `dank`.
3. Retry only the affected service:

   ```bash
   ./scripts/deploy.sh dankserver --service frank-medium
   ./scripts/deploy.sh dankclaw --service frank-large
   ./scripts/deploy.sh dankclaw --service frank-small
   ```
4. Verify the matching health endpoint before declaring recovery.

Do not stop the other host's inference service during recovery. The two
Dankclaw endpoints are independent processes on different devices: restarting
frank-large does not disturb frank-small, and the reverse holds too — with one
exception worth knowing, that the deployer waits for port 7770 to be released
before it starts the replacement container. For benchmark work, follow the
separate stop/benchmark/restart procedure in [[Testing Policy]] and the
repository rules.

## See Also

- [Deployment Flow](../Architecture/Deployment%20Flow.md)
- [[index]]
- [Dankclaw deployment architecture](../Architecture/Network%20Architecture.md)