Frank
8212d4451a
Mechanical identity rename across the knowledge base, AGENTS.md, README, the
hermes config example, and homepage services.json. No behavior change; served
aliases on the LLM endpoints change with the deploy of commit 69f87c8.
131 lines
4.4 KiB
Markdown
131 lines
4.4 KiB
Markdown
---
|
|
type: guide
|
|
title: LLM Deployment
|
|
description: Scoped deployment and update process for the two inference services.
|
|
tags:
|
|
- llm
|
|
- deployment
|
|
- systemd
|
|
timestamp: 2026-09-03
|
|
---
|
|
|
|
# Deployment and Update Process
|
|
|
|
This page describes how to deploy and update the LLM servers on Dankserver and
|
|
Dankclaw. The process uses Git as the single source of truth. A change to
|
|
SearXNG, Caddy, or another container service does **not** restart inference.
|
|
|
|
## Fleet
|
|
|
|
| Target | Host | Service | Port |
|
|
|---|---|---|---:|
|
|
| Dankserver | `dank@dankserver` | `frank-small` | 7770 |
|
|
| Dankclaw | `dank@dankclaw` | `frank-large` | 7770 |
|
|
| Dankclaw | `dank@dankclaw` | `frank-npu` (FLM on the XDNA2 NPU) | 7771 |
|
|
|
|
## Scoped workflow
|
|
|
|
```mermaid
|
|
graph LR
|
|
A[Edit LLM files] --> B[Run tests]
|
|
B --> C[Commit main]
|
|
C --> D[deploy.sh selects matching service]
|
|
D --> E[Target pulls ff-only]
|
|
E --> F[Relink one unit]
|
|
F --> G[Restart one inference service]
|
|
G --> H[Wait for that service health]
|
|
```
|
|
|
|
Run the unified deployer from the repository root:
|
|
|
|
```bash
|
|
./scripts/deploy.sh dankserver --service frank-small
|
|
./scripts/deploy.sh dankclaw --service frank-large
|
|
./scripts/deploy.sh dankclaw --dry-run
|
|
```
|
|
|
|
Without `--service`, `scripts/deploy-plan.py` maps changed files to the service
|
|
on the selected host. The following files select the corresponding service:
|
|
|
|
- `scripts/frank-small.env`, `frank-small.service`, and
|
|
`serve-frank-small.sh` select `frank-small` on Dankserver.
|
|
- `scripts/frank-large.env`, `frank-large.service`, and
|
|
`serve-frank-large.sh` select `frank-large` on Dankclaw.
|
|
- `scripts/frank-npu.env`, `frank-npu.service`,
|
|
and `serve-frank-npu.sh` select `frank-npu`
|
|
on Dankclaw.
|
|
- `scripts/chat-template.jinja` selects the LLM on the selected host only.
|
|
- Documentation, tests, deployment tooling, and unrelated container changes
|
|
do not select either inference service.
|
|
|
|
The target-side `scripts/deploy-remote.sh` relinks the selected unit into
|
|
`/etc/systemd/system/`, reloads systemd, enables it, restarts it, and waits for
|
|
that service's health endpoint (`http://127.0.0.1:7770/health` for the two
|
|
inference services; `/v1/version` for the NPU sight+sound endpoint — FLM has no
|
|
`/health` route). The wait budget is 20 minutes because cold model loads can be
|
|
slow. A failed handler does not advance its per-service state.
|
|
|
|
## Unit and artifact paths
|
|
|
|
Both units point into the stable target checkout:
|
|
|
|
```text
|
|
/home/dank/serverAppConfigs/scripts/frank-small.service
|
|
/home/dank/serverAppConfigs/scripts/frank-large.service
|
|
```
|
|
|
|
The unit files and serve scripts validate their model/runtime artifacts before
|
|
starting. Keep `REMOTE_DIR` stable; moving the checkout requires updating both
|
|
the unit absolute paths and the deployment tests in the same change.
|
|
|
|
## NPU unit
|
|
|
|
`frank-npu` is an inactive, historical Dankclaw endpoint. It is
|
|
relinked only when explicitly selected with `--service frank-npu`; the
|
|
large-model handler never restarts it. This is intentional: the NPU is not part
|
|
of the active inference path and should not be disturbed by a large-model
|
|
change.
|
|
|
|
## Isolation guarantee
|
|
|
|
The remote runner has one explicit service dispatch. It does not run a
|
|
host-wide LLM loop, `podman-compose up` for unrelated services, or a blanket
|
|
systemd restart. Each successful service is recorded under the target user's
|
|
state directory outside Git:
|
|
|
|
```text
|
|
~/.local/state/serverAppConfigs/deploy-state/
|
|
```
|
|
|
|
On a later ordinary deployment, a service remains pending only while its
|
|
recorded revision differs from the checkout and the planner still finds a
|
|
change in that service's inputs.
|
|
|
|
## Manual recovery
|
|
|
|
If the LLM handler fails after the target pull:
|
|
|
|
1. Inspect `journalctl -u frank-small -n 200` or
|
|
`journalctl -u frank-large -n 200`.
|
|
2. Confirm the model files, custom runtime binary, and chat template are
|
|
readable by `dank`.
|
|
3. Retry only the affected service:
|
|
|
|
```bash
|
|
./scripts/deploy.sh dankserver --service frank-small
|
|
# or
|
|
./scripts/deploy.sh dankclaw --service frank-large
|
|
```
|
|
|
|
4. Verify the matching health endpoint before declaring recovery.
|
|
|
|
Do not stop the other host's inference service during recovery. For benchmark
|
|
work, follow the separate stop/benchmark/restart procedure in [[Testing Policy]]
|
|
and the repository rules.
|
|
|
|
## See Also
|
|
|
|
- [Deployment Flow](../Architecture/Deployment%20Flow.md)
|
|
- [[index]]
|
|
- [Dankclaw deployment architecture](../Architecture/Network%20Architecture.md)
|