Frank
a5fe045b58
Structural repair: - 13 pages were the same document twice (each exactly 2x its HEAD size), a second frontmatter block whose closing fence had been glued onto a body line by a frontmatter rewrite that re-read its own output. Un-doubled by cutting at the second frontmatter and keeping the copy whose body is a superset; the orphan census reports 0 lost tokens against HEAD. - log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first — 66 dated headings merged into 34, titles demoted to bold leads, every content line verified present. The file had been descending at the head and ascending at the tail. Ownership moves: - New `LLM/frank-medium.md` is the locator for the retired Dankserver endpoint: era windows quoted from each era page's own banner, no numbers restated, and the two unresolved date conflicts recorded instead of silently resolved. `frank-archon.md` links to it instead of restating the pre-rename window. - The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone; `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor verdicts and cross-split spec decode, and the four links that pointed at the stubs were repointed at it or at the drafter page. - Authentik's three stub pages fold into `Authentik.md`, with the process detail and the outpost reference split out so the parent stays under the length hint. Verified corrections (host census + live probe, 2026-10-19): - The MXFP4 checkpoint two pages called the "on-disk rollback" is gone: /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on the era page that owns that payload; the swap page now links to it. - frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2 arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run census is kept once (155 requests) instead of twice (~120 and 155). - Fleet shapes re-confirmed against the live endpoints before touching any page: 3x200000 fp8 KV, 2x200704, 2x262144. Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
179 lines
7.8 KiB
Markdown
179 lines
7.8 KiB
Markdown
---
|
|
type: howto
|
|
title: LLM Deployment
|
|
description: Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot).
|
|
tags:
|
|
- llm
|
|
- deployment
|
|
- systemd
|
|
timestamp: 2026-09-03
|
|
---
|
|
|
|
# Deployment and Update Process
|
|
|
|
This page describes how to deploy and update the LLM servers on Dankserver and
|
|
Dankclaw. The process uses Git as the single source of truth. A change to
|
|
SearXNG, Caddy, or another container service does **not** restart inference.
|
|
|
|
## Fleet
|
|
|
|
| Target | Host | Service | Port |
|
|
|---|---|---|---:|
|
|
| Dankserver | `dank@dankserver` | `frank-archon` | 7770 |
|
|
| Dankclaw | `dank@dankclaw` | `frank-dragoon` | 7770 |
|
|
| Dankclaw | `dank@dankclaw` | `frank-zealot` | 7771 |
|
|
|
|
## Scoped workflow
|
|
|
|
```mermaid
|
|
graph LR
|
|
A[Edit LLM files] --> B[Run tests]
|
|
B --> C[Commit main]
|
|
C --> D[deploy.sh selects matching service]
|
|
D --> E[Target pulls ff-only]
|
|
E --> F[Relink one unit]
|
|
F --> G[Restart one inference service]
|
|
G --> H[Wait for that service health]
|
|
```
|
|
|
|
Run the unified deployer from the repository root:
|
|
|
|
```bash
|
|
./scripts/deploy.sh dankserver --service frank-archon
|
|
./scripts/deploy.sh dankclaw --service frank-dragoon
|
|
./scripts/deploy.sh dankclaw --dry-run
|
|
```
|
|
|
|
Without `--service`, `scripts/deploy-plan.py` maps changed files to the service
|
|
on the selected host. The following files select the corresponding service:
|
|
|
|
- `scripts/frank-archon.env`, `frank-archon.service`, and
|
|
`serve-frank-archon.sh` select `frank-archon` on Dankserver
|
|
(`scripts/deploy-plan.py:422-428`). The quadlet-era directory
|
|
`dankserver/frank-medium/` left the tree on 2026-09-26 and plans nothing
|
|
(`scripts/deploy-plan.py:374-377`).
|
|
- `scripts/qwen-sharp-v22.4.0.jinja` selects `frank-dragoon` — a planner rule
|
|
left from the llama.cpp era when dragoon loaded it via
|
|
`--chat-template-file` (`scripts/deploy-plan.py:415-421`).
|
|
- `scripts/frank-dragoon.env`, `frank-dragoon.service`, and
|
|
`serve-frank-dragoon.sh` select `frank-dragoon` on Dankclaw.
|
|
- `scripts/frank-zealot.env`, `frank-zealot.service`, and
|
|
`serve-frank-zealot.sh` select `frank-zealot` on Dankclaw.
|
|
- Documentation, tests, deployment tooling, and unrelated container changes
|
|
do not select any inference service.
|
|
|
|
The target-side `scripts/deploy-remote.sh` installs the selected service.
|
|
All three LLM services are **root units** whose repo unit file is relinked
|
|
into `/etc/systemd/system/` (`relink_unit`), daemon-reloaded, enabled and
|
|
restarted (`deploy-remote.sh:1158` `deploy_llm_dragoon`, `:1254`
|
|
`deploy_llm_zealot`, `:1315` `deploy_llm_archon`). The dragoon and archon
|
|
handlers wait for port 7770 to be released before the replacement starts
|
|
(`wait_port_free`; the 2026-09-16/26 cutover lessons). The archon handler
|
|
additionally hard-gates on the env file, the digest-pinned image ref, and the
|
|
payload before touching the unit (refuse rather than restart-loop). Each
|
|
handler then waits on that service's health endpoint —
|
|
`http://127.0.0.1:7770/health` for frank-dragoon (budget 1200 s) and
|
|
frank-archon (1800 s), `http://127.0.0.1:7771/health` for frank-zealot
|
|
(900 s) — and refuses a false green: the deploy is only green when the unit
|
|
itself reports a live MainPID behind the health answer
|
|
(`deploy-remote.sh:1395-1403`, `:1285`). A failed handler does not advance
|
|
its per-service state.
|
|
|
|
## Unit and artifact paths
|
|
|
|
The root units point into the stable target checkout:
|
|
```text
|
|
/home/dank/serverAppConfigs/scripts/frank-archon.service
|
|
/home/dank/serverAppConfigs/scripts/frank-dragoon.service
|
|
/home/dank/serverAppConfigs/scripts/frank-zealot.service
|
|
```
|
|
All three services keep their unit, env and serve script as this flat trio
|
|
under `scripts/` — there is no LLM Quadlet left (the `frank-medium`
|
|
quadlet-era directory left the tree 2026-09-26).
|
|
|
|
The unit files and serve scripts validate their model/runtime artifacts before
|
|
starting. Keep `REMOTE_DIR` stable; moving the checkout requires updating both
|
|
the unit absolute paths and the deployment tests in the same change.
|
|
|
|
## Retired endpoints
|
|
|
|
`frank-npu` (the FastFlowLM XDNA2 NPU endpoint) was retired on 2026-09-11
|
|
with the `amd_iommu=off` decision and is no longer a deployable service — the
|
|
planner has no `frank-npu` entry. Its runbook is [[frank-npu]].
|
|
|
|
## Isolation guarantee
|
|
|
|
The remote runner has one explicit service dispatch. It does not run a
|
|
host-wide LLM loop, `podman-compose up` for unrelated services, or a blanket
|
|
systemd restart. Each successful service is recorded under the target user's
|
|
state directory outside Git:
|
|
|
|
```text
|
|
~/.local/state/serverAppConfigs/deploy-state/
|
|
```
|
|
|
|
On a later ordinary deployment, a service remains pending only while its
|
|
recorded revision differs from the checkout and the planner still finds a
|
|
change in that service's inputs.
|
|
|
|
## Locally built images
|
|
|
|
`build_compose_image` in `scripts/deploy-remote.sh` builds the images that have
|
|
no registry digest — `comfyui-rocm`, `franksdank/loki`, `franksdank/alloy` and
|
|
the voice pair `franksdank/frank-tts` / `franksdank/frank-tts-engine` — and
|
|
**skips** the build when a `cfhash` label on the installed image matches a hash
|
|
it computes from the checkout now. That skip is what keeps the engine's
|
|
ten-minute `git clone --recursive` off every re-deploy, so the rule has to be
|
|
honest about what the image bakes:
|
|
|
|
- The hash covers the `Containerfile` **plus the files its `COPY` lines pull
|
|
in**, named as `extra=(…)` in each case arm. Hashing the `Containerfile`
|
|
alone was wrong in practice: editing `dankclaw/voiceservice/app.py` or
|
|
`tts-engine-warm.sh` left the label matching, `podman build` never ran, and
|
|
the redeployed container still carried the previous code while `/health`
|
|
answered — invisible to every gate. Corrected 2026-10-18;
|
|
`tests/frank_tts_engine.bats` drives the function against a stub `podman`
|
|
(first deploy builds, unchanged tree does not, a `COPY`'d file rebuilds).
|
|
- A base-image assumption is part of the image's identity too. The engine's
|
|
runtime stage creates its unprivileged user with `useradd`, not `adduser`,
|
|
because the pinned `ubuntu:24.04` base ships no `adduser` — checked by
|
|
running the command against the digest on 2026-10-18. That failure mode is a
|
|
build that dies at the `RUN` line, so it is loud, unlike the stale-hash one.
|
|
|
|
Both engines keep serving during a rebuild: `build_compose_image` runs after
|
|
`install_quadlet` only for the LLM-style handlers, and the voice deploy gates on
|
|
recorded health rather than a fixed sleep — see [[Frank TTS Engine]].
|
|
|
|
## Manual recovery
|
|
|
|
If the LLM handler fails after the target pull:
|
|
|
|
1. Inspect `journalctl -u frank-dragoon -n 200`, `journalctl -u frank-zealot
|
|
-n 200` or `journalctl -u frank-archon -n 200` (all three are root units);
|
|
frank-dragoon's engine boot ledger is in `/home/dank/logs/frank_dragoon.log`,
|
|
frank-archon's in `podman logs frank-archon` — not in the journal.
|
|
2. Confirm the model files, the runtime binary and (for frank-archon) the
|
|
digest-pinned container image are present and readable by `dank`.
|
|
3. Retry only the affected service:
|
|
|
|
```bash
|
|
./scripts/deploy.sh dankserver --service frank-archon
|
|
./scripts/deploy.sh dankclaw --service frank-dragoon
|
|
./scripts/deploy.sh dankclaw --service frank-zealot
|
|
```
|
|
4. Verify the matching health endpoint before declaring recovery.
|
|
|
|
Do not stop the other host's inference service during recovery. The two
|
|
Dankclaw endpoints are independent processes on different devices: restarting
|
|
frank-dragoon does not disturb frank-zealot, and the reverse holds too — with one
|
|
exception worth knowing, that the deployer waits for port 7770 to be released
|
|
before it starts the replacement container. For benchmark work, follow the
|
|
separate stop/benchmark/restart procedure in [[Testing Policy]] and the
|
|
repository rules.
|
|
|
|
## See Also
|
|
|
|
- [Deployment Flow](../Architecture/Deployment%20Flow.md)
|
|
- [[index]]
|
|
- [Dankclaw deployment architecture](../Architecture/Network%20Architecture.md)
|