serverAppConfigs/serverAppConfigs-docs/LLM/deployment.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
howto	LLM Deployment	Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot).	llm deployment systemd	2026-09-03

Deployment and Update Process

This page describes how to deploy and update the LLM servers on Dankserver and Dankclaw. The process uses Git as the single source of truth. A change to SearXNG, Caddy, or another container service does not restart inference.

Fleet

Target	Host	Service	Port
Dankserver	dank@dankserver	frank-archon	7770
Dankclaw	dank@dankclaw	frank-dragoon	7770
Dankclaw	dank@dankclaw	frank-zealot	7771

Scoped workflow

graph LR
    A[Edit LLM files] --> B[Run tests]
    B --> C[Commit main]
    C --> D[deploy.sh selects matching service]
    D --> E[Target pulls ff-only]
    E --> F[Relink one unit]
    F --> G[Restart one inference service]
    G --> H[Wait for that service health]

Run the unified deployer from the repository root:

./scripts/deploy.sh dankserver --service frank-archon
./scripts/deploy.sh dankclaw --service frank-dragoon
./scripts/deploy.sh dankclaw --dry-run

Without --service, scripts/deploy-plan.py maps changed files to the service on the selected host. The following files select the corresponding service:

• scripts/frank-archon.env, frank-archon.service, and serve-frank-archon.sh select frank-archon on Dankserver (scripts/deploy-plan.py:422-428). The quadlet-era directory dankserver/frank-medium/ left the tree on 2026-09-26 and plans nothing (scripts/deploy-plan.py:374-377).
• scripts/qwen-sharp-v22.4.0.jinja selects frank-dragoon — a planner rule left from the llama.cpp era when dragoon loaded it via --chat-template-file (scripts/deploy-plan.py:415-421).
• scripts/frank-dragoon.env, frank-dragoon.service, and serve-frank-dragoon.sh select frank-dragoon on Dankclaw.
• scripts/frank-zealot.env, frank-zealot.service, and serve-frank-zealot.sh select frank-zealot on Dankclaw.
• Documentation, tests, deployment tooling, and unrelated container changes do not select any inference service.

The target-side scripts/deploy-remote.sh installs the selected service. All three LLM services are root units whose repo unit file is relinked into /etc/systemd/system/ (relink_unit), daemon-reloaded, enabled and restarted (deploy-remote.sh:1158 deploy_llm_dragoon, :1254 deploy_llm_zealot, :1315 deploy_llm_archon). The dragoon and archon handlers wait for port 7770 to be released before the replacement starts (wait_port_free; the 2026-09-16/26 cutover lessons). The archon handler additionally hard-gates on the env file, the digest-pinned image ref, and the payload before touching the unit (refuse rather than restart-loop). Each handler then waits on that service's health endpoint — http://127.0.0.1:7770/health for frank-dragoon (budget 1200 s) and frank-archon (1800 s), http://127.0.0.1:7771/health for frank-zealot (900 s) — and refuses a false green: the deploy is only green when the unit itself reports a live MainPID behind the health answer (deploy-remote.sh:1395-1403, :1285). A failed handler does not advance its per-service state.

Unit and artifact paths

The root units point into the stable target checkout:

/home/dank/serverAppConfigs/scripts/frank-archon.service
/home/dank/serverAppConfigs/scripts/frank-dragoon.service
/home/dank/serverAppConfigs/scripts/frank-zealot.service

All three services keep their unit, env and serve script as this flat trio under scripts/ — there is no LLM Quadlet left (the frank-medium quadlet-era directory left the tree 2026-09-26).

The unit files and serve scripts validate their model/runtime artifacts before starting. Keep REMOTE_DIR stable; moving the checkout requires updating both the unit absolute paths and the deployment tests in the same change.

Retired endpoints

frank-npu (the FastFlowLM XDNA2 NPU endpoint) was retired on 2026-09-11 with the amd_iommu=off decision and is no longer a deployable service — the planner has no frank-npu entry. Its runbook is frank-npu.

Isolation guarantee

The remote runner has one explicit service dispatch. It does not run a host-wide LLM loop, podman-compose up for unrelated services, or a blanket systemd restart. Each successful service is recorded under the target user's state directory outside Git:

~/.local/state/serverAppConfigs/deploy-state/

On a later ordinary deployment, a service remains pending only while its recorded revision differs from the checkout and the planner still finds a change in that service's inputs.

Locally built images

build_compose_image in scripts/deploy-remote.sh builds the images that have no registry digest — comfyui-rocm, franksdank/loki, franksdank/alloy and the voice pair franksdank/frank-tts / franksdank/frank-tts-engine — and skips the build when a cfhash label on the installed image matches a hash it computes from the checkout now. That skip is what keeps the engine's ten-minute git clone --recursive off every re-deploy, so the rule has to be honest about what the image bakes:

• The hash covers the Containerfile plus the files its COPY lines pull in, named as extra=(…) in each case arm. Hashing the Containerfile alone was wrong in practice: editing dankclaw/voiceservice/app.py or tts-engine-warm.sh left the label matching, podman build never ran, and the redeployed container still carried the previous code while /health answered — invisible to every gate. Corrected 2026-10-18; tests/frank_tts_engine.bats drives the function against a stub podman (first deploy builds, unchanged tree does not, a COPY'd file rebuilds).
• A base-image assumption is part of the image's identity too. The engine's runtime stage creates its unprivileged user with useradd, not adduser, because the pinned ubuntu:24.04 base ships no adduser — checked by running the command against the digest on 2026-10-18. That failure mode is a build that dies at the RUN line, so it is loud, unlike the stale-hash one.

Both engines keep serving during a rebuild: build_compose_image runs after install_quadlet only for the LLM-style handlers, and the voice deploy gates on recorded health rather than a fixed sleep — see Frank TTS Engine.

Manual recovery

If the LLM handler fails after the target pull:

1. Inspect journalctl -u frank-dragoon -n 200, journalctl -u frank-zealot -n 200 or journalctl -u frank-archon -n 200 (all three are root units); frank-dragoon's engine boot ledger is in /home/dank/logs/frank_dragoon.log, frank-archon's in podman logs frank-archon — not in the journal.

2. Confirm the model files, the runtime binary and (for frank-archon) the digest-pinned container image are present and readable by dank.

3. Retry only the affected service:

   ./scripts/deploy.sh dankserver --service frank-archon
   ./scripts/deploy.sh dankclaw --service frank-dragoon
   ./scripts/deploy.sh dankclaw --service frank-zealot
   

4. Verify the matching health endpoint before declaring recovery.

Do not stop the other host's inference service during recovery. The two Dankclaw endpoints are independent processes on different devices: restarting frank-dragoon does not disturb frank-zealot, and the reverse holds too — with one exception worth knowing, that the deployer waits for port 7770 to be released before it starts the replacement container. For benchmark work, follow the separate stop/benchmark/restart procedure in Testing Policy and the repository rules.

See Also

• Deployment Flow
• index
• Dankclaw deployment architecture