Frank
903d54c389
Update target maps (AGENTS, README, deployment, deploy.sh) from the retired ROCmFPX llama.cpp stack to the active vLLM-ROCm CIRU INT4 service on port 7770 with MTP speculative decoding. Align Dankserver to the running vllm-awq-qwen36 (AWQ, port 34401). Refresh KB pages: Dankclaw system specs (gpu-mem-util 0.95, CIRU venv), quantization choices (INT4 primary, MXFP4 experiment), DFlash/DSpark Dankclaw section, and the KB index topology.
134 lines
4.4 KiB
Markdown
134 lines
4.4 KiB
Markdown
# Deployment and Update Process
|
|
|
|
This page describes how to deploy and update the LLM servers on Dankserver and Dankclaw. The process uses git as the single source of truth. All script edits flow through this repository.
|
|
|
|
## Overview
|
|
|
|
Two servers run LLM inference services. Each server pulls its scripts from this repository. The deployment process moves a committed change from the repository to a running service.
|
|
|
|
| Target | Host | Runtime | Service | Port |
|
|
|--------|------|---------|---------|------|
|
|
| Dankclaw | dank@dankclaw | vLLM-ROCm (CIRU INT4) | frank-default-large | 7770 |
|
|
| Dankserver | dank@dankserver | vLLM (Docker) | vllm-awq-qwen36 | 34401 |
|
|
|
|
## Workflow
|
|
|
|
The deployment workflow has six steps.
|
|
|
|
```mermaid
|
|
graph LR
|
|
A[Edit scripts locally] --> B[Commit]
|
|
B --> C[Push to origin<br>Gitea on Dankclaw]
|
|
C --> D[Pull on target<br>git pull]
|
|
D --> E[Restart systemd service]
|
|
E --> F[Verify health endpoint]
|
|
```
|
|
|
|
1. Fetch the current script from the target machine.
|
|
2. Commit the fetched version in this repository.
|
|
3. Edit the script locally in the working tree.
|
|
4. Commit and push the change to origin.
|
|
5. Pull the change on the target machine.
|
|
6. Restart the service and verify that it is healthy.
|
|
|
|
## Deploy Script
|
|
|
|
The `scripts/deploy.sh` script executes steps 4 through 6. Run it from the repository root.
|
|
|
|
Run this command to deploy to Dankclaw:
|
|
|
|
```
|
|
./scripts/deploy.sh dankclaw
|
|
```
|
|
|
|
Run this command to deploy to Dankserver:
|
|
|
|
```
|
|
./scripts/deploy.sh dankserver
|
|
```
|
|
|
|
Add the `--dry-run` flag to preview without making changes:
|
|
|
|
```
|
|
./scripts/deploy.sh dankclaw --dry-run
|
|
```
|
|
|
|
The script does these actions:
|
|
|
|
1. Verifies that the local working tree has no uncommitted changes.
|
|
2. Pushes the `main` branch to origin.
|
|
3. Pulls the repository on the target machine.
|
|
4. Installs or updates the systemd unit.
|
|
5. Reloads systemd and restarts the service.
|
|
6. Waits for the health endpoint to respond.
|
|
|
|
## Manual Update
|
|
|
|
If you cannot run the deploy script, update a server manually. Complete each step on the target machine.
|
|
|
|
### Update Dankclaw
|
|
|
|
1. Pull the repository.
|
|
2. Symlink the unit file, if it changed.
|
|
3. Reload and restart the service.
|
|
|
|
Run these commands on Dankclaw:
|
|
|
|
```
|
|
cd /home/dank/vllmScripts
|
|
git pull origin main
|
|
sudo ln -sf /home/dank/vllmScripts/scripts/frank-default-large.service /etc/systemd/system/frank-default-large.service
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl restart frank-default-large
|
|
curl -sf http://127.0.0.1:7770/health
|
|
```
|
|
|
|
### Update Dankserver
|
|
|
|
Run these commands on Dankserver:
|
|
|
|
```
|
|
cd /home/dank/vLLM
|
|
git pull origin main
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl restart vllm-qwen36
|
|
curl -sf http://127.0.0.1:8000/health
|
|
```
|
|
|
|
## Service Units
|
|
|
|
### Dankclaw — frank-default-large
|
|
|
|
The active service on Dankclaw runs a vLLM-ROCm runtime for the Ling-3.0-flash CIRU INT4 (W4A16) model. It serves under the alias `frank-default-large` on port 7770 with native MTP speculative decoding. The unit file is `scripts/frank-default-large.service` in the repository. The machine symlinks it to `/etc/systemd/system/frank-default-large.service`. The service launches through `scripts/serve-ling3-vllm.sh`, which sources the CIRU runtime and passes the production settings (host `0.0.0.0`, port 7770, gpu memory utilization 0.95).
|
|
|
|
### Dankserver — vllm-awq-qwen36
|
|
|
|
The active vLLM service on Dankserver runs in a Podman/Docker container and serves the Qwen3.6-35B-A3B AWQ 4-bit model on port 34401. The unit file is a physical file on the machine, not a symlink into the repository.
|
|
|
|
> Note: the failed `vllm-qwen36.service` (FP8) and the retired `llama-server.service` (ROCmFPX llama.cpp) are not active. Verify the intended service before a restart. See the repository `git log` for the latest decision.
|
|
|
|
## Validation
|
|
|
|
The deploy script checks the health endpoint after a restart. For deeper checks, run the validation script on the target.
|
|
|
|
Run this command on Dankserver to test vLLM throughput:
|
|
|
|
```
|
|
./scripts/validate-vllm.sh
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
If the health check fails after a deploy:
|
|
|
|
1. Read the service logs with `journalctl -u <service> -n 200`.
|
|
2. Confirm that the model files exist in the cache.
|
|
3. Confirm that the GPU has free memory.
|
|
4. Check the service status with `systemctl status <service>`.
|
|
|
|
## See Also
|
|
|
|
- [AGENTS.md](../AGENTS.md) — the git workflow rules
|
|
- [README.md](../README.md) — repository overview
|
|
- [index.md](index.md) — knowledge base index
|