dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: reflect Ling-3 vLLM (frank-default-large) on Dankclaw

Browse Source 
Update target maps (AGENTS, README, deployment, deploy.sh) from the retired
ROCmFPX llama.cpp stack to the active vLLM-ROCm CIRU INT4 service on port 7770
with MTP speculative decoding. Align Dankserver to the running
vllm-awq-qwen36 (AWQ, port 34401). Refresh KB pages: Dankclaw system specs
(gpu-mem-util 0.95, CIRU venv), quantization choices (INT4 primary, MXFP4
experiment), DFlash/DSpark Dankclaw section, and the KB index topology.
This commit is contained in:
Frank committed 2026-08-10 23:37:27 -07:00
1 parent 52ad985978
commit 903d54c389
9 files changed
+55 -54
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/deployment.md
+10 -10
View File
@@ -8,8 +8,8 @@ Two servers run LLM inference services. Each server pulls its scripts from this
| Target | Host | Runtime | Service | Port |
|--------|------|---------|---------|------|
| Dankclaw | dank@dankclaw | ROCmFPX llama.cpp | llama-server | 8080 |
| Dankserver | dank@dankserver | vLLM (Docker) | vllm-qwen36 | 8000 |
| Dankclaw | dank@dankclaw | vLLM-ROCm (CIRU INT4) | frank-default-large | 7770 |
| Dankserver | dank@dankserver | vLLM (Docker) | vllm-awq-qwen36 | 34401 |
## Workflow
@@ -77,10 +77,10 @@ Run these commands on Dankclaw:
```
cd /home/dank/vllmScripts
git pull origin main
sudo ln -sf /home/dank/vllmScripts/scripts/llama-server.service /etc/systemd/system/llama-server.service
sudo ln -sf /home/dank/vllmScripts/scripts/frank-default-large.service /etc/systemd/system/frank-default-large.service
sudo systemctl daemon-reload
sudo systemctl restart llama-server
curl -sf http://127.0.0.1:8080/health
sudo systemctl restart frank-default-large
curl -sf http://127.0.0.1:7770/health
```
### Update Dankserver
@@ -97,15 +97,15 @@ curl -sf http://127.0.0.1:8000/health
## Service Units
### Dankclaw — llama-server
### Dankclaw — frank-default-large
The active service on Dankclaw runs the ROCmFPX llama.cpp runtime. It serves the Ling-3.0-flash ROCmFP4 MTP model with `draft-mtp` speculative decoding. The unit file is `scripts/llama-server.service` in the repository. The machine symlinks it to `/etc/systemd/system/llama-server.service`.
The active service on Dankclaw runs a vLLM-ROCm runtime for the Ling-3.0-flash CIRU INT4 (W4A16) model. It serves under the alias `frank-default-large` on port 7770 with native MTP speculative decoding. The unit file is `scripts/frank-default-large.service` in the repository. The machine symlinks it to `/etc/systemd/system/frank-default-large.service`. The service launches through `scripts/serve-ling3-vllm.sh`, which sources the CIRU runtime and passes the production settings (host `0.0.0.0`, port 7770, gpu memory utilization 0.95).
### Dankserver — vllm-qwen36
### Dankserver — vllm-awq-qwen36
The vLLM service on Dankserver runs in a Docker container. The unit file is a physical file on the machine, not a symlink into the repository. The `vllm-qwen36.sh` launcher in the repository passes the model and flags to `docker run`.
The active vLLM service on Dankserver runs in a Podman/Docker container and serves the Qwen3.6-35B-A3B AWQ 4-bit model on port 34401. The unit file is a physical file on the machine, not a symlink into the repository.
> Note: a second unit, `vllm-awq-qwen36.service`, currently runs the AWQ 4-bit model on port 34401. Verify the intended service before a restart. See the repository `git log` for the latest decision.
> Note: the failed `vllm-qwen36.service` (FP8) and the retired `llama-server.service` (ROCmFPX llama.cpp) are not active. Verify the intended service before a restart. See the repository `git log` for the latest decision.
## Validation