Frank
903d54c389
Update target maps (AGENTS, README, deployment, deploy.sh) from the retired ROCmFPX llama.cpp stack to the active vLLM-ROCm CIRU INT4 service on port 7770 with MTP speculative decoding. Align Dankserver to the running vllm-awq-qwen36 (AWQ, port 34401). Refresh KB pages: Dankclaw system specs (gpu-mem-util 0.95, CIRU venv), quantization choices (INT4 primary, MXFP4 experiment), DFlash/DSpark Dankclaw section, and the KB index topology.
63 lines
2.7 KiB
Markdown
63 lines
2.7 KiB
Markdown
# AGENTS.md — vllmScripts Agent Rules
|
|
|
|
## Overview
|
|
|
|
This project manages deployment scripts for two LLM inference servers. Both run vLLM. Dankclaw runs a vLLM-ROCm service for Ling-3.0-flash (CIRU INT4). Dankserver runs a vLLM service for Qwen3.6. See [docs/deployment.md](docs/deployment.md) for the full deployment and update process.
|
|
|
|
## Git Workflow
|
|
|
|
All script edits must flow through git. Do not edit files on the remote machine directly.
|
|
|
|
### Protocol
|
|
|
|
1. **Source the script from the target machine** — fetch it from the server where it runs
|
|
2. **Place it in this repo** — commit the fetched version
|
|
3. **Edit here** — make all changes locally in the working tree
|
|
4. **Commit and push** — push to origin (Gitea on dankclaw)
|
|
5. **Pull on the target** — `git pull` on the remote machine before restarting
|
|
6. **Restart the service** — use the updated script on the target
|
|
|
|
## Deployment & Update Process
|
|
|
|
Use `scripts/deploy.sh` to deploy committed changes to a target. It pushes `main`, pulls on the target, restarts the service, and checks the health endpoint.
|
|
|
|
```
|
|
./scripts/deploy.sh dankclaw # vLLM (frank-default-large, port 7770)
|
|
./scripts/deploy.sh dankserver # vLLM (vllm-awq-qwen36, port 34401)
|
|
./scripts/deploy.sh dankclaw --dry-run # preview without changes
|
|
```
|
|
|
|
Deployment workflow:
|
|
|
|
```mermaid
|
|
graph LR
|
|
A[Edit locally] --> B[Commit]
|
|
B --> C[Push to origin]
|
|
C --> D[deploy.sh: pull on target]
|
|
D --> E[restart service]
|
|
E --> F[health check]
|
|
```
|
|
|
|
Target map:
|
|
|
|
| Target | Host | Runtime | Service | Port |
|
|
|--------|------|---------|---------|------|
|
|
| Dankclaw | dank@dankclaw | vLLM-ROCm (CIRU INT4) | frank-default-large | 7770 |
|
|
| Dankserver | dank@dankserver | vLLM (Docker) | vllm-awq-qwen36 | 34401 |
|
|
|
|
## Why
|
|
|
|
Direct edits on the remote drift from version control. Future agents will not know what changed. Git is the single source of truth.
|
|
|
|
> User request → Fetch script from remote → Commit here → Edit → Commit → Push → Pull on remote → Restart
|
|
|
|
## Summary
|
|
|
|
| Context | Action |
|
|
|---------|--------|
|
|
| Editing a deployment script | **Work through git** — fetch, commit, edit, push, pull, restart. Never edit on the remote directly. |
|
|
| Script exists only on remote | **Pull it into the repo first** — commit the fetched version before making changes. |
|
|
| Unstaged local changes | **Commit or discard** — check `git status` before starting work. Ask the user if unsure. |
|
|
| Deploying a committed change | **Run `scripts/deploy.sh <target>`** — it pushes, pulls on the target, restarts the service, and checks health. See [docs/deployment.md](docs/deployment.md). |
|
|
| Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
|