Frank
35c44c4123
Retire vllm-awq-qwen36 (AWQ) from target maps, README, deployment, KB index, Dankserver specs, DFlash and FP8 docs. New dankserver runtime: upstream llama.cpp, port 7770, alias frank-default-small, 3x262144 ctx, DFlash Q8_0 draft, MoE experts offloaded to CPU, attention+KV on GPU (K q8_0, V f16). deploy.sh now maps dankserver to frank-default-small and symlinks its unit. models.json: dankserver baseUrl -> :7770, contextWindow -> 262144.
63 lines
2.8 KiB
Markdown
63 lines
2.8 KiB
Markdown
# AGENTS.md — vllmScripts Agent Rules
|
|
|
|
## Overview
|
|
|
|
This project manages deployment scripts for two LLM inference servers. Dankclaw runs a vLLM-ROCm service for Ling-3.0-flash (CIRU INT4). Dankserver runs a llama.cpp service for Qwen3.6-35B-A3B (HauhauCS uncensored Q6_K_P) with a DFlash draft and CPU-offloaded MoE experts. See [docs/deployment.md](docs/deployment.md) for the full deployment and update process.
|
|
|
|
## Git Workflow
|
|
|
|
All script edits must flow through git. Do not edit files on the remote machine directly.
|
|
|
|
### Protocol
|
|
|
|
1. **Source the script from the target machine** — fetch it from the server where it runs
|
|
2. **Place it in this repo** — commit the fetched version
|
|
3. **Edit here** — make all changes locally in the working tree
|
|
4. **Commit and push** — push to origin (Gitea on dankclaw)
|
|
5. **Pull on the target** — `git pull` on the remote machine before restarting
|
|
6. **Restart the service** — use the updated script on the target
|
|
|
|
## Deployment & Update Process
|
|
|
|
Use `scripts/deploy.sh` to deploy committed changes to a target. It pushes `main`, pulls on the target, restarts the service, and checks the health endpoint.
|
|
|
|
```
|
|
./scripts/deploy.sh dankclaw # vLLM (frank-default-large, port 7770)
|
|
./scripts/deploy.sh dankserver # llama.cpp (frank-default-small, port 7770)
|
|
./scripts/deploy.sh dankclaw --dry-run # preview without changes
|
|
```
|
|
|
|
Deployment workflow:
|
|
|
|
```mermaid
|
|
graph LR
|
|
A[Edit locally] --> B[Commit]
|
|
B --> C[Push to origin]
|
|
C --> D[deploy.sh: pull on target]
|
|
D --> E[restart service]
|
|
E --> F[health check]
|
|
```
|
|
|
|
Target map:
|
|
|
|
| Target | Host | Runtime | Service | Port |
|
|
|--------|------|---------|---------|------|
|
|
| Dankclaw | dank@dankclaw | vLLM-ROCm (CIRU INT4) | frank-default-large | 7770 |
|
|
| Dankserver | dank@dankserver | llama.cpp (Q6_K_P + DFlash, MoE offload) | frank-default-small | 7770 |
|
|
|
|
## Why
|
|
|
|
Direct edits on the remote drift from version control. Future agents will not know what changed. Git is the single source of truth.
|
|
|
|
> User request → Fetch script from remote → Commit here → Edit → Commit → Push → Pull on remote → Restart
|
|
|
|
## Summary
|
|
|
|
| Context | Action |
|
|
|---------|--------|
|
|
| Editing a deployment script | **Work through git** — fetch, commit, edit, push, pull, restart. Never edit on the remote directly. |
|
|
| Script exists only on remote | **Pull it into the repo first** — commit the fetched version before making changes. |
|
|
| Unstaged local changes | **Commit or discard** — check `git status` before starting work. Ask the user if unsure. |
|
|
| Deploying a committed change | **Run `scripts/deploy.sh <target>`** — it pushes, pulls on the target, restarts the service, and checks health. See [docs/deployment.md](docs/deployment.md). |
|
|
| Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
|