Add AGENTS.md, deploy script, update README with service status and file locations

This commit is contained in:
Frank committed 2026-08-09 09:47:24 -07:00
1 parent 469bfaf979
commit 6bddf0a72a
3 files changed
+212 -4

No files matched your search

Executable
+98
View File
@@ -0,0 +1,98 @@
# vLLM Scripts — Agent Instructions
## Project Overview
Scripts for running large language models via vLLM on two local machines:
- **dankserver**: NVIDIA GPU + System RAM (frank-default-small)
- **dankclaw**: AMD Strix Halo Shared Memory (frank-default-large)
## Directory Structure
```
vllmScripts/
├── AGENTS.md # This file — agent instructions
├── README.md # Human-readable project overview
├── deploy.sh # Automated deployment script
├── scripts/ # Executable vLLM launch scripts
│ ├── validate-vllm.sh # Validation tool
│ └── *.sh # Inference server scripts
├── tests/ # Test suite
│ └── test_validator.sh
├── docs/ # Knowledge base
│ ├── README.md
│ └── *.md # Research notes
└── failedAttempts/ # Archived scripts (for reference)
```
## Running Tests
```bash
# Test validator on dankserver
bash tests/test_validator.sh dankserver
# Test validator on dankclaw
bash tests/test_validator.sh dankclaw
# Run all tests
bash tests/test_validator.sh dankserver && bash tests/test_validator.sh dankclaw
```
## Deployment
```bash
# Deploy all scripts to all targets
bash deploy.sh all
# Deploy specific script to dankserver
bash deploy.sh dankserver vllm-qwen36.sh
# Deploy specific script to dankclaw
bash deploy.sh dankclaw vllm-thinkingcap.sh
```
## Git Workflow
### Branching Strategy
- `main`: Stable, working scripts only
- `vllm-qwen36`: Active development for Qwen3.6 models
- Feature branches: `feature/<description>` for experiments
### Worktree Pattern
For parallel development:
```bash
# Create worktree for feature
git worktree add ../vllm-feature feature/qwen36-awq
# When done, merge back
git worktree remove ../vllm-feature
```
### Commit Guidelines
- Use descriptive commit messages: `git commit -m "Add AWQ DFlash script"`
- Commit before deploying: `git add -A && git commit -m "Update for deployment"`
- Push to remote: `git push origin <branch>`
### Deployment Flow
1. Make changes locally
2. Run tests: `bash tests/test_validator.sh dankserver`
3. Commit: `git add -A && git commit -m "Description"`
4. Deploy: `bash deploy.sh dankserver`
5. Verify: `bash scripts/validate-vllm.sh -t dankserver`
## Remote Servers
### dankserver
- SSH: `dank@dankserver`
- Scripts: `~/vllm/`
- Service: `vllm-awq-qwen36.service`
- Model: frank-default-small (AWQ 4bit)
- Port: 34401
### dankclaw
- SSH: `dank@dankclaw`
- Scripts: `~/vllm/`
- Service: `vllm-thinkingcap.service`
- Model: frank-default-large (FP8)
- Port: 3090
## Troubleshooting
- Service failed: `systemctl status <service>`
- Container stuck: `docker stop <name> && docker rm <name>`
- OOM error: Reduce `--max-model-len` or increase `--cpu-offload-gb`
- Prefix caching conflict: Remove `--enable-prefix-caching` from script