Add AGENTS.md, deploy script, update README with service status and file locations
This commit is contained in:
1 parent
469bfaf979
commit
6bddf0a72a
3 files changed
+212
-4
No files matched your search
@@ -0,0 +1,98 @@
|
||||
# vLLM Scripts — Agent Instructions
|
||||
|
||||
## Project Overview
|
||||
Scripts for running large language models via vLLM on two local machines:
|
||||
- **dankserver**: NVIDIA GPU + System RAM (frank-default-small)
|
||||
- **dankclaw**: AMD Strix Halo Shared Memory (frank-default-large)
|
||||
|
||||
## Directory Structure
|
||||
```
|
||||
vllmScripts/
|
||||
├── AGENTS.md # This file — agent instructions
|
||||
├── README.md # Human-readable project overview
|
||||
├── deploy.sh # Automated deployment script
|
||||
├── scripts/ # Executable vLLM launch scripts
|
||||
│ ├── validate-vllm.sh # Validation tool
|
||||
│ └── *.sh # Inference server scripts
|
||||
├── tests/ # Test suite
|
||||
│ └── test_validator.sh
|
||||
├── docs/ # Knowledge base
|
||||
│ ├── README.md
|
||||
│ └── *.md # Research notes
|
||||
└── failedAttempts/ # Archived scripts (for reference)
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
```bash
|
||||
# Test validator on dankserver
|
||||
bash tests/test_validator.sh dankserver
|
||||
|
||||
# Test validator on dankclaw
|
||||
bash tests/test_validator.sh dankclaw
|
||||
|
||||
# Run all tests
|
||||
bash tests/test_validator.sh dankserver && bash tests/test_validator.sh dankclaw
|
||||
```
|
||||
|
||||
## Deployment
|
||||
```bash
|
||||
# Deploy all scripts to all targets
|
||||
bash deploy.sh all
|
||||
|
||||
# Deploy specific script to dankserver
|
||||
bash deploy.sh dankserver vllm-qwen36.sh
|
||||
|
||||
# Deploy specific script to dankclaw
|
||||
bash deploy.sh dankclaw vllm-thinkingcap.sh
|
||||
```
|
||||
|
||||
## Git Workflow
|
||||
|
||||
### Branching Strategy
|
||||
- `main`: Stable, working scripts only
|
||||
- `vllm-qwen36`: Active development for Qwen3.6 models
|
||||
- Feature branches: `feature/<description>` for experiments
|
||||
|
||||
### Worktree Pattern
|
||||
For parallel development:
|
||||
```bash
|
||||
# Create worktree for feature
|
||||
git worktree add ../vllm-feature feature/qwen36-awq
|
||||
|
||||
# When done, merge back
|
||||
git worktree remove ../vllm-feature
|
||||
```
|
||||
|
||||
### Commit Guidelines
|
||||
- Use descriptive commit messages: `git commit -m "Add AWQ DFlash script"`
|
||||
- Commit before deploying: `git add -A && git commit -m "Update for deployment"`
|
||||
- Push to remote: `git push origin <branch>`
|
||||
|
||||
### Deployment Flow
|
||||
1. Make changes locally
|
||||
2. Run tests: `bash tests/test_validator.sh dankserver`
|
||||
3. Commit: `git add -A && git commit -m "Description"`
|
||||
4. Deploy: `bash deploy.sh dankserver`
|
||||
5. Verify: `bash scripts/validate-vllm.sh -t dankserver`
|
||||
|
||||
## Remote Servers
|
||||
|
||||
### dankserver
|
||||
- SSH: `dank@dankserver`
|
||||
- Scripts: `~/vllm/`
|
||||
- Service: `vllm-awq-qwen36.service`
|
||||
- Model: frank-default-small (AWQ 4bit)
|
||||
- Port: 34401
|
||||
|
||||
### dankclaw
|
||||
- SSH: `dank@dankclaw`
|
||||
- Scripts: `~/vllm/`
|
||||
- Service: `vllm-thinkingcap.service`
|
||||
- Model: frank-default-large (FP8)
|
||||
- Port: 3090
|
||||
|
||||
## Troubleshooting
|
||||
- Service failed: `systemctl status <service>`
|
||||
- Container stuck: `docker stop <name> && docker rm <name>`
|
||||
- OOM error: Reduce `--max-model-len` or increase `--cpu-offload-gb`
|
||||
- Prefix caching conflict: Remove `--enable-prefix-caching` from script
|
||||
Reference in new issue
Block a user