# vLLM Scripts — Agent Instructions

## Project Overview
Scripts for running large language models via vLLM on two local machines:
- **dankserver**: NVIDIA GPU + System RAM (frank-default-small)
- **dankclaw**: AMD Strix Halo Shared Memory (frank-default-large)

## Directory Structure
```
vllmScripts/
├── AGENTS.md          # This file — agent instructions
├── README.md          # Human-readable project overview
├── deploy.sh          # Automated deployment script
├── scripts/           # Executable vLLM launch scripts
│   ├── validate-vllm.sh  # Validation tool
│   └── *.sh             # Inference server scripts
├── tests/             # Test suite
│   └── test_validator.sh
├── docs/              # Knowledge base
│   ├── README.md
│   └── *.md           # Research notes
└── failedAttempts/    # Archived scripts (for reference)
```

## Running Tests
```bash
# Test validator on dankserver
bash tests/test_validator.sh dankserver

# Test validator on dankclaw
bash tests/test_validator.sh dankclaw

# Run all tests
bash tests/test_validator.sh dankserver && bash tests/test_validator.sh dankclaw
```

## Deployment
```bash
# Deploy all scripts to all targets
bash deploy.sh all

# Deploy specific script to dankserver
bash deploy.sh dankserver vllm-qwen36.sh

# Deploy specific script to dankclaw
bash deploy.sh dankclaw vllm-thinkingcap.sh
```

## Git Workflow

### Branching Strategy
- `main`: Stable, working scripts only
- `dankserver/*`: Server-specific development
  - `dankserver/qwen36-awq`: Qwen3.6 AWQ experiments
  - `dankserver/*`: Feature branches for dankserver
- `dankclaw/*`: Claw-specific development
  - `dankclaw/*`: Feature branches for dankclaw

### Worktree Pattern
For parallel development:
```bash
# Create worktree for feature
git worktree add ../vllm-feature feature/qwen36-awq

# When done, merge back
git worktree remove ../vllm-feature
```

### Commit Guidelines
- Use descriptive commit messages: `git commit -m "Add AWQ DFlash script"`
- Commit before deploying: `git add -A && git commit -m "Update for deployment"`
- Push to remote: `git push origin <branch>`

### Deployment Flow
1. Make changes locally
2. Run tests: `bash tests/test_validator.sh dankserver`
3. Commit: `git add -A && git commit -m "Description"`
4. Deploy: `bash deploy.sh dankserver`
5. Verify: `bash scripts/validate-vllm.sh -t dankserver`

## Remote Servers

### dankserver
- SSH: `dank@dankserver`
- Scripts: `~/vllm/`
- Service: `vllm-awq-qwen36.service`
- Model: frank-default-small (AWQ 4bit)
- Port: 34401
- Status: Disabled (pending bug fixes)

### dankclaw
- SSH: `dank@dankclaw`
- Scripts: `~/vllm/`
- Service: `vllm-thinkingcap.service`
- Model: frank-default-large (FP8)
- Port: 3090
- Status: Running (you are the deployed inference)

## Troubleshooting
- Service failed: `systemctl status <service>`
- Container stuck: `docker stop <name> && docker rm <name>`
- OOM error: Reduce `--max-model-len` or increase `--cpu-offload-gb`
- Prefix caching conflict: Remove `--enable-prefix-caching` from script