serverAppConfigs/AGENTS.md

vLLM Scripts — Agent Instructions

Project Overview

Scripts for running large language models via vLLM on two local machines:

• dankserver: NVIDIA GPU + System RAM (frank-default-small)
• dankclaw: AMD Strix Halo Shared Memory (frank-default-large)

Directory Structure

vllmScripts/
├── AGENTS.md          # This file — agent instructions
├── README.md          # Human-readable project overview
├── deploy.sh          # Automated deployment script
├── scripts/           # Executable vLLM launch scripts
│   ├── validate-vllm.sh  # Validation tool
│   └── *.sh             # Inference server scripts
├── tests/             # Test suite
│   └── test_validator.sh
├── docs/              # Knowledge base
│   ├── README.md
│   └── *.md           # Research notes
└── failedAttempts/    # Archived scripts (for reference)

Running Tests

# Test validator on dankserver
bash tests/test_validator.sh dankserver

# Test validator on dankclaw
bash tests/test_validator.sh dankclaw

# Run all tests
bash tests/test_validator.sh dankserver && bash tests/test_validator.sh dankclaw

Deployment

# Deploy all scripts to all targets
bash deploy.sh all

# Deploy specific script to dankserver
bash deploy.sh dankserver vllm-qwen36.sh

# Deploy specific script to dankclaw
bash deploy.sh dankclaw vllm-thinkingcap.sh

Git Workflow

Branching Strategy

• main: Stable, working scripts only
• vllm-qwen36: Active development for Qwen3.6 models
• Feature branches: feature/<description> for experiments

Worktree Pattern

For parallel development:

# Create worktree for feature
git worktree add ../vllm-feature feature/qwen36-awq

# When done, merge back
git worktree remove ../vllm-feature

Commit Guidelines

• Use descriptive commit messages: git commit -m "Add AWQ DFlash script"
• Commit before deploying: git add -A && git commit -m "Update for deployment"
• Push to remote: git push origin <branch>

Deployment Flow

1. Make changes locally
2. Run tests: bash tests/test_validator.sh dankserver
3. Commit: git add -A && git commit -m "Description"
4. Deploy: bash deploy.sh dankserver
5. Verify: bash scripts/validate-vllm.sh -t dankserver

Remote Servers

dankserver

• SSH: dank@dankserver
• Scripts: ~/vllm/
• Service: vllm-awq-qwen36.service
• Model: frank-default-small (AWQ 4bit)
• Port: 34401

dankclaw

• SSH: dank@dankclaw
• Scripts: ~/vllm/
• Service: vllm-thinkingcap.service
• Model: frank-default-large (FP8)
• Port: 3090

Troubleshooting

• Service failed: systemctl status <service>
• Container stuck: docker stop <name> && docker rm <name>
• OOM error: Reduce --max-model-len or increase --cpu-offload-gb
• Prefix caching conflict: Remove --enable-prefix-caching from script