Frank
841d872aa7
- llama-server.service: dankclaw migrated to ROCmFPX runtime (Ling-3.0-flash, draft-mtp, no-mmap/direct-io) from regular llama.cpp Qwen3.6-27B-DFlash - serve-vllm-qwen36-dflash.sh: dankserver version (relies on image 'vllm serve' entrypoint; HF cache mounted read-write) - validate-vllm.sh: dankserver multi-stream vLLM validation (4 unique prompts, 1-4 streams, prompt+completion speed) - remove superseded scripts: serve-llamacpp-dflash.sh, validate.sh, vllm-awq-qwen36-dflash-ampere.sh, vllm-qwen36.service
vLLM Scripts
Scripts for running large language models via vLLM on two local machines.
Targets
| Host | User | Codename | Architecture |
|---|---|---|---|
| dankserver | dank | DankServer | NVIDIA GPU + System RAM |
| dankclaw | dank | DankClaw | AMD Strix Halo (Shared Memory) |
Models
| Model | Target | Format |
|---|---|---|
| Qwen3.6 35B A3B FP8 | dankserver | FP8 |
| Qwen3.6 27B FP8 | dankclaw | FP8 |
Structure
docs/— Project knowledge base with research notes on vLLM configuration, hardware specifics, and deployment notesscripts/— Executable vLLM launch scripts (coming soon)
Knowledge Base
See docs/README.md for the project knowledge base index.