docs: polish pass — fix stale names, duplicates, branding, and README
- Normalize 'VLLM' to 'vLLM' in prose across 6 docs (32 occurrences; wikilink targets untouched) - Remove duplicated Environment Variables block in 27B Dankclaw doc - Update stale model refs Qwen3-30B-A3B -> Qwen3.6-35B-A3B in example commands - Fix VLLM Index: -- separators to em-dashes; correct model names on links - README: document active services (ROCmFPX llama-server on Dankclaw, vLLM AWQ on Dankserver), replace broken [[docs/README.md]] link with docs/index.md
This commit is contained in:
1 parent
993b6af576
commit
f650fff0aa
8 files changed
+62
-58
No files matched your search
@@ -1,26 +1,39 @@
|
||||
# vLLM Scripts
|
||||
|
||||
Scripts for running large language models via vLLM on two local machines.
|
||||
Scripts for running LLM inference on two local machines.
|
||||
|
||||
## Targets
|
||||
|
||||
| Host | User | Codename | Architecture |
|
||||
|------|------|----------|-------------|
|
||||
| dankserver | dank | DankServer | NVIDIA GPU + System RAM |
|
||||
| dankclaw | dank | DankClaw | AMD Strix Halo (Shared Memory) |
|
||||
| dankserver | dank | Dankserver | NVIDIA RTX 3090 + 62 GB RAM |
|
||||
| dankclaw | dank | Dankclaw | AMD Strix Halo APU (shared memory) |
|
||||
|
||||
## Active Services
|
||||
|
||||
| Service | Host | Runtime | Model | Port |
|
||||
|---------|------|---------|-------|------|
|
||||
| llama-server | dankclaw | ROCmFPX llama.cpp | Ling-3.0-flash ROCmFP4 MTP | 8080 |
|
||||
| vllm-awq-qwen36 | dankserver | vLLM (Docker) | Qwen3.6-35B-A3B AWQ 4-bit | 34401 |
|
||||
|
||||
## Models
|
||||
|
||||
| Model | Target | Format |
|
||||
|-------|--------|--------|
|
||||
| Qwen3.6 35B A3B FP8 | dankserver | FP8 |
|
||||
| Qwen3.6 27B FP8 | dankclaw | FP8 |
|
||||
| Ling-3.0-flash ROCmFP4-MTP | dankclaw | ROCmFP4 GGUF |
|
||||
| Qwen3.6 35B A3B AWQ 4bit | dankserver | AWQ |
|
||||
| Qwen3.6 35B A3B FP8 | dankserver | FP8 (documented, not active) |
|
||||
| Qwen3.6 27B FP8 | dankclaw | FP8 (documented, not active) |
|
||||
|
||||
## Structure
|
||||
|
||||
- `docs/` — Project knowledge base with research notes on vLLM configuration, hardware specifics, and deployment notes
|
||||
- `scripts/` — Executable vLLM launch scripts (coming soon)
|
||||
- `docs/` — Project knowledge base with research notes on vLLM configuration, hardware specifics, speculative decoding, and deployment notes
|
||||
- `scripts/` — Deployment and launch scripts (deploy.sh, serve-vllm-qwen36-dflash.sh, validate-vllm.sh, systemd unit files)
|
||||
|
||||
## Deployment
|
||||
|
||||
See [docs/deployment.md](docs/deployment.md) for the deploy/update process. Run `./scripts/deploy.sh <dankclaw|dankserver>` after pushing committed changes.
|
||||
|
||||
## Knowledge Base
|
||||
|
||||
See [[docs/README.md]] for the project knowledge base index.
|
||||
See [docs/index.md](docs/index.md) for the project knowledge base index.
|
||||
Reference in new issue
Block a user