docs: boot policy (only PP units enabled), deploy.sh dankserver hazard, stale-claim cleanup

- deployment.md: Boot Policy (2026-08-17) section (disabled units table,
  re-enable recipes, stale frank-restart-small.sh helper, broken
  deploy.sh dankserver target); Update Dankserver -> frank-pp-worker;
  condensed retired 35B history block and 2026-08-13 incident
- AGENTS.md: boot-policy note, target map fix (Q6_K, split 0.68),
  deploy.sh dankserver warning, obsolete quirk entries condensed
- README.md: rewrite active services/models tables (PP head + worker)
- index.md: topology fix (qwythos:9b, Q6_K split 0.68, NPU disabled)
- qwen3.8-spec-matrix.md: superseded configs merged into compact
  history; stale inline env block removed; bundled-head MTP note
- FastFlowLM, PP Deployment: boot-policy + enablement notes
- System Specs x2, DFlash doc, 35B FP8 doc, Ling docs: stale active
  claims fixed, retired banners, never-shipped plan stubbed
This commit is contained in:
Frank committed 2026-08-17 03:30:00 -07:00
1 parent 3f88b8e1ed
commit 968066ff23
13 files changed
+190 -246

No files matched your search

+23 -19
View File
@@ -1,11 +1,13 @@
# vLLM Scripts
Scripts for running LLM inference on two local machines.
Deployment scripts for the LLM inference fleet: a llama.cpp
pipeline-parallel Qwen3.8-27B split across two machines, plus the
FastFlowLM NPU endpoint on Dankclaw.
## Targets
| Host | User | Codename | Architecture |
|------|------|----------|-------------|
|------|------|----------|--------------|
| dankserver | dank | Dankserver | NVIDIA RTX 3090 + 62 GB RAM |
| dankclaw | dank | Dankclaw | AMD Strix Halo APU (shared memory) |
@@ -13,30 +15,32 @@ Scripts for running LLM inference on two local machines.
| Service | Host | Runtime | Model | Port |
|---------|------|---------|-------|------|
| frank-default-large | dankclaw | vLLM-ROCm (CIRU) | Ling-3.0-flash CIRU INT4 MTP | 7770 |
| frank-default-small | dankserver | llama.cpp (ngram-mod) | Qwen3.6-35B-A3B uncensored Q6_K_P | 7770 |
| frank-default-large | dankclaw | llama.cpp PP head (RPC to Dankserver) | Qwen3.8-27B-Uncensored Q6_K, standalone MTP draft | 7770 |
| frank-pp-worker | dankserver | ggml-rpc worker (RTX 3090) | receives the head's tensor share | 50052 |
> The retired ROCmFPX llama.cpp service on Dankclaw was replaced by the vLLM service above. The retired `vllm-awq-qwen36` (AWQ 4-bit) service on Dankserver was replaced by the llama.cpp service above. Legacy unit files remain in `scripts/` for reference.
## Models
| Model | Target | Format |
|-------|--------|--------|
| Ling-3.0-flash CIRU INT4 (MTP) | dankclaw | INT4 W4A16 (Quark/vLLM) |
| Qwen3.6 35B A3B uncensored Q6_K_P (ngram-mod speculative decoding) | dankserver | GGUF Q6_K_P (llama.cpp, MoE offload) |
| Qwen3.6 35B A3B AWQ 4bit | dankserver | AWQ (retired) |
| Qwen3.6 35B A3B FP8 | dankserver | FP8 (documented, not active) |
| Qwen3.6 27B FP8 | dankclaw | FP8 (documented, not active) |
Boot policy (2026-08-17): only these two units are enabled. The
FastFlowLM NPU endpoint (`frank-default-small`, qwythos:9b, port 52625)
is disabled for boot; re-enable it from `scripts/frank-default-small.service`
if needed. Retired units: the Dankserver llama.cpp 35B service,
`llamaserver`, `ollama`, `vllm-awq-qwen36`, `vllm-qwen36`,
`frank-large-ling` (Ling trial), and the vLLM-ROCm CIRU path. Their unit
files remain in `scripts/` for reference.
## Structure
- `docs/` — Project knowledge base with research notes on vLLM configuration, hardware specifics, speculative decoding, and deployment notes
- `scripts/` — Deployment and launch scripts (deploy.sh, serve-llama-frank-small.sh, serve-ling3-vllm.sh, systemd unit files)
- `docs/` — knowledge base: deployment runbook, PP runbook, spec matrix,
NPU notes, research archive
- `scripts/` — serve scripts, env files, systemd units, deploy and test
tools
## Deployment
See [docs/deployment.md](docs/deployment.md) for the deploy/update process. Run `./scripts/deploy.sh <dankclaw|dankserver>` after pushing committed changes.
See [docs/deployment.md](docs/deployment.md). Run
`./scripts/deploy.sh dankclaw` for the PP head. The `dankserver` target
is broken (it targets the removed `frank-default-small` unit); restart
the worker with `sudo systemctl restart frank-pp-worker` until the
script is fixed.
## Knowledge Base
See [docs/index.md](docs/index.md) for the project knowledge base index.
See [docs/index.md](docs/index.md) for the project knowledge base index.