dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: one knowledge base for both workloads

Browse Source 
- docs/index.md: LLM Fleet section (all 34 pages linked), merge note,
  See Also now internal
- docs/LLM/index.md: retitled, dangling [[Dankserver]]/[[Dankclaw]]
  links fixed, sibling section replaced with internal links
- docs/LLM/deployment.md: rewritten for the unified deploy flow,
  manual-update paths point at the merged checkout, incident history
  kept intact
- Deployment Flow / Dankclaw Stack / Gitea: stale vllmScripts
  references and retired-topology notes updated to the 2026-08-22
  state
- Historical docs (frank-redo, PP deployment, FastFlowLM) keep their
  dated runbook paths as-is (records of retired topologies)
This commit is contained in:
Frank
2026-08-22 15:50:39 -07:00
parent 0461988884
commit e367f0f43c
6 changed files with 177 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Architecture/Deployment Flow.md
+7 -4
View File
@@ -1,6 +1,8 @@
# Deployment Flow
Deployment mirrors the `vllmScripts` repository: git is the single source of truth, and a `deploy.sh` script moves committed changes to the target hosts over SSH.
Git is the single source of truth. A `deploy.sh` script moves committed
changes to the target hosts over SSH. One run per target covers both
workloads on that host: the podman stack and the LLM service.
## Workflow
@@ -16,8 +18,9 @@ graph LR
1. Edit files locally in this repository.
2. Commit and push.
3. Run `./scripts/deploy.sh <dankclaw|dankserver>`.
4. The script pulls the repo on the target and restarts the podman stack.
5. It verifies service health after restart.
4. The script pulls the repo on the target and restarts the podman stack
and the LLM service.
5. It verifies service health after restart (homepage and LLM endpoint).
## Deploy Script
@@ -54,4 +57,4 @@ The suite validates compose YAML, env consistency, Caddyfiles, docs completeness
- [[Storage Policy]]
- [[Testing Policy]]
- [vllmScripts deployment docs](../../../vllmScripts/docs/deployment.md) — the model this follows
- [LLM deployment docs](../LLM/deployment.md) — the LLM half of the same flow
docs/Dankclaw/Dankclaw Stack.md
+11 -11
View File
@@ -1,8 +1,8 @@
# Dankclaw Stack
Dankclaw (192.168.0.143) runs the AI, search, and development stack. It
hosts the vLLM inference endpoint (`frank-default-large`, `:52625`, from
the `vllmScripts` repo), the agent tools, the privacy search, and the
hosts the LLM inference endpoint (`frank-default-large`, `:52625`, CIRU
ROCmFPX llama.cpp), the agent tools, the privacy search, and the
observability stack. The llama.cpp endpoint (`frank-default-small`,
`:7770`) runs on Dankserver (192.168.0.235).
@@ -17,7 +17,7 @@ graph LR
S --> V[Gluetun VPN]
O --> L[llama.cpp dankserver :7770]
O --> H[Hermes]
O --> E[vLLM dankclaw :52625]
O --> E[CIRU llama.cpp :52625]
```
## Services
@@ -32,14 +32,14 @@ graph LR
| Grafana | grafana.dankclaw.local | 3002 | [[Grafana]] |
| Prometheus | prometheus.dankclaw.local | 9090 | [[Prometheus]] |
| Homepage | dankclaw.local | 80/443 | — |
| frank-default-large (host service, Dankclaw) | — | 52625 | [vllmScripts](../../vllmScripts/docs/index.md) |
| frank-default-small (host service, Dankserver) | — | 7770 | [vllmScripts](../../vllmScripts/docs/index.md) |
| frank-default-large (host service, Dankclaw) | — | 52625 | [[deployment]] |
| frank-default-small (host service, Dankserver) | — | 7770 | [[deployment]] |
The last two rows are bare-metal systemd services, not containers. The
`vllmScripts` repository owns their scripts and knowledge base. The live
mapping (verified 2026-08-19) is the reverse of the 2026-08-17 note:
`frank-default-large` runs vLLM on Dankclaw; `frank-default-small` runs
llama.cpp FastMTP on Dankserver.
The last two rows are bare-metal systemd services, not containers. This
repo owns their scripts, units, and knowledge base (see
[docs/LLM/index.md](../LLM/index.md)). Current mapping (2026-08-22):
`frank-default-large` runs the CIRU ROCmFPX llama.cpp on Dankclaw;
`frank-default-small` runs the upstream llama.cpp on Dankserver.
## Boot Persistence
@@ -58,4 +58,4 @@ Deploy the SearXNG trio with `scripts/deploy-searxng-quadlet.sh dankclaw` and op
- [[index]] — repository index
- [[Network Architecture]] — routing and proxy
- [[Observability]] — Grafana, Prometheus, logs
- [vllmScripts knowledge base](../../vllmScripts/docs/index.md) — the model-serving endpoints on this host
- [LLM fleet knowledge base](../LLM/index.md) — the model-serving endpoints on this host
docs/Dankclaw/Gitea.md
+4 -4
View File
@@ -18,10 +18,10 @@ The containers share the private bridge network `systemd-gitea`. The quadlet net
| Repository | Purpose |
|------------|---------|
| `dank/serverAppConfigs` | this repo — Podman stack configs |
| `dank/vllmScripts` | model-serving scripts: llama.cpp FastMTP on Dankserver (`:7770`), vLLM on Dankclaw (`:52625`) |
| `dank/serverAppConfigs` | this repo — Podman stacks + LLM fleet |
| `dank/vllmScripts` | retired 2026-08-22 — merged into `dank/serverAppConfigs` (history preserved) |
The vllmScripts knowledge base: [../vllmScripts/docs/index.md](../../vllmScripts/docs/index.md)
The LLM fleet knowledge base: [docs/LLM/index.md](../LLM/index.md)
## Management
@@ -83,7 +83,7 @@ Back up `/opt/gitea/data/` (tar) or `pg_dump` the database. See the knowledge ba
- **The files in the data directory are owned by host UID 100999**, not `dank` (rootless ID mapping: container uid 1000 → host 100999). Edit them inside the container with `podman exec --user 1000 gitea-server ...`.
- **The host auto-reboots.** `unattended-upgrades` reboots Dankclaw when a package needs it. The quadlet units make Gitea survive that. See the 2026-08-16 incident below.
- **Port 3000 vs 3001.** The live host port is 3000. The full-stack [`dankclaw/compose.yaml`](../compose.yaml) maps 3001. Align the two before deploying the full stack.
- **Legacy compose retired.** `/opt/gitea/podman-compose.yml` is retired to `/opt/gitea/.deprecated/`. Do not run it; its containers would collide with the quadlet units on ports 3000, 2222, 80, and 443.
- **Legacy compose retired.** `/opt/gitea/podman-compose.yml` is retired to `/opt/gitea/.deprecated/`. Do not run it. Its containers collide with the quadlet units on ports 3000, 2222, 80, and 443.
- **Rootless cannot bind ports below 1024 by default.** `gitea-caddy` publishes 80 and 443. Without a host sysctl, `rootlessport` fails with `permission denied` and the unit crash-loops under `Restart=always`. The fix is a one-time sysctl: `net.ipv4.ip_unprivileged_port_start=80`, persisted in `/etc/sysctl.d/99-rootless-ports.conf`. Applied 2026-08-18 after a 13,000-restart crash loop.
## Incident: 2026-08-16 — down after unattended reboot
docs/LLM/deployment.md
+73 -35
View File
@@ -1,10 +1,19 @@
# Deployment and Update Process
This page describes how to deploy and update the LLM servers on Dankserver and Dankclaw. The process uses git as the single source of truth. All script edits flow through this repository.
This page describes how to deploy and update the LLM servers on Dankserver
and Dankclaw. The process uses git as the single source of truth. All script
edits flow through this repository.
> 2026-08-22: the former `vllmScripts` repository is merged into this repo.
> Both LLM services now run from the `/home/dank/serverAppConfigs` checkout
> on each host. The retired checkouts (`/home/dank/vLLM` on Dankserver,
> `/home/dank/vllmScripts` on Dankclaw) are archived on the hosts.
## Overview
Two servers run LLM inference services. Each server pulls its scripts from this repository. The deployment process moves a committed change from the repository to a running service.
Two servers run LLM inference services. Each server pulls its scripts from
this repository. The deployment process moves a committed change from the
repository to a running service.
| Target | Host | Runtime | Service | Port |
|--------|------|---------|---------|------|
@@ -13,38 +22,42 @@ Two servers run LLM inference services. Each server pulls its scripts from this
## Workflow
The deployment workflow has six steps.
The deployment workflow has seven steps.
```mermaid
graph LR
A[Edit scripts locally] --> B[Commit]
B --> C[Push to origin<br>Gitea on Dankclaw]
C --> D[Pull on target<br>git pull]
D --> E[Restart systemd service]
E --> F[Verify health endpoint]
D --> E[Restart podman stack]
E --> F[Relink + restart LLM unit]
F --> G[Verify health endpoints]
```
1. Fetch the current script from the target machine.
2. Commit the fetched version in this repository.
3. Edit the script locally in the working tree.
4. Commit and push the change to origin.
5. Pull the change on the target machine.
6. Restart the service and verify that it is healthy.
4. Run the tests (`./tests/run.sh` must pass).
5. Commit and push the change to origin.
6. Pull the change on the target machine.
7. Restart the services and verify that they are healthy.
## Deploy Script
The `scripts/deploy.sh` script executes steps 4 through 6. Run it from the repository root.
The `scripts/deploy.sh` script is the unified deployer. One run per target
covers both workloads on that host: the podman stack and the LLM service.
Run it from the repository root.
Run this command to deploy to Dankserver:
```
FRANK_SUDO_PW=... ./scripts/deploy.sh dankserver
./scripts/deploy.sh dankserver
```
Run this command to deploy to Dankclaw:
```
FRANK_SUDO_PW=... ./scripts/deploy.sh dankclaw
./scripts/deploy.sh dankclaw
```
Add the `--dry-run` flag to preview without making changes:
@@ -58,12 +71,18 @@ The script does these actions:
1. Verifies that the local working tree has no uncommitted changes.
2. Pushes the `main` branch to origin.
3. Pulls the repository on the target machine.
4. Symlinks the systemd unit, reloads systemd, enables, and restarts the service.
5. Waits for the health endpoint to respond (over ssh).
4. Restarts the podman stack (Dankclaw only. The Dankserver media stack
runs as bare containers).
5. Relinks the LLM systemd units, reloads systemd, enables, and restarts
the active LLM service.
6. Waits for the Caddy homepage and the LLM health endpoint (over ssh).
Set `FRANK_SUDO_PW` when running headless, so that remote sudo reads the
password from stdin (`sudo -S`). The passwordless restart helpers from the
retired topology are obsolete and are not used.
Set `FRANK_SUDO_PW` when running headless on a host without passwordless
sudo, so that remote sudo reads the password from stdin (`sudo -S`). The
password is never stored in this repo. Both fleet hosts currently allow
passwordless sudo for the deploy user (verified 2026-08-22). The
passwordless restart helpers from the retired topology are obsolete and are
not used.
Health endpoints:
@@ -77,9 +96,9 @@ Health endpoints:
Run these commands on Dankserver:
```
cd /home/dank/vLLM
cd /home/dank/serverAppConfigs
git pull origin main
sudo ln -sf /home/dank/vLLM/scripts/frank-default-small.service /etc/systemd/system/frank-default-small.service
sudo ln -sf /home/dank/serverAppConfigs/scripts/frank-default-small.service /etc/systemd/system/frank-default-small.service
sudo systemctl daemon-reload
sudo systemctl restart frank-default-small
curl -sf http://127.0.0.1:7770/health
@@ -90,9 +109,9 @@ curl -sf http://127.0.0.1:7770/health
Run these commands on Dankclaw:
```
cd /home/dank/vllmScripts
cd /home/dank/serverAppConfigs
git pull origin main
sudo ln -sf /home/dank/vllmScripts/scripts/frank-default-large.service /etc/systemd/system/frank-default-large.service
sudo ln -sf /home/dank/serverAppConfigs/scripts/frank-default-large.service /etc/systemd/system/frank-default-large.service
sudo systemctl daemon-reload
sudo systemctl restart frank-default-large
curl -sf http://127.0.0.1:52625/health
@@ -117,11 +136,12 @@ launches through `scripts/serve-frank-small.sh`, which reads tunables from
The active service on Dankclaw is the CIRU ROCmFPX llama.cpp runtime serving
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` (v3 five-file payload: GGUF
+ FFN/GDN .pfs + Output-K8 proxy + BF16 mmproj) under the alias `frank-default-large` on port 52625. It runs
+ FFN/GDN .pfs + Output-K8 proxy + BF16 mmproj) under the alias
`frank-default-large` on port 52625. It runs
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` (ROCm
7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP
(`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context
(1 slot; concurrent requests queue, the server never preempts a running
(1 slot. Concurrent requests queue. The server never preempts a running
generation), batch 4096 / ubatch 4096, 16 context checkpoints, 8 GiB
prompt cache, and reasoning in deepseek format. The unit file is
`scripts/frank-default-large.service`. It launches through
@@ -129,11 +149,12 @@ prompt cache, and reasoning in deepseek format. The unit file is
`scripts/frank-default-large.env`. See [[ciru-activefpx-promptforge]].
> Restored 2026-08-21. The intermediate DFlash2 Vulkan deployment was
> retired the same day; see [[dflash2-large-vulkan]] for the archived runbook.
> retired the same day. See [[dflash2-large-vulkan]] for the archived runbook.
## Validation
The deploy script checks the health endpoint after a restart. For deeper checks, run a completion request against the endpoint:
The deploy script checks the health endpoint after a restart. For deeper
checks, run a completion request against the endpoint:
```
curl -s http://127.0.0.1:7770/v1/chat/completions \
@@ -152,19 +173,36 @@ If the health check fails after a deploy:
## Incident: HIP Runtime Crash From the `TEMP` Env Variable (2026-08-13)
On 2026-08-13 the old `frank-default-large` service crash-looped for hours on Dankclaw. Root cause: the env files exported `TEMP=1.0` (a sampling default). The ROCm HIP runtime reads `TEMP` as its JIT kernel-cache directory, used the path `1.0`, failed, and segfaulted on a NULL object. Fix: rename the sampling variable to `TEMPERATURE` (commit `227ce01`).
On 2026-08-13 the old `frank-default-large` service crash-looped for hours
on Dankclaw. Root cause: the env files exported `TEMP=1.0` (a sampling
default). The ROCm HIP runtime reads `TEMP` as its JIT kernel-cache
directory, used the path `1.0`, failed, and segfaulted on a NULL object.
Fix: rename the sampling variable to `TEMPERATURE` (commit `227ce01`).
If a service segfaults at load with no obvious cause, compare the exported env against the HIP runtime's env reads. Do not name any tunable `TEMP`, `HOST`, or `GPU_MAX_HEAP_SIZE`.
If a service segfaults at load with no obvious cause, compare the exported
env against the HIP runtime's env reads. Do not name any tunable `TEMP`,
`HOST`, or `GPU_MAX_HEAP_SIZE`.
## Incident: Jobs "Released Prematurely" Under Parallel Load (2026-08-13)
Historical: on the retired topology, parallel jobs on the Dankserver 35B service died with a cuBLAS crash under researcher load. The affected service is retired. A client abort logs `W srv stop: cancel task, id_task = N`; a natural stop logs only `slot release` with `truncated = 0`. The server never preempts a running generation.
Historical: on the retired topology, parallel jobs on the Dankserver 35B
service died with a cuBLAS crash under researcher load. The affected
service is retired. A client abort logs `W srv stop: cancel task,
id_task = N`. A natural stop logs only `slot release` with
`truncated = 0`. The server never preempts a running generation.
## Incident: frank-default-small Crash Loop After Distro Upgrade (2026-08-21)
On 2026-08-21 a release upgrade moved Dankserver from Ubuntu 24.04 to Ubuntu 26.04 (Resolute Raccoon). The upgrade removed the CUDA 12 runtime packages: `libcudart12`, `libcublas12`, `libcublaslt12`, and `nvidia-cuda-toolkit`. The `llama-server` binary links `libcudart.so.12` and `libcublas.so.12`. On every start the binary exited with code 127. systemd restart-looped the service continuously. The health endpoint on port 7770 stayed down.
On 2026-08-21 a release upgrade moved Dankserver from Ubuntu 24.04 to
Ubuntu 26.04 (Resolute Raccoon). The upgrade removed the CUDA 12 runtime
packages: `libcudart12`, `libcublas12`, `libcublaslt12`, and
`nvidia-cuda-toolkit`. The `llama-server` binary links
`libcudart.so.12` and `libcublas.so.12`. On every start the binary exited
with code 127. systemd restart-looped the service continuously. The health
endpoint on port 7770 stayed down.
Fix: reinstall the runtime packages. The same versions remain in the 26.04 repository.
Fix: reinstall the runtime packages. The same versions remain in the 26.04
repository.
sudo apt-get install -y libcudart12 libcublas12 libcublaslt12
@@ -174,13 +212,13 @@ Verify that the binary resolves all libraries:
Start the service and check the health endpoint on port 7770.
After any release upgrade on Dankserver, run the `ldd` check above. A missing CUDA 12 library means the packages need a reinstall.
After any release upgrade on Dankserver, run the `ldd` check above. A
missing CUDA 12 library means the packages need a reinstall.
## See Also
- [AGENTS.md](../AGENTS.md) — the git workflow rules
- [README.md](../README.md) — repository overview
- [index.md](index.md) — knowledge base index
- [AGENTS.md](../../AGENTS.md) — the git workflow rules
- [README.md](../../README.md) — repository overview
- [index.md](index.md) — LLM fleet knowledge base index
- [frank-redo-2026-08-17.md](frank-redo-2026-08-17.md) — the full redo runbook
- [serverAppConfigs knowledge base](../serverAppConfigs/docs/index.md) —
the sibling Podman app stack on the same hosts
- [docs/index.md](../index.md) — the project knowledge base (podman stacks)
docs/LLM/index.md
+16 -9
View File
@@ -1,10 +1,15 @@
# vLLM Scripts — Knowledge Base
# LLM Fleet — Knowledge Base
Documentation for the LLM inference fleet on two local machines
(2026-08-22): upstream llama.cpp on Dankserver (frank-default-small) and
CIRU ROCmFPX llama.cpp with the PromptForge v3 runtime on Dankclaw
(frank-default-large).
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
> `docs/` root to `docs/LLM/`. The fleet scripts and units now live in the
> repo's `scripts/` directory.
## Deployment Topology
```mermaid
@@ -43,11 +48,11 @@ frank-default-large, port 52625]
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[ciru-activefpx-promptforge]] — the v3 CIRU PromptForge runtime on
Dankclaw (frank-default-large, active; restored 2026-08-22)
Dankclaw (frank-default-large, active. Restored 2026-08-22)
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment
(deployed and reverted 2026-08-22)
- [[Dankserver]] — the upstream llama.cpp box (frank-default-small)
- [[Dankclaw]] — the CIRU ROCmFPX box (frank-default-large)
- [[Dankserver System Specs]] — the upstream llama.cpp box (frank-default-small)
- [[Dankclaw System Specs]] — the CIRU ROCmFPX box (frank-default-large)
## CIRU Reference (Dankclaw)
@@ -88,10 +93,12 @@ frank-default-large, port 52625]
## Pipeline Parallelism (archive)
- [[Pipeline Parallelism]] — index of multi-node inference research. The PP
pair is retired; these pages are research history.
pair is retired. These pages are research history.
## Sibling projects
## Related
- [serverAppConfigs](../serverAppConfigs/docs/index.md) — the Podman app
stack on the same two hosts. Its OpenWebUI consumes the `:7770` endpoint
on Dankserver; its Gitea instance hosts this repository.
- [Project knowledge base](../index.md) — the Podman app stacks on the same
two hosts. OpenWebUI consumes the `:7770` endpoint on Dankserver. The
Gitea instance on Dankclaw hosts this repository.
- [Deployment Flow](../Architecture/Deployment%20Flow.md) — the unified
git-based deploy for both workloads
docs/index.md
+66 -5
View File
@@ -2,7 +2,14 @@
Podman deployment configuration for two homelab servers: Dankserver and Dankclaw.
This repository tracks the container stacks for both hosts. Git is the single source of truth. Edits flow through this repo, then deploy to the targets with `scripts/deploy.sh`. This mirrors the workflow of the `vllmScripts` repository.
This repository is the single source of truth for both hosts. It tracks the
container stacks and the LLM inference fleet. Edits flow through this repo,
then deploy to the targets with `scripts/deploy.sh` (one script per target
covers both workloads).
> 2026-08-22: the former `vllmScripts` repository (LLM fleet) is merged
> into this repo with its full git history. Its knowledge base now lives at
> [docs/LLM/index.md](LLM/index.md).
## Hosts
@@ -74,10 +81,64 @@ This repository tracks the container stacks for both hosts. Git is the single so
- [[Jellyfin]] — media server research brief
- [[ARR-Stack]] — automated download stack research brief
## LLM Fleet
The LLM inference fleet on the two hosts. Full index:
[docs/LLM/index.md](LLM/index.md).
### Deployment and Topology
- [[deployment]] — unified deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[ciru-activefpx-promptforge]] — the v3 CIRU PromptForge runtime on Dankclaw (active)
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
- [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21)
- [[Dankserver System Specs]] — the upstream llama.cpp box hardware
- [[Dankclaw System Specs]] — the CIRU ROCmFPX box hardware
- [[Dankclaw Improvements]] — host-level changes on the CIRU box
### Benchmarks and Sizing
- [[frank-default-large-benchmark]] — backend/spec benchmark behind the old head
- [[frank-default-small-benchmark]] — the small-model benchmark suite
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
- [[qwen3.8-strix-optimizations]] — Strix Halo tuning notes
### Memory and Hardware
- [[RAM CPU Offloading in VLLM]] — KV cache swap between GPU VRAM and system RAM
- [[Hybrid CPU GPU Processing Dankserver]] — NVIDIA GPU compute with system RAM offload
- [[Shared Memory Architecture Dankclaw]] — AMD unified memory and ROCm configuration
- [[dankserver-vram-growth]] — VRAM/RAM growth behind the retired cuBLAS crash
- [[Quantization by Hardware Architecture]] — which quant format fits which GPU/CPU
### Retired Topology and History
- [[Qwen3.8 PP Deployment]] — the retired pipeline-parallel head + worker
- [[qwen3.8-spec-matrix]] — Qwen3.8-27B spec history (superseded by FastMTP)
- [[FastFlowLM NPU Inference]] — the retired FastFlowLM NPU endpoint
- [[DFlash DSpark Speculative Decoding]] — retired DFlash/DSpark drafting
- [[ling-3.0-flash-trial]] — the Ling-3.0-Flash trial runbook
- [[ling-3.0-flash-vllm-plan]] — the Ling-3.0-Flash vLLM plan (retired)
### vLLM Reference (historical)
- [[VLLM Overview]] — installation, CLI, API, architecture, quantization, parallelism
- [[VLLM Index]] — quick reference with commands, endpoints, and parameter table
- [[vllm_research_findings]] — research findings from the vLLM era
### Pipeline Parallelism (archive)
- [[Ethernet Bandwidth Math]] — the bandwidth budget for the PP pair
- [[Expected Performance]] — expected PP throughput
- [[llama.cpp RPC]] — the RPC transport between head and worker
- [[Mixed-Vendor Experiments]] — cross-vendor PP experiments
- [[Qwen3.6 27B Spec Methods]] — spec-decoding methods for the 27B
- [[Sample Script Set]] — reference scripts for the PP pair
- [[vLLM Pipeline Parallelism]] — vLLM's PP implementation
## See Also
- [vllmScripts repository](../vllmScripts/docs/index.md) — the model-serving
scripts on the two hosts (2026-08-17 redo): `frank-default-large`
(llama.cpp FastMTP, Dankserver, port 7770) and `frank-default-small`
(vLLM, Dankclaw, port 52625). Git workflow model for this repo.
- [[AGENTS.md]] — repository rules
- [docs/LLM/index.md](LLM/index.md) — the LLM fleet knowledge base