docs: one knowledge base for both workloads

- docs/index.md: LLM Fleet section (all 34 pages linked), merge note,
  See Also now internal
- docs/LLM/index.md: retitled, dangling [[Dankserver]]/[[Dankclaw]]
  links fixed, sibling section replaced with internal links
- docs/LLM/deployment.md: rewritten for the unified deploy flow,
  manual-update paths point at the merged checkout, incident history
  kept intact
- Deployment Flow / Dankclaw Stack / Gitea: stale vllmScripts
  references and retired-topology notes updated to the 2026-08-22
  state
- Historical docs (frank-redo, PP deployment, FastFlowLM) keep their
  dated runbook paths as-is (records of retired topologies)
This commit is contained in:
Frank committed 2026-08-22 15:50:39 -07:00
1 parent 0461988884
commit e367f0f43c
6 files changed
+177 -68

No files matched your search

+66 -5
View File
@@ -2,7 +2,14 @@
Podman deployment configuration for two homelab servers: Dankserver and Dankclaw.
This repository tracks the container stacks for both hosts. Git is the single source of truth. Edits flow through this repo, then deploy to the targets with `scripts/deploy.sh`. This mirrors the workflow of the `vllmScripts` repository.
This repository is the single source of truth for both hosts. It tracks the
container stacks and the LLM inference fleet. Edits flow through this repo,
then deploy to the targets with `scripts/deploy.sh` (one script per target
covers both workloads).
> 2026-08-22: the former `vllmScripts` repository (LLM fleet) is merged
> into this repo with its full git history. Its knowledge base now lives at
> [docs/LLM/index.md](LLM/index.md).
## Hosts
@@ -74,10 +81,64 @@ This repository tracks the container stacks for both hosts. Git is the single so
- [[Jellyfin]] — media server research brief
- [[ARR-Stack]] — automated download stack research brief
## LLM Fleet
The LLM inference fleet on the two hosts. Full index:
[docs/LLM/index.md](LLM/index.md).
### Deployment and Topology
- [[deployment]] — unified deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[ciru-activefpx-promptforge]] — the v3 CIRU PromptForge runtime on Dankclaw (active)
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
- [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21)
- [[Dankserver System Specs]] — the upstream llama.cpp box hardware
- [[Dankclaw System Specs]] — the CIRU ROCmFPX box hardware
- [[Dankclaw Improvements]] — host-level changes on the CIRU box
### Benchmarks and Sizing
- [[frank-default-large-benchmark]] — backend/spec benchmark behind the old head
- [[frank-default-small-benchmark]] — the small-model benchmark suite
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
- [[qwen3.8-strix-optimizations]] — Strix Halo tuning notes
### Memory and Hardware
- [[RAM CPU Offloading in VLLM]] — KV cache swap between GPU VRAM and system RAM
- [[Hybrid CPU GPU Processing Dankserver]] — NVIDIA GPU compute with system RAM offload
- [[Shared Memory Architecture Dankclaw]] — AMD unified memory and ROCm configuration
- [[dankserver-vram-growth]] — VRAM/RAM growth behind the retired cuBLAS crash
- [[Quantization by Hardware Architecture]] — which quant format fits which GPU/CPU
### Retired Topology and History
- [[Qwen3.8 PP Deployment]] — the retired pipeline-parallel head + worker
- [[qwen3.8-spec-matrix]] — Qwen3.8-27B spec history (superseded by FastMTP)
- [[FastFlowLM NPU Inference]] — the retired FastFlowLM NPU endpoint
- [[DFlash DSpark Speculative Decoding]] — retired DFlash/DSpark drafting
- [[ling-3.0-flash-trial]] — the Ling-3.0-Flash trial runbook
- [[ling-3.0-flash-vllm-plan]] — the Ling-3.0-Flash vLLM plan (retired)
### vLLM Reference (historical)
- [[VLLM Overview]] — installation, CLI, API, architecture, quantization, parallelism
- [[VLLM Index]] — quick reference with commands, endpoints, and parameter table
- [[vllm_research_findings]] — research findings from the vLLM era
### Pipeline Parallelism (archive)
- [[Ethernet Bandwidth Math]] — the bandwidth budget for the PP pair
- [[Expected Performance]] — expected PP throughput
- [[llama.cpp RPC]] — the RPC transport between head and worker
- [[Mixed-Vendor Experiments]] — cross-vendor PP experiments
- [[Qwen3.6 27B Spec Methods]] — spec-decoding methods for the 27B
- [[Sample Script Set]] — reference scripts for the PP pair
- [[vLLM Pipeline Parallelism]] — vLLM's PP implementation
## See Also
- [vllmScripts repository](../vllmScripts/docs/index.md) — the model-serving
scripts on the two hosts (2026-08-17 redo): `frank-default-large`
(llama.cpp FastMTP, Dankserver, port 7770) and `frank-default-small`
(vLLM, Dankclaw, port 52625). Git workflow model for this repo.
- [[AGENTS.md]] — repository rules
- [docs/LLM/index.md](LLM/index.md) — the LLM fleet knowledge base