Doc audit: fix all broken links + correct stale facts across the KB
Fanned out 5 read-only doc-auditor subagents (one per service group) that fact-checked the knowledge base against the live system + repo config, researched official setup docs online, and verified progress. Applied: - All 34 broken links fixed (real pages / external URLs / correct relative paths); 0 broken links remain. No .AGENTTRASH links (the 4 mentions are accurate descriptive text, not links). - Fact-check fixes: Readarr retired (May 2026), Radarr v4->v6, Calibre 9.2.1 live, Calibre-Web 0.6.25 live, Copyparty v1.20.21, LLM index ncmoe 38 + 786432 ctx, frank-default-small active (not frank-pp-worker), Hermes/SearXNG/Grafana hostnames, Loki/Alloy local images, Observability diagram (Promtail->Alloy), VLLM retired note, DFlash status, benchmark live-drift notes, Homepage test count 116.
This commit is contained in:
1 parent
ac7c8ee442
commit
90c4ac69a7
32 files changed
+57
-48
No files matched your search
@@ -160,7 +160,7 @@ The DFlash draft is not a standalone model. It must pair with its target model.
|
||||
|
||||
## Deployment on Dankserver (NVIDIA RTX 3090)
|
||||
|
||||
The Dankserver llama.cpp deployment (`frank-default-small`, Qwen3.6-35B-A3B) is retired (2026-08-16) and its unit disabled (2026-08-17). The RTX 3090 now runs the PP worker for the Dankclaw head (see [[Qwen3.8 PP Deployment]]). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
|
||||
The Dankserver llama.cpp deployment (`frank-default-small`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
|
||||
|
||||
> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-default-small` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
|
||||
|
||||
|
||||
@@ -60,7 +60,7 @@ Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.0
|
||||
|
||||
### Active Services
|
||||
|
||||
The active service on Dankserver is `frank-pp-worker.service`: the ggml-rpc worker (RTX 3090) that receives the PP head's tensor share (45 of 65 blocks, ~23.8 GiB of 24 GiB at steady state). The old llama.cpp Qwen3.6-35B-A3B service (`frank-default-small`) is retired (2026-08-16) and disabled (2026-08-17). See [[Qwen3.8 PP Deployment]].
|
||||
The active service on Dankserver is `frank-default-small.service`: the llama.cpp Qwen3.6-35B-A3B (Q6_K_P + ngram-mod) on port 7770. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
|
||||
|
||||
## Hardware Relevance
|
||||
|
||||
|
||||
@@ -189,7 +189,7 @@ For the 2-node pipeline-parallel split (RTX 3090 24 GB + Strix Halo 96 GB):
|
||||
- **Kept:** Qwen official config.json (HF) — exact architecture params (64 layers, 5120 hidden, 4 KV heads, head_dim 256) [8](https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/config.json)
|
||||
- **Kept:** bartowski/Qwen_Qwen3.6-27B-GGUF — Q8_0 at 29.12 GB, imatrix calibration [2](https://huggingface.co/bartowski/Qwen_Qwen3.6-27B-GGUF)
|
||||
- **Kept:** Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp — DFlash 2B draft GGUF model [4](https://huggingface.co/Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp)
|
||||
- **Kept:** frank-default-large-benchmark.md — Project spec benchmarks (MTP, DFlash, ngram-mod comparison) [5](docs/frank-default-large-benchmark.md)
|
||||
- **Kept:** frank-default-large-benchmark.md — Project spec benchmarks (MTP, DFlash, ngram-mod comparison) [5](../frank-default-large-benchmark.md)
|
||||
- **Kept:** a4lg/Qwen3.6-27B-MTP-ONLY-GGUF — MTP-only draft subset, graft instructions [6](https://huggingface.co/a4lg/Qwen3.6-27B-MTP-ONLY-GGUF)
|
||||
- **Kept:** llama.cpp tools/rpc/README.md — RPC backend, weight/KV split, remote host setup [7](https://github.com/ggerganov/llama.cpp/blob/master/tools/rpc/README.md)
|
||||
- **Kept:** Unsloth Qwen3.6-27B-MTP-GGUF — MTP command-line usage for llama.cpp [9](https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF)
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
> **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
|
||||
> See [frank-redo-2026-08-17.md](frank-redo-2026-08-17.md) and
|
||||
> [deployment.md](deployment.md) for the active configuration.
|
||||
> See [frank-redo-2026-08-17.md](../frank-redo-2026-08-17.md) and
|
||||
> [deployment.md](../deployment.md) for the active configuration.
|
||||
|
||||
# Qwen3.8-27B Pipeline-Parallel Deployment (Dankclaw + Dankserver)
|
||||
|
||||
Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX
|
||||
3090 (worker) using llama.cpp RPC. This page is the design record and the
|
||||
morning-of runbook. Research background:
|
||||
[Qwen3.8 Strix optimizations](../../qwen3.8-strix-optimizations.md).
|
||||
[Qwen3.8 Strix optimizations](../qwen3.8-strix-optimizations.md).
|
||||
|
||||
## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68)
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# vLLM Index
|
||||
|
||||
> **RETIRED (2026-08-18).** The vLLM service on Dankclaw was replaced by the CIRU ROCmFPX llama.cpp runtime (v3). These pages are historical reference. See [[ciru-activefpx-promptforge]].
|
||||
|
||||
## Documentation Overview
|
||||
|
||||
This index contains all documentation for vLLM deployment on Dankserver and Dankclaw.
|
||||
|
||||
@@ -253,7 +253,7 @@ ngram draft costs its own forward pass / occupancy without adding enough
|
||||
accepted tokens on top of MTP. Baseline (no ngram) remains the production
|
||||
config (`--spec-type draft-mtp` only). Keep ngram-mod off.
|
||||
|
||||
The harness lives at [scripts/bench-spec-stack.sh](../scripts/bench-spec-stack.sh);
|
||||
The harness lives at [scripts/bench-spec-stack.sh](../../scripts/bench-spec-stack.sh);
|
||||
it boots the v3 stack on a scratch port and reads authoritative
|
||||
`print_timing` lines. It requires the production service to be stopped.
|
||||
|
||||
|
||||
@@ -105,6 +105,8 @@ service runs 700416 cells (13,680 MiB).
|
||||
|
||||
## Chosen State
|
||||
|
||||
> **Live drift (2026-08-22 audit):** the deployed `frank-default-small` now runs `--n-cpu-moe 38`, `--batch-size 4096`, `--ubatch-size 2048` (ncmoe raised from 36 for stability; batch lowered). The values below are the original benchmark recommendation.
|
||||
|
||||
Prefill speed is the priority, so the state keeps the top prefill tier and
|
||||
moves one expert layer to CPU for stability:
|
||||
|
||||
|
||||
+2
-2
@@ -19,7 +19,7 @@ graph LR
|
||||
B[upstream llama.cpp b10121
|
||||
frank-default-small, port 7770]
|
||||
C[Qwen3.6-35B-A3B-Uncensored Q6_K_P
|
||||
ngram-mod, 36 MoE experts on CPU, 700K ctx]
|
||||
ngram-mod, 38 MoE experts on CPU, 786432 ctx]
|
||||
A --> B --> C
|
||||
end
|
||||
|
||||
@@ -92,7 +92,7 @@ frank-default-large, port 7770]
|
||||
|
||||
## Pipeline Parallelism (archive)
|
||||
|
||||
- [[Pipeline Parallelism]] — index of multi-node inference research. The PP
|
||||
- [[Pipeline Parallelism/index]] — index of multi-node inference research. The PP
|
||||
pair is retired. These pages are research history.
|
||||
|
||||
## Related
|
||||
|
||||
Reference in new issue
Block a user