PP kit + Strix research: Qwen3.8-27B RPC split (Dankclaw head / Dankserver worker)
- scripts: serve-pp-head/worker, envs, systemd units, pp-test.sh orchestrator, pp-bench.py - head design verified in b10433 source + live 4B smoke test: * no --device with --rpc (RPC device silently dropped otherwise) * default device selection = [RPC0, ROCm0] (HIP registers before Vulkan) * default split proportional to reported free mem = ~20/80 here, pin 1,1 * -dio required for large model over RPC - docs: qwen3.8-strix-optimizations (ranked no-quant-downgrade levers, quant landscape, 2026-08-16 test matrix), PP deployment runbook, index updates, Dankclaw Improvements corrections (Vulkan0, no power_profile sysfs) - worker built at 9b05354ec in /home/dank/llama-cpp-rpc worktree (CUDA+RPC) - second head build build-vulkan-rpc (HIP off) for backend A/B
This commit is contained in:
1 parent
c775077016
commit
9544c36afc
13 files changed
+1037
-5
No files matched your search
@@ -53,6 +53,7 @@ Vulkan0 backend, port 7770, PARALLEL=2]
|
||||
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Dense model (27B) on Strix Halo APU
|
||||
- [[frank-default-large-benchmark]] — backend/spec benchmark; chose MTP sidecar (2026-08-14) over DFlash (parallel premature-EOS bug)
|
||||
- [[qwen3.8-spec-matrix]] — Qwen3.8-27B UD-Q8_K_XL spec matrix; production = --spec-default (ngram-mod), PARALLEL=2, Vulkan0 backend (2026-08-15)
|
||||
- [[qwen3.8-strix-optimizations]] — Qwen3.8-27B Strix speed research: ranked levers that keep UD-Q8_K_XL (PP to 3090, MTP tuning, same-bit Q8_0 A/B, prefill path, OS), quant landscape, 2026-08-16 test matrix
|
||||
- [[frank-default-small-benchmark]] — ncmoe and batch/ubatch sweep that drove the Dankserver state
|
||||
- [[dankserver-vram-growth]] — VRAM/RAM growth investigation behind the recurring cuBLAS crash on frank-default-small
|
||||
- [[Dankclaw Improvements]] — researched enhancements: self-hosted agentic tools, runtime tuning, server/OS settings
|
||||
@@ -71,3 +72,4 @@ Vulkan0 backend, port 7770, PARALLEL=2]
|
||||
- [[Pipeline Parallelism/Mixed-Vendor Experiments]] — NVIDIA + Strix Halo real-world runs
|
||||
- [[Pipeline Parallelism/Expected Performance]] — estimated tok/s over 1/2.5/10 GbE
|
||||
- [[Pipeline Parallelism/Sample Script Set]] — educational two-node Qwen3.6-27B Q8 split
|
||||
- [[Pipeline Parallelism/Qwen3.8 PP Deployment]] — production design + runbook: Qwen3.8-27B head (Strix) / worker (RTX 3090) over RPC, b10433, scripts in scripts/
|
||||
Reference in new issue
Block a user