PP kit + Strix research: Qwen3.8-27B RPC split (Dankclaw head / Dankserver worker)
- scripts: serve-pp-head/worker, envs, systemd units, pp-test.sh orchestrator, pp-bench.py - head design verified in b10433 source + live 4B smoke test: * no --device with --rpc (RPC device silently dropped otherwise) * default device selection = [RPC0, ROCm0] (HIP registers before Vulkan) * default split proportional to reported free mem = ~20/80 here, pin 1,1 * -dio required for large model over RPC - docs: qwen3.8-strix-optimizations (ranked no-quant-downgrade levers, quant landscape, 2026-08-16 test matrix), PP deployment runbook, index updates, Dankclaw Improvements corrections (Vulkan0, no power_profile sysfs) - worker built at 9b05354ec in /home/dank/llama-cpp-rpc worktree (CUDA+RPC) - second head build build-vulkan-rpc (HIP off) for backend A/B
This commit is contained in:
1 parent
c775077016
commit
9544c36afc
13 files changed
+1037
-5
No files matched your search
@@ -6,7 +6,7 @@ tags:
|
||||
- pipeline-parallelism
|
||||
- distributed
|
||||
- index
|
||||
timestamp: 2026-08-14
|
||||
timestamp: 2026-08-15
|
||||
---
|
||||
|
||||
# Pipeline Parallelism
|
||||
@@ -58,6 +58,14 @@ graph LR
|
||||
- [[Expected Performance]] — estimated tokens per second for a 27B Q8 split
|
||||
across an RTX 3090 and a Strix Halo NUC over 1/2.5/10 GbE.
|
||||
|
||||
## Deployment
|
||||
|
||||
- [[Qwen3.8 PP Deployment]] — the real Qwen3.8-27B split: head on Dankclaw
|
||||
(Strix), worker on Dankserver (RTX 3090), b10433 on both sides, scripts in
|
||||
`scripts/`, morning-of test runbook. Verified gotchas: no `--device` with
|
||||
`--rpc`, default split is proportional to reported free memory (pin 1,1),
|
||||
the iGPU side defaults to the ROCm backend.
|
||||
|
||||
## Sample Scripts
|
||||
|
||||
- [[Sample Script Set]] — a two-node llama.cpp RPC split of
|
||||
|
||||
Reference in new issue
Block a user