docs: llama.cpp stack at production state (2026-08-16) + sibling-project links
- spec matrix: new 'Production state' section — PP head (RPC0+ROCm0, split 0.40/0.60), pool 401400 / n_ctx_slot 200704 live, MTP n_max 4 over ngram-mod fallback (journal acceptance stats), f16 KV. 2026-08-15 standalone state marked superseded (Vulkan0 verdict was standalone-only; PP head runs ROCm0 by default). - deployment.md: fix frank-default-large section (MTP not 'no draft', ROCm0 not Vulkan0, 200704/slot not 262144, f16 not q8_0 KV, serve-pp-head.sh not serve-llama-frank-large.sh). - PP deployment doc: production-state section; test-day open questions resolved with journal evidence. - index.md: NPU line now qwen3.6-moe:35b-a3b @262k ctx; Sibling projects section -> serverAppConfigs. - AGENTS.md: overview reflects PP+MTP reality; sibling pointer.
This commit is contained in:
1 parent
f5a78db017
commit
7fe271e65d
5 files changed
+139
-31
No files matched your search
@@ -5,6 +5,33 @@ Split Qwen3.8-27B UD-Q8_K_XL across the Strix Halo iGPU (head) and the RTX
|
||||
morning-of runbook. Research background:
|
||||
[Qwen3.8 Strix optimizations](../../qwen3.8-strix-optimizations.md).
|
||||
|
||||
## Production state (promoted 2026-08-16)
|
||||
|
||||
This split is the production `frank-default-large` service (renamed from
|
||||
`frank-pp-head`, commit `48142f5`). Current state, verified against the
|
||||
live journal (2026-08-16, 15:03 load):
|
||||
|
||||
- Split `0.40,0.60` (not the test-day `1,1`): 40% of the layers on the
|
||||
3090 — its maximum under the 2x200700 f16 KV budget — 60% on the iGPU.
|
||||
- Pool `401400`, `PARALLEL=2`, live `n_ctx_slot = 200704`.
|
||||
- MTP enabled: `--spec-type draft-mtp --spec-draft-n-max 4` + ngram-mod
|
||||
fallback (48/64/24). Journal acceptance: MTP mean len 3.00
|
||||
(per-position 0.730/0.541/0.411/0.323).
|
||||
- Head GPU side: **ROCm0** (no `--device` flag; HIP registers before
|
||||
Vulkan). KV at load: 15680 MiB ROCm0 + 9408 MiB RPC0.
|
||||
- Unit `frank-default-large.service`, launched through
|
||||
`serve-pp-head.sh` + `frank-default-large.env`.
|
||||
|
||||
Open questions from the test day, resolved:
|
||||
|
||||
1. MTP (in the last layer group, iGPU side) drafts cleanly across the RPC
|
||||
boundary — see the acceptance figures above.
|
||||
2. Two parallel slots hold under production load: 150k-token prompts
|
||||
prefill and generate, slots turn over every few minutes, no RPC
|
||||
faults observed.
|
||||
3. Long prompts (~160k tokens) prefill on the ROCm0 side with no
|
||||
RPC-boundary anomaly in production traffic.
|
||||
|
||||
## Topology
|
||||
|
||||
```
|
||||
|
||||
Reference in new issue
Block a user