docs(NPU page): era-tag host-RAM and PP-rate restatements
One fact, one owner (de-dup campaign C4): - `FastFlowLM NPU Inference.md` carried the host RAM figures three times (29 GiB budget input kept as era data; the 2026-10-18 `free -g` re-reads of 121/246 GiB cut -> [[Dankclaw System Specs]]/[[Dankserver System Specs]]). - The 14-18 t/s PP-split decode is this page's own solo-held era measurement: kept, tagged to the 2026-08 era with [[frank-redo-2026-08-17]]. - Promotion section: `frank-zealot` "now runs the NPU endpoint" era-tagged (the name has been reused twice since; the NPU unit is `frank-npu`, masked). - Repaired the Test Runbook splice introduced by the range edit above. Refs: vault de-dup campaign, cluster C4.
This commit is contained in:
1 parent
a518fc7c22
commit
e32df1b04f
2 files changed
+46
-17
No files matched your search
@@ -57,7 +57,10 @@ Key properties, per the project README and docs:
|
||||
|
||||
## NPU Hardware on Dankclaw
|
||||
|
||||
Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU rated at 50 TOPS (vendor spec: "NPU TOPS — Up to 50 TOPS", [amd.com product page](https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html)). It sits beside the Radeon 8060S iGPU and shares the same unified memory pool — 30 GiB exposed to the OS in the 2026-08 reading below; the box presents **121 GiB** today (`free -g` total 121, read 2026-10-18).
|
||||
Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU
|
||||
with 50 TOPS of compute and, crucially, **no dedicated VRAM** - it shares system memory
|
||||
via the IOMMU SVA mapping. The budget this page reasons about - 29 GiB usable at
|
||||
4800 MT/s LPDDR5X - is the 2026-08 host state; the live figure is [[Dankclaw System Specs]].
|
||||
|
||||
Live verification on Dankclaw, 2026-08-16:
|
||||
|
||||
@@ -89,10 +92,9 @@ Budget math for this fleet:
|
||||
| Dankserver (Ryzen 7 7700X) | 62 GB | no NPU |
|
||||
| FLM bench box (Kraken Point) | 32 or 96 GB | ~16 or ~48 GB |
|
||||
|
||||
2026-10-18 re-read: Dankclaw's OS now reports **121 GiB** (`free -g`) and
|
||||
Dankserver **246 GiB** (`free -g`), so the half-of-RAM rule would price the NPU
|
||||
budget at ~60 GiB and the 7700X box still has no NPU. The 30 GiB / 62 GB rows
|
||||
are the 2026-08-16 state; the table is moot while no NPU endpoint is deployed.
|
||||
The half-of-RAM gate scales with host RAM - live figures in [[Dankclaw System Specs]]
|
||||
and [[Dankserver System Specs]]. The live gate on Dankclaw is still `flm validate`
|
||||
(it owns the current number).
|
||||
|
||||
Weight sizes (approximate, Q4-class):
|
||||
|
||||
@@ -159,7 +161,10 @@ Qwen3.6-35B-A3B (Q4_KS), FLM v0.9.45, Kraken Point **96 GB DRAM**:
|
||||
| Decode t/s | 13.65 | 13.41 | 13.09 | 12.51 | 11.24 | 9.51 |
|
||||
| Prefill t/s | 78.98 | 118.04 | 156.43 | 197.93 | 218.84 | 221.96 |
|
||||
|
||||
Context: the current PP split (head + 3090 worker) decodes at 14-18 t/s for the 27B dense model. A 35B-A3B at ~10-14 t/s on an NPU with 48 GB of RAM budget is not a speed win over the existing topology.
|
||||
- PP and NPU together cost 2.2 GiB + NPU model + ~2 GiB of headroom. The then-current
|
||||
PP split (head + 3090 worker, 2026-08 era - see [[frank-redo-2026-08-17]]) decoded at
|
||||
14-18 t/s for the 27B dense model, so an NPU offload would free the head or upgrade
|
||||
the worker's model.
|
||||
|
||||
The vendor also claims "over 10x more power-efficient" versus GPU. That is a marketing claim. We have not measured it.
|
||||
|
||||
@@ -219,7 +224,12 @@ Notes:
|
||||
|
||||
## Promotion (2026-08-16)
|
||||
|
||||
`frank-zealot` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
|
||||
`frank-zealot` then ran the NPU endpoint on Dankclaw (2026-08; the name was reused
|
||||
twice after this window - see the banner - and the NPU unit today is `frank-npu`,
|
||||
masked). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`),
|
||||
wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`.
|
||||
`LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap
|
||||
`FLM_MODEL` in the env to switch.
|
||||
|
||||
### Boot policy (2026-08-17)
|
||||
|
||||
@@ -256,11 +266,13 @@ The 4-stream wall times form a near-perfect 10.7 s ladder. Each stream self-repo
|
||||
Consequence: the NPU endpoint is a single lane. It suits low-frequency background work (triage, summarization, agent side-tasks). It does not suit interactive chat for several simultaneous users. The 27B llama.cpp service, by contrast, runs PARALLEL=2 with true concurrent slots.
|
||||
|
||||
## Test Runbook
|
||||
|
||||
This runbook targets a retired endpoint; keep it as the procedure if the NPU is
|
||||
ever revived. Do not run large model loads while the production service holds
|
||||
most of the unified pool — 30 GiB exposed to the OS in the 2026-08 reading,
|
||||
**121 GiB** today (`free -g`, read 2026-10-18). Check free RAM first, or stop
|
||||
ever revived.
|
||||
|
||||
**Host RAM availability.** The gate is half of system RAM by default - a relic of the
|
||||
install: the endpoint ran on a machine that presented 30 GiB of system RAM; on today's
|
||||
box that ceiling is far larger (live figure: [[Dankclaw System Specs]]). Check free RAM
|
||||
first, or stop
|
||||
the service the way the bench policy requires (`sudo systemctl stop
|
||||
frank-dragoon && sudo systemctl mask frank-dragoon` — self-healing would
|
||||
otherwise restart it). The NPU shares the unified pool with everything else on
|
||||
|
||||
Reference in new issue
Block a user