docs(NPU page): era-tag host-RAM and PP-rate restatements

One fact, one owner (de-dup campaign C4):

- `FastFlowLM NPU Inference.md` carried the host RAM figures three times
  (29 GiB budget input kept as era data; the 2026-10-18 `free -g` re-reads
  of 121/246 GiB cut -> [[Dankclaw System Specs]]/[[Dankserver System Specs]]).
- The 14-18 t/s PP-split decode is this page's own solo-held era measurement:
  kept, tagged to the 2026-08 era with [[frank-redo-2026-08-17]].
- Promotion section: `frank-zealot` "now runs the NPU endpoint" era-tagged
  (the name has been reused twice since; the NPU unit is `frank-npu`, masked).
- Repaired the Test Runbook splice introduced by the range edit above.

Refs: vault de-dup campaign, cluster C4.
This commit is contained in:
Frank committed 2026-10-19 01:58:03 -07:00
1 parent a518fc7c22
commit e32df1b04f
2 files changed
+46 -17

No files matched your search

@@ -57,7 +57,10 @@ Key properties, per the project README and docs:
## NPU Hardware on Dankclaw
Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU rated at 50 TOPS (vendor spec: "NPU TOPS — Up to 50 TOPS", [amd.com product page](https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html)). It sits beside the Radeon 8060S iGPU and shares the same unified memory pool — 30 GiB exposed to the OS in the 2026-08 reading below; the box presents **121 GiB** today (`free -g` total 121, read 2026-10-18).
Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU
with 50 TOPS of compute and, crucially, **no dedicated VRAM** - it shares system memory
via the IOMMU SVA mapping. The budget this page reasons about - 29 GiB usable at
4800 MT/s LPDDR5X - is the 2026-08 host state; the live figure is [[Dankclaw System Specs]].
Live verification on Dankclaw, 2026-08-16:
@@ -89,10 +92,9 @@ Budget math for this fleet:
| Dankserver (Ryzen 7 7700X) | 62 GB | no NPU |
| FLM bench box (Kraken Point) | 32 or 96 GB | ~16 or ~48 GB |
2026-10-18 re-read: Dankclaw's OS now reports **121 GiB** (`free -g`) and
Dankserver **246 GiB** (`free -g`), so the half-of-RAM rule would price the NPU
budget at ~60 GiB and the 7700X box still has no NPU. The 30 GiB / 62 GB rows
are the 2026-08-16 state; the table is moot while no NPU endpoint is deployed.
The half-of-RAM gate scales with host RAM - live figures in [[Dankclaw System Specs]]
and [[Dankserver System Specs]]. The live gate on Dankclaw is still `flm validate`
(it owns the current number).
Weight sizes (approximate, Q4-class):
@@ -159,7 +161,10 @@ Qwen3.6-35B-A3B (Q4_KS), FLM v0.9.45, Kraken Point **96 GB DRAM**:
| Decode t/s | 13.65 | 13.41 | 13.09 | 12.51 | 11.24 | 9.51 |
| Prefill t/s | 78.98 | 118.04 | 156.43 | 197.93 | 218.84 | 221.96 |
Context: the current PP split (head + 3090 worker) decodes at 14-18 t/s for the 27B dense model. A 35B-A3B at ~10-14 t/s on an NPU with 48 GB of RAM budget is not a speed win over the existing topology.
- PP and NPU together cost 2.2 GiB + NPU model + ~2 GiB of headroom. The then-current
PP split (head + 3090 worker, 2026-08 era - see [[frank-redo-2026-08-17]]) decoded at
14-18 t/s for the 27B dense model, so an NPU offload would free the head or upgrade
the worker's model.
The vendor also claims "over 10x more power-efficient" versus GPU. That is a marketing claim. We have not measured it.
@@ -219,7 +224,12 @@ Notes:
## Promotion (2026-08-16)
`frank-zealot` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
`frank-zealot` then ran the NPU endpoint on Dankclaw (2026-08; the name was reused
twice after this window - see the banner - and the NPU unit today is `frank-npu`,
masked). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`),
wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`.
`LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap
`FLM_MODEL` in the env to switch.
### Boot policy (2026-08-17)
@@ -256,11 +266,13 @@ The 4-stream wall times form a near-perfect 10.7 s ladder. Each stream self-repo
Consequence: the NPU endpoint is a single lane. It suits low-frequency background work (triage, summarization, agent side-tasks). It does not suit interactive chat for several simultaneous users. The 27B llama.cpp service, by contrast, runs PARALLEL=2 with true concurrent slots.
## Test Runbook
This runbook targets a retired endpoint; keep it as the procedure if the NPU is
ever revived. Do not run large model loads while the production service holds
most of the unified pool — 30 GiB exposed to the OS in the 2026-08 reading,
**121 GiB** today (`free -g`, read 2026-10-18). Check free RAM first, or stop
ever revived.
**Host RAM availability.** The gate is half of system RAM by default - a relic of the
install: the endpoint ran on a machine that presented 30 GiB of system RAM; on today's
box that ceiling is far larger (live figure: [[Dankclaw System Specs]]). Check free RAM
first, or stop
the service the way the bench policy requires (`sudo systemctl stop
frank-dragoon && sudo systemctl mask frank-dragoon` — self-healing would
otherwise restart it). The NPU shares the unified pool with everything else on