agent: repoint roles to frank-medium/dankclaw-3090; sync niri tweaks, memories, managed skills

- config.yml: default/commit -> frank-medium (dankserver); task/tiny/smol/vision -> frank-small (dankclaw-3090); vision xhigh
- models.yml: add llamacpp (dankclaw-3090) provider; frank-large preserve_thinking + context notes
- niri: focus-ring/border/shadow layout, hot-corners off, re-add noctalia.kdl include; blankpad eDP-1 scale 1.0
- memories: flush rollouts; retire frank_large + marreta_authentik skills; add auth-chain/kv-pool/quadlet-log skills
- managed-skills: 40+ new fleet runbook skills
- agent/skills: import built-in skill set
This commit is contained in:
Frank committed 2026-09-17 12:27:08 -07:00
1 parent 83c9c92b3f
commit 135bb2194d
103 files changed
+5333 -170

No files matched your search

@@ -0,0 +1,29 @@
---
name: ik-llama-hybrid-moe-performance
description: Diagnose and tune ik_llama.cpp hybrid CPU/GPU MoE decode/prefill on the dankclaw 3090+Strix Halo frank-large deployment; includes the barrier-split root-cause method and the systemd-unit test-interference trap
---
# ik_llama.cpp hybrid MoE performance campaign (dankclaw frank-large)
Use when tuning or debugging decode/prefill speed of the ik_llama.cpp hybrid deployment (RTX 3090 + Strix Halo LPDDR, CPU-resident MoE experts) on dankclaw, or when a config experiment gives surprising numbers.
## Measured envelope (2026-09-16)
- Decode 3.8-4.3 tok/s, flat across thread counts (T1 ≈ T16). Prefill ~192 tok/s at 1.8k-32k prompts = the CPU AVX512-VNNI compute ceiling for CPU experts (batch amortizes expert reads; compute-bound, NOT memory-bound).
- Decode root cause: ~100 CPU↔GPU backend splits/token; GPU at 5% SM during decode (launch/sync-bound over the PCIe x4 dock); T16 profile shows ~90% of cycles in `pause` (ggml_barrier spin) because threads wait on the serial critical path. Memory placement (mmap/anon/THP/deferred) does NOT matter — all variants land 3.8-4.6.
- CUDA-graph capture aborts: qwen4exp's hc broadcasts use per-layer `ggml_repeat_4d` whose CUDA path does a capture-illegal op. Until ik gains capture-safe repeat/hc ops (mainline PR #28901 direction), graphs stay dead. Mainline llama.cpp itself runs 1.0 tok/s on this model (CPU MoE kernels 4x slower than ik iqk) — do not switch for speed.
- MTP/speculative impossible for mradermacher i1 GGUFs: the NextN head is stripped (no blk.48 tensors).
## Tuning levers, ranked
1. `-ncmoe 42` puts 6 layers' experts on the 3090: decode 4.54, prefill (small prompt) 32 — but VRAM 22/24.6 GiB at 262k ctx; needs an OOM-margin decision from the operator.
2. `--prefetch-experts` is a no-op when the page cache is resident.
3. `-thp` huge-mmap usually fails (`Cannot allocate memory`) and falls back silently.
## Traps
- The frank-large root systemd unit has Restart=always: `pkill`ing llama-server respawns it in ~4 s. For manual experiments: `sudo systemctl stop frank-large.service` first (root unit — NOT `systemctl --user`, that is stale halogen-era docs); verify `pgrep -xc llama-server` == 1 and 'HTTP server listening' in the current log before trusting any measurement.
- Any `-ot`/`-ncmoe` override disables mmap for the whole model load in ik (`create_tensors` returns false).
- `-ncmoe N` without `GGML_CUDA_NO_PINNED_WEIGHTS=1` page-locks ~108 GiB (co-tenant killer — the 2026-09-15 incident).
- `--defer-experts` defers only PLE; under `-ngl 999` the experts still target CUDA0 → cudaMalloc OOM (~79.7 GiB on a 24 GiB card).
- Never run `-j16` builds concurrently with decode measurements — both results are garbage.