docs: distill the managed skill library into the vault (waves 1-7)

All ~173 managed skills read in full and merged, one polished reference
page per skill-cluster:

- LLM cross-cutting (17 pages): GGUF Compatibility, Llama Cpp Serving
  Sizing, Llama Cpp Multi-Device Splits, PCIe Link and Atomics, RDNA4
  Container Serving, RDNA4 MoE Expert Offload VRAM Budget, vLLM Boot
  Forensics, vLLM Porting and Compile Triage, LLM Bench Windows, LLM
  Performance Diagnosis, Model Swap and Rollback, Tcclaviger Fork Ops,
  Heterogeneous Split Triage, FreeToken Builds + deltas folded into the
  retired runbooks (GLM DFlash2 port laws, Lucebox dual-R9700, R9V
  qualification pipeline, FLM pack registration, ik hybrid envelope,
  frank-medium split rebalance, llama.cpp GGUF preflight, batch-ubatch
  split section, bench-window D-state/stale-read notes, second-endpoint
  bring-up).
- Architecture (3): Stale Signal Audit, Quadlet Lifecycle, Deploy State
  Triage.
- Operations (6 + index): Remote Job Runner, Payload and Disk Reclaim,
  Container Host Reachability, Fleet Ops Playbook, Fleet Audit and
  Posture, SSH and Remote Access.
- Infrastructure: Observability Provisioning + Homepage Status Probes;
  updates to Authentik, Anubis, LAN-DNS (per-client DoH bypass, Porkbun
  zone ops), Swap Configuration, OpenWebUI, Hermes (pre-existing page
  preserved and extended).
- FrankBook-Pro: ARR-Stack filled (API split, schema-first probing,
  auth-posture, auth-config trap); qBittorrent E2E verification.
- Desktop/Agent/Research: Noctalia and Niri, OMP Harness, Agent Docs and
  Edits, Command Center Dotfiles, PICO-8 Asset Standards.
- Indexes: LLM Cross-Cutting References section, Operations/ and Agent/
  index pages, root-index Skill Distillation section, AGENTS.md pointer.
- log.md carries the wave plan.

check-vault-links.py 0 unresolved (185 pages); ./tests/run.sh 366/366.
This commit is contained in:
Frank committed 2026-10-05 21:19:15 -07:00
1 parent 8618ec6dbe
commit 33eae71d33
43 files changed
+5436 -2

No files matched your search

@@ -146,3 +146,70 @@ systemctl --user start frank-medium`) and stop/mask frank-primary. The
_Filled in after the cutover: health, real completion, tool call, Open WebUI
completion, measured prefill/decode, `systemctl --user is-enabled/is-active`
state._
## Dual-R9700 follow-up (2026-09-28 probe era)
After this single-card runbook was retired, the DS41 engine got a dual-R9700
attempt on the `feat/ds41-engine` branch (`main` rejects the checkpoint with
`unsupported model architecture 'deepseek41'`). Findings that outlive the
probe:
**Budget both cards explicitly, or you waste half the box.** The engine
sizes the streamed-expert cache LAST, after the model, so uncapped it absorbs
every spare byte on the stream device — symptom: card1 at 33.0/34.2 GB
looking busy while card0 sits at 14.2/34.2 GB; the busy card is holding a
cache of things still streaming from NVMe. Pin all four:
| env | card | meaning |
|---|---|---|
| `LUCE_EXPERT_BUDGET_MB` | hip:0 | primary hot experts (~11.96 MiB each) |
| `LUCE_EXPERT_SECONDARY_BUDGET_MB` | hip:1 | secondary experts (needs `LUCE_DS4_MOE_TP*`) |
| `LUCE_EXPERT_STREAM_CACHE_MB` | hip:1 | cap the auto-sized stream cache |
| `LUCE_EXPERT_STREAM_DEVICE=1` | hip:1 | which card holds the slots and computes them |
Working full-card shape: 21000 + 20000 + 8000 → card0 ~28.6/31.86 GiB,
card1 ~27.3/31.86 GiB, resident experts 3,360 of 15,360.
**Expert TP DOES work on gfx1201+gfx1201.** A retired quadlet recorded
`LUCE_DS4_MOE_TP_INPROC` as "qualified for gfx1151+gfx1201, not
gfx1201+gfx1201" — that is WRONG; peer access negotiates and the secondary
tier comes up clean: `[deepseek4-moe-tp] enabled mode=in-process local=hip:0
secondary=hip:1`. It is the only lever that materially raises resident
expert count, and resident count is the entire bottleneck. Confirm every run
with `expert ownership:` and `hybrid experts ready:` lines.
**The placement file must live under the mounted models dir.** JSON
`{"owners": [...]}`, 15,360 entries indexed `layer*384+expert`, values
`primary`/`secondary`/`stream`. It is a ranking, not a promise: each device
keeps what fits its budget and demotes the rest. Anywhere outside
`/models/...` gives `cannot open expert placement` and a SILENT single-tier
fallback — a silent-wrongness failure, not a crash.
**V4.1 refuses layer split and layer-major prefill** — only `--ds4-prefill
exact` (token-by-token) is legal on a 32 GiB card and `--ds4-fused-decode`
is refused there. With ~22% of experts resident each token pulls ~187 cold
experts × ~12 MiB; if `/proc/<pid>/io` `read_bytes` climbs into terabytes
the model is larger than the box's RAM and you are in page-cache thrash at
the NVMe ceiling — a capacity verdict, not a tuning problem. Low GPU util
with a pegged link is the EXPECTED signature of PCIe-expert-stream-bound
decode, not a broken run.
**Harness traps, each of which produced a wrong number:** never `env -i` the
podman call (rootless podman runs as `dank`; `env -i HOME=/root` makes
PODMAN try `/root/.config` and die `stat /root/.config: permission denied`
before the container exists — pass container env with `-e`); never put a
large prompt on curl's argv (`Argument list too long` at ~128 KB — use
`--data-binary @file`); size prompts by growing the file until `wc -c`
reaches the byte target AND assert depth against the engine's own
`usage.prompt_tokens` (nominal 4 chars/token was really 5.73; a chunk loop
that appends a fixed ~152-byte line per nominal 200-byte chunk undershot
30%); the server runs as the container's PID 1, so tear down with
`podman rm -f`, never host `pkill`; drop page cache between arms
(`sync; echo 3 > /proc/sys/vm/drop_caches`) or the first arm reads warm and
every later one is a lie.
Read the placement log — it is the whole diagnosis
(`arch=deepseek41 layers=40 embd=5120 experts=384 used=6 ff=2304
hash_layers=0` — hash_layers=0 is correct for V4.1; `engram: layers=[1,14]
… 2 table(s) embedded (not mapped)` — the 189.13 GiB Engram tables are never
mapped, so they do not evict the expert page cache).