docs: distill the managed skill library into the vault (waves 1-7)
All ~173 managed skills read in full and merged, one polished reference page per skill-cluster: - LLM cross-cutting (17 pages): GGUF Compatibility, Llama Cpp Serving Sizing, Llama Cpp Multi-Device Splits, PCIe Link and Atomics, RDNA4 Container Serving, RDNA4 MoE Expert Offload VRAM Budget, vLLM Boot Forensics, vLLM Porting and Compile Triage, LLM Bench Windows, LLM Performance Diagnosis, Model Swap and Rollback, Tcclaviger Fork Ops, Heterogeneous Split Triage, FreeToken Builds + deltas folded into the retired runbooks (GLM DFlash2 port laws, Lucebox dual-R9700, R9V qualification pipeline, FLM pack registration, ik hybrid envelope, frank-medium split rebalance, llama.cpp GGUF preflight, batch-ubatch split section, bench-window D-state/stale-read notes, second-endpoint bring-up). - Architecture (3): Stale Signal Audit, Quadlet Lifecycle, Deploy State Triage. - Operations (6 + index): Remote Job Runner, Payload and Disk Reclaim, Container Host Reachability, Fleet Ops Playbook, Fleet Audit and Posture, SSH and Remote Access. - Infrastructure: Observability Provisioning + Homepage Status Probes; updates to Authentik, Anubis, LAN-DNS (per-client DoH bypass, Porkbun zone ops), Swap Configuration, OpenWebUI, Hermes (pre-existing page preserved and extended). - FrankBook-Pro: ARR-Stack filled (API split, schema-first probing, auth-posture, auth-config trap); qBittorrent E2E verification. - Desktop/Agent/Research: Noctalia and Niri, OMP Harness, Agent Docs and Edits, Command Center Dotfiles, PICO-8 Asset Standards. - Indexes: LLM Cross-Cutting References section, Operations/ and Agent/ index pages, root-index Skill Distillation section, AGENTS.md pointer. - log.md carries the wave plan. check-vault-links.py 0 unresolved (185 pages); ./tests/run.sh 366/366.
This commit is contained in:
1 parent
8618ec6dbe
commit
33eae71d33
43 files changed
+5436
-2
No files matched your search
@@ -146,3 +146,70 @@ systemctl --user start frank-medium`) and stop/mask frank-primary. The
|
||||
_Filled in after the cutover: health, real completion, tool call, Open WebUI
|
||||
completion, measured prefill/decode, `systemctl --user is-enabled/is-active`
|
||||
state._
|
||||
|
||||
## Dual-R9700 follow-up (2026-09-28 probe era)
|
||||
|
||||
After this single-card runbook was retired, the DS41 engine got a dual-R9700
|
||||
attempt on the `feat/ds41-engine` branch (`main` rejects the checkpoint with
|
||||
`unsupported model architecture 'deepseek41'`). Findings that outlive the
|
||||
probe:
|
||||
|
||||
**Budget both cards explicitly, or you waste half the box.** The engine
|
||||
sizes the streamed-expert cache LAST, after the model, so uncapped it absorbs
|
||||
every spare byte on the stream device — symptom: card1 at 33.0/34.2 GB
|
||||
looking busy while card0 sits at 14.2/34.2 GB; the busy card is holding a
|
||||
cache of things still streaming from NVMe. Pin all four:
|
||||
|
||||
| env | card | meaning |
|
||||
|---|---|---|
|
||||
| `LUCE_EXPERT_BUDGET_MB` | hip:0 | primary hot experts (~11.96 MiB each) |
|
||||
| `LUCE_EXPERT_SECONDARY_BUDGET_MB` | hip:1 | secondary experts (needs `LUCE_DS4_MOE_TP*`) |
|
||||
| `LUCE_EXPERT_STREAM_CACHE_MB` | hip:1 | cap the auto-sized stream cache |
|
||||
| `LUCE_EXPERT_STREAM_DEVICE=1` | hip:1 | which card holds the slots and computes them |
|
||||
|
||||
Working full-card shape: 21000 + 20000 + 8000 → card0 ~28.6/31.86 GiB,
|
||||
card1 ~27.3/31.86 GiB, resident experts 3,360 of 15,360.
|
||||
|
||||
**Expert TP DOES work on gfx1201+gfx1201.** A retired quadlet recorded
|
||||
`LUCE_DS4_MOE_TP_INPROC` as "qualified for gfx1151+gfx1201, not
|
||||
gfx1201+gfx1201" — that is WRONG; peer access negotiates and the secondary
|
||||
tier comes up clean: `[deepseek4-moe-tp] enabled mode=in-process local=hip:0
|
||||
secondary=hip:1`. It is the only lever that materially raises resident
|
||||
expert count, and resident count is the entire bottleneck. Confirm every run
|
||||
with `expert ownership:` and `hybrid experts ready:` lines.
|
||||
|
||||
**The placement file must live under the mounted models dir.** JSON
|
||||
`{"owners": [...]}`, 15,360 entries indexed `layer*384+expert`, values
|
||||
`primary`/`secondary`/`stream`. It is a ranking, not a promise: each device
|
||||
keeps what fits its budget and demotes the rest. Anywhere outside
|
||||
`/models/...` gives `cannot open expert placement` and a SILENT single-tier
|
||||
fallback — a silent-wrongness failure, not a crash.
|
||||
|
||||
**V4.1 refuses layer split and layer-major prefill** — only `--ds4-prefill
|
||||
exact` (token-by-token) is legal on a 32 GiB card and `--ds4-fused-decode`
|
||||
is refused there. With ~22% of experts resident each token pulls ~187 cold
|
||||
experts × ~12 MiB; if `/proc/<pid>/io` `read_bytes` climbs into terabytes
|
||||
the model is larger than the box's RAM and you are in page-cache thrash at
|
||||
the NVMe ceiling — a capacity verdict, not a tuning problem. Low GPU util
|
||||
with a pegged link is the EXPECTED signature of PCIe-expert-stream-bound
|
||||
decode, not a broken run.
|
||||
|
||||
**Harness traps, each of which produced a wrong number:** never `env -i` the
|
||||
podman call (rootless podman runs as `dank`; `env -i HOME=/root` makes
|
||||
PODMAN try `/root/.config` and die `stat /root/.config: permission denied`
|
||||
before the container exists — pass container env with `-e`); never put a
|
||||
large prompt on curl's argv (`Argument list too long` at ~128 KB — use
|
||||
`--data-binary @file`); size prompts by growing the file until `wc -c`
|
||||
reaches the byte target AND assert depth against the engine's own
|
||||
`usage.prompt_tokens` (nominal 4 chars/token was really 5.73; a chunk loop
|
||||
that appends a fixed ~152-byte line per nominal 200-byte chunk undershot
|
||||
30%); the server runs as the container's PID 1, so tear down with
|
||||
`podman rm -f`, never host `pkill`; drop page cache between arms
|
||||
(`sync; echo 3 > /proc/sys/vm/drop_caches`) or the first arm reads warm and
|
||||
every later one is a lie.
|
||||
|
||||
Read the placement log — it is the whole diagnosis
|
||||
(`arch=deepseek41 layers=40 embd=5120 experts=384 used=6 ff=2304
|
||||
hash_layers=0` — hash_layers=0 is correct for V4.1; `engram: layers=[1,14]
|
||||
… 2 table(s) embedded (not mapped)` — the 189.13 GiB Engram tables are never
|
||||
mapped, so they do not evict the expert page cache).
|
||||
Reference in new issue
Block a user