docs: distill the managed skill library into the vault (waves 1-7)
All ~173 managed skills read in full and merged, one polished reference page per skill-cluster: - LLM cross-cutting (17 pages): GGUF Compatibility, Llama Cpp Serving Sizing, Llama Cpp Multi-Device Splits, PCIe Link and Atomics, RDNA4 Container Serving, RDNA4 MoE Expert Offload VRAM Budget, vLLM Boot Forensics, vLLM Porting and Compile Triage, LLM Bench Windows, LLM Performance Diagnosis, Model Swap and Rollback, Tcclaviger Fork Ops, Heterogeneous Split Triage, FreeToken Builds + deltas folded into the retired runbooks (GLM DFlash2 port laws, Lucebox dual-R9700, R9V qualification pipeline, FLM pack registration, ik hybrid envelope, frank-medium split rebalance, llama.cpp GGUF preflight, batch-ubatch split section, bench-window D-state/stale-read notes, second-endpoint bring-up). - Architecture (3): Stale Signal Audit, Quadlet Lifecycle, Deploy State Triage. - Operations (6 + index): Remote Job Runner, Payload and Disk Reclaim, Container Host Reachability, Fleet Ops Playbook, Fleet Audit and Posture, SSH and Remote Access. - Infrastructure: Observability Provisioning + Homepage Status Probes; updates to Authentik, Anubis, LAN-DNS (per-client DoH bypass, Porkbun zone ops), Swap Configuration, OpenWebUI, Hermes (pre-existing page preserved and extended). - FrankBook-Pro: ARR-Stack filled (API split, schema-first probing, auth-posture, auth-config trap); qBittorrent E2E verification. - Desktop/Agent/Research: Noctalia and Niri, OMP Harness, Agent Docs and Edits, Command Center Dotfiles, PICO-8 Asset Standards. - Indexes: LLM Cross-Cutting References section, Operations/ and Agent/ index pages, root-index Skill Distillation section, AGENTS.md pointer. - log.md carries the wave plan. check-vault-links.py 0 unresolved (185 pages); ./tests/run.sh 366/366.
This commit is contained in:
1 parent
8618ec6dbe
commit
33eae71d33
43 files changed
+5436
-2
No files matched your search
@@ -271,6 +271,83 @@ graph TD
|
||||
6. Step up in size: 9B, then gpt-oss-20b.
|
||||
7. Attempt `qwen3.6-moe:35b-a3b` only after the smaller runs succeed, and only with the production service stopped. Watch `free -h` during the model download and load.
|
||||
|
||||
## Registering packs absent from the catalog (NPU2)
|
||||
|
||||
Use when adding a non-catalog NPU2 pack (an extraction or fine-tuned model)
|
||||
to the FLM registry. Registry facts: user registry
|
||||
`/home/dank/.config/flm/model_list.json`, shape
|
||||
`{"model_path": "models", "models": {"<tag>": {"<size>": {name, url,
|
||||
file_url, size, flm_min_version, files[], vlm, default_context_length,
|
||||
max_prefill_len, details{format:"NPU2", family, …}, label[]}}}}`; serve
|
||||
resolves tags through `FLM_CONFIG_PATH` at startup, and `flm list` with the
|
||||
env var set lists custom tags (`⏬` = not downloaded).
|
||||
|
||||
1. Back up the registry JSON first (`cp -a`).
|
||||
2. Insert the entry via a python json edit + atomic `os.replace`. Set
|
||||
`files` to the FULL HF file list — **including `vision_weight.q4nx` even
|
||||
for text-only packs**: the qwen3.5-family loader opens it unconditionally
|
||||
and a missing file makes the first request HANG silently (no error log).
|
||||
3. **Do not use `flm pull <tag>` for custom entries** — it throws
|
||||
`json.exception out_of_range key '<tag>:<size>' not found` while building
|
||||
the download list. Place files manually into
|
||||
`/home/dank/.config/flm/models/<PackName>/` with
|
||||
`curl -sfL "https://huggingface.co/<repo>/resolve/main/<file>?download=true"`
|
||||
— plain `raw/main/<file>` returns 135-byte LFS pointer stubs, not
|
||||
weights; verify byte counts against the HF tree API before trusting a
|
||||
download.
|
||||
4. Kernel packs: copy from the same-shape family pack (shape-match
|
||||
`config.json` on hidden_size/num_hidden_layers/head_dim first) into
|
||||
`/opt/fastflowlm/share/flm/xclbins/<PackName>/` via `sudo cp -a`
|
||||
(root-owned dir).
|
||||
5. Deploy/restart the serve unit (the deploy path, not ad-hoc restarts) so
|
||||
the new process loads with all files present.
|
||||
6. Verify with a REAL completion request (`model: "<tag>:<size>"`) —
|
||||
`/v1/models` alone proves registration, not loadability.
|
||||
|
||||
Hard constraints: only ONE `flm serve` process can hold the NPU hardware
|
||||
context — a second concurrent serve process fails every load with
|
||||
`DRM_IOCTL_AMDXDNA_CREATE_HWCTX err=-22` (aie2 mgmt msg opcode 0x2 status
|
||||
0x2000003); never run scratch/verify instances while production holds the
|
||||
NPU. Tag switching happens INSIDE one serve process; the first request
|
||||
after a model switch pays ~6–11 s cold-load cost (visible in
|
||||
`usage.load_duration`). Over ssh, never `pkill -f "<pattern>"` with a
|
||||
pattern present in your own command line — use bracket-guarded patterns
|
||||
(`[q]wen3.5:4b`) or exact PIDs.
|
||||
|
||||
## Tag load-failure diagnosis (200 with an error body)
|
||||
|
||||
`POST /v1/chat/completions` answers **HTTP 200** with a flat
|
||||
`{"error":"Failed to load <tag> model!"}` and no `choices`; `/v1/completions`
|
||||
returns a nested OpenAI-shaped `server_error` (code 500) also under HTTP
|
||||
200. A status-code-only client reads this as a successful empty completion —
|
||||
**always check for the `error` key**. Two distinct causes, distinguished via
|
||||
`journalctl -u <flm-service> --full`:
|
||||
|
||||
1. **Unknown tag** (not in the registry): immediate error, nothing loaded.
|
||||
Correct behavior, not a bug.
|
||||
2. **Registered tag whose kernel pack is missing**: FLM resolves the tag,
|
||||
downloads missing weight files on the fly (first request hangs ~90–110 s),
|
||||
then fails `Failed to load model: No such file
|
||||
'/home/dank/.config/flm/xclbins/<PackName>/<kernel>.xclbin'`; subsequent
|
||||
attempts fail fast (~2 s). Root cause: the serve script exports
|
||||
`FLM_XCLBIN_PATH=/home/dank/.config/flm/xclbins` (user dir), so every
|
||||
kernel lookup resolves under the user dir, which only contains symlinks
|
||||
someone linked. **A tag listed in `/v1/models` is NOT proof it is
|
||||
loadable on this host.** Fix: one symlink per missing pack
|
||||
(`ln -s /opt/fastflowlm/share/flm/xclbins/<PackName>
|
||||
/home/dank/.config/flm/xclbins/<PackName>`) — verified live: after the
|
||||
symlink the first request loaded and answered in 7.4 s wall.
|
||||
|
||||
Switch-cost behavior: the `model` field routes per-request (no restart
|
||||
needed); a request whose model differs from the resident one triggers a full
|
||||
reload (7–11 s wall for an 8-token completion on ~4B-class packs);
|
||||
alternating tags pays the swap each time. Single-lane NPU: concurrent
|
||||
requests queue FIFO behind the current lock holder (`NPU Locked!` →
|
||||
`NPU Lock Released!`). Probe recipe: `/v1/models` for registered tags; grep
|
||||
the journal for `Failed to load model` (the quoted xclbin path names the
|
||||
missing pack dir exactly); diff `ls ~/.config/flm/xclbins/` (symlinks =
|
||||
installed) against `/opt/fastflowlm/share/flm/xclbins/` (available).
|
||||
|
||||
## Incident: qwen3.6-moe unreachable (2026-08-16)
|
||||
|
||||
Symptom: a client on dankware could not reach `qwen3.6-moe:35b-a3b` on port 52625. The journal shows three separate causes in the 14:19-15:03 window.
|
||||
|
||||
Reference in new issue
Block a user