Frank
9bfa0be85d
- 31 new managed skills (anubis/arr/authentik/hermes/freetoken/podman/llm-swap runbooks etc.) now tracked. - Removed superseded skills: omp_harness_config, peon_ping_integration (replaced by user-authored peon-ping-* managed skills), rocmfp4-strix-benchmark-orchestration. - New memory-root skills: niri_compositor_ops, omp_git_workflow, omp_model_config, quadlet-media-migration, homelab-service-triage, hermes-dankclaw-ops, frank-default-large-ops, caddy-auth-dns-playbook.
28 lines
2.9 KiB
Markdown
28 lines
2.9 KiB
Markdown
---
|
||
name: flm-npu-pack-registration
|
||
description: "Runbook for registering a custom FastFlowLM NPU2 model pack on a tag that isn't in FLM's built-in catalog (Dankclaw :7771), including the LFS-pointer and vision-weight traps and the NPU hwctx contention rule."
|
||
---
|
||
|
||
# FLM custom-tag pack registration (Dankclaw NPU endpoint)
|
||
|
||
Use when adding a non-catalog NPU2 pack (e.g. an extraction or fine-tuned model) to the FLM registry served by `frank-npu` on port 7771. Companion diagnosis skill: `flm-tag-load-failure-diagnosis`.
|
||
|
||
## Registry facts
|
||
- User registry: `/home/dank/.config/flm/model_list.json`, shape `{"model_path": "models", "models": {"<tag>": {"<size>": {name, url, file_url, size, flm_min_version, files[], vlm, default_context_length, max_prefill_len, details{format:"NPU2", family, ...}, label[]}}}}}`.
|
||
- Serve resolves tags through `FLM_CONFIG_PATH` at startup; `flm list` with the env var set lists custom tags (`⏬` = not downloaded).
|
||
|
||
## Procedure
|
||
1. Back up the registry JSON first (`cp -a`).
|
||
2. Insert the entry via a python json edit + atomic `os.replace`. Set `files` to the FULL HF file list — **including `vision_weight.q4nx`** even for text-only packs: the qwen3.5-family loader opens it unconditionally and a missing file makes the first request HANG silently (no error log).
|
||
3. **Do not use `flm pull <tag>` for custom entries** — it throws `json.exception out_of_range key '<tag>:<size>' not found` while building the download list. Place files manually into `/home/dank/.config/flm/models/<PackName>/` with:
|
||
`curl -sfL "https://huggingface.co/<repo>/resolve/main/<file>?download=true" -o <file>`
|
||
Plain `raw/main/<file>` returns 135-byte LFS pointer stubs, not weights — verify byte counts against the HF tree API before trusting a download.
|
||
4. Kernel packs: copy from the same-shape family pack (shape-match `config.json` on hidden_size/num_hidden_layers/head_dim first) into `/opt/fastflowlm/share/flm/xclbins/<PackName>/` via `sudo cp -a` (root-owned dir).
|
||
5. Deploy/restart the serve unit (deploy path, not ad-hoc restarts) so the new process loads with all files present.
|
||
6. Verify with a REAL completion request on 7771 (`model: "<tag>:<size>"`) — `/v1/models` alone proves registration, not loadability.
|
||
|
||
## Hard constraints
|
||
- Only ONE `flm serve` process can hold the NPU hardware context. A second concurrent serve process fails every load with `DRM_IOCTL_AMDXDNA_CREATE_HWCTX err=-22` (aie2 mgmt msg opcode 0x2 status 0x2000003). Never run scratch/verify instances while production holds the NPU — verify via the deployed service instead.
|
||
- Tag switching happens INSIDE one serve process; first request after a model switch pays ~6–11 s cold-load cost (visible in `usage.load_duration`).
|
||
- Over ssh, never `pkill -f "<pattern>"` with a pattern present in your own command line — it kills your own session; use bracket-guarded patterns (`[q]wen3.5:4b`) or exact PIDs.
|