Frank
9bfa0be85d
- 31 new managed skills (anubis/arr/authentik/hermes/freetoken/podman/llm-swap runbooks etc.) now tracked. - Removed superseded skills: omp_harness_config, peon_ping_integration (replaced by user-authored peon-ping-* managed skills), rocmfp4-strix-benchmark-orchestration. - New memory-root skills: niri_compositor_ops, omp_git_workflow, omp_model_config, quadlet-media-migration, homelab-service-triage, hermes-dankclaw-ops, frank-default-large-ops, caddy-auth-dns-playbook.
2.9 KiB
2.9 KiB
name, description
| name | description |
|---|---|
| flm-npu-pack-registration | Runbook for registering a custom FastFlowLM NPU2 model pack on a tag that isn't in FLM's built-in catalog (Dankclaw :7771), including the LFS-pointer and vision-weight traps and the NPU hwctx contention rule. |
FLM custom-tag pack registration (Dankclaw NPU endpoint)
Use when adding a non-catalog NPU2 pack (e.g. an extraction or fine-tuned model) to the FLM registry served by frank-npu on port 7771. Companion diagnosis skill: flm-tag-load-failure-diagnosis.
Registry facts
- User registry:
/home/dank/.config/flm/model_list.json, shape{"model_path": "models", "models": {"<tag>": {"<size>": {name, url, file_url, size, flm_min_version, files[], vlm, default_context_length, max_prefill_len, details{format:"NPU2", family, ...}, label[]}}}}}. - Serve resolves tags through
FLM_CONFIG_PATHat startup;flm listwith the env var set lists custom tags (⏬= not downloaded).
Procedure
- Back up the registry JSON first (
cp -a). - Insert the entry via a python json edit + atomic
os.replace. Setfilesto the FULL HF file list — includingvision_weight.q4nxeven for text-only packs: the qwen3.5-family loader opens it unconditionally and a missing file makes the first request HANG silently (no error log). - Do not use
flm pull <tag>for custom entries — it throwsjson.exception out_of_range key '<tag>:<size>' not foundwhile building the download list. Place files manually into/home/dank/.config/flm/models/<PackName>/with:curl -sfL "https://huggingface.co/<repo>/resolve/main/<file>?download=true" -o <file>Plainraw/main/<file>returns 135-byte LFS pointer stubs, not weights — verify byte counts against the HF tree API before trusting a download. - Kernel packs: copy from the same-shape family pack (shape-match
config.jsonon hidden_size/num_hidden_layers/head_dim first) into/opt/fastflowlm/share/flm/xclbins/<PackName>/viasudo cp -a(root-owned dir). - Deploy/restart the serve unit (deploy path, not ad-hoc restarts) so the new process loads with all files present.
- Verify with a REAL completion request on 7771 (
model: "<tag>:<size>") —/v1/modelsalone proves registration, not loadability.
Hard constraints
- Only ONE
flm serveprocess can hold the NPU hardware context. A second concurrent serve process fails every load withDRM_IOCTL_AMDXDNA_CREATE_HWCTX err=-22(aie2 mgmt msg opcode 0x2 status 0x2000003). Never run scratch/verify instances while production holds the NPU — verify via the deployed service instead. - Tag switching happens INSIDE one serve process; first request after a model switch pays ~6–11 s cold-load cost (visible in
usage.load_duration). - Over ssh, never
pkill -f "<pattern>"with a pattern present in your own command line — it kills your own session; use bracket-guarded patterns ([q]wen3.5:4b) or exact PIDs.