Frank
135bb2194d
- config.yml: default/commit -> frank-medium (dankserver); task/tiny/smol/vision -> frank-small (dankclaw-3090); vision xhigh - models.yml: add llamacpp (dankclaw-3090) provider; frank-large preserve_thinking + context notes - niri: focus-ring/border/shadow layout, hot-corners off, re-add noctalia.kdl include; blankpad eDP-1 scale 1.0 - memories: flush rollouts; retire frank_large + marreta_authentik skills; add auth-chain/kv-pool/quadlet-log skills - managed-skills: 40+ new fleet runbook skills - agent/skills: import built-in skill set
4.2 KiB
name, description
| name | description |
|---|---|
| fleet-model-cache-purge-refresh | Clear and re-fetch model payloads on a Dank-fleet host that serves production LLM endpoints — inventory, protect live payloads, verify every fetch, restore the service, and use the refresh to detect upstream updates. |
Clearing a model cache on a serving host is a two-sided operation: the operator wants no stale payload, and the host must keep serving. Do it in this order, on the target host.
1. Inventory before touching anything
du -sh /home/dank/models/* | sort -h # payload dirs, biggest last
grep -rlE "models/|<model-name>" /home/dank/.config/containers/systemd/ /etc/systemd/system/*.service
Classify every dir: live service payload (referenced by a unit/env), retired runtime
payload, unreferenced model dir, tool/script dir, logs (evidence — keep). Check the
service env files (scripts/*.env) for MODEL_DIR/MODEL paths, not just unit Volume= lines:
a payload can be mounted by path and named by env. models/logs and small tool dirs are not cache.
2. Record fingerprints first
ls -l --block-size=1 <dir> gives sizes; for the LLM payloads compare each to the Hub's declared
size (/api/models/<repo>/tree/main?recursive=true&blobs=true). They can differ — see §6. If a
file is irreplaceable upstream (delisted revision, gated repo) say so before deleting it.
3. Protect live payloads by delete-then-refetch, never delete-only
Deleting a live payload leaves the endpoint unable to start. Know the recovery path before
deleting: some engines fetch their own weights (halogen: HALOGEN_DOWNLOAD, triggered when the
checkpoint is absent), others need a manual fetch (llama.cpp GGUF services: the serve script
refuses to start without the weights).
Bind-mount trap: podman refuses a mount whose source directory is missing —
Error: statfs /home/dank/models/<dir>: no such file or directory, with the unit looping on
Restart=on-failure. Recreate the directory empty (mkdir -p), and the engine's first-start
fetch runs.
4. Fetch with verification (never trust an exit code)
curl -o FILE URL exits 0 on 401/403 and writes the error body — a gated repo produced three
153-byte shards with identical sha256s and a cheerful DONE line. Always: -fL -C -, compare the
byte count against the Hub's declared size, and compare sha256sum against the LFS oid. Public
repos means no token in argv (it is world-readable in /proc/<pid>/cmdline).
Parts trap: mradermacher-style repos ship large quants as .part1ofN … .partNofN — a filter on
endswith('.gguf') makes those quants look absent. Fetch the parts, verify each, then
cat part1 part2 part3 > name.gguf in numeric order; the assembled file's size equals the sum
(concatenation is deterministic, so per-part hashes verify the whole).
5. Restore, then verify behaviour — not health alone
Put the quadlet back (mv from trash + systemctl --user daemon-reload), reset-failed, start;
root units need sudo. Watch the engine's own fetch/load lines in the journal, then check
/health, /v1/models, and a real completion on every restored endpoint. Read the engine's
memory ledger (kv pool / weights / blocks left) rather than assuming the config still fits.
6. A refresh is also an update check
Clearing and re-fetching only guarantees freshness at fetch time — if upstream has not changed,
the bytes come back identical, which is itself the finding. Compare recorded fingerprints against
the new local sizes and the Hub's declared sizes: sidecars are often versioned in place, so a
stale copy is invisible from its file name (observed: overlay.hgn 2,477,677,120 → 2,572,466,560
while the main checkpoint returned byte-identical).
7. Close out
- Clear test payloads once the benchmark using them is done, and keep the payload the next experiment needs.
- Leave the operator's decision points explicit: unreferenced-but-licensed dirs, other services' caches, and whether they want the payloads absent at their next run or fresh as of now.
- Record sizes/verdicts in the vault (runbook +
log.md); neverrmwithout explicit permission, and remember a same-filesystem trash reclaims no space.