Frank
135bb2194d
- config.yml: default/commit -> frank-medium (dankserver); task/tiny/smol/vision -> frank-small (dankclaw-3090); vision xhigh - models.yml: add llamacpp (dankclaw-3090) provider; frank-large preserve_thinking + context notes - niri: focus-ring/border/shadow layout, hot-corners off, re-add noctalia.kdl include; blankpad eDP-1 scale 1.0 - memories: flush rollouts; retire frank_large + marreta_authentik skills; add auth-chain/kv-pool/quadlet-log skills - managed-skills: 40+ new fleet runbook skills - agent/skills: import built-in skill set
74 lines
4.2 KiB
Markdown
74 lines
4.2 KiB
Markdown
---
|
|
name: fleet-model-cache-purge-refresh
|
|
description: "Clear and re-fetch model payloads on a Dank-fleet host that serves production LLM endpoints — inventory, protect live payloads, verify every fetch, restore the service, and use the refresh to detect upstream updates."
|
|
---
|
|
|
|
Clearing a model cache on a serving host is a two-sided operation: the operator wants no stale
|
|
payload, and the host must keep serving. Do it in this order, on the target host.
|
|
|
|
## 1. Inventory before touching anything
|
|
|
|
```bash
|
|
du -sh /home/dank/models/* | sort -h # payload dirs, biggest last
|
|
grep -rlE "models/|<model-name>" /home/dank/.config/containers/systemd/ /etc/systemd/system/*.service
|
|
```
|
|
|
|
Classify every dir: **live service payload** (referenced by a unit/env), **retired runtime
|
|
payload**, **unreferenced model dir**, **tool/script dir**, **logs** (evidence — keep). Check the
|
|
service env files (`scripts/*.env`) for `MODEL_DIR`/`MODEL` paths, not just unit `Volume=` lines:
|
|
a payload can be mounted by path and named by env. `models/logs` and small tool dirs are not cache.
|
|
|
|
## 2. Record fingerprints first
|
|
|
|
`ls -l --block-size=1 <dir>` gives sizes; for the LLM payloads compare each to the Hub's declared
|
|
size (`/api/models/<repo>/tree/main?recursive=true&blobs=true`). They can differ — see §6. If a
|
|
file is irreplaceable upstream (delisted revision, gated repo) say so before deleting it.
|
|
|
|
## 3. Protect live payloads by delete-then-refetch, never delete-only
|
|
|
|
Deleting a live payload leaves the endpoint unable to start. Know the recovery path *before*
|
|
deleting: some engines fetch their own weights (halogen: `HALOGEN_DOWNLOAD`, triggered when the
|
|
checkpoint is absent), others need a manual fetch (llama.cpp GGUF services: the serve script
|
|
refuses to start without the weights).
|
|
|
|
**Bind-mount trap:** podman refuses a mount whose source directory is missing —
|
|
`Error: statfs /home/dank/models/<dir>: no such file or directory`, with the unit looping on
|
|
`Restart=on-failure`. Recreate the directory **empty** (`mkdir -p`), and the engine's first-start
|
|
fetch runs.
|
|
|
|
## 4. Fetch with verification (never trust an exit code)
|
|
|
|
`curl -o FILE URL` exits 0 on 401/403 and writes the error body — a gated repo produced three
|
|
153-byte shards with identical sha256s and a cheerful DONE line. Always: `-fL -C -`, compare the
|
|
byte count against the Hub's declared size, and compare `sha256sum` against the LFS oid. Public
|
|
repos means **no token in argv** (it is world-readable in `/proc/<pid>/cmdline`).
|
|
|
|
**Parts trap:** mradermacher-style repos ship large quants as `.part1ofN … .partNofN` — a filter on
|
|
`endswith('.gguf')` makes those quants look absent. Fetch the parts, verify each, then
|
|
`cat part1 part2 part3 > name.gguf` in numeric order; the assembled file's size equals the sum
|
|
(concatenation is deterministic, so per-part hashes verify the whole).
|
|
|
|
## 5. Restore, then verify behaviour — not health alone
|
|
|
|
Put the quadlet back (`mv` from trash + `systemctl --user daemon-reload`), `reset-failed`, `start`;
|
|
root units need `sudo`. Watch the engine's own fetch/load lines in the journal, then check
|
|
`/health`, `/v1/models`, and a **real completion** on every restored endpoint. Read the engine's
|
|
memory ledger (kv pool / weights / blocks left) rather than assuming the config still fits.
|
|
|
|
## 6. A refresh is also an update check
|
|
|
|
Clearing and re-fetching only guarantees freshness *at fetch time* — if upstream has not changed,
|
|
the bytes come back identical, which is itself the finding. Compare recorded fingerprints against
|
|
the new local sizes and the Hub's declared sizes: sidecars are often versioned **in place**, so a
|
|
stale copy is invisible from its file name (observed: `overlay.hgn` 2,477,677,120 → 2,572,466,560
|
|
while the main checkpoint returned byte-identical).
|
|
|
|
## 7. Close out
|
|
|
|
- Clear test payloads once the benchmark using them is done, and keep the payload the *next*
|
|
experiment needs.
|
|
- Leave the operator's decision points explicit: unreferenced-but-licensed dirs, other services'
|
|
caches, and whether they want the payloads absent at their next run or fresh as of now.
|
|
- Record sizes/verdicts in the vault (runbook + `log.md`); never `rm` without explicit permission,
|
|
and remember a same-filesystem trash reclaims no space.
|