Frank
135bb2194d
- config.yml: default/commit -> frank-medium (dankserver); task/tiny/smol/vision -> frank-small (dankclaw-3090); vision xhigh - models.yml: add llamacpp (dankclaw-3090) provider; frank-large preserve_thinking + context notes - niri: focus-ring/border/shadow layout, hot-corners off, re-add noctalia.kdl include; blankpad eDP-1 scale 1.0 - memories: flush rollouts; retire frank_large + marreta_authentik skills; add auth-chain/kv-pool/quadlet-log skills - managed-skills: 40+ new fleet runbook skills - agent/skills: import built-in skill set
99 lines
4.5 KiB
Markdown
99 lines
4.5 KiB
Markdown
---
|
||
name: hf-gguf-payload-fetch-verify
|
||
description: "Inventory, fetch, hash-verify and assemble a Hugging Face GGUF payload (including mradermacher's .partNofM fragments and gated mirrors) before pointing an LLM service at it — on the Dank fleet or anywhere else."
|
||
---
|
||
|
||
# Fetch and verify an HF GGUF payload
|
||
|
||
Use before any model swap where the weights come from Hugging Face, especially
|
||
large (>32 GiB) quantised models. The failure modes below all look like success.
|
||
|
||
## 1. Inventory — query by quant name, never by extension
|
||
|
||
```bash
|
||
curl -s "https://huggingface.co/api/models/<org>/<repo>/tree/main?recursive=true&blobs=true" \
|
||
| python3 -c "
|
||
import json,sys
|
||
for e in json.load(sys.stdin):
|
||
p=e.get('path','')
|
||
if 'Q4_K_S' in p: # the QUANT, not '.gguf'
|
||
print(p, e.get('size',0), (e.get('lfs') or {}).get('oid','-'))
|
||
"
|
||
```
|
||
|
||
**Large quants ship as `.partNofM` fragments.** `mradermacher/*-i1-GGUF` repos
|
||
hold e.g. Q4_K_M as `part1of3/2of3/3of3` (37.00+37.00+36.97 = 110.97 GiB) and
|
||
Q4_K_S the same way, while the smaller rungs (Q2_K, Q3_K_*, IQ3_*, IQ4_XS) are
|
||
single `.gguf` files. Filtering on `endswith('.gguf')` therefore makes Q4_K_S
|
||
look like it does not exist — do not conclude a quant is missing from an
|
||
extension-filtered listing.
|
||
|
||
Check both mradermacher repos: `…-i1-GGUF` (imatrix) and `…-GGUF` (plain), and
|
||
the model's own repo. Gated mirrors exist (`orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF`
|
||
carries the same ladder) and return **403 even with a valid HF token** until
|
||
that account accepts the gate — prefer the ungated copy rather than chasing access.
|
||
|
||
## 2. Fetch — `-f` is mandatory, secrets stay out of argv
|
||
|
||
```bash
|
||
# token out of band: mode-600 config, never -H on the command line
|
||
printf 'header = "Authorization: Bearer %s"\n' "$(cat ~/.cache/huggingface/token)" > /tmp/hf.curlrc
|
||
chmod 600 /tmp/hf.curlrc
|
||
curl -fL -C - --retry 10 --retry-delay 5 --retry-max-time 21600 -K /tmp/hf.curlrc \
|
||
-o "$DEST/$name" "$BASE/$name"
|
||
```
|
||
|
||
- `curl -o FILE URL` **exits 0 on HTTP 401/403** and writes the error body as the
|
||
artifact. A gated repo yields identical small files (153 bytes) that hash the
|
||
same and a `DOWNLOAD_DONE` in seconds for a 104 GiB payload.
|
||
- A token in `-H "Authorization: Bearer …"` is visible in `/proc/<pid>/cmdline`
|
||
to any `ps`/`pgrep -a` on the host. Never `pgrep -a` a running fetch.
|
||
|
||
## 3. Verify, then assemble
|
||
|
||
```bash
|
||
# per part: byte count must equal the Hub's declared size, sha256 must equal lfs.oid
|
||
sha256sum "$PARTS"/*.part* | sort
|
||
|
||
# concatenate in NUMERIC order, then verify the total and the container magic
|
||
cat "$PARTS"/X.gguf.part1of3 "$PARTS"/X.gguf.part2of3 "$PARTS"/X.gguf.part3of3 > "$OUT"
|
||
stat -c%s "$OUT" # must equal the sum of the part sizes
|
||
head -c 4 "$OUT" # must print GGUF
|
||
```
|
||
|
||
The assembled file has no published hash (only the parts do), so the per-part
|
||
LFS oids plus an exact total are the verification chain. If a size check fails,
|
||
suspect your own arithmetic before the payload: the sum of the parts is the
|
||
expected value, and it is easy to mis-add by hand (`python3 -c "print(a+b+c)"`).
|
||
|
||
## 4. Housekeeping
|
||
|
||
- Failure artifacts (error pages) and stray directories: move to
|
||
`~/.AGENTTRASH/YYYYMMDD-topic`, never `rm`; list them and wait for approval to
|
||
purge.
|
||
- Keep the `.partNofM` fragments after assembly when disk allows: they are the
|
||
only copies that can be re-verified against the Hub. Note the total they cost
|
||
and ask before purging.
|
||
- Never delete the payload being replaced as a side effect of a swap: upstreams
|
||
delist quants (both mradermacher and orcarouter have delisted Q4_K_M for
|
||
Qwen3.8-Flash-Next-Uncensored), and the local file may be the last copy.
|
||
|
||
## 5. Waiting for a long fetch
|
||
|
||
Do not `pgrep -f <script>` where the pattern appears in your own command line —
|
||
the loop matches itself and spins its full iteration count while the download is
|
||
fine. Watch the artifact instead:
|
||
|
||
```bash
|
||
while [ "$(stat -c%s "$DEST/$name" 2>/dev/null || echo 0)" -lt "$EXPECTED" ]; do sleep 30; done
|
||
```
|
||
|
||
## 6. Before choosing a smaller quant
|
||
|
||
A smaller quant buys memory and ~3–5 % decode, and costs quality. It does **not**
|
||
change what the runtime can express: on llama.cpp, per-tensor cross-device
|
||
placement is refused whenever `--override-tensor` is set
|
||
(`has_tensor_overrides()` gates `pipeline_parallel`, which sets `n_copies`), and a
|
||
layer split can only move contiguous layer ranges. State which of the two you are
|
||
buying — headroom or a design change — before spending 100 GiB of bandwidth.
|