---
name: hf-gguf-payload-fetch-verify
description: "Inventory, fetch, hash-verify and assemble a Hugging Face GGUF payload (including mradermacher's .partNofM fragments and gated mirrors) before pointing an LLM service at it — on the Dank fleet or anywhere else."
---

# Fetch and verify an HF GGUF payload

Use before any model swap where the weights come from Hugging Face, especially
large (>32 GiB) quantised models. The failure modes below all look like success.

## 1. Inventory — query by quant name, never by extension

```bash
curl -s "https://huggingface.co/api/models/<org>/<repo>/tree/main?recursive=true&blobs=true" \
 | python3 -c "
import json,sys
for e in json.load(sys.stdin):
    p=e.get('path','')
    if 'Q4_K_S' in p:                       # the QUANT, not '.gguf'
        print(p, e.get('size',0), (e.get('lfs') or {}).get('oid','-'))
"
```

**Large quants ship as `.partNofM` fragments.** `mradermacher/*-i1-GGUF` repos
hold e.g. Q4_K_M as `part1of3/2of3/3of3` (37.00+37.00+36.97 = 110.97 GiB) and
Q4_K_S the same way, while the smaller rungs (Q2_K, Q3_K_*, IQ3_*, IQ4_XS) are
single `.gguf` files. Filtering on `endswith('.gguf')` therefore makes Q4_K_S
look like it does not exist — do not conclude a quant is missing from an
extension-filtered listing.

Check both mradermacher repos: `…-i1-GGUF` (imatrix) and `…-GGUF` (plain), and
the model's own repo. Gated mirrors exist (`orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF`
carries the same ladder) and return **403 even with a valid HF token** until
that account accepts the gate — prefer the ungated copy rather than chasing access.

## 2. Fetch — `-f` is mandatory, secrets stay out of argv

```bash
# token out of band: mode-600 config, never -H on the command line
printf 'header = "Authorization: Bearer %s"\n' "$(cat ~/.cache/huggingface/token)" > /tmp/hf.curlrc
chmod 600 /tmp/hf.curlrc
curl -fL -C - --retry 10 --retry-delay 5 --retry-max-time 21600 -K /tmp/hf.curlrc \
     -o "$DEST/$name" "$BASE/$name"
```

- `curl -o FILE URL` **exits 0 on HTTP 401/403** and writes the error body as the
  artifact. A gated repo yields identical small files (153 bytes) that hash the
  same and a `DOWNLOAD_DONE` in seconds for a 104 GiB payload.
- A token in `-H "Authorization: Bearer …"` is visible in `/proc/<pid>/cmdline`
  to any `ps`/`pgrep -a` on the host. Never `pgrep -a` a running fetch.

## 3. Verify, then assemble

```bash
# per part: byte count must equal the Hub's declared size, sha256 must equal lfs.oid
sha256sum "$PARTS"/*.part* | sort

# concatenate in NUMERIC order, then verify the total and the container magic
cat "$PARTS"/X.gguf.part1of3 "$PARTS"/X.gguf.part2of3 "$PARTS"/X.gguf.part3of3 > "$OUT"
stat -c%s "$OUT"          # must equal the sum of the part sizes
head -c 4 "$OUT"          # must print GGUF
```

The assembled file has no published hash (only the parts do), so the per-part
LFS oids plus an exact total are the verification chain. If a size check fails,
suspect your own arithmetic before the payload: the sum of the parts is the
expected value, and it is easy to mis-add by hand (`python3 -c "print(a+b+c)"`).

## 4. Housekeeping

- Failure artifacts (error pages) and stray directories: move to
  `~/.AGENTTRASH/YYYYMMDD-topic`, never `rm`; list them and wait for approval to
  purge.
- Keep the `.partNofM` fragments after assembly when disk allows: they are the
  only copies that can be re-verified against the Hub. Note the total they cost
  and ask before purging.
- Never delete the payload being replaced as a side effect of a swap: upstreams
  delist quants (both mradermacher and orcarouter have delisted Q4_K_M for
  Qwen3.8-Flash-Next-Uncensored), and the local file may be the last copy.

## 5. Waiting for a long fetch

Do not `pgrep -f <script>` where the pattern appears in your own command line —
the loop matches itself and spins its full iteration count while the download is
fine. Watch the artifact instead:

```bash
while [ "$(stat -c%s "$DEST/$name" 2>/dev/null || echo 0)" -lt "$EXPECTED" ]; do sleep 30; done
```

## 6. Before choosing a smaller quant

A smaller quant buys memory and ~3–5 % decode, and costs quality. It does **not**
change what the runtime can express: on llama.cpp, per-tensor cross-device
placement is refused whenever `--override-tensor` is set
(`has_tensor_overrides()` gates `pipeline_parallel`, which sets `n_copies`), and a
layer split can only move contiguous layer ranges. State which of the two you are
buying — headroom or a design change — before spending 100 GiB of bandwidth.