agent: repoint roles to frank-medium/dankclaw-3090; sync niri tweaks, memories, managed skills
- config.yml: default/commit -> frank-medium (dankserver); task/tiny/smol/vision -> frank-small (dankclaw-3090); vision xhigh - models.yml: add llamacpp (dankclaw-3090) provider; frank-large preserve_thinking + context notes - niri: focus-ring/border/shadow layout, hot-corners off, re-add noctalia.kdl include; blankpad eDP-1 scale 1.0 - memories: flush rollouts; retire frank_large + marreta_authentik skills; add auth-chain/kv-pool/quadlet-log skills - managed-skills: 40+ new fleet runbook skills - agent/skills: import built-in skill set
This commit is contained in:
1 parent
83c9c92b3f
commit
135bb2194d
103 files changed
+5333
-170
No files matched your search
@@ -0,0 +1,98 @@
|
||||
---
|
||||
name: hf-gguf-payload-fetch-verify
|
||||
description: "Inventory, fetch, hash-verify and assemble a Hugging Face GGUF payload (including mradermacher's .partNofM fragments and gated mirrors) before pointing an LLM service at it — on the Dank fleet or anywhere else."
|
||||
---
|
||||
|
||||
# Fetch and verify an HF GGUF payload
|
||||
|
||||
Use before any model swap where the weights come from Hugging Face, especially
|
||||
large (>32 GiB) quantised models. The failure modes below all look like success.
|
||||
|
||||
## 1. Inventory — query by quant name, never by extension
|
||||
|
||||
```bash
|
||||
curl -s "https://huggingface.co/api/models/<org>/<repo>/tree/main?recursive=true&blobs=true" \
|
||||
| python3 -c "
|
||||
import json,sys
|
||||
for e in json.load(sys.stdin):
|
||||
p=e.get('path','')
|
||||
if 'Q4_K_S' in p: # the QUANT, not '.gguf'
|
||||
print(p, e.get('size',0), (e.get('lfs') or {}).get('oid','-'))
|
||||
"
|
||||
```
|
||||
|
||||
**Large quants ship as `.partNofM` fragments.** `mradermacher/*-i1-GGUF` repos
|
||||
hold e.g. Q4_K_M as `part1of3/2of3/3of3` (37.00+37.00+36.97 = 110.97 GiB) and
|
||||
Q4_K_S the same way, while the smaller rungs (Q2_K, Q3_K_*, IQ3_*, IQ4_XS) are
|
||||
single `.gguf` files. Filtering on `endswith('.gguf')` therefore makes Q4_K_S
|
||||
look like it does not exist — do not conclude a quant is missing from an
|
||||
extension-filtered listing.
|
||||
|
||||
Check both mradermacher repos: `…-i1-GGUF` (imatrix) and `…-GGUF` (plain), and
|
||||
the model's own repo. Gated mirrors exist (`orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF`
|
||||
carries the same ladder) and return **403 even with a valid HF token** until
|
||||
that account accepts the gate — prefer the ungated copy rather than chasing access.
|
||||
|
||||
## 2. Fetch — `-f` is mandatory, secrets stay out of argv
|
||||
|
||||
```bash
|
||||
# token out of band: mode-600 config, never -H on the command line
|
||||
printf 'header = "Authorization: Bearer %s"\n' "$(cat ~/.cache/huggingface/token)" > /tmp/hf.curlrc
|
||||
chmod 600 /tmp/hf.curlrc
|
||||
curl -fL -C - --retry 10 --retry-delay 5 --retry-max-time 21600 -K /tmp/hf.curlrc \
|
||||
-o "$DEST/$name" "$BASE/$name"
|
||||
```
|
||||
|
||||
- `curl -o FILE URL` **exits 0 on HTTP 401/403** and writes the error body as the
|
||||
artifact. A gated repo yields identical small files (153 bytes) that hash the
|
||||
same and a `DOWNLOAD_DONE` in seconds for a 104 GiB payload.
|
||||
- A token in `-H "Authorization: Bearer …"` is visible in `/proc/<pid>/cmdline`
|
||||
to any `ps`/`pgrep -a` on the host. Never `pgrep -a` a running fetch.
|
||||
|
||||
## 3. Verify, then assemble
|
||||
|
||||
```bash
|
||||
# per part: byte count must equal the Hub's declared size, sha256 must equal lfs.oid
|
||||
sha256sum "$PARTS"/*.part* | sort
|
||||
|
||||
# concatenate in NUMERIC order, then verify the total and the container magic
|
||||
cat "$PARTS"/X.gguf.part1of3 "$PARTS"/X.gguf.part2of3 "$PARTS"/X.gguf.part3of3 > "$OUT"
|
||||
stat -c%s "$OUT" # must equal the sum of the part sizes
|
||||
head -c 4 "$OUT" # must print GGUF
|
||||
```
|
||||
|
||||
The assembled file has no published hash (only the parts do), so the per-part
|
||||
LFS oids plus an exact total are the verification chain. If a size check fails,
|
||||
suspect your own arithmetic before the payload: the sum of the parts is the
|
||||
expected value, and it is easy to mis-add by hand (`python3 -c "print(a+b+c)"`).
|
||||
|
||||
## 4. Housekeeping
|
||||
|
||||
- Failure artifacts (error pages) and stray directories: move to
|
||||
`~/.AGENTTRASH/YYYYMMDD-topic`, never `rm`; list them and wait for approval to
|
||||
purge.
|
||||
- Keep the `.partNofM` fragments after assembly when disk allows: they are the
|
||||
only copies that can be re-verified against the Hub. Note the total they cost
|
||||
and ask before purging.
|
||||
- Never delete the payload being replaced as a side effect of a swap: upstreams
|
||||
delist quants (both mradermacher and orcarouter have delisted Q4_K_M for
|
||||
Qwen3.8-Flash-Next-Uncensored), and the local file may be the last copy.
|
||||
|
||||
## 5. Waiting for a long fetch
|
||||
|
||||
Do not `pgrep -f <script>` where the pattern appears in your own command line —
|
||||
the loop matches itself and spins its full iteration count while the download is
|
||||
fine. Watch the artifact instead:
|
||||
|
||||
```bash
|
||||
while [ "$(stat -c%s "$DEST/$name" 2>/dev/null || echo 0)" -lt "$EXPECTED" ]; do sleep 30; done
|
||||
```
|
||||
|
||||
## 6. Before choosing a smaller quant
|
||||
|
||||
A smaller quant buys memory and ~3–5 % decode, and costs quality. It does **not**
|
||||
change what the runtime can express: on llama.cpp, per-tensor cross-device
|
||||
placement is refused whenever `--override-tensor` is set
|
||||
(`has_tensor_overrides()` gates `pipeline_parallel`, which sets `n_copies`), and a
|
||||
layer split can only move contiguous layer ranges. State which of the two you are
|
||||
buying — headroom or a design change — before spending 100 GiB of bandwidth.
|
||||
Reference in new issue
Block a user