fix(frank-large): verify exactly the fetched file set, not the whole Hub tree
The verification looped over every LFS file under UD-IQ4_XS/ and MTP/ in the live Hub tree — the five MTP head variants we do not fetch (BF16, Q4_K_M, Q8_0, shared-BF16, shared-Q4_K_M) printed MISSING, set ok=False, and set -e killed the script silently between verification and assembly after a 3-hour download. Verify exactly the FILES set instead (passed to python as argv), and report NOT IN HUB TREE when an expected file has vanished upstream. Also fix the layout comment: the transport parts live in UD-IQ4_XS/, not parts/.
This commit is contained in:
1 parent
7010f5b754
commit
660db661b6
2 files changed
+15
-11
No files matched your search
@@ -22,7 +22,7 @@
|
||||
# Qwen3.8-Flash-Next-UD-IQ4_XS.gguf 93,682,584,224 main model
|
||||
# mtp/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf 2,786,568,256 draft head
|
||||
# vision/mmproj-F16.gguf 904,004,000 projector
|
||||
# parts/ the three verified transport parts (kept for --verify-only)
|
||||
# UD-IQ4_XS/ the three verified transport parts (kept for --verify-only)
|
||||
# SHA256SUMS
|
||||
set -euo pipefail
|
||||
|
||||
@@ -57,15 +57,17 @@ FILES=(
|
||||
"mmproj-F16.gguf 904004000 1f7b7f0b984cf065c604360c29c8098362ed61b290db0ff12c6f360bb1a8a980"
|
||||
)
|
||||
|
||||
if [[ "$mode" == "download" ]]; then
|
||||
for line in "${FILES[@]}"; do
|
||||
read -r rel size _ <<<"$line"
|
||||
RELS=()
|
||||
for line in "${FILES[@]}"; do
|
||||
read -r rel size _ <<<"$line"
|
||||
RELS+=("$rel")
|
||||
if [[ "$mode" == "download" ]]; then
|
||||
dl "$rel"
|
||||
done
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
echo "verifying parts against the live Hub tree API"
|
||||
python3 - "$MODEL_DIR" "$REPO" <<'PY'
|
||||
python3 - "$MODEL_DIR" "$REPO" "${RELS[@]}" <<'PY'
|
||||
import hashlib, json, os, sys, urllib.request
|
||||
|
||||
model_dir, repo = sys.argv[1], sys.argv[2]
|
||||
@@ -84,9 +86,11 @@ def sha256(path):
|
||||
return h.hexdigest()
|
||||
|
||||
ok = True
|
||||
for rel, (size, oid) in sorted(want.items()):
|
||||
if not rel.startswith(("UD-IQ4_XS/", "MTP/")) and rel != "mmproj-F16.gguf":
|
||||
continue
|
||||
expected = set(sys.argv[3:])
|
||||
for rel in sorted(expected):
|
||||
if rel not in want:
|
||||
print(f"NOT IN HUB TREE {rel}"); ok = False; continue
|
||||
size, oid = want[rel]
|
||||
path = os.path.join(model_dir, rel)
|
||||
if not os.path.isfile(path):
|
||||
print(f"MISSING {rel}"); ok = False; continue
|
||||
|
||||
@@ -79,7 +79,7 @@ One root unit, one process, one published port:
|
||||
- `mtp/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf` — MTP head (2.6 GiB)
|
||||
- `vision/mmproj-F16.gguf` — projector (904 MB, not loaded while
|
||||
`ENABLE_VISION=0`)
|
||||
- `parts/` + `SHA256SUMS` — transport parts and the assembled digest
|
||||
- `UD-IQ4_XS/` + `SHA256SUMS` — transport parts and the assembled digest
|
||||
- **Fetch**: `scripts/download-frank-large-unsloth.sh` — resumable
|
||||
(`curl -fL -C -`), each part verified against its **LFS sha256 fetched
|
||||
live from the Hub tree API** (not a file that could drift), then the
|
||||
|
||||
Reference in new issue
Block a user