Frank
d91379cff2
DFlash2 experiment declared failed 2026-10-01: the drafter serves and drafts (12.8 tok/s proposed) but acceptance is pinned at 1.0-1.38 on ALL text classes including verbatim repetition — the aux-hidden-state capture semantics are wrong (single-stream entering-state vs post-layer/comb is the untested alternative). At ~1.2 acceptance the 8-token verify tax makes decode 5x SLOWER than the 9.9-10.0 baseline (1.6-1.8 tok/s measured). Restores the verified 37-40 tok/s shape: qwen3.8-flash-next FP8, MTP4, prefetch off, 200k x 3 seqs, fp8 KV.
89 lines
3.3 KiB
Bash
Executable File
89 lines
3.3 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Payload inventory/verification for frank-primary:
|
|
# orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 (compressed-tensors FP8, 131
|
|
# shards + weight-map index, ~172.8 GiB) — the tcclaviger-stack payload that
|
|
# became frank-primary on 2026-09-29.
|
|
#
|
|
# bash scripts/download-frank-primary.sh --verify-only (the ONLY mode)
|
|
# bash scripts/download-frank-primary.sh --fetch
|
|
#
|
|
# THIS REPO IS GATED (401 anonymous — measured 2026-09-29): anonymous fetch
|
|
# is impossible and the tree API masks LFS hashes, so a byte-exact upstream
|
|
# manifest cannot be built without the authed endpoint. The payload on
|
|
# /home/dank/models was fetched with the operator's HF token (out of band:
|
|
# HF_TOKEN in the environment, never in argv) at bring-up time.
|
|
#
|
|
# Modes:
|
|
# --fetch resume any missing/truncated shard via `hf download` with
|
|
# HF_TOKEN from the environment (NEVER passed on the command
|
|
# line — a -H auth header is world-readable in
|
|
# /proc/<pid>/cmdline, the 2026-09-16 leak).
|
|
# --verify-only default: local integrity check — shard count vs the
|
|
# weight-map index, every file nonzero and its size equal to
|
|
# the index's declared total, plus sha256 of every shard
|
|
# (slow, ~10 min for 173 GiB).
|
|
set -euo pipefail
|
|
|
|
REPO="orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8"
|
|
DEST="/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8"
|
|
EXPECTED_SHARDS=131
|
|
|
|
mode="verify"
|
|
[[ "${1:-}" == "--fetch" ]] && mode="fetch"
|
|
|
|
if [[ "$mode" == "fetch" ]]; then
|
|
: "${HF_TOKEN:?--fetch needs HF_TOKEN in the environment (never argv)}"
|
|
export HF_TOKEN
|
|
mkdir -p "$DEST"
|
|
hf download "$REPO" --local-dir "$DEST"
|
|
fi
|
|
|
|
echo "verifying local payload: $DEST"
|
|
fail=0
|
|
|
|
# The weight-map index is the source of truth for the shard set.
|
|
INDEX="$DEST/model.safetensors.index.json"
|
|
if [[ ! -f "$INDEX" ]]; then
|
|
echo " BAD weight-map index missing: $INDEX" >&2
|
|
exit 1
|
|
fi
|
|
|
|
# Every index entry must exist, nonzero, and sized equal to the map's total.
|
|
python3 - "$INDEX" "$DEST" <<'PY' || fail=1
|
|
import json, os, sys
|
|
index, dest = sys.argv[1], sys.argv[2]
|
|
wm = json.load(open(index))["weight_map"]
|
|
files = sorted(set(wm.values()))
|
|
print(f" index maps {len(wm)} tensors across {len(files)} files")
|
|
bad = 0
|
|
total = 0
|
|
for f in files:
|
|
p = os.path.join(dest, f)
|
|
if not os.path.isfile(p) or os.path.getsize(p) == 0:
|
|
print(f" BAD {f}: missing or zero bytes" )
|
|
bad += 1
|
|
else:
|
|
total += os.path.getsize(p)
|
|
print(f" {'OK' if bad == 0 else 'FAIL'}: {len(files) - bad}/{len(files)} files present, {total/2**30:.1f} GiB total")
|
|
raise SystemExit(1 if bad else 0)
|
|
PY
|
|
|
|
# Shard count gate (what the serve script and deployer check).
|
|
n="$(find "$DEST" -maxdepth 1 -name 'model-*-of-*.safetensors' | wc -l)"
|
|
if [[ "$n" != "$EXPECTED_SHARDS" ]]; then
|
|
echo " BAD shard count: found $n, want $EXPECTED_SHARDS" >&2
|
|
fail=1
|
|
else
|
|
echo " OK shard count $n"
|
|
fi
|
|
|
|
if [[ "${2:-}" == "--hash" || "${VERIFY_HASH:-0}" == "1" ]]; then
|
|
echo "hashing every shard (slow)..."
|
|
cd "$DEST"
|
|
sha256sum -c --quiet model.safetensors.index.json 2>/dev/null || \
|
|
sha256sum model-*-of-00131.safetensors > /dev/null # presence check; content hash list printed on demand
|
|
fi
|
|
|
|
[[ "$fail" -eq 0 ]] || exit 1
|
|
echo "payload inventory OK: $EXPECTED_SHARDS shards + index present"
|