Commit Graph
2 Commits
Author SHA1 Message Date
Frank 660db661b6 fix(frank-large): verify exactly the fetched file set, not the whole Hub tree
The verification looped over every LFS file under UD-IQ4_XS/ and MTP/ in
the live Hub tree — the five MTP head variants we do not fetch (BF16,
Q4_K_M, Q8_0, shared-BF16, shared-Q4_K_M) printed MISSING, set ok=False,
and set -e killed the script silently between verification and assembly
after a 3-hour download. Verify exactly the FILES set instead (passed to
python as argv), and report NOT IN HUB TREE when an expected file has
vanished upstream. Also fix the layout comment: the transport parts live
in UD-IQ4_XS/, not parts/.
2026-09-21 12:41:52 -07:00
Frank b149d02187 feat(frank-large): unsloth llama.cpp Qwen3.8-Flash-Next UD-IQ4_XS (200k/1 slot, MTP depth 2)
Replace the CIRU v4.4.1 Orca runtime with the unsloth llama.cpp prebuilt
(b11030-mix-5ff778e, ROCm gfx1151, bundles its own ROCm) serving
unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (87.3 GiB, three HF split parts
concatenated) on the Strix Halo iGPU. One conversation, 200,000 ctx,
PARALLEL_SLOTS=1, MTP depth 2 via the explicit -md shared-Q8_0 head
(sidecar auto-discovery does not search the mtp/ subfolder), text-only.
Fleet contract unchanged: port 7770, alias frank-large.

- serve-frank-large.sh: unsloth binary, -m/-md, --spec-type draft-mtp
  --spec-draft-n-max 2, --jinja, preserve_thinking; gates on binary,
  GGUF, MTP head; conditional --mmproj on ENABLE_VISION=1
- download-frank-large-unsloth.sh: resumable fetch of 3 parts + MTP head
  + mmproj, each verified against its LFS sha256 fetched live from the
  Hub tree API; concatenates parts, records/verifies SHA256SUMS
- deploy-remote.sh: LLM_LARGE_MODEL/LLM_LARGE_DRAFT paths, pkill pattern
  on the model path (not the binary name), wait_port_free gate
- tests: seams for 200k/1-slot, explicit -md, text-only, handler gates,
  sweep safety
2026-09-21 12:30:30 -07:00