fix(frank-large): load the model as its native 3-way split, not a concatenated file
The first cutover crash-looped the unit with 'invalid split file name': the fetch script concatenated the three split parts into a single GGUF, but part 1's header carries ggml.file_count=3, so the byte-correct concatenation still advertises itself as a split and the loader demands a -NNNNN-of-NNNNN filename. - serve-frank-large.sh: -m points at part 1; llama.cpp loads the split natively (parts 2-3 auto-discovered in the same directory) - deploy-remote.sh: LLM_LARGE_MODEL gate path follows - download-frank-large-unsloth.sh: assembly removed; the parts ARE the canonical payload, verified against live Hub LFS oids - docs: runbook model section + troubleshooting note, AGENTS.md, log - the redundant 87 GiB assembled file is staged on dankclaw at ~/.AGENTTRASH/20260921-frank-large-assembled/
This commit is contained in:
@@ -279,8 +279,8 @@ The unsloth llama.cpp prebuilt `app-b11030-mix-5ff778e` (ROCm gfx1151,
|
||||
`/home/dank/GitPackages/unsloth-llama-b11030/`, bundles its own ROCm runtime —
|
||||
no host SDK gate, no venv) as a **root systemd unit** (`scripts/
|
||||
frank-large.service`, `User=dank`), serving
|
||||
`unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS` (87.3 GiB main GGUF, three HF
|
||||
split parts concatenated, + a 2.6 GiB MTP head, all sha256-verified) on port
|
||||
`unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS` (87.3 GiB main GGUF as three HF
|
||||
split parts, loaded natively as a split via `-m` on part 1, + a 2.6 GiB MTP head, all sha256-verified) on port
|
||||
7770 from the gfx1151 iGPU. One stream, 200,000 ctx (`PARALLEL_SLOTS=1`,
|
||||
operator's requested shape; the trained window is 262,144), MTP depth 2
|
||||
(`SPEC_DRAFT_N_MAX=2`, the model card's default), reasoning extracted to
|
||||
|
||||
Reference in New Issue
Block a user