dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-small: swap the payload to Ornith-1.5-9B Q8_0

Browse Source
This commit is contained in:
Frank committed 2026-10-05 17:55:36 -07:00
1 parent be840b4881
commit 69f10a17bb
1 file changed
+12 -9
Download Patch File Download Diff File Expand all files Collapse all files
scripts/download-frank-small.sh
+12 -9
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download the frank-small payload for llama.cpp on the RTX 3090: # Download the frank-small payload for llama.cpp on the RTX 3090:
# bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF:Q6_K plus its vision projector # ornith-ai/Ornith-1.5-9B-GGUF:Q8_0 plus its vision projector
# (the GGUF carries no `v.*` tensors — the tower lives only in the mmproj, so # (the GGUF carries no `v.*` tensors — the tower lives only in the mmproj, so
# without this file the endpoint is text-only by construction). Public repo — # without this file the endpoint is text-only by construction). Public repo —
# no auth header, and never a token on the command line (it is world-readable # no auth header, and never a token on the command line (it is world-readable
@@ -11,9 +11,12 @@
# then byte count and the Hub LFS sha256. Idempotent: complete files are # then byte count and the Hub LFS sha256. Idempotent: complete files are
# skipped. # skipped.
# #
# 2026-10-03: the rung is Q6_K again — the freed third slot funds exact f16 V # 2026-10-05 (Frank): the rung is Ornith-1.5-9B Q8_0. The payload moved off
# AND the higher quant. The Q5_K_M file this script served from 2026-10-02 # MiMo-V2.6 (Q6_K) to Ornith because it is the fleet's own model and it is the
# stays on disk at /home/dank/models/MiMo-Q5KM/ (6876124704 B, LFS oid # only one the harness's own prompt recipes were written against; Q8_0 was the
# rung chosen so the quant does not cost the throughput this endpoint already
# runs at. The Q5_K_M file this script served from 2026-10-02 stays on disk at
# /home/dank/models/MiMo-Q5KM/ (6876124704 B, LFS oid
# 04cf46f2b5584ef922697d6c063f94879434fb86c467f1a09e52daf0cddaf35f) as the # 04cf46f2b5584ef922697d6c063f94879434fb86c467f1a09e52daf0cddaf35f) as the
# rollback target; re-fetch it from the same repo with the same recipe. # rollback target; re-fetch it from the same repo with the same recipe.
# #
@@ -21,7 +24,7 @@
# scripts/download-frank-small.sh [DEST_DIR] [--verify-only] # scripts/download-frank-small.sh [DEST_DIR] [--verify-only]
set -euo pipefail set -euo pipefail
REPO="bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF" REPO="ornith-ai/Ornith-1.5-9B-GGUF"
BASE="https://huggingface.co/${REPO}/resolve/main" BASE="https://huggingface.co/${REPO}/resolve/main"
DEST="" DEST=""
@@ -32,12 +35,12 @@ for arg in "$@"; do
*) DEST="$arg" ;; *) DEST="$arg" ;;
esac esac
done done
DEST="${DEST:-/home/dank/models/MiMo-Q6K}" DEST="${DEST:-/home/dank/models/Ornith-Q8}"
# name bytes sha256 # name bytes sha256 (the sha256 is the LFS oid in the repo's pointer file)
FILES=( FILES=(
"MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf 7793710624 ef96d05a2ddf2cbb450d1af1ac3860ec769d3575bafa70692ee5609bda3fad7d" "Ornith-1.5-9B-Q8_0.gguf 9786060384 22086870b009dbe9815ee752c48a82de930118a7c5ce5599590892ae03b8b010"
"mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf 918166048 ff348f3180a63188aa7285db85f550fe38acb61dd013c599eb8bad08d2cc2576" "mmproj-Ornith-1.5-9B-BF16.gguf 921704672 626f9f90627402a6bf4a999111d0fbd69b5fcca7aa8ba089d69e5f10e8858e1d"
) )
mkdir -p "$DEST" mkdir -p "$DEST"