frank-small: Q5_K_M -> Q6_K rung at the held 2x200704 f16-KV shape

The freed third slot funded exact V; the remaining ~3.5 GiB of honest
nvidia-smi free (3,571, not the 4,046 total-minus-used I quoted - the
driver holds ~450 MiB of reserve) funds the +875 MiB of Q6_K weights.
Projected landing ~2.7 GiB free, inside the 2-3 GiB floor; measure on
boot, step the pool one 12,288-token increment if it lands under 2 GiB.

- payload dir MiMo-Q6K/ made self-contained (mmproj hardlinked from
  MiMo-Q5KM/, link-count-2 verified)
- env/serve/deployer/bats repointed; download script now fetches and
  verifies Q6_K (7,793,710,624 B, ef96d05a...) with Q5_K_M documented as
  the rollback; payload sha verified live on the host before deploy
- seam green 14/14 against the stub engine
This commit is contained in:
Frank committed 2026-10-03 11:00:19 -07:00
1 parent b80886e318
commit 6ec0b1a85d
5 files changed
+35 -25

No files matched your search

+8 -7
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Download the frank-small payload for llama.cpp on the RTX 3090:
# bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF:Q5_K_M plus its vision projector
# bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF:Q6_K plus its vision projector
# (the GGUF carries no `v.*` tensors — the tower lives only in the mmproj, so
# without this file the endpoint is text-only by construction). Public repo —
# no auth header, and never a token on the command line (it is world-readable
@@ -11,10 +11,11 @@
# then byte count and the Hub LFS sha256. Idempotent: complete files are
# skipped.
#
# The Q6_K file this endpoint served until 2026-10-01 night stays on disk as
# the rollback target (7793710624 B, LFS oid
# ef96d05a2ddf2cbb450d1af1ac3860ec769d3575bafa70692ee5609bda3fad7d) — re-fetch
# it from the same repo with the same recipe if it is ever needed again.
# 2026-10-03: the rung is Q6_K again — the freed third slot funds exact f16 V
# AND the higher quant. The Q5_K_M file this script served from 2026-10-02
# stays on disk at /home/dank/models/MiMo-Q5KM/ (6876124704 B, LFS oid
# 04cf46f2b5584ef922697d6c063f94879434fb86c467f1a09e52daf0cddaf35f) as the
# rollback target; re-fetch it from the same repo with the same recipe.
#
# Usage:
# scripts/download-frank-small.sh [DEST_DIR] [--verify-only]
@@ -31,11 +32,11 @@ for arg in "$@"; do
*) DEST="$arg" ;;
esac
done
DEST="${DEST:-/home/dank/models/MiMo-Q5KM}"
DEST="${DEST:-/home/dank/models/MiMo-Q6K}"
# name bytes sha256
FILES=(
"MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf 6876124704 04cf46f2b5584ef922697d6c063f94879434fb86c467f1a09e52daf0cddaf35f"
"MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf 7793710624 ef96d05a2ddf2cbb450d1af1ac3860ec769d3575bafa70692ee5609bda3fad7d"
"mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf 918166048 ff348f3180a63188aa7285db85f550fe38acb61dd013c599eb8bad08d2cc2576"
)