docs: how the Uncensored payload is really fetched (it is shipped in parts)

Correcting a claim I made from a bad query: Q4_K_S does exist for this finetune.
`mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF` ships the large quants as
`.partNofM` fragments — Q4_K_M 37.00+37.00+36.97 = 110.97 GiB (which is how the
deployed file was assembled) and Q4_K_S 35.00+35.00+34.09 = 104.09 GiB — while
the smaller rungs are single files. A tree listing filtered on `.gguf` makes
Q4_K_S invisible, which is exactly how I concluded it did not exist.

Also records the fetch trap that hid it for a moment: `curl -o FILE URL` exits 0
on 401/403, so a gated repo produced three 153-byte "shards" with identical
sha256 values and a cheerful DOWNLOAD_DONE in 20 s for 104 GiB. -f plus the
Hub's declared size plus the LFS sha256 is the standard the fleet's own
download-frank-medium-gguf.sh already meets.
This commit is contained in:
Frank
2026-09-16 20:13:44 -07:00
parent ad18d2ddd8
commit 88f0d79021
2 changed files with 26 additions and 0 deletions
@@ -42,10 +42,19 @@ From the 111 GiB GGUF (`mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M`,
every 4th — `full_attention_interval = 4`) and 36 are GDN/linear.
- Only the 12 attention layers carry KV. At the engine's measured 28 KiB per
position, a 200k-token conversation reads **5.5 GiB of KV per token**.
- A layer split moves **whole layers**, so it moves ~1.62 GiB of experts with
every layer (75 GiB / 48). Attention layers are spread across the whole depth
(every 4th), so no contiguous layer range can hold them all.
The GGUF comes from `mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF`, which
ships this quant as three `.partNofM` fragments (37.00 + 37.00 + 36.97 =
110.97 GiB) concatenated in numeric order. The same repo carries the smaller
rungs — i1-Q4_K_S 104.09 GiB (also as parts), i1-IQ4_XS 90.78, i1-Q3_K_L 90.02,
i1-Q3_K_M 87.67, i1-IQ3_M 83.39, i1-IQ3_S 82.78, i1-Q3_K_S 82.66, i1-Q2_K
74.92 GiB — which is where a "does a smaller quant buy anything?" test would
start (it buys memory, ~5 % decode, and costs quality; see the write-up below).
That is why the interesting split is *not* "put the experts somewhere else" —
experts are re-read once per token but they are only ~1.7 GB of traffic — it is
"get KV onto the 936 GB/s pool", because KV is the O(context) term.