frank-small: Swift-1.5-27B Uncensored RCO-IQ3_XXS with MTP back on (two 131072 slots, ubatch 512, q8_0 KV, publisher's BF16 mmproj)

scripts/frank-small.env

@@ -1,71 +1,55 @@
 # frank-small — llama.cpp (master, CUDA sm_86) serving
-# ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF:IQ3_S **-mtp** on the RTX 3090,
-# :7771.
+# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF:RCO-IQ3_XXS
+# **with the MTP head in the file** on the RTX 3090, :7771.
 #
-# 2026-10-05 (Frank, evening): the payload moved to Swift-1.5 Qwen3.8-27B
-# GSQ-RCO IQ3_S WITH the MTP head in the file, each conversation capped at
-# 131,072 tokens, KV at FP8 class, vision on. Header-verified facts that
-# re-price the shape (read from the Hub file 2026-10-05, HTTP range + struct
-# parse): qwen35 hybrid like MiMo/Ornith but bigger — block_count 65 (64 real
-# layers + the blk.64 MTP head), full_attention_interval 4 => SIXTEEN
-# full-attention layers (MiMo had 8, Ornith 9), 24 Q heads / 4 KV heads,
-# k/v length 256, native window 262144, dense, nextn_predict_layers=1 with
-# blk.64.nextn.{enorm,hnorm,shared_head_norm} (the -mtp file's head tensors
-# are Q6_K/F32 per the publisher RECIPE, "MTP_OK" verified).
+# 2026-10-06 (Frank, evening): the payload moved to ajgazin's UNCELERSED
+# Swift-1.5-Qwen3.8-27B at RCO-IQ3_XXS (10,442,827,392 B) — 1.56 GiB
+# lighter than the IQ3_S-mtp it replaces — with MTP BACK ON (draft-mtp,
+# depth 3), TWO 131,072 slots, and ubatch 512. The uncensored finetune
+# changes behavior by design (publisher: 23/100 refusals vs Swift 1.5's
+# 98/100, KL 0.0884 from Swift 1.5) — Frank's order. The RCO tier is the
+# same per-tensor ALLOCATION recipe as the IQ3_S file (histogram: iq3_s
+# 97 / bf16 96 / iq3_xxs 83 / iq2_s 72 / iq4_xs 33 / iq2_xs 32 / iq2_xxs
+# 30 / q2_k 28 / q4_k 19 / q6_k 8 / iq1_s 4 / iq1_m 4 — all stock types,
+# stock llama.cpp loads it); publisher KLD: Mean 0.002631, 90 % 0.0421
+# (kld_results.csv). RCO files are "more experimental" per the README —
+# the quality floor is the IQ3_S rung, not a promise.
 #
-# THE KV CONTRACT IS NOW q8_0 K + q8_0 V — Frank's order for THIS swap
-# ("k/v at FP8", 2026-10-05), superseding the same-day f16/f16 rule, which
-# was written for the Ornith 9B fit ("no further quantization anywhere on
-# this endpoint"). llama.cpp has no true fp8 KV dtype; q8_0 is the 8-bit
-# class (1.0625 B/elem => 8.5 effective bits) and the only FP8-class lever
-# this runtime moves. Arithmetic that makes the order load-bearing:
-#   KV rate = 16 attn layers x 4 heads x 256 x 1.0625 B, K and V again
-#   => 34 KiB/token (f16 would be 64 KiB/token — halogen's 27B card quotes
-#   the same f16 number; at f16 the 131,072 slot alone would cost 8.0 GiB).
+# The architecture is unchanged (same model family): qwen35 hybrid —
+# block_count 65 (64 real layers + the blk.64 MTP head),
+# full_attention_interval 4 => SIXTEEN full-attention layers, 24 Q heads
+# / 4 KV heads, k/v length 256, native window 262144, dense,
+# nextn_predict_layers=1. The MTP head ships in every main GGUF of this
+# repo (README); --spec-type draft-mtp loads it against the main model's
+# own weights.
 #
-# 2026-10-06 (Frank): drop the MTP drafter (skip ngram too — on a qwen35
-# hybrid, ngram speculation disables recurrent-state rollback: fleet lesson
-# from the frank-large chain measurement), drop ubatch to 512, and take TWO
-# conversations at 131,072 — "should be a very tight fit", accepted
-# explicitly, i.e. the 2-3 GiB floor is knowingly violated. Measured at the
-# 1x131072 MTP boot (2026-10-05): 19,404 used / 4,721 MiB free.
+# THE KV CONTRACT STAYS q8_0 K + q8_0 V (Frank, 2026-10-05): 34 KiB/token
+# (16 attn layers x 4 heads x 256 x 1.0625 B, K and V).
 #
-# Why TWO slots at 131,072 fits, by arithmetic (pool = 2 x 131,072 =
-# 262,144; the per-conversation cap stays Frank's 128k):
+# Why TWO slots + MTP fits now (it did not under IQ3_S):
+#   card 24,576 MiB. Weights 9,963 MiB (10,442,827,392 B; IQ3_S was
+#   11,561). Projector ~1,290 MiB (the PUBLISHER's OWN mmproj-BF16.gguf,
+#   931,145,888 B, at the measured ~1.45x card law — no substitution).
+#   Compute buffers at ubatch 512 ~1.3 GiB. CUDA ctx ~500 MiB. DeltaNet
+#   state ~150 MiB/slot x 2. MTP draft ctx back: ~0.4 GiB. Non-KV ~13.9
+#   GiB. Two q8_0 slots = 8,704 MiB. Predicted free ~2.6-3.2 GiB — back
+#   over the old 2-3 GiB floor that the IQ3_S two-slot boot violated
+#   (measured there: 1,397 MiB free). MEASURE nvidia-smi free at boot
+#   (quote the free field) and let the runbook carry the number.
 #
-#   card 24,576 MiB. Weights 11,561 MiB (the IQ3_S-mtp file; the GSQ
-#   "IQ3_S" tier is a per-tensor ALLOCATION mix — histogram IQ2_XS 144 /
-#   IQ3_XXS 96 / Q6_K 78 / F32 360 / BF16 96 / Q3_K_M 39 ... all stock
-#   types, stock llama.cpp loads it). Projector ~1,350 MiB (927,607,488 B
-#   F16 at the measured ~1.45x card law). Compute buffers at ubatch 512
-#   ~1.3 GiB (inferred from the measured 616 MiB/doubling rung on the 9B).
-#   CUDA ctx ~500 MiB. DeltaNet state ~150 MiB/slot x 2. NO draft context
-#   (the MTP draft ctx cost ~0.3-0.5 GiB and is gone with speculation).
-#   Non-KV ~15.5 GiB. Two q8_0 slots = 8,704 MiB (34 KiB/token x 131,072
-#   each). Predicted free at boot ~0.3-1.3 GiB. Compute buffers allocate
-#   at graph capture during BOOT, so a miss is loud and fast; MEASURE
-#   nvidia-smi free at boot (quote the free field) and let the runbook
-#   carry the number.
-#
-# Payload history (for the rollback trail): the Ornith-1.5-9B Q8_0 swap of
-# earlier on 2026-10-05 (commits 77decb5..380aafa) was committed but never
-# deployed — the live service was still MiMo-V2.6 Q6_K (2 x 200,704, f16
-# KV) when Swift cut over. Rollback payloads on disk: Ornith-Q8/ and
-# MiMo-Q6K/ (one MODEL_FILE + shape edit back; see the runbook).
+# Payload history (rollback trail): /home/dank/models/Swift15-IQ3S/ (the
+# IQ3_S-mtp that was live until this swap), Ornith-Q8/, MiMo-Q6K/ all
+# stay on disk; one MODEL_FILE + shape edit back each.
 LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server
-MODEL_FILE=/home/dank/models/Swift15-IQ3S/Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf
+MODEL_FILE=/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf
 
-# Vision: no Swift repo ships a projector (ukisai's release is
-# language-model-only by their own README), so the projector comes from the
-# SHARED BASE — unsloth's mmproj-F16.gguf built from Qwen/Qwen3.8-27B,
-# stored as mmproj-Qwen3.8-27B-F16.gguf (927,607,488 B, LFS sha256
-# cbb841a9..., fetched by scripts/download-frank-small.sh). The substitution
-# is proven, not assumed: Swift-1.5's vision stack (tower + merger) was
-# hash-compared against the base on 2026-10-05 — 333/333 visual tensor names
-# and 6/6 sampled tensors byte-identical — so the base projector IS the
-# model's own projector (Swift's RL/OPD post-training touched text only).
+# Vision: the publisher SHIPS the projector — mmproj-BF16.gguf
+# (931,145,888 B) from the same repo, fetched by
+# scripts/download-frank-small.sh. No substitution needed for this
+# payload (the unsloth base-tower proof of 2026-10-05 remains the
+# fallback argument for the IQ3_S rollback payload).
 # Empty string = text-only on purpose.
-MMPROJ_FILE=/home/dank/models/Swift15-IQ3S/mmproj-Qwen3.8-27B-F16.gguf
+MMPROJ_FILE=/home/dank/models/Swift15-UncIQ3XXS/mmproj-BF16.gguf
 
 # Fleet endpoint: same port and alias as every frank-small before it, so the
 # harness models.yml id and the homepage probe do not move.
@@ -100,13 +84,14 @@ UBATCH_SIZE=512
 CACHE_TYPE_K=q8_0
 CACHE_TYPE_V=q8_0
 
-# Speculative decoding is OFF (Frank, 2026-10-06): no draft-mtp, and
-# ngram-mod is skipped too — on a qwen35 hybrid ngram disables
-# recurrent-state rollback, so a rejected draft would corrupt the DeltaNet
-# state. SPEC_TYPE=none strips BOTH speculative flags in the serve script
-# (flags move as a set). The -mtp file still carries the head; it is simply
-# not drafted with. Measured MTP reference (2026-10-05): 44.8 % acceptance
-# at the recipe temperature => 58.9 t/s, 52.1 % at temp 0.8 => 66.1 t/s.
-SPEC_TYPE=none
+# Speculative decoding is ON (Frank, 2026-10-06 evening): draft-mtp at
+# depth 3 against the head IN the file (the -mtp variant ships the
+# blk.64.nextn.* tensors). ngram-mod stays skipped — on a qwen35 hybrid
+# it disables recurrent-state rollback, so a rejected draft would
+# corrupt the DeltaNet state; MTP needs no such bypass (its draft ctx
+# re-derives state from the target's own weights). The flags move as a
+# set: the serve script emits BOTH only when SPEC_TYPE names a drafter.
+SPEC_TYPE=draft-mtp
+SPEC_DRAFT_N_MAX=3
 
 NGL=99