@@ -1,7 +1,7 @@
---
type: reference
title: "frank-secondary on Dankclaw — llama.cpp, Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2 x 131072 , draft-mtp, q8_0 KV"
description: "The active frank-secondary runbook (2026-10-05 evening; MTP back ON on the lighter RCO-IQ3_XXS since 2026-10-06 ) — llama.cpp master (CUDA sm_86) serving ajgazin's Swift-1.5-Qwen3.8-27B Uncensored RCO-IQ3_XXS (MTP head in the file, drafted) on the RTX 3090: TWO 131,072 -token conversations at Frank's 128 k cap, q8_0 K + q8_0 V (llama.cpp's FP8 class), vision through the publisher's own BF16 projector, and the full two-slot + drafter arithmetic."
title: "frank-secondary on Dankclaw — llama.cpp, Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2 x 114688 , draft-mtp, q8_0 KV"
description: "The active frank-secondary runbook (live since 2026-10-05 evening; renamed from frank-small 2026-10-06, per-slot lowered to 112k and ubatch raised to 1024 the same evening ) — llama.cpp master (CUDA sm_86) serving ajgazin's Swift-1.5-Qwen3.8-27B Uncensored RCO-IQ3_XXS (MTP head in the file, drafted) on the RTX 3090: TWO 114,688 -token conversations at Frank's 11 2k cap, q8_0 K + q8_0 V (llama.cpp's FP8 class), vision through the publisher's own BF16 projector, and the full two-slot + drafter arithmetic."
tags:
- llm
- frank-secondary
@@ -15,26 +15,25 @@ tags:
timestamp: 2026-10-06
---
# frank-secondary — llama.cpp Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2 × 31,072 , draft-mtp, q8_0 KV
# frank-secondary — llama.cpp Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2 × 14,688 , draft-mtp, q8_0 KV
> **ACTIVE since 2026-10-05 (evening); reshaped twice on 2026-10-06. ** A
> 27B replaces the 9B on the same 24 GiB card, bought by Frank's explicit
> contract for the swap: context capped at 128k per conversation, KV at
> FP8 class, vision on, "as many of those contexts as you can". T he
> morning order ("drop mtp, skip ngram, drop ubatch to 512, let's get 2
> contexts at 128k — should be a very tight fit") bought the second slot
> by spending the drafter (~0.4 GiB assumed) and ubatch headroom ( ~0.3
> GiB): 1,397 MiB mea sur ed free. The evening order — swap to the
> UNCELERSED Swift-1.5 at RCO-IQ3_XXS and turn MTP back on — landed
> **489 MiB free ** : the lighter quant (−
> second slot, but the MTP draft ctx costs * * ~2.4 GiB measured, not the
> ~0.4 GiB assumed** — the drafter's KV is built against the target's
> full context (the 262,144 draft window), not the 3-token proposal
> depth. The 2–
> supersedes BOTH earlier 2026-10-05 payloads: the live MiMo-V2.6 Q6_K
> shape ([[frank-secondary-llama-mimo-q6k-2x200704]], now retired) and the
> Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but never
> deployed.
> **ACTIVE since 2026-10-05 (evening); reshaped three times on
> 2026-10-06, renamed from frank-small the same night.** A 27B replaces
> the 9B on the same 24 GiB card: KV at FP8 class, vision on. The
> morning order (drop MTP, ubatch 512, TWO 128k contexts) bought t he
> second slot — 1,397 MiB free. The evening order swapped to the
> UNCELERSED Swift-1.5 at RCO-IQ3_XXS and put MTP back ON — landing
> **489 MiB free ** : the MTP draft ctx costs * * ~2.4 GiB, not the ~0.4
> GiB as sum ed** (the drafter's KV is built against the target's full
> context, not the 3-token proposal depth), paid for by the lighter
> quant (−
> 131,072 → **114,688 ** (Frank's 112k cap) and raised ubatch 512 →
> 1024 — landing **1,751 MiB free ** , back above the morning's number
> with the prefill knob raised. This supersedes BOTH earlier 2026-10-05
> payloads: the live MiMo-V2.6 Q6_K shape
> ([[frank-secondary-llama-mimo-q6k-2x200704]], now retired) and the
> Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but
> never deployed.
## Identity
@@ -64,11 +63,12 @@ escape hatch is the same as every payload here: revert and redeploy.
## Shape and KV contract
`--ctx-size 262144 --parallel 2` — TWO conversations at Frank's 128k cap
(the GSQ publisher's own benchmark protocol ran at 131,072 per stream).
`CTX_SIZE` stays divisible by `PARALLEL` ; the seam enforces it, and the
per-slot number (`262144 / 2 = 131,072` ) is the product contract —
models.yml stays at 131,072.
`--ctx-size 229376 --parallel 2` — TWO conversations at Frank's 112k
cap (2026-10-06 late: cut from the 128k the morning/evening shapes ran
at, to fund ubatch 1024). `CTX_SIZE` stays divisible by `PARALLEL` ; the
seam enforces it, and the per-slot number (`229376 / 2 = 114,688` ) is
the product contract — models.yml follows at 114,688 (a longer prompt
is a 400, never a truncation).
**KV at FP8 class: `q8_0` K + `q8_0` V (Frank, 2026-10-05 evening).**
llama.cpp has no true fp8 KV dtype; q8_0 is the 8-bit class (1.0625 B/elem
@@ -81,18 +81,25 @@ pinned: quantized V requires it, and the build's
`GGML_CUDA_FA_ALL_QUANTS=ON` (2026-09-17) makes the path real rather than
a silent f16 fallback.
**Two slots + drafter, by arithmetic and by order (2026-10-06).** The
morning order bought the second slot by spending the drafter and ubatch
headroom (1,397 MiB free). The evening swap put the drafter BACK and
changed the arithmetic: the draft ctx is built against the target's FULL
context (262,144 — the draft window, not the 3-token proposal depth), so
it costs * * ~2.4 GiB measured** (23,636 − 22,728 + 1,561), not the ~0.4 GiB
the one-slot era assumed. The lighter quant pays for it: **23,636 used /
489 MiB free** (nvidia-smi `free` field) — the tightest fit in
frank-secondary's history, the 2– knowingly violated, and the
512× and recovers. Compute buffers
allocate at graph capture during BOOT, so a miss is loud and fast — the
failure mode is a crash loop, not silent corruption.
**Two slots + drafter, by arithmetic and by order (2026-10-06 morning /
evening, the 2 × morning order bought the second slot
by spending the drafter and ubatch headroom (1,397 MiB free). The evening
swap put the drafter BACK and changed the arithmetic: the draft ctx is
built against the target's FULL context (262,144 — the draft window, not
the 3-token proposal depth), so it costs * * ~2.4 GiB measured** (23,636 −
22,728 + 1,561), not the ~0.4 GiB the one-slot era assumed. The lighter
quant pays for it: **23,636 used / 489 MiB free** (nvidia-smi `free`
field) — the tightest fit in frank-secondary's history, the 2–
knowingly violated, and the 512×
and recovers.
**The late reshape (2 ×
per-slot cut frees 2 × ×
512 → 1024 spends part of it on compute buffers (a prefill knob — decode
never moves with it). Measured boot: **22,374 used / 1,751 MiB free ** .
The drafter's own KV shrank with its target (draft window 229,376).
Compute buffers allocate at graph capture during BOOT, so a miss is loud
and fast — the failure mode is a crash loop, not silent corruption.
## Speculative decoding — ON since 2026-10-06 evening (draft-mtp, depth 3)
@@ -161,7 +168,7 @@ leave room for thinking.
- **Vision ** : 512× `Red` again on this shape.
- Deploy: commit 812d9eb, e2e gate 33/33 green on the live fleet.
## Measured: the 2 × live )
## Measured: the 2 × historical )
- **Boot ledger ** : `creating MTP draft context against the target model
'…/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf'`,
@@ -186,6 +193,29 @@ leave room for thinking.
slot prefix cache is live.
- Deploy: commit da36fa3, e2e gate 33/33 green on the live fleet.
## Measured: the 2 ×
- **argv ** (from `/proc/<MainPID>/cmdline` ): `--ctx-size 229376
--parallel 2` (per-slot 114,688), ` --batch-size 4096 --ubatch-size
1024`, ` --spec-type draft-mtp`. Deploys 87721ab (the rename — old
`frank-small.service` stopped + disabled first, by hand: `disable` on
a relinked unit removes only its symlink) then f2575a1 (the shape);
health gate + MainPID check green, e2e 32 passed / 1 flaky-retry.
- **VRAM: 22,374 used / 1,751 MiB free ** (nvidia-smi `free` field) —
the per-slot cut (≈1.05 GiB of KV) paid for the ubatch-1024 buffers
and netted back ~1.25 GiB against the evening's 489 MiB; back above
the morning's 1,397 MiB.
- **Decode + MTP acceptance ** (live `timings` ): recipe (temp 1.0) →
**58.2 t/s ** (430/1005, 42.8 %); temp 0 → **66.9 t/s ** (475/873,
54.4 %). The 2 × − –
acceptance is text-dependent and these prompts differ from that
measurement's, so the comparison is CONFOUNDED, not a shape verdict;
a clean A/B needs stop + mask (bench policy).
- **Smoke ** : `READY` completion + `reasoning_content` populated; the
harness reached :7771 under the new alias (models.yml provider/id
`frank-secondary` , config.yml modelRoles rewritten; per-slot
`contextWindow: 114688` ).
## Client contract
- Reasoning in `reasoning_content` (`--reasoning-format deepseek` );
@@ -209,14 +239,17 @@ leave room for thinking.
## Rollback
The 2 ×
da36fa3 and redeploy: the env trio then ships `SPEC_TYPE=none` ,
`CTX_SIZE=262144` , `PARALLEL=2` , `UBATCH_SIZE=512` against
`/home/dank/models/Swift15-IQ3S/` (the IQ3_S-mtp payload, still on disk,
measured 48.0 tok/s no-spec). The 1 ×
back (812d9eb's parent): the same IQ3_S-mtp file wi th
`SPEC_TYPE=draft-mtp` , `UBATCH_SIZE=1024` , `CTX_SIZE=131072` , `PARALLEL=1`
(measured 58.9–
** The immediate predecessor shape (2 ×
payload) is ONE revert away** — revert f2575a1 and redeploy
( `CTX_SIZE=262144` , `UBATCH_SIZE=512` ). The 2 ×
IQ3_S-mtp shape is one revert further — revert da36fa3 next: the env
trio then ships `SPEC_TYPE=none` , `CTX_SIZE=262144` , `PARALLEL=2` ,
`UBATCH_SIZE=512` against `/home/dank/models/Swift15-IQ3S/` ( the
IQ3_S-mtp payload, still on disk, measured 48.0 tok/s no-spec). The
1 ×
the same IQ3_S-mtp file with `SPEC_TYPE=draft-mtp` , `UBATCH_SIZE=1024` ,
`CTX_SIZE=131072` , `PARALLEL=1` (measured 58.9–
drafting).
Three payloads on disk, all fetchable by the same script's git-history
recipes:
@@ -260,3 +293,13 @@ recipes:
target's full 262,144 context, not the 3-token proposal depth) plus the
second 131,072 slot: 489 MiB free, vision `Red` at a 128-token budget,
decode 63.3 t/s recipe / 72.9 temp 0 (48.0 no-spec).
- 2026-10-06 (late, Frank): the endpoint renamed frank-small →
**frank-secondary ** (87721ab — scripts, unit, seam, runbooks,
identifiers; historical references to the OLD FreeToken/CUDA
frank-small, and to the Dankserver service that became frank-medium,
stay frank-small) and the per-slot window cut 131,072 → 114,688 to
fund ubatch 512 → 1024 (f2575a1). Live unit transition done by hand
before the deploy: `systemctl stop frank-small` + `systemctl disable
frank-small`, then ` deploy.sh dankclaw --service frank-secondary`.
1,751 MiB free; decode 58.2 t/s recipe / 66.9 temp 0; models.yml +
config.yml retargeted.
Frank