serverAppConfigs/serverAppConfigs-docs/LLM/frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	frank-zealot on Dankclaw — llama.cpp, Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2 x 114688, draft-mtp, q8_0 KV	The active frank-zealot runbook (live since 2026-10-05 evening; renamed from frank-small 2026-10-06, per-slot lowered to 112k and ubatch raised to 1024 the same evening) — llama.cpp master (CUDA sm_86) serving ajgazin's Swift-1.5-Qwen3.8-27B Uncensored RCO-IQ3_XXS (MTP head in the file, drafted) on the RTX 3090: TWO 114,688-token conversations at Frank's 112k cap, q8_0 K + q8_0 V (llama.cpp's FP8 class), vision through the publisher's own BF16 projector, and the full two-slot + drafter arithmetic.	llm frank-zealot dankclaw llama-cpp gguf rtx3090 qwen35 mtp vision	2026-10-06

frank-zealot — llama.cpp Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2 × 114,688, draft-mtp, q8_0 KV

ACTIVE since 2026-10-05 (evening); reshaped three times on 2026-10-06, renamed from frank-small the same night. A 27B replaces the 9B on the same 24 GiB card: KV at FP8 class, vision on. The morning order (drop MTP, ubatch 512, TWO 128k contexts) bought the second slot — 1,397 MiB free. The evening order swapped to the UNCELERSED Swift-1.5 at RCO-IQ3_XXS and put MTP back ON — landing 489 MiB free: the MTP draft ctx costs ~2.4 GiB, not the ~0.4 GiB assumed (the drafter's KV is built against the target's full context, not the 3-token proposal depth), paid for by the lighter quant (−1.56 GiB). The late order dropped the per-slot window 131,072 → 114,688 (Frank's 112k cap) and climbed the prefill ladder ubatch 512 → 1024 → 2048 — landing 1,005 MiB free (1,751 at the 1024 rung; the 2048 rung cost 746 MiB, ~130 over the 616 MiB measured prediction). This supersedes BOTH earlier 2026-10-05 payloads: the live MiMo-V2.6 Q6_K shape (frank-zealot-llama-mimo-q6k-2x200704, now retired) and the Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but never deployed.

Identity

Host / device	Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB
Endpoint	http://192.168.0.143:7771 — alias frank-zealot, /v1 OpenAI surface
Runtime	llama.cpp master (0.5.0-dev, commit 5fc4f3c), CUDA sm_86 build …/llama-master/build-cuda/bin/llama-server (GGML_CUDA_FA_ALL_QUANTS=ON — required for quantized V)
Unit	root systemd frank-zealot.service (scripts/frank-zealot.service), User=dank, Restart=on-failure, LimitMEMLOCK=infinity
Serve script	scripts/serve-frank-zealot.sh, tunables scripts/frank-zealot.env
Weights	/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf — ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF RCO-IQ3_XXS -mtp, 10,442,827,392 B, sha256 e13d3e74… (qwen35 hybrid: 64 real layers, 16 full-attention + 48 Gated DeltaNet, native 262,144; MTP head in the file: blk.64.nextn.*, nextn_predict_layers=1, and DRAFTED with it)
Projector	…/Swift15-UncIQ3XXS/mmproj-BF16.gguf — PUBLISHER-SHIPPED in the same repo, 931,145,888 B, sha256 19acd7fc… (27-layer ViT → 5120)
Fetch	scripts/download-frank-zealot.sh — -f, byte count + sha256 verified per file (Xet repo: the Hub APIs expose no LFS oid, so the shas were computed locally after the first verified fetch; byte counts match the Hub's declared sizes)
Seam	tests/frank_zealot_unit.bats (argv assertions against a stub engine: payload path, slot geometry, KV pair, spec pair (both flags present), batch pair, projector opt-out)

Why this payload (2026-10-06 evening)

Frank asked for the UNCELERSED Swift-1.5 with MTP back on. The RCO-IQ3_XXS is 1.56 GiB lighter than the IQ3_S-mtp it replaced — and that lighter weight is exactly what funds the MTP draft context (~2.4 GiB measured) PLUS the second 131,072 slot, while keeping the q8_0 KV contract and the two-slot shape Frank ordered that morning.

Uncensored, intentionally: the publisher reports 23/100 refusals vs Swift 1.5's 98/100 (KL 0.0884 against the censored base). The refusal escape hatch is the same as every payload here: revert and redeploy.

Shape and KV contract

--ctx-size 229376 --parallel 2 — TWO conversations at Frank's 112k cap (2026-10-06 late: cut from the 128k the morning/evening shapes ran at, to fund ubatch 1024). CTX_SIZE stays divisible by PARALLEL; the seam enforces it, and the per-slot number (229376 / 2 = 114,688) is the product contract — models.yml follows at 114,688 (a longer prompt is a 400, never a truncation).

KV at FP8 class: q8_0 K + q8_0 V (Frank, 2026-10-05 evening). llama.cpp has no true fp8 KV dtype; q8_0 is the 8-bit class (1.0625 B/elem ⇒ 8.5 effective bits) and the only FP8-class lever this runtime moves. The order supersedes the same-day f16/f16 rule, which was written for the Ornith 9B fit — at 64 KiB/token f16 the single 131,072 slot alone would cost 8.0 GiB of KV. At q8_0 the rate is 34 KiB/token (16 attn layers × 4 KV heads × 256 dim × 1.0625 B, K and again V). --flash-attn on is pinned: quantized V requires it, and the build's GGML_CUDA_FA_ALL_QUANTS=ON (2026-09-17) makes the path real rather than a silent f16 fallback.

Two slots + drafter, by arithmetic and by order (2026-10-06 morning / evening, the 2 × 131,072 era). The morning order bought the second slot by spending the drafter and ubatch headroom (1,397 MiB free). The evening swap put the drafter BACK and changed the arithmetic: the draft ctx is built against the target's FULL context (262,144 — the draft window, not the 3-token proposal depth), so it costs ~2.4 GiB measured (23,636 − 22,728 + 1,561), not the ~0.4 GiB the one-slot era assumed. The lighter quant pays for it: 23,636 used / 489 MiB free (nvidia-smi free field) — the tightest fit in frank-zealot's history, the 2–3 GiB floor knowingly violated, and the 512×512 vision transient (34 MiB) still fits and recovers.

The late reshape (2 × 114,688, ubatch 2048) is the current fit. The per-slot cut frees 2 × 16,384 tokens × 34 KiB ≈ 1.05 GiB of KV; the ubatch rungs 512 → 1024 → 2048 spend it back on compute buffers (a prefill knob — decode never moves with it; the rungs cost ~0.3 and ~0.75 GiB measured). Measured boot at 2048: 23,120 used / 1,005 MiB free. The drafter's own KV shrank with its target (draft window 229,376). Compute buffers allocate at graph capture during BOOT, so a miss is loud and fast — the failure mode is a crash loop, not silent corruption.

Speculative decoding — ON since 2026-10-06 evening (draft-mtp, depth 3)

Frank's evening order: MTP back on. The MTP head ships IN the file — the -mtp GGUF carries blk.64.nextn.* (qwen35.nextn_predict_layers=1) — and --spec-type draft-mtp loads it against the main model's own weights (no --model-draft, no second draft model — two context-backed drafters cannot share a process). SPEC_DRAFT_N_MAX=3 is the proposal depth. ngram-mod stays skipped, not swapped in: on a qwen35 hybrid, ngram speculation logs "recurrent state rollback is not compatible with 'ngram-mod' — disabling rollback support" (fleet lesson from frank-dragoon's chain measurement, 2026-10-03). A rejected draft on Gated DeltaNet layers cannot be rewound by KV truncation — without the rollback checkpoints the DeltaNet state carries phantom transitions and output degrades silently. The flags move as a set: the serve script emits BOTH --spec-type and --spec-draft-n-max together or neither (SPEC_TYPE=none strips both).

Vision contract

--mmproj is the entire vision path (the GGUF carries no vision tensors). Probes: a solid-color PNG ≥ 256 px answers its exact color — a tiny probe undersupplies image tokens and the model invents ("video"); the legacy top-level images field is IGNORED, only the OpenAI image_url content part reaches the model. New on this payload: the uncensored model THINKS more than the censored Swift 1.5 — a 64-token budget was eaten entirely by reasoning_content (finish_reason: length, empty content); 128 tokens clears it. Token budgets for vision probes must leave room for thinking.

Measured: the 1 × 131,072 MTP boot (2026-10-05, historical)

• Boot ledger: creating MTP draft context against the target model '…/Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf', loaded multimodal model, '…/Swift15-IQ3S/mmproj-Qwen3.8-27B-F16.gguf', n_slots = 1, n_ctx_slot = 131072 — every expected line, no q8→f16 fallback.
• VRAM: 19,404 used / 4,721 MiB free (nvidia-smi free field). The 4.5 GiB prediction held; f16 KV would have left ~700 MiB, so this free number doubles as the q8_0 proof.
• Decode + MTP acceptance (draft counts from the live timings): temp 0.8 → 66.1 tok/s, 182/349 accepted (52.1 %); temp 0 → 66.9 tok/s, 166/318; deployed recipe (temp 1.0/top_p 0.95/top_k 20) → 58.9 tok/s, 159/355 (44.8 %) — temperature buys fluency, costs acceptance and rate. For scale: MiMo Q6_K measured 94.4 tok/s on the same card — this 27B at IQ3_S runs ~2/3 the rate with ~3× the capability class.
• Vision: 512×512 solid-red PNG → Red, finish_reason: stop.
• Template gate: reasoning_effort: xhigh → 200 with reasoning_content populated; high → HTTP 500 (Jinja raise) — the models.yml fold is mandatory, exactly as the MiMo eras found.
• Streaming completion: 200. No first-request silence window (the halogen-era 45–60 s stall has no analogue here).

Measured: the 2 × 131,072 no-spec boot (2026-10-06 morning, historical)

• Boot ledger: loaded multimodal model, '…/Swift15-IQ3S/ mmproj-Qwen3.8-27B-F16.gguf' and NO draft-context line (speculation off). /slots: 2 slots, n_ctx 131,072 each (pool 262,144).
• VRAM: 22,728 used / 1,397 MiB free (nvidia-smi free field) — the predicted 0.3–1.3 GiB window held; the old 2-3 GiB floor is knowingly violated by Frank's order. After a 512×512 image probe: 1,363 MiB — the vision transients fit.
• Decode without the drafter: 48.0 tok/s (temp 0) / 47.8 (deployed recipe temp 1.0). draft_n absent from timings (no drafter). The MTP era's 58.9–66.9 tok/s is the honest price of the second context: −18–28 %.
• Vision: 512×512 solid-red PNG → Red again on this shape.
• Deploy: commit 812d9eb, e2e gate 33/33 green on the live fleet.

Measured: the 2 × 131,072 draft-mtp boot (2026-10-06 evening, historical)

• Boot ledger: creating MTP draft context against the target model '…/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf', loaded multimodal model, '…/Swift15-UncIQ3XXS/mmproj-BF16.gguf', n_slots = 2, n_ctx_slot = 131072 — every expected line, no q8→f16 fallback.
• VRAM: 23,636 used / 489 MiB free (nvidia-smi free field) — the draft ctx measured ~2.4 GiB, ~6× the ~0.4 GiB the one-slot era assumed; the lighter quant (−1.56 GiB) plus the morning's no-spec base still fund both slots. After a 512×512 image probe: 489 MiB — the vision transient (34 MiB) fits and recovers.
• Decode + MTP acceptance (draft counts from the live timings, 512-token essay): temp 0 → 72.9 tok/s, 333/534 accepted (62.4 %); deployed recipe (temp 1.0/top_p 0.95/top_k 20) → 63.3 tok/s, 305/612 (49.8 %). e2e traffic in the boot window read 0.52–0.67 acceptance. Against the no-spec shape (48.0 tok/s): +31 % at the recipe temp — the drafter earned its 2.4 GiB.
• Vision: 512×512 solid-red PNG → Red at a 128-token budget (64 tokens: all thinking, empty content). prompt_tokens: 321 confirms the image part reached the model.
• Prompt cache: cached_tokens: 317 of 321 on the second probe — the slot prefix cache is live.
• Deploy: commit da36fa3, e2e gate 33/33 green on the live fleet.

Measured: the 2 × 114,688 / ubatch-1024 boot (2026-10-06 late, historical)

• argv (from /proc/<MainPID>/cmdline): --ctx-size 229376 --parallel 2 (per-slot 114,688), --batch-size 4096 --ubatch-size 1024, --spec-type draft-mtp. Deploys 87721ab (the rename — old frank-small.service stopped + disabled first, by hand: disable on a relinked unit removes only its symlink) then f2575a1 (the shape); health gate + MainPID check green, e2e 32 passed / 1 flaky-retry.
• VRAM: 22,374 used / 1,751 MiB free (nvidia-smi free field) — the per-slot cut (≈1.05 GiB of KV) paid for the ubatch-1024 buffers and netted back ~1.25 GiB against the evening's 489 MiB; back above the morning's 1,397 MiB.
• Decode + MTP acceptance (live timings): recipe (temp 1.0) → 58.2 t/s (430/1005, 42.8 %); temp 0 → 66.9 t/s (475/873, 54.4 %). The 2 × 131,072 boot measured 63.3 / 72.9 — −8–10 %, but acceptance is text-dependent and these prompts differ from that measurement's, so the comparison is CONFOUNDED, not a shape verdict; a clean A/B needs stop + mask (bench policy).
• Smoke: READY completion + reasoning_content populated; the harness reached :7771 under the new alias (models.yml provider/id frank-zealot, config.yml modelRoles rewritten; per-slot contextWindow: 114688).

Measured: ubatch 2048 (2026-10-06 midnight, live)

• argv: --ctx-size 229376 --parallel 2 --batch-size 4096 --ubatch-size 2048 --spec-type draft-mtp (commit d8da7a9, Frank's "set ubatch to 2k"; deploy green, e2e gate 33/33).
• VRAM: 23,120 used / 1,005 MiB free at boot — the 1024 → 2048 rung cost 746 MiB, ~130 over the 616 MiB the rung measured on 2026-10-05 (rung cost is shape-dependent, not a constant). The 512×512 solid-red vision probe (Red) then took ~240 MiB and kept it: settled 763 MiB free — on this boot the vision encoder's buffers stay resident after the request (the evening boot saw its 34 MiB transient return).
• Decode (ubatch-invariant knob, same prompt as the 1024 rung): recipe → 52.0 t/s (388/1133, 34.2 %) vs 58.2 t/s / 42.8 % at 1024 — acceptance variance at temp 1.0, not a ubatch effect.
• Prefill (THE ubatch knob): 1,209 t/s @ 3,760 tokens — first recorded number for this shape; future rungs compare against this (no same-prompt 1024 baseline exists; the law predicts ≈ +4.6 % 1024 → 2048).
• Headroom left ~0.7–1.0 GiB. ubatch 4096 (= the logical batch, no micro-splitting) doubles the buffer growth again — not affordable; 2048 is the ceiling for this shape.

Client contract

• Reasoning in reasoning_content (--reasoning-format deepseek); thinking tokens COUNT against max_tokens. This payload thinks more than the censored Swift 1.5 (uncensored finetune) — short max_tokens budgets can be eaten entirely by thinking (finish_reason: length, empty content); the 32,768 maxTokens in models.yml is comfortably above.
• The embedded chat template gates reasoning_effort to {xhigh, medium, low} — anything else is a raise/HTTP 500, so models.yml compat.reasoningEffortMap folds the harness levels onto that set (mandatory, same as every Qwen3.8-template endpoint).
• Sampling rides the wire (extraBody): the publisher recipe temp 1.0 / top_p 0.95 / top_k 20 / min_p 0 (evaluated at ctx 131,072).
• The RCO-IQ3_XXS tier is a per-tensor allocation mix (IQ3_S 97 / BF16 96 / IQ3_XXS 83 / IQ2_S 72 / IQ4_XS 33 / IQ2_XS 32 / IQ2_XXS 30 / Q2_K 28 / Q4_K 19 / Q6_K 8 / IQ1_S 4 / IQ1_M 4) — all stock quant types; stock llama.cpp loads it. Publisher KLD vs Swift BF16: mean 0.002631, 90th 0.042136, max 34.49 — RCO is the publisher's "more experimental" family.

Rollback

Predecessor shape (same 2 × 114,688, ubatch 1024) is ONE revert away — revert d8da7a9 and redeploy. The 2 × 131,072 / ubatch-512 era is one revert further (f2575a1). The 2 × 131,072 no-spec IQ3_S-mtp shape is one revert further — revert da36fa3 next: the env trio then ships SPEC_TYPE=none, CTX_SIZE=262144, PARALLEL=2, UBATCH_SIZE=512 against /home/dank/models/Swift15-IQ3S/ (the IQ3_S-mtp payload, still on disk, measured 48.0 tok/s no-spec). The 1 × 131,072 MTP shape is two reverts further back (812d9eb's parent): the same IQ3_S-mtp file with SPEC_TYPE=draft-mtp, UBATCH_SIZE=1024, CTX_SIZE=131072, PARALLEL=1 (measured 58.9–66.9 tok/s with the head drafting).

Three payloads on disk, all fetchable by the same script's git-history recipes:

1. Swift-1.5 IQ3_S-mtp — the payload live until this cutover: /home/dank/models/Swift15-IQ3S/; the IQ3_XXS swap above.
2. MiMo-V2.6 Q6_K 2×200,704 f16 KV — the payload that was LIVE until the 2026-10-05 cutover: /home/dank/models/MiMo-Q6K/; revert frank-zealot-llama-mimo-q6k-2x200704 (env trio at commit e5893dc or 380aafa's parent), one MODEL_FILE + shape edit.
3. Ornith-1.5-9B Q8_0 — the committed-but-never-deployed swap: /home/dank/models/Ornith-Q8/; revert to 380aafa's tree (ONE native 262,144 slot, f16/f16 KV).

Trail

• The Ornith-1.5-9B Q8_0 swap (77decb5 → 380aafa, earlier 2026-10-05) was committed and pushed but its deploy never completed (blocked at the whole-tree docs gate); the live service was still MiMo Q6_K when Swift cut over. Both Ornith payloads were verified on disk before Swift.
• Swift header facts (block_count 65, interval 4, KV geometry, nextn) were read from the Hub file by HTTP range + struct parse — no download needed — before the payload was fetched.
• Cutover 2026-10-05 ~23:26: the radiance-cutover session (parallel) label-committed both sets at Frank's direction — 1919ad7 (frank-archon radiance) then baf33d0 (this swap: trio, seam, runbooks, fleet docs). Deploy ran deploy.sh dankclaw --service frank-zealot; the service dispatch and health gate went green; the first e2e run failed ONE spec (12-crawl, net::ERR_ABORTED on www.franksdank.cloud — a transient, unrelated to frank-zealot), green on rerun.
• The unit's Description= (MiMo era) was stale at cutover time and was updated in-repo afterward — the relinked unit picks it up at the next deploy/restart, not retroactively.
• 2026-10-06 (Frank): drop MTP, skip ngram, ubatch 512, TWO 128k contexts (812d9eb). The tight fit landed: 1,397 MiB free, vision Red, decode 48.0 t/s no-spec. Model of the trade: the drafter's ~0.4 GiB + ubatch ~0.3 GiB bought the second 4,352 MiB KV slot.
• 2026-10-06 (Frank, evening): swap to the UNCELERSED Swift-1.5 at RCO-IQ3_XXS, MTP back on (da36fa3). The lighter quant (−1.56 GiB) funds the draft ctx (~2.4 GiB measured — the drafter's KV scales with the target's full 262,144 context, not the 3-token proposal depth) plus the second 131,072 slot: 489 MiB free, vision Red at a 128-token budget, decode 63.3 t/s recipe / 72.9 temp 0 (48.0 no-spec).
• 2026-10-06 (late, Frank): the endpoint renamed frank-small → frank-zealot (87721ab — scripts, unit, seam, runbooks, identifiers; historical references to the OLD FreeToken/CUDA frank-small, and to the Dankserver service that became frank-medium, stay frank-small) and the per-slot window cut 131,072 → 114,688 to fund ubatch 512 → 1024 (f2575a1). Live unit transition done by hand before the deploy: systemctl stop frank-small + systemctl disable frank-small, then deploy.sh dankclaw --service frank-zealot. 1,751 MiB free; decode 58.2 t/s recipe / 66.9 temp 0; models.yml + config.yml retargeted.
• 2026-10-06 (midnight, Frank): "set ubatch to 2k" → UBATCH_SIZE=2048 (d8da7a9). Cost 746 MiB: 23,120 used / 1,005 free at boot, settling 763 MiB free after a vision probe (encoder buffers stay resident on this boot). Decode 52.0 t/s recipe on the shared prompt (58.2 at the 1024 rung — acceptance noise, decode is ubatch-invariant); prefill 1,209 t/s @ 3,760 tokens (first recorded baseline). 2048 is the ceiling for this shape; 4096 would not fit.