llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot

The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.

Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.
This commit is contained in:
Frank committed 2026-10-10 17:03:14 -07:00
1 parent 5ced474eb9
commit 55f87fcdce
195 files changed
+2085 -2064

No files matched your search

@@ -0,0 +1,338 @@
---
type: reference
title: "frank-zealot on Dankclaw — llama.cpp, Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2 x 114688, draft-mtp, q8_0 KV"
description: "The active frank-zealot runbook (live since 2026-10-05 evening; renamed from frank-small 2026-10-06, per-slot lowered to 112k and ubatch raised to 1024 the same evening) — llama.cpp master (CUDA sm_86) serving ajgazin's Swift-1.5-Qwen3.8-27B Uncensored RCO-IQ3_XXS (MTP head in the file, drafted) on the RTX 3090: TWO 114,688-token conversations at Frank's 112k cap, q8_0 K + q8_0 V (llama.cpp's FP8 class), vision through the publisher's own BF16 projector, and the full two-slot + drafter arithmetic."
tags:
- llm
- frank-zealot
- dankclaw
- llama-cpp
- gguf
- rtx3090
- qwen35
- mtp
- vision
timestamp: 2026-10-06
---
# frank-zealot — llama.cpp Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2 × 114,688, draft-mtp, q8_0 KV
> **ACTIVE since 2026-10-05 (evening); reshaped three times on
> 2026-10-06, renamed from frank-small the same night.** A 27B replaces
> the 9B on the same 24 GiB card: KV at FP8 class, vision on. The
> morning order (drop MTP, ubatch 512, TWO 128k contexts) bought the
> second slot — 1,397 MiB free. The evening order swapped to the
> UNCELERSED Swift-1.5 at RCO-IQ3_XXS and put MTP back ON — landing
> **489 MiB free**: the MTP draft ctx costs **~2.4 GiB, not the ~0.4
> GiB assumed** (the drafter's KV is built against the target's full
> context, not the 3-token proposal depth), paid for by the lighter
> quant (−1.56 GiB). The late order dropped the per-slot window
> 131,072 → **114,688** (Frank's 112k cap) and climbed the prefill ladder
> ubatch 512 → 1024 → **2048** — landing **1,005 MiB free** (1,751 at
> the 1024 rung; the 2048 rung cost 746 MiB, ~130 over the 616 MiB
> measured prediction). This supersedes BOTH earlier 2026-10-05
> payloads: the live MiMo-V2.6 Q6_K shape
> ([[frank-zealot-llama-mimo-q6k-2x200704]], now retired) and the
> Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but
> never deployed.
## Identity
| | |
|---|---|
| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
| Endpoint | `http://192.168.0.143:7771` — alias `frank-zealot`, `/v1` OpenAI surface |
| Runtime | llama.cpp master (0.5.0-dev, commit 5fc4f3c), CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` (`GGML_CUDA_FA_ALL_QUANTS=ON` — required for quantized V) |
| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
| Serve script | [`scripts/serve-frank-zealot.sh`](../../scripts/serve-frank-zealot.sh), tunables [`scripts/frank-zealot.env`](../../scripts/frank-zealot.env) |
| Weights | `/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf` — ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF **RCO-IQ3_XXS -mtp**, 10,442,827,392 B, sha256 `e13d3e74…` (qwen35 hybrid: 64 real layers, 16 full-attention + 48 Gated DeltaNet, native 262,144; **MTP head in the file**: blk.64.nextn.*, `nextn_predict_layers=1`, and DRAFTED with it) |
| Projector | `…/Swift15-UncIQ3XXS/mmproj-BF16.gguf` — PUBLISHER-SHIPPED in the same repo, 931,145,888 B, sha256 `19acd7fc…` (27-layer ViT → 5120) |
| Fetch | [`scripts/download-frank-zealot.sh`](../../scripts/download-frank-zealot.sh) — `-f`, byte count + sha256 verified per file (Xet repo: the Hub APIs expose no LFS oid, so the shas were computed locally after the first verified fetch; byte counts match the Hub's declared sizes) |
| Seam | `tests/frank_zealot_unit.bats` (argv assertions against a stub engine: payload path, slot geometry, KV pair, spec pair (both flags present), batch pair, projector opt-out) |
## Why this payload (2026-10-06 evening)
Frank asked for the UNCELERSED Swift-1.5 with MTP back on. The RCO-IQ3_XXS
is 1.56 GiB lighter than the IQ3_S-mtp it replaced — and that lighter
weight is exactly what funds the MTP draft context (~2.4 GiB measured)
PLUS the second 131,072 slot, while keeping the q8_0 KV contract and the
two-slot shape Frank ordered that morning.
**Uncensored, intentionally:** the publisher reports 23/100 refusals vs
Swift 1.5's 98/100 (KL 0.0884 against the censored base). The refusal
escape hatch is the same as every payload here: revert and redeploy.
## Shape and KV contract
`--ctx-size 229376 --parallel 2` — TWO conversations at Frank's 112k
cap (2026-10-06 late: cut from the 128k the morning/evening shapes ran
at, to fund ubatch 1024). `CTX_SIZE` stays divisible by `PARALLEL`; the
seam enforces it, and the per-slot number (`229376 / 2 = 114,688`) is
the product contract — models.yml follows at 114,688 (a longer prompt
is a 400, never a truncation).
**KV at FP8 class: `q8_0` K + `q8_0` V (Frank, 2026-10-05 evening).**
llama.cpp has no true fp8 KV dtype; q8_0 is the 8-bit class (1.0625 B/elem
⇒ 8.5 effective bits) and the only FP8-class lever this runtime moves.
The order supersedes the same-day f16/f16 rule, which was written for the
Ornith 9B fit — at 64 KiB/token f16 the single 131,072 slot alone would
cost 8.0 GiB of KV. At q8_0 the rate is **34 KiB/token** (16 attn layers ×
4 KV heads × 256 dim × 1.0625 B, K and again V). `--flash-attn on` is
pinned: quantized V requires it, and the build's
`GGML_CUDA_FA_ALL_QUANTS=ON` (2026-09-17) makes the path real rather than
a silent f16 fallback.
**Two slots + drafter, by arithmetic and by order (2026-10-06 morning /
evening, the 2 × 131,072 era).** The morning order bought the second slot
by spending the drafter and ubatch headroom (1,397 MiB free). The evening
swap put the drafter BACK and changed the arithmetic: the draft ctx is
built against the target's FULL context (262,144 — the draft window, not
the 3-token proposal depth), so it costs **~2.4 GiB measured** (23,636 −
22,728 + 1,561), not the ~0.4 GiB the one-slot era assumed. The lighter
quant pays for it: **23,636 used / 489 MiB free** (nvidia-smi `free`
field) — the tightest fit in frank-zealot's history, the 2–3 GiB floor
knowingly violated, and the 512×512 vision transient (34 MiB) still fits
and recovers.
**The late reshape (2 × 114,688, ubatch 2048) is the current fit.** The
per-slot cut frees 2 × 16,384 tokens × 34 KiB ≈ 1.05 GiB of KV; the
ubatch rungs 512 → 1024 → 2048 spend it back on compute buffers (a
prefill knob — decode never moves with it; the rungs cost ~0.3 and
~0.75 GiB measured). Measured boot at 2048: **23,120 used / 1,005 MiB
free**. The drafter's own KV shrank with its target (draft window
229,376). Compute buffers allocate at graph capture during BOOT, so a
miss is loud and fast — the failure mode is a crash loop, not silent
corruption.
## Speculative decoding — ON since 2026-10-06 evening (draft-mtp, depth 3)
**Frank's evening order: MTP back on.** The MTP head ships IN the file —
the `-mtp` GGUF carries `blk.64.nextn.*` (`qwen35.nextn_predict_layers=1`)
— and `--spec-type draft-mtp` loads it against the main model's own
weights (no `--model-draft`, no second draft model — two context-backed
drafters cannot share a process). `SPEC_DRAFT_N_MAX=3` is the proposal
depth. **ngram-mod stays skipped, not swapped in:** on a qwen35 hybrid,
ngram speculation logs *"recurrent state rollback is not compatible with
'ngram-mod' — disabling rollback support"* (fleet lesson from
frank-dragoon's chain measurement, 2026-10-03). A rejected draft on Gated
DeltaNet layers cannot be rewound by KV truncation — without the rollback
checkpoints the DeltaNet state carries phantom transitions and output
degrades silently. The flags move as a set: the serve script emits BOTH
`--spec-type` and `--spec-draft-n-max` together or neither
(`SPEC_TYPE=none` strips both).
## Vision contract
`--mmproj` is the entire vision path (the GGUF carries no vision tensors).
Probes: a solid-color PNG **≥ 256 px** answers its exact color — a tiny
probe undersupplies image tokens and the model invents ("video"); the
legacy top-level `images` field is IGNORED, only the OpenAI `image_url`
content part reaches the model. **New on this payload: the uncensored
model THINKS more than the censored Swift 1.5** — a 64-token budget was
eaten entirely by `reasoning_content` (`finish_reason: length`, empty
`content`); 128 tokens clears it. Token budgets for vision probes must
leave room for thinking.
## Measured: the 1 × 131,072 MTP boot (2026-10-05, historical)
- **Boot ledger**: `creating MTP draft context against the target model
'…/Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf'`, `loaded multimodal
model, '…/Swift15-IQ3S/mmproj-Qwen3.8-27B-F16.gguf'`, `n_slots = 1,
n_ctx_slot = 131072` — every expected line, no q8→f16 fallback.
- **VRAM: 19,404 used / 4,721 MiB free** (nvidia-smi `free` field). The
4.5 GiB prediction held; f16 KV would have left ~700 MiB, so this free
number doubles as the q8_0 proof.
- **Decode + MTP acceptance** (draft counts from the live `timings`):
temp 0.8 → 66.1 tok/s, 182/349 accepted (52.1 %); temp 0 → 66.9 tok/s,
166/318; **deployed recipe (temp 1.0/top_p 0.95/top_k 20) → 58.9 tok/s,
159/355 (44.8 %)** — temperature buys fluency, costs acceptance and
rate. For scale: MiMo Q6_K measured 94.4 tok/s on the same card — this
27B at IQ3_S runs ~2/3 the rate with ~3× the capability class.
- **Vision**: 512×512 solid-red PNG → `Red`, `finish_reason: stop`.
- **Template gate**: `reasoning_effort: xhigh` → 200 with
`reasoning_content` populated; `high` → HTTP 500 (Jinja raise) — the
models.yml fold is mandatory, exactly as the MiMo eras found.
- Streaming completion: 200. No first-request silence window (the
halogen-era 45–60 s stall has no analogue here).
## Measured: the 2 × 131,072 no-spec boot (2026-10-06 morning, historical)
- **Boot ledger**: `loaded multimodal model, '…/Swift15-IQ3S/
mmproj-Qwen3.8-27B-F16.gguf'` and NO draft-context line (speculation
off). `/slots`: **2 slots, n_ctx 131,072 each** (pool 262,144).
- **VRAM: 22,728 used / 1,397 MiB free** (nvidia-smi `free` field) — the
predicted 0.3–1.3 GiB window held; the old 2-3 GiB floor is knowingly
violated by Frank's order. After a 512×512 image probe: 1,363 MiB —
the vision transients fit.
- **Decode without the drafter**: 48.0 tok/s (temp 0) / 47.8 (deployed
recipe temp 1.0). `draft_n` absent from `timings` (no drafter). The MTP
era's 58.9–66.9 tok/s is the honest price of the second context:
−18–28 %.
- **Vision**: 512×512 solid-red PNG → `Red` again on this shape.
- Deploy: commit 812d9eb, e2e gate 33/33 green on the live fleet.
## Measured: the 2 × 131,072 draft-mtp boot (2026-10-06 evening, historical)
- **Boot ledger**: `creating MTP draft context against the target model
'…/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf'`,
`loaded multimodal model, '…/Swift15-UncIQ3XXS/mmproj-BF16.gguf'`,
`n_slots = 2, n_ctx_slot = 131072` — every expected line, no q8→f16
fallback.
- **VRAM: 23,636 used / 489 MiB free** (nvidia-smi `free` field) — the
draft ctx measured ~2.4 GiB, ~6× the ~0.4 GiB the one-slot era
assumed; the lighter quant (−1.56 GiB) plus the morning's no-spec base
still fund both slots. After a 512×512 image probe: 489 MiB — the vision
transient (34 MiB) fits and recovers.
- **Decode + MTP acceptance** (draft counts from the live `timings`,
512-token essay): temp 0 → **72.9 tok/s**, 333/534 accepted (62.4 %);
deployed recipe (temp 1.0/top_p 0.95/top_k 20) → **63.3 tok/s**,
305/612 (49.8 %). e2e traffic in the boot window read 0.52–0.67
acceptance. Against the no-spec shape (48.0 tok/s): **+31 % at the
recipe temp** — the drafter earned its 2.4 GiB.
- **Vision**: 512×512 solid-red PNG → `Red` at a 128-token budget (64
tokens: all thinking, empty content). `prompt_tokens: 321` confirms the
image part reached the model.
- **Prompt cache**: `cached_tokens: 317` of 321 on the second probe — the
slot prefix cache is live.
- Deploy: commit da36fa3, e2e gate 33/33 green on the live fleet.
## Measured: the 2 × 114,688 / ubatch-1024 boot (2026-10-06 late, historical)
- **argv** (from `/proc/<MainPID>/cmdline`): `--ctx-size 229376
--parallel 2` (per-slot 114,688), `--batch-size 4096 --ubatch-size
1024`, `--spec-type draft-mtp`. Deploys 87721ab (the rename — old
`frank-small.service` stopped + disabled first, by hand: `disable` on
a relinked unit removes only its symlink) then f2575a1 (the shape);
health gate + MainPID check green, e2e 32 passed / 1 flaky-retry.
- **VRAM: 22,374 used / 1,751 MiB free** (nvidia-smi `free` field) —
the per-slot cut (≈1.05 GiB of KV) paid for the ubatch-1024 buffers
and netted back ~1.25 GiB against the evening's 489 MiB; back above
the morning's 1,397 MiB.
- **Decode + MTP acceptance** (live `timings`): recipe (temp 1.0) →
**58.2 t/s** (430/1005, 42.8 %); temp 0 → **66.9 t/s** (475/873,
54.4 %). The 2 × 131,072 boot measured 63.3 / 72.9 — −8–10 %, but
acceptance is text-dependent and these prompts differ from that
measurement's, so the comparison is CONFOUNDED, not a shape verdict;
a clean A/B needs stop + mask (bench policy).
- **Smoke**: `READY` completion + `reasoning_content` populated; the
harness reached :7771 under the new alias (models.yml provider/id
`frank-zealot`, config.yml modelRoles rewritten; per-slot
`contextWindow: 114688`).
## Measured: ubatch 2048 (2026-10-06 midnight, live)
- **argv**: `--ctx-size 229376 --parallel 2 --batch-size 4096
--ubatch-size 2048 --spec-type draft-mtp` (commit d8da7a9, Frank's
"set ubatch to 2k"; deploy green, e2e gate 33/33).
- **VRAM: 23,120 used / 1,005 MiB free at boot** — the 1024 → 2048 rung
cost **746 MiB**, ~130 over the 616 MiB the rung measured on
2026-10-05 (rung cost is shape-dependent, not a constant). The
512×512 solid-red vision probe (`Red`) then took ~240 MiB **and kept
it**: settled **763 MiB free** — on this boot the vision encoder's
buffers stay resident after the request (the evening boot saw its
34 MiB transient return).
- **Decode (ubatch-invariant knob, same prompt as the 1024 rung)**:
recipe → 52.0 t/s (388/1133, 34.2 %) vs 58.2 t/s / 42.8 % at 1024 —
acceptance variance at temp 1.0, not a ubatch effect.
- **Prefill (THE ubatch knob)**: **1,209 t/s @ 3,760 tokens** — first
recorded number for this shape; future rungs compare against this
(no same-prompt 1024 baseline exists; the law predicts ≈ +4.6 %
1024 → 2048).
- **Headroom left ~0.7–1.0 GiB.** ubatch 4096 (= the logical batch, no
micro-splitting) doubles the buffer growth again — not affordable;
**2048 is the ceiling for this shape.**
## Client contract
- Reasoning in `reasoning_content` (`--reasoning-format deepseek`);
thinking tokens COUNT against `max_tokens`. **This payload thinks more
than the censored Swift 1.5** (uncensored finetune) — short `max_tokens`
budgets can be eaten entirely by thinking (`finish_reason: length`,
empty content); the 32,768 `maxTokens` in models.yml is comfortably
above.
- The embedded chat template gates `reasoning_effort` to
`{xhigh, medium, low}` — anything else is a raise/HTTP 500, so
`models.yml` `compat.reasoningEffortMap` folds the harness levels onto
that set (mandatory, same as every Qwen3.8-template endpoint).
- Sampling rides the wire (`extraBody`): the publisher recipe temp 1.0 /
top_p 0.95 / top_k 20 / min_p 0 (evaluated at ctx 131,072).
- The RCO-IQ3_XXS tier is a **per-tensor allocation mix** (IQ3_S 97 /
BF16 96 / IQ3_XXS 83 / IQ2_S 72 / IQ4_XS 33 / IQ2_XS 32 / IQ2_XXS 30 /
Q2_K 28 / Q4_K 19 / Q6_K 8 / IQ1_S 4 / IQ1_M 4) — all stock quant types;
stock llama.cpp loads it. Publisher KLD vs Swift BF16: mean 0.002631,
90th 0.042136, max 34.49 — RCO is the publisher's "more experimental"
family.
## Rollback
**Predecessor shape (same 2 × 114,688, ubatch 1024) is ONE revert
away** — revert d8da7a9 and redeploy. The 2 × 131,072 / ubatch-512
era is one revert further (f2575a1). The 2 × 131,072 no-spec
IQ3_S-mtp shape is one revert further — revert da36fa3 next: the env
trio then ships `SPEC_TYPE=none`, `CTX_SIZE=262144`, `PARALLEL=2`,
`UBATCH_SIZE=512` against `/home/dank/models/Swift15-IQ3S/` (the
IQ3_S-mtp payload, still on disk, measured 48.0 tok/s no-spec). The
1 × 131,072 MTP shape is two reverts further back (812d9eb's parent):
the same IQ3_S-mtp file with `SPEC_TYPE=draft-mtp`, `UBATCH_SIZE=1024`,
`CTX_SIZE=131072`, `PARALLEL=1` (measured 58.9–66.9 tok/s with the head
drafting).
Three payloads on disk, all fetchable by the same script's git-history
recipes:
1. **Swift-1.5 IQ3_S-mtp** — the payload live until this cutover:
`/home/dank/models/Swift15-IQ3S/`; the IQ3_XXS swap above.
2. **MiMo-V2.6 Q6_K 2×200,704 f16 KV** — the payload that was LIVE until
the 2026-10-05 cutover: `/home/dank/models/MiMo-Q6K/`; revert
[[frank-zealot-llama-mimo-q6k-2x200704]] (env trio at commit e5893dc or
380aafa's parent), one `MODEL_FILE` + shape edit.
3. **Ornith-1.5-9B Q8_0** — the committed-but-never-deployed swap:
`/home/dank/models/Ornith-Q8/`; revert to 380aafa's tree (ONE native
262,144 slot, f16/f16 KV).
## Trail
- The Ornith-1.5-9B Q8_0 swap (77decb5 → 380aafa, earlier 2026-10-05) was
committed and pushed but its deploy never completed (blocked at the
whole-tree docs gate); the live service was still MiMo Q6_K when Swift
cut over. Both Ornith payloads were verified on disk before Swift.
- Swift header facts (block_count 65, interval 4, KV geometry, nextn) were
read from the Hub file by HTTP range + struct parse — no download needed
— before the payload was fetched.
- Cutover 2026-10-05 ~23:26: the radiance-cutover session (parallel)
label-committed both sets at Frank's direction — 1919ad7 (frank-archon
radiance) then baf33d0 (this swap: trio, seam, runbooks, fleet docs).
Deploy ran `deploy.sh dankclaw --service frank-zealot`; the service
dispatch and health gate went green; the first e2e run failed ONE spec
(12-crawl, `net::ERR_ABORTED` on `www.franksdank.cloud` — a transient,
unrelated to frank-zealot), green on rerun.
- The unit's `Description=` (MiMo era) was stale at cutover time and was
updated in-repo afterward — the relinked unit picks it up at the next
deploy/restart, not retroactively.
- 2026-10-06 (Frank): drop MTP, skip ngram, ubatch 512, TWO 128k contexts
(812d9eb). The tight fit landed: 1,397 MiB free, vision `Red`, decode
48.0 t/s no-spec. Model of the trade: the drafter's ~0.4 GiB + ubatch
~0.3 GiB bought the second 4,352 MiB KV slot.
- 2026-10-06 (Frank, evening): swap to the UNCELERSED Swift-1.5 at
RCO-IQ3_XXS, MTP back on (da36fa3). The lighter quant (−1.56 GiB) funds
the draft ctx (~2.4 GiB measured — the drafter's KV scales with the
target's full 262,144 context, not the 3-token proposal depth) plus the
second 131,072 slot: 489 MiB free, vision `Red` at a 128-token budget,
decode 63.3 t/s recipe / 72.9 temp 0 (48.0 no-spec).
- 2026-10-06 (late, Frank): the endpoint renamed frank-small →
**frank-zealot** (87721ab — scripts, unit, seam, runbooks,
identifiers; historical references to the OLD FreeToken/CUDA
frank-small, and to the Dankserver service that became frank-medium,
stay frank-small) and the per-slot window cut 131,072 → 114,688 to
fund ubatch 512 → 1024 (f2575a1). Live unit transition done by hand
before the deploy: `systemctl stop frank-small` + `systemctl disable
frank-small`, then `deploy.sh dankclaw --service frank-zealot`.
1,751 MiB free; decode 58.2 t/s recipe / 66.9 temp 0; models.yml +
config.yml retargeted.
- 2026-10-06 (midnight, Frank): "set ubatch to 2k" → `UBATCH_SIZE=2048`
(d8da7a9). Cost 746 MiB: 23,120 used / 1,005 free at boot, settling
**763 MiB free after a vision probe** (encoder buffers stay resident
on this boot). Decode 52.0 t/s recipe on the shared prompt (58.2 at
the 1024 rung — acceptance noise, decode is ubatch-invariant);
prefill 1,209 t/s @ 3,760 tokens (first recorded baseline). 2048 is
the ceiling for this shape; 4096 would not fit.