---
type: reference
title: "frank-secondary on Dankclaw — llama.cpp, Qwen3.5-9B Q6_K, 2 x 200704, f16 KV (ACTIVE)"
description: "The ACTIVE frank-secondary runbook (since 2026-10-07) — llama.cpp master (CUDA sm_86) serving HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090: two conversations of 200,704 tokens with exact f16 KV, vision through the publisher's BF16 projector, MTP impossible (no tensors in the file), and the measured geometry that priced the shape."
tags:
  - llm
  - frank-secondary
  - dankclaw
  - llama-cpp
  - gguf
  - rtx3090
timestamp: 2026-10-07
---

# frank-secondary — llama.cpp Qwen3.5-9B Q6_K, 2 × 200,704, f16 KV

Live since 2026-10-07, replacing the
[[frank-secondary-llama-gemma4-q4km-200704]] era (Q4_K_M on ONE 200,704
slot). The swap is a capacity trade: the 9B dense-hybrid rung costs ~7 GiB of
weights where the 26B MoE cost ~16.6 GiB, and the freed room buys the second
conversation slot and unquantized KV back.

## Identity

| | |
|---|---|
| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
| Endpoint | `http://192.168.0.143:7771` — served model id **`frank-zealot`** (`ALIAS` in the env; the SERVICE keeps the `frank-secondary` name), `/v1` OpenAI surface |
| Runtime | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` (`GGML_CUDA_FA_ALL_QUANTS=ON`); `qwen35` is registered in the build's own `src/llama-arch.cpp` and `src/models/qwen35.cpp` exists at that commit |
| Unit | root systemd `frank-secondary.service` ([`scripts/frank-secondary.service`](../../scripts/frank-secondary.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
| Serve script | [`scripts/serve-frank-secondary.sh`](../../scripts/serve-frank-secondary.sh), tunables [`scripts/frank-secondary.env`](../../scripts/frank-secondary.env) |
| Weights | `/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf` — HauhauCS **Q6_K**, 7,359,259,008 B, sha256 `c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986` |
| Projector | `mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf`, 921,704,448 B, sha256 `05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934` — the model is natively multimodal and the Q6_K file carries no vision tensors, so this file IS the vision capability |
| Fetch | [`scripts/download-frank-secondary.sh`](../../scripts/download-frank-secondary.sh) — `-f`, byte-count + sha256 verified (the hashes are computed locally: the repo is Xet-backed, so the Hub publishes file OIDs, not file sha256), idempotent, `--verify-only` |
| Seam | `tests/frank_secondary_unit.bats` (argv assertions against a stub engine) |

## Payload geometry (read from the GGUF header, not the model card)

Range-read of the file's own metadata + tensor-info section:

|Key|Value|
|---|---|
|`general.architecture`|`qwen35` (dense hybrid: Gated DeltaNet + softmax attention)|
|`qwen35.block_count`|32, `full_attention_interval` 4 ⇒ **8 full-attention layers (3, 7, 11, …, 31) + 24 DeltaNet layers**|
|attention|16 heads, **4 KV heads**, `key_length` = `value_length` = 256|
|`qwen35.ssm.*`|`conv_kernel` 4, `state_size` 128, `group_count` 16, `time_step_rank` 32, `inner_size` 4096|
|context / vocab|262,144 native / 248,320|
|tensor types|Q6_K × 250 + F32 × 177 — standard types only, so the stock master build loads it (the ROCmFPX-style custom-quant lock-in does not apply)|
|MTP|**no `nextn`/`mtp` tensors exist** — in-file MTP is impossible; the card's "MTP support" line describes the base architecture, not this quantized export|
|chat template|supports `enable_thinking`, `tools`/`tool_call`, image/video content parts; it has **no `reasoning_effort` variable** (unused Jinja variables are ignored, so the harness's effort map is inert here, not an error source)|

## Shape

`--ctx-size 401408 --parallel 2` — llama.cpp divides the pool across slots,
so each conversation gets `n_ctx_slot = 200,704` (Frank's cap; the harness
`contextWindow` quotes the per-slot number). `--flash-attn on`,
**`--cache-type-k f16 --cache-type-v f16`** (exact KV), batch 4096 /
**ubatch 2048**, `--log-verbosity 4`, `--jinja --reasoning auto
--reasoning-format deepseek`, `--mmproj`, **speculative decoding OFF**
(`SPEC_TYPE=none` strips both spec flags — the file has no MTP tensors).

**KV rate law.** The pool scales with the 8 full-attention layers: 8 × 4 KV
heads × 256 dim = 8,192 elements/token per cache, so f16 K + f16 V bills
32,768 B/token = **32 KiB/token** → 12.25 GiB at the 401,408 pool. The 24
DeltaNet layers bill a fixed recurrent state **per slot**, not per token —
that is what makes two 200k slots affordable where the 26B MoE fit only one.

**Priced from boots, not formulas.** MiMo-V2.6-Distill-Qwen-9B (identical
qwen35 geometry, 7,793,710,624 B, same mmproj class) at 2 × 200,704 f16 K +
f16 V measured **21,276 used / 2,849 free** at ubatch 1024 and **21,846 /
2,279** at ubatch 2048 — see [[frank-secondary-llama-mimo-q6k-2x200704]].
This file is 415 MiB lighter in weights and its projector is 3 MiB heavier,
so the projection is ~21,430 used / ~2,700 free.

Ubatch 2048 is the measured CUDA optimum for this card and arch (+4.6 %
prefill at 8.9k depth, decode flat, cost 616 MiB); `1024` is the documented
fallback rung if a future shape lands under the 2 GiB floor.

## Measured boot (2026-10-07, deploy `ec6184d`)

`nvidia-smi`: **21,436 MiB used / 2,689 MiB free** — the MiMo-priced
projection (~21,430 / ~2,700) was within 11 MiB. Engine ledger
(`--log-verbosity 4`, `journalctl -u frank-secondary`):

|Line|Value|
|---|---|
|`llama_context`|`n_ctx = 401408`, `n_ctx_seq = 200704`, `kv_unified = false`|
|`llama_kv_cache`|`CUDA0 KV buffer size = 12544.00 MiB` — `size = 12544.00 MiB (200704 cells, 8 layers, 2/2 seqs), K (f16): 6272.00 MiB, V (f16): 6272.00 MiB`|
|`sched_reserve`|`CUDA0 compute buffer size = 1136.06 MiB`, `CUDA_Host compute buffer size = 848.07 MiB`|
|`clip_model_loader` (mmproj)|`model size: 878.99 MiB` + `reserve_compute_meta: CUDA0 248.10 MiB, CPU 24.93 MiB`|
|`srv load_model`|`n_slots = 2, n_ctx_slot = 200704`|
|`print_info`|`n_ctx_train = 262144`, `n_head_kv = 4`|

The `32 KiB/token` law is confirmed exactly: 401,408 × 32 KiB = 12,544 MiB,
which is what the engine printed. The `n_ctx_seq (200704) < n_ctx_train
(262144)` line is informational (a context **below** the trained window is
safe; the Gemma4 era's warning was the opposite case, a drafter running 53 %
**past** its window). Weights load in ~6 s from page cache; `/health` answers
`{"status":"ok"}` and `/v1/models` serves the alias `frank-zealot`.

**Wire rename (2026-10-09):** `ALIAS` moved `frank-secondary` → `frank-zealot`,
so the served id matches the harness model id and the family-facing name.
Consumers re-keyed in the same change: OpenWebUI (`model` row, `access_grant`
row, `openai.api_configs` index-2 `model_ids`) and `models.yml` (already
renamed 2026-10-08). llama.cpp does NOT validate the requested model name —
it answers and echoes the served alias — so a stale client keeps working;
the picker and any id-keyed grant is what breaks. See [[OpenWebUI]].

The deploy's Playwright gate passed (33 tests) and the fleet e2e set is
unchanged.

## Client contract

llama.cpp surface: model id `frank-secondary`, native tools via `--jinja`,
thinking in **`reasoning_content`**, vision via OpenAI `image_url` content
parts (probe with a ≥ 256 px image, or the Qwen-VL tower answers "video").
The publisher's recipe ships in `models.yml` `extraBody`: thinking mode
`temperature 0.6 / top_p 0.95 / top_k 20 / min_p 0` (non-thinking mode is
0.7/0.8/20/0). The card's advice "maintain at least 128K context to preserve
thinking" is satisfied by the 200,704 slot.

**Compaction hub**: `models.yml` routes `compactionModel` for all three
endpoints here. With `--parallel 2` a compaction request no longer serializes
behind a live generation (the Gemma4 era's `--parallel 1` law), and the
200,704 slot still fits the worst-case compaction input (~133k tokens).

## Rollback

|Target|How|
|---|---|
|Gemma4-26B-A4B Q4_K_M, 1 × 200,704, f16 K + q8_0 V|revert the swap commit — payload stays at `/home/dank/models/Gemma4-26B-A4B/`; see [[frank-secondary-llama-gemma4-q4km-200704]]|
|MiMo-V2.6 Q6_K, 2 × 200,704, f16 KV|`MODEL_FILE=/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf` + its mmproj — measured 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048|
|Swift-1.5-27B IQ3_XXS 2 × 114,688 + MTP|payload on disk at `/home/dank/models/Swift15-UncIQ3XXS/`|

## When it misbehaves

Same decision tree as the predecessors: `No CUDA GPUs are available` ⇒
`nvidia-modprobe -u -c=0`; a 400 naming 200,704 ⇒ the conversation outgrew
one slot, not the pool; queue behind two long generations ⇒ the slot ceiling
is two, by contract; a boot that OOMs ⇒ read the verbosity-4 ledger in
`journalctl -u frank-secondary` (KV MiB, compute buffers) before touching
flags — the engine prints the exact figures the shape needs.

**`status=203/EXEC` boot loop**: the rewrite of `serve-frank-secondary.sh`
shipped without its `#!/usr/bin/env bash` line, and systemd failed the
`ExecStart` before the script ran — 10 s restarts, `Mem peak: 1.9M`, and
`nvidia-smi` flat at 1 MiB used (the GPU is never touched). Journal signature:
`Failed at step EXEC spawning /home/dank/…/serve-frank-secondary.sh`. A
shebang is not decoration on this host: every LLM serve script is a direct
`ExecStart`, so `head -1` is part of the deploy check (commit `ec6184d`).