frank-small: Swift-1.5 IQ3_S-mtp swap (parallel session) + fleet docs index

The parallel session's uncommitted swap, committed at Frank's direction (commit whatever's in the working tree): - llama.cpp master on the RTX 3090 now serves ukisai's Swift-1.5-Qwen3.8-27B GSQ-RCO IQ3_S WITH the in-file MTP head, ONE 131,072-token slot at Frank's 128k cap, q8_0 K + q8_0 V, vision through the base-model projector; MiMo-V2.6 Q6_K (2x200704 f16 KV) retired to rollback target. - env/serve/download + seam twins updated together (deployer payload gates rode the previous commit's deploy-remote.sh). - docs: new swift15 runbook; MiMo Q6K runbook retired; AGENTS.md topology and both vault indexes follow (root index ACTIVE/RETIRED flip, test 135 seam green). - comfyui: two qwen-image t2i workflow drops under dankclaw/comfyui/ workflows/ (12 KiB).
2026-10-05 23:26:59 -07:00
1 parent 1919ad7f31
commit baf33d016f
@@ -24,9 +24,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
-| Dankserver | `frank-primary` | **FreeToken PR #587** (branch `rocm714_torch214`, head `77a7201`; local image `frank-primary-freetoken:pr587-77a7201` built on the host from `scripts/docker/frank-primary-freetoken.Dockerfile` — digest-pinned rocm/pytorch rocm7.14 torch-2.11.0 base, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards, modelopt block-FP8, loaded straight from safetensors), **ONE R9700 (second card idle)**, all experts in **pinned host RAM** (~120 GiB) behind a GPU LRU (`--moe-cache-rate 0.10`), PLE table 47.7 GiB pinned (root unit + `LimitMEMLOCK=infinity`), **KV pool `--num-tokens 299968`** (24 KiB/token, multiple-of-64 law of the `qsa_sparse` auto backend), 2 slots, `--max-output-tokens 36864`, **parsers AUTO** (thinking → `reasoning_content`, tools qwen3.8 dialect), vision tower host-streamed. Measured: boot ~2.3 min cold / 92 s warm, decode **17.1 tok/s** single-stream; **TP=2 does not boot on ROCm** (pynccl JIT); `/metrics` 404 (Prometheus scrape reads down). Deploy gate reads the `/health` BODY (`status:"ok"`) — FreeToken 200s while loading. Rollback tag `frank-primary-vllm-shapeG-20261003` |7770|
+| Dankserver | `frank-primary` | **radiance** (StillDeadcode v1.0.7 line; image `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned INDEX digest, 287 MiB, `libr4d` gfx1201 kernels, plain `podman run`, root systemd unit) | `Qwen/Qwen3.8-Flash-Next-FP8` official (172.8 GB, 131 shards, sha256-verified) converted **no-recipe** to `qwen4exp-fp8-native.rad` (~161 GiB) — **native modelopt block-FP8, no requantisation** — on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, `--expert-vs-cache-ratio 0.82` = the 99 %-hit point), **PLE 47.7 GiB pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV 7.32 KiB/token**, **MTP spec decode depth 3**, **2 × 200,000** (first boot; native 262,144), froggeric v22.5 template, vision + tools + `reasoning_content` auto, `/metrics` scraped. Cutover 2026-10-05 (rollback `frank-primary-freetoken-pr587-20261005`, FreeToken one-card image + payload on disk) | 7770 |
 | Dankclaw | `frank-large` | **halogen container** (`ghcr.io/peonist-ai/halogen@sha256:dc0a39a0…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | `peonist-ai/halogen-qwen3.8-27b` **`qwen3.8-27b-p1w4d-d2.hgn`** (35,865,565,184 B; the `.hgn` format loads in NO other engine — llama.cpp, vLLM and transformers all refuse it), **DFlash2 spec decode (`HALOGEN_DRAFTER=2`)**, **1 slot × 200,700 ctx** — `HALOGEN_KV_SLOTS=1` is load-bearing: batching XOR speculation, slots>1 silently disables the drafter (measured: 8 slots at serial ~6 t/s/stream), d2 KV (2× fp8, 64 KiB/token → 12.2 GiB pool, engine sizes it and prints a boot ledger naming any clamp), **TEXT-ONLY** (the engine ignores the vision encoder — vision is frank-small's job), **censored base** (the ROCmFPX AGENT rollback is the refusal escape hatch), **no server-side sampler: absent temperature is GREEDY**, so the card's thinking recipe (temp 1.0 / top_p 0.95) rides the wire in `models.yml`. Publisher card 31.7 t/s mean across eight text workloads vs the retired ROCmFPX endpoint's measured 27.6–28.1. Prompt over the slot AND `max_tokens` over `HALOGEN_MAX_TOKENS_CAP` are both 400s naming the limit, never truncations. Serve passes NO `--ulimit nofile` (rootless runc cannot raise Dankclaw's 524288 hard limit — see Key Lessons). Cutover 2026-10-05|7770|
-| Dankclaw | `frank-small` | llama.cpp master, CUDA sm_86 build (root systemd unit, relinked) | bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** (9B dense qwen3_5 hybrid, 7.26 GiB GGUF, no MTP weights so no spec decode) + the repo's f16 mmproj for **vision**, on the RTX 3090, **2 × 200,704** (pool 401,408 — NOT the native window) with **fully unquantized KV (f16 K + f16 V)** under flash attention, **batch 4096 / ubatch 2048** (doubled 2026-10-05: measured +4.6 % prefill at 8,928 tokens for 616 MiB), 21,846 used / **2,279 free** of 24,576 MiB, 94.4 tok/s decode. Resized 2026-10-03 (3 × 180,224 + q8_0 V → 2 × 200,704 + f16 V → + Q6_K rung: the freed slots paid for exact values AND the higher quant; the rung costs a measured −10 % decode — weight-bandwidth-bound); rollback `MODEL_FILE=…/MiMo-Q5KM/…Q5_K_M.gguf` (buys the 10 % back) or `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (Q5 only) | 7771 |
+| Dankclaw | `frank-small` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF **IQ3_S -mtp** (27B dense qwen3_5 hybrid, 12,120,016,896 B; a per-tensor allocation mix of stock types; **MTP head IN the file** — `--spec-type draft-mtp`, depth 3) + unsloth's Qwen3.8-27B **F16 mmproj** for **vision** (no Swift projector ships; the base vision stack was hash-proven byte-identical to Swift's), on the RTX 3090, **1 × 131,072** (Frank's 128k cap; a second 131,072 slot misses the 2-3 GiB floor) with **q8_0 K + q8_0 V (FP8 class, 34 KiB/token)** under flash attention, batch 4096 / ubatch 1024. Supersedes BOTH same-day payloads: the live MiMo-V2.6 Q6_K 2×200,704 f16-KV shape and the Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but never deployed. | 7771 |
 Retired endpoints (`frank-npu`, and every prior runtime) are in
 the [runbook index](#runbook-index). The full swap history lives in
@@ -264,7 +264,7 @@ merely wedged kernel usually still answers ARP.
     `sudo systemctl stop frank-large && sudo systemctl mask frank-large`
   - Dankclaw `frank-small` (root unit, llama.cpp on the 3090):
     `sudo systemctl stop frank-small && sudo systemctl mask frank-small`
-   - Dankserver `frank-primary` (root unit, FreeToken on one R9700):
+  - Dankserver `frank-primary` (root unit, radiance on both R9700s, TP=2):
     `sudo systemctl stop frank-primary && sudo systemctl mask frank-primary`
 2. Bench.
   NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
@@ -299,8 +299,9 @@ a role pointing at a provider the file does not define fails at call time. As
 of 2026-10-05 **two of the three endpoints accept images** (`input: [text,
 image]`) — `frank-large` went TEXT-ONLY with the halogen swap (the engine
 ignores the vision encoder; the BF16-mmproj era ended 2026-10-05) — while
-`frank-primary` takes them natively (FreeToken builds the qwen3.8 vision
+`frank-primary` takes them natively (radiance runs the checkpoint's
-tower and streams its block weights from host RAM two at a time by default),
+27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
 are the engine core; `--mm-max-patches` available),
 and `frank-small` through
 `mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf` passed as `--mmproj` — the MiMo
 GGUF itself carries no vision tensors. `frank-small`'s `contextWindow` is
@@ -378,132 +379,145 @@ rollback (payloads removed 2026-10-03; `scripts/build-gufo.sh` stays).
 ### frank-small — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
-**llama.cpp master** (CUDA sm_86 build
+**llama.cpp master** (0.5.0-dev, commit `5fc4f3c`; CUDA sm_86 build
-`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`) as a
+`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`,
-**root systemd unit** (`scripts/frank-small.service`, `relink_unit`ed into
+`GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit**
-`/etc/systemd/system/`), serving
+(`scripts/frank-small.service`, `relink_unit`ed into
-`bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF` at **Q6_K** (7,793,710,624 B,
+`/etc/systemd/system/`), serving `ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF`
-sha256 `ef96d05a…`) **plus `mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf`**
+at **IQ3_S -mtp** (12,120,016,896 B, sha256 `9aecf1cd…`) **plus
-(918,166,048 B, sha256 `ff348f…`), both `/home/dank/models/MiMo-Q6K/` (the
+`mmproj-Qwen3.8-27B-F16.gguf`** (927,607,488 B, sha256 `cbb841a9…`), both
-projector is a hardlink of the Q5KM copy — the dir is self-contained), on
+`/home/dank/models/Swift15-IQ3S/`, on port 7771. Cutover 2026-10-05
-port 7771. The rung went Q5_K_M → Q6_K on 2026-10-03 the same morning as the
+(evening) on Frank's explicit contract: **128k context cap, KV at FP8,
-reslice, funded by the freed slots. The runtime arrived 2026-10-01 night for
+vision on, as many 128k contexts as fit** — the answer is ONE, by
-one reason:
+arithmetic. The same-day payloads it supersedes: the LIVE MiMo-V2.6 Q6_K
-**asymmetric KV** — `--cache-type-k f16` + `--cache-type-v q8_0` is the fleet's
+shape and the Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa)
-only per-tensor KV lever, and SGLang's `--kv-cache-dtype` moves K and V
+but never deployed (blocked at the docs gate; the docs debt is what this
-together. On 2026-10-03 Frank spent the headroom that lever bought: the reslice
+section settles).
 from three slots to two made room for **fully unquantized V**
 (`CACHE_TYPE_V=f16`); both flags stay explicit in the serve script. Vision came
 2026-10-02 via `--mmproj`: the GGUF carries no `v.*` tensors, so the projector
 file is the entire vision path (and ~1.4 GiB of the card).
-**Architecture: qwen3_5 hybrid** — 32 layers, 8 full-attention + 24 Gated
+**Architecture: qwen3_5 hybrid, bigger** — block_count 65 (64 real layers +
-DeltaNet, native 262,144 window. KV exists only in the 8 attention layers
+the blk.64 MTP head), `full_attention_interval 4` ⇒ **16 full-attention +
-(4 KV heads × 256 dim × 2 B ≈ 2 KiB/token/layer, ~16 KiB/token for f16 K+V);
+48 Gated DeltaNet**, native 262,144 window, dense. KV exists only in the 16
-the DeltaNet state is a fixed per-slot buffer, so slot count — not context —
+attention layers (4 KV heads × 256 dim): 64 KiB/token at f16, **34 KiB/token
-drives that term.
+at q8_0**. The DeltaNet state is a fixed per-slot buffer (~150 MiB/slot at
 48 linear layers × 48 value heads). Header facts were read from the Hub
 file by HTTP range + struct parse before anything was downloaded.
-**The shape (2026-10-03):** `--ctx-size 401408 --parallel 2` = **TWO
+**The shape (2026-10-05):** `--ctx-size 131072 --parallel 1` = ONE
-conversations of 200,704 tokens each** (llama.cpp divides the pool:
+conversation at Frank's 128k cap (Swift's own benchmark protocol ran at
-`n_ctx_slot = CTX_SIZE / PARALLEL`), `--flash-attn on` (the seam pins it),
+131,072), `--flash-attn on` pinned, **q8_0 K + q8_0 V** (llama.cpp's FP8
-**f16 K + f16 V**, all layers on the GPU, `--mmproj` for vision, and since
+class — no true fp8 KV dtype exists; 8.5 effective bits; the order
-2026-10-05 **batch 4096 / ubatch 2048** (doubled; the boot-pair A/B is in the
+SUPERSEDES the same-day f16/f16 rule written for the Ornith 9B fit), all
-runbook). Measured trade at 8,928 prompt tokens: prefill 4,024 → **4,209 t/s,
+layers on the GPU, `--mmproj` for vision, batch 4096 / ubatch 1024 (lower
-+4.6 %**, decode flat (91.9–93.0 vs 93.8–94.3 — noise), and a 279-token prompt,
+ubatch first, Frank's standing law; 2048 is the first A/B if boot lands
-which sits below either ubatch, did not move at all. The old "ubatch 2048
+high). Non-KV ≈ 15.5 GiB (weights 11,561 + projector ~1,350 + buffers
-failed the scratch boot" note belongs to the Q5_K_M predecessor and does NOT
+~1.6k + CUDA ~500 + DeltaNet ~150/slot + MTP draft ctx ~0.3-0.5k); one q8_0
-generalize: this shape boots 2048 clean. Measured VRAM **21,846 used / 2,279
+slot = 4,352 MiB ⇒ free ≈ 4.5 GiB; a SECOND slot wants +4,352 ⇒ ~0.2 GiB —
-free** of 24,576 MiB against **21,276 / 2,849** for the 1024 arm, reproduced
+far under the 2-3 GiB floor. The pool is capped by the 128k law, not the
-byte-identically the same hour — the doubling costs **616 MiB** and lands at
+card; leftover headroom is honest slack.
 the BOTTOM of the 2–3 GiB floor Frank asked for (`4096/1536` is the middle rung
 if that margin is ever wanted; non-KV 9,302 MiB, up from 8,732 by exactly the
 compute buffers; f16 K+V is 32 KiB/token across the 8 attention layers;
 quote `nvidia-smi`'s `free` field, never `total − used` — the driver holds
 ~450 MiB of reserve the subtraction hides). The 2026-10-02 3 × 180,224 +
 q8_0-V shape stays one env edit away (`CTX_SIZE=540672 PARALLEL=3
 CACHE_TYPE_V=q8_0`, and it must pair with Q5_K_M — Q6_K breaks the floor
 there). Three native windows need 18.4 GiB of KV alone and were never on the
 table.
-Verified at deploy (2026-10-03): boot logs `n_slots = 2, n_ctx_slot = 200704`
+**Speculative decoding is the file's own MTP head**: the `-mtp` GGUF
-and `loaded multimodal model`; decode **94.44 tok/s** at temp 0 on the probe
+carries `blk.64.nextn.*` (`qwen35.nextn_predict_layers=1`), so
-that measured 104.84 on Q5_K_M — **the Q6_K rung costs a measured −10 %**:
+`--spec-type draft-mtp --spec-draft-n-max 3` loads it against the main
-decode on this card is weight-bandwidth-bound and Q6_K streams ~10 % more
+model's weights — no `--model-draft`, no second draft KV pool. Flags move
-bytes/token; health 200, e2e 33/33; native `tool_calls`; thinking in
+as a set (a no-spec baseline strips BOTH). Acceptance reads from the
-`reasoning_content`; a **512×512** solid-red PNG answers `Red` through an
+response `timings` (`draft_n`/`draft_n_accepted`) on real traffic, never a
-`image_url` content part (two traps: the legacy top-level `images` field is
+synthetic repeat.
 ignored, and a TINY probe image undersupplies image tokens — a 10×10 PNG made
 the model claim "it's a video"; probe with ≥256 px). A prompt over the slot
 is a **400 `exceed_context_size_error`**, which is why `models.yml` sets
 `contextWindow: 200704`.
-Runbook: `serverAppConfigs-docs/LLM/frank-small-llama-mimo-q6k-2x200704.md`
+**Vision without a Swift projector**: no Swift repo ships one (ukisai's
-(the boot-pair table that prices pool changes lives in the retired
+README is explicit), so the projector is unsloth's mmproj built from the
-[[frank-small-llama-mimo-q5km]]).
+SHARED BASE `Qwen/Qwen3.8-27B` — legal because Swift-1.5's vision stack was
-Rollback: `MODEL_FILE=/home/dank/models/MiMo-Q5KM/MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf`
+hash-proven byte-identical to the base (333/333 tensor names, 6/6 sampled
-buys back 875 MiB and the −10 % decode rate (3,571 free, 104.84 tok/s, both
+tensors: merger fc1/fc2, blocks 0/13/26) on 2026-10-05. The probe traps
-measured); the older shape `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0`
+carry over: ≥256 px probe images or the model answers "video"; only the
-pairs with Q5 only; SGLang rollback in
+OpenAI `image_url` content part reaches the model.
 [[frank-small-sglang-mimo-v26]] (venv on disk at
 `/home/dank/GitPackages/sglang-cu128/`, INT8 payload staged to
 `~dank/.AGENTTRASH/20261002-retired-llm-payloads/`).
-### frank-primary — FreeToken (PR #587 ROCm), Qwen3.8-Flash-Next FP8 (Dankserver, ACTIVE)
+The GSQ-RCO "IQ3_S" tier is a **per-tensor allocation mix** of stock quant
 types (IQ2_XS 144 / IQ3_XXS 96 / Q6_K 78 / F32 360 / BF16 96 / Q3_K_M 39 …)
 — stock llama.cpp loads it; the tensor-type histogram is the law (see Key
 Lessons). Publisher KLD vs Swift BF16: 0.0513 development / 0.024–0.052
 held-out; the MTP packaging is NOT separately evaluated by the publisher.
-The **FreeToken engine at PR #587 head `77a7201`** (branch
+Runbook:
-`rocm714_torch214`, built on the host into the local image
+`serverAppConfigs-docs/LLM/frank-small-llama-swift15-iq3s-mtp-131072.md`.
-`localhost/frank-primary-freetoken:pr587-77a7201` by
+Rollback: `/home/dank/models/MiMo-Q6K/` (the LIVE payload until the cutover;
-`scripts/docker/frank-primary-freetoken.Dockerfile` — the PR's own
+revert to the 2×200,704 f16 shape — [[frank-small-llama-mimo-q6k-2x200704]])
-docs/install_amd.md stack: digest-pinned
+or `/home/dank/models/Ornith-Q8/` (the committed-never-deployed Q8_0;
-`rocm/pytorch:rocm7.14_ubuntu24.04_py3.12_pytorch_release_2.11.0` base =
+revert to 380aafa's tree).
-torch 2.11.0+rocm7.14/triton 3.7, `--no-build-isolation --no-deps -e .`,
+
-curated portable deps, `flashlib==0.3.0 --no-deps`, and a `libhiprtc.so`
+### frank-primary — Radiance (StillDeadcode), Qwen3.8-Flash-Next FP8 native, TP=2 (Dankserver, ACTIVE)
-dev-symlink the pip-ROCm wheel lacks) serving
+
-**`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8`** (~172.8 GiB, 131 shards,
+The **radiance engine** (StillDeadcode, Apache-2.0; image
-standard modelopt block-FP8 — 75,264 `weight_scale_inv` tensors, no
+`stilldeadcode/radiance@sha256:4c3f484564bb2872784e4…` digest-pinned,
-`input_scale`, loaded directly from HF safetensors, no FTW conversion) on
+287 MiB, `libr4d@0.5.0` kernels prebuilt for gfx1201, engine as the image
-**ONE R9700 (gfx1201)** — the second card sits idle. Plain `podman run`
+entrypoint) serving the **official `Qwen/Qwen3.8-Flash-Next-FP8`
-through `scripts/serve-frank-primary.sh` under the root systemd unit
+checkpoint at NATIVE precision** — modelopt block-FP8 throughout, no
 requantisation, no calibration. Plain `podman run` through
 `scripts/serve-frank-primary.sh` under the root systemd unit
 (`frank-primary.service`, `User=dank`, `LimitMEMLOCK=infinity`); the
-container name is the unit name (`frank-primary`; the vLLM-era
+container name is the unit name (`frank-primary`). Cutover 2026-10-05;
-`frank-primary-qwen` is in the legacy cleanup list). Cutover 2026-10-05;
+rollback tag `frank-primary-freetoken-pr587-20261005` (the FreeToken trio
-rollback tag `frank-primary-vllm-shapeG-20261003` (the tcclaviger vLLM trio
+ Dockerfile are in the tag; the image and the orcarouter payload stay on
-and its 362-line env lore live in that tag; the tcclaviger image stays in
+disk until the window closes), older rollback `frank-primary-vllm-shapeG-20261003`.
 dank's rootless storage).
-**Shape**: all expert weights in **pinned host RAM** (~120 GiB) behind a
+**Payload chain**: official checkpoint → staging dir whose `config.json`
-GPU LRU (`--moe-cache-rate 0.10`, ~12 GiB — the single-card ceiling given
+drops `quantization_config` → `stack-qwen38-official-fp8.py` (two
-the KV contract), the ~47.7 GiB PLE ngram table pinned in host RAM
+value-preserving reshapes: experts byte-stacked into the plugin's stacked
-(→ root unit + memlock law), **KV pool `--num-tokens 299968`**
+`gate_up_proj`/`down_proj` layout with `_scale_inv` at the same index — 49
-(24 KiB/token f16 across the 12 full-attention layers ≈ 7.0 GiB; **must be
+stacks × 512 experts, no requantisation; PLE's 128 scaleless fp8 shards
-a multiple of 64** — the auto attention backend is `qsa_sparse`, which
+decoded to one bf16 shard with the checkpoint's own bf16 scale
-overrides page size to 64; 300,000 was refused at first boot), 2 decode
+1.9931793212890625e-04) → `rad-convert` with ONE `--quant` rule — pure
-slots, `--max-output-tokens 36864` (the harness maxTokens; requests that
+copy for every weight, and the rtn rule re-encodes the bf16 PLE at that
-omit one get this default). **Parsers are AUTO**: thinking lands in
+same fixed scale, an exact round-trip that restores the original codes and
-`reasoning_content` natively, tool calls in the qwen3.8 dialect — models.yml
+lands the plugin's one-element scale plane →
-flips `reasoningContentField` to `reasoning_content` on the harness side.
+`/home/dank/models/rad/qwen4exp-fp8-native.rad` (~161 GiB, floor
-Vision works (models.yml keeps `input: [text, image]`); the vision tower's
+`FRANK_PRIMARY_RAD_MIN_MIB=158000`). The staging drop is the plugin
-blocks stream from host RAM two at a time by default.
+SELECTION law (spec §2.4): the descriptor is `(arch_id, quant)` matched
 exactly, no shipped plugin claims `(qwen4exp, "fp8")`, the shipped
 `qwen4exp_fp8` plugin claims `(qwen4exp, "")` — and the descriptor is only
 the selection key, per-weight encodings are data, so nothing else is
 requantised.
 Raw serve AND unmodified convert both refuse; that is the design. The
 checkpoint itself is sha256-verified against the Hub (131/131 shards,
 2026-10-05). Built by `scripts/build-frank-primary-rad.sh` (`--plan-only`
 first — its declare phase is the engine's own, so a resolving plan boots).
-**Measured at cutover**: boot ~2.3 min cold-ish / 92 s warm → /health
+**Shape (both R9700s, TP=2)**: `--tp-wire wht6` (fused lossy all-reduce,
-status ok; decode **17.1 tok/s** single-stream temp 0 (the vLLM incumbent
+5.6–6.3 % prefill / ~−1.3 % 8-stream decode at TP2), `--max-num-seqs 2`
-measured ~30 tok/s on TWO cards; the KV contract caps the single-card LRU,
+(Frank's first boot; raise from the ledger + slot occupancy), `--max-model-len
-and **TP=2 exists but does not boot on ROCm** — the TP allreduce JIT
+200000` (native 262,144; the harness contextWindow must quote this — a longer
-(`pynccl.cu`) can't resolve `cuda_runtime.h` for gfx1201; probed
+prompt is refused, never truncated), `--placement expert_tiered
-2026-10-06, signature in the runbook). `/metrics` is a 404 — the Dankclaw
+--host-pool-mib 40960` (native FP8 experts are 112.8 GiB total, 56.4 GiB
-Prometheus llm-fleet scrape of :7770 reads down until repointed.
+per rank slice — the pinned pool is the price of native precision),
-`ft ctl cache --kv/--moe` resizes pools live (grow only; shrink needs a
+`--expert-vs-cache-ratio 0.82` (the measured 99 %-hit point; 0.87 collapses
-restart); `ft ctl stats` reports occupancy.
+to 0 %), `--kv-cache-dtype fp8` (**7.32 KiB/token**; a 200K sequence is
 1.43 GiB), `--num-speculative-tokens 3` (the checkpoint's own MTP head —
 3,101 `mtp.*` tensors, probe depth 3), `--max-num-batched-tokens 2048`,
 prefix cache 4 GiB pinned host + 64 GiB disk at
 `/home/dank/radiance-state`, and **`--ngram-placement ram`**: the 47.7 GiB
 PLE n-gram table (51.2 GB = 29 % of all parameters) serves from pinned
 host RAM. Froggeric v22.5 chat template vendored
 (`scripts/qwen38-froggeric-v22.5.jinja`, `--override-chat-template`).
-**Deploy gate law (changed for FreeToken)**: `/health` returns **200 with
+**The pinned-RAM ledger is why this is a root unit**: 2 × 40 GiB pools +
-`{"status":"loading"}`** while the engine loads — the FastAPI front end
+47.7 GiB PLE + 4 GiB prefix host ≈ **134 GiB** charged against
-binds before the scheduler is up, so a bare `curl` false-greens (measured:
+`RLIMIT_MEMLOCK`. `ttm.pages_limit`/`pages_max` cap GPU-mappable pinned
-"healthy after 6 s" mid-page-in). The deployer loops until the body says
+pages (pre-cutover cap 123 GiB — BELOW the plan); the serve script's
-`"status":"ok"`; the MainPID stray-port guard is unchanged. The image gate
+TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to start until
-is `podman image exists` (a local build carries no RepoDigests — the old
+both knobs are raised in the stop window (sysfs write first, kernel
-digest grep refused exactly this shape; pinning lives in the Dockerfile).
+cmdline + reboot only if read-only).
-Runbook: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`.
+**Contract facts** (source-verified): the HTTP listener binds AFTER the
 full load, so a bare `curl /health` gate is honest (`/health` == `/ping`,
 200 serving / 503 stopping); sampling params temperature…preserve_thinking
 accepted, **unknown params 400** (models.yml's extraBody set is fully
 inside it); `response_format` supported; `logprobs` REFUSED; reasoning in
 `reasoning_content` and tool calls in the qwen3.8 dialect via the auto
 chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
 `--mm-max-patches`); `/metrics` carries vLLM-compatible names +
 `radiance:*` families — the Dankclaw Prometheus scrape is re-added.
 Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`.
 Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`.
 Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
 Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`.
@@ -950,8 +964,8 @@ Active:
 | Service | Runbook |
 |---------|---------|
 | frank-large (halogen container, Qwen3.8-27B .hgn + DFlash2, 1×200,700, text-only, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md` |
-| frank-small (llama.cpp CUDA, MiMo-V2.6-Distill-Qwen-9B Q6_K, 2×200,704 f16 KV, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-llama-mimo-q6k-2x200704.md` |
+| frank-small (llama.cpp CUDA, Swift-1.5-27B GSQ-RCO IQ3_S-mtp, 1×131,072 q8_0 KV, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-llama-swift15-iq3s-mtp-131072.md` |
-| frank-primary (FreeToken PR #587 ROCm, Qwen3.8-Flash-Next FP8, 1×R9700, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md` |
+| frank-primary (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md` |
 | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
 | Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
@@ -961,6 +975,9 @@ Retired (rollback targets and history):
 |---------|------|--------|---------|
 | ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
 | tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-primary`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-primary-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
 | FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-primary`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-primary-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-primary-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-primary-freetoken-pr587.md` |
 | llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-small`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-small-llama-mimo-q6k-2x200704.md` |
 | llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-small`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-small-llama-swift15-iq3s-mtp-131072.md` |
 | ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` |
 | gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` |
 | vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |
@@ -0,0 +1,64 @@
 {
  "3": {
    "class_type": "CLIPTextEncode",
    "_meta": { "title": "Positive prompt" },
    "inputs": {
      "text": "A detailed illustration of a 1930s Soviet industrial design bureau at night: a well-measured engineer at a drafting table, a warm desk lamp, rolled blueprints, gears and calipers on the desk, deep red and cream palette, gouache painterly style, intricate mechanical detail, soft shadows",
      "clip": [ "6", 0 ]
    }
  },
  "4": {
    "class_type": "CLIPTextEncode",
    "_meta": { "title": "Negative prompt" },
    "inputs": {
      "text": "lowres, blurry, deformed, extra limbs, watermark, signature, jpeg artifacts",
      "clip": [ "6", 0 ]
    }
  },
  "5": {
    "class_type": "UNETLoader",
    "_meta": { "title": "Qwen Image 2.1 (int8)" },
    "inputs": { "unet_name": "qwen_image_2.1_int8_convrot.safetensors", "weight_dtype": "default" }
  },
  "6": {
    "class_type": "CLIPLoader",
    "_meta": { "title": "Qwen3-VL 8B text encoder (w4a8)" },
    "inputs": { "clip_name": "qwen3vl_8b_w4a8.safetensors", "type": "qwen_image" }
  },
  "7": {
    "class_type": "VAELoader",
    "_meta": { "title": "VAE" },
    "inputs": { "vae_name": "qwen_image_2.1_vae_bf16.safetensors" }
  },
  "8": {
    "class_type": "EmptySD3LatentImage",
    "_meta": { "title": "Empty Latent 1024x1024" },
    "inputs": { "width": 1024, "height": 1024, "batch_size": 1 }
  },
  "9": {
    "class_type": "KSampler",
    "_meta": { "title": "KSampler" },
    "inputs": {
      "model": [ "5", 0 ],
      "seed": 1930,
      "steps": 20,
      "cfg": 1.0,
      "sampler_name": "euler",
      "scheduler": "simple",
      "positive": [ "3", 0 ],
      "negative": [ "4", 0 ],
      "latent_image": [ "8", 0 ],
      "denoise": 1.0
    }
  },
  "10": {
    "class_type": "VAEDecode",
    "_meta": { "title": "VAE Decode" },
    "inputs": { "samples": [ "9", 0 ], "vae": [ "7", 0 ] }
  },
  "11": {
    "class_type": "SaveImage",
    "_meta": { "title": "Save Image (art)" },
    "inputs": { "filename_prefix": "qwen21-art", "images": [ "10", 0 ] }
  }
 }
@@ -0,0 +1,64 @@
 {
  "3": {
    "class_type": "CLIPTextEncode",
    "_meta": { "title": "Positive prompt (legible text)" },
    "inputs": {
      "text": "Clean minimalist movie-poster design, dark charcoal background with a subtle vintage grain. Centered large bold white sans-serif text reading \"SOLIDARITY OF THE SOVIET COMPUTER SYSTEMS\", perfectly legible, crisp letterforms, even letter spacing. Below it, a smaller line of yellow text reading \"DANKCLAW DANKSERVER EST 2026\", equally legible. Flat graphic design, high contrast, generous margins, no other words, no watermark",
      "clip": [ "6", 0 ]
    }
  },
  "4": {
    "class_type": "CLIPTextEncode",
    "_meta": { "title": "Negative prompt" },
    "inputs": {
      "text": "blurry, illegible, misspelled, warped letters, extra text, watermark, signature, lowres, jpeg artifacts",
      "clip": [ "6", 0 ]
    }
  },
  "5": {
    "class_type": "UNETLoader",
    "_meta": { "title": "Qwen Image 2.1 (int8)" },
    "inputs": { "unet_name": "qwen_image_2.1_int8_convrot.safetensors", "weight_dtype": "default" }
  },
  "6": {
    "class_type": "CLIPLoader",
    "_meta": { "title": "Qwen3-VL 8B text encoder (w4a8)" },
    "inputs": { "clip_name": "qwen3vl_8b_w4a8.safetensors", "type": "qwen_image" }
  },
  "7": {
    "class_type": "VAELoader",
    "_meta": { "title": "VAE" },
    "inputs": { "vae_name": "qwen_image_2.1_vae_bf16.safetensors" }
  },
  "8": {
    "class_type": "EmptySD3LatentImage",
    "_meta": { "title": "Empty Latent 832x1216 poster" },
    "inputs": { "width": 832, "height": 1216, "batch_size": 1 }
  },
  "9": {
    "class_type": "KSampler",
    "_meta": { "title": "KSampler" },
    "inputs": {
      "model": [ "5", 0 ],
      "seed": 2026,
      "steps": 25,
      "cfg": 1.0,
      "sampler_name": "euler",
      "scheduler": "simple",
      "positive": [ "3", 0 ],
      "negative": [ "4", 0 ],
      "latent_image": [ "8", 0 ],
      "denoise": 1.0
    }
  },
  "10": {
    "class_type": "VAEDecode",
    "_meta": { "title": "VAE Decode" },
    "inputs": { "samples": [ "9", 0 ], "vae": [ "7", 0 ] }
  },
  "11": {
    "class_type": "SaveImage",
    "_meta": { "title": "Save Image (text poster)" },
    "inputs": { "filename_prefix": "qwen21-text", "images": [ "10", 0 ] }
  }
 }
@@ -1,32 +1,36 @@
 #!/usr/bin/env bash
 # Download the frank-small payload for llama.cpp on the RTX 3090:
-# ornith-ai/Ornith-1.5-9B-GGUF:Q8_0 plus its vision projector
+# ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF:IQ3_S **with the MTP head in the
-# (the GGUF carries no `v.*` tensors — the tower lives only in the mmproj, so
+# file** (the -mtp variant), plus a vision projector. No Swift repo ships a
-# without this file the endpoint is text-only by construction). Public repo —
+# projector — ukisai's release is language-model-only ("A Swift 27B vision
-# no auth header, and never a token on the command line (it is world-readable
+# projector has not been verified for this release") — so this script fetches
-# in /proc/<pid>/cmdline).
+# unsloth's mmproj-F16.gguf built from the shared base Qwen/Qwen3.8-27B.
 # That substitution is proven, not assumed: on 2026-10-05 Swift-1.5's vision
 # stack was hash-compared against the base over HTTP range reads — 333/333
 # visual tensor names and 6/6 sampled tensors byte-identical (merger fc1/fc2,
 # blocks 0/13/26) — so the base-model projector IS the model's own projector.
 # Public repos — no auth header, and never a token on the command line (it is
 # world-readable in /proc/<pid>/cmdline).
 #
 # Fresh downloads, no resume (`curl -C -` once produced a 213 MB oversized file
-# on a 31 GiB payload); -f/--fail-with-body so an HTTP 401 is not a success,
+# on a 31 GiB payload); -f/--fail-with-body so an HTTP error is not a success,
 # then byte count and the Hub LFS sha256. Idempotent: complete files are
-# skipped.
+# skipped. The projector is stored under a self-documenting dest name; the
 # URL name and sha stay pinned in the table below.
 #
-# 2026-10-05 (Frank): the rung is Ornith-1.5-9B Q8_0. The payload moved off
+# 2026-10-05 (Frank, evening): the payload moved to Swift-1.5-27B GSQ-RCO
-# MiMo-V2.6 (Q6_K) to Ornith because it is the fleet's own model and it is the
+# IQ3_S-mtp — a 27B on the 24 GiB card: ONE 131,072-token slot, q8_0 K + q8_0
-# only one the harness's own prompt recipes were written against; Q8_0 was the
+# V, the in-file MTP head drafting 3. This supersedes the Ornith-1.5-9B Q8_0
-# rung chosen so the quant does not cost the throughput this endpoint already
+# swap (commits 77decb5..380aafa, same day), which was committed but never
-# runs at. The Q5_K_M file this script served from 2026-10-02 stays on disk at
+# deployed — the live service was still MiMo-V2.6 Q6_K when Swift cut over.
-# /home/dank/models/MiMo-Q5KM/ (6876124704 B, LFS oid
+# Rollback payloads stay on disk: /home/dank/models/Ornith-Q8/ and
-# 04cf46f2b5584ef922697d6c063f94879434fb86c467f1a09e52daf0cddaf35f) as the
+# /home/dank/models/MiMo-Q6K/ (the payload that was live until this swap).
-# rollback target; re-fetch it from the same repo with the same recipe.
+# Their re-fetch recipes are in git history (predecessors of this script).
 #
 # Usage:
 #   scripts/download-frank-small.sh [DEST_DIR] [--verify-only]
 set -euo pipefail
 REPO="ornith-ai/Ornith-1.5-9B-GGUF"
 BASE="https://huggingface.co/${REPO}/resolve/main"
 DEST=""
 VERIFY_ONLY=0
 for arg in "$@"; do
@@ -35,19 +39,20 @@ for arg in "$@"; do
        *) DEST="$arg" ;;
    esac
 done
-DEST="${DEST:-/home/dank/models/Ornith-Q8}"
+DEST="${DEST:-/home/dank/models/Swift15-IQ3S}"
-# name bytes sha256 (the sha256 is the LFS oid in the repo's pointer file)
+# repo | hub-name | dest-name | bytes | sha256 (the Hub LFS oid)
 FILES=(
-  "Ornith-1.5-9B-Q8_0.gguf 9786060384 22086870b009dbe9815ee752c48a82de930118a7c5ce5599590892ae03b8b010"
+  "ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF|Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf|Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf|12120016896|9aecf1cd41b2cb2f32a74e0d889e33855ebef43b26f43b43feb5720239e677e5"
-  "mmproj-Ornith-1.5-9B-BF16.gguf 921704672 626f9f90627402a6bf4a999111d0fbd69b5fcca7aa8ba089d69e5f10e8858e1d"
+  "unsloth/Qwen3.8-27B-GGUF|mmproj-F16.gguf|mmproj-Qwen3.8-27B-F16.gguf|927607488|cbb841a9ee0636b2ec172f5bb8df2ea8dfeb01e90fe7c6126581d662a0b4e43e"
 )
 mkdir -p "$DEST"
 status=0
 for entry in "${FILES[@]}"; do
-  read -r name size sha <<<"$entry"
+  IFS='|' read -r repo hubname destname size sha <<<"$entry"
-  f="$DEST/$name"
+  url="https://huggingface.co/${repo}/resolve/main/${hubname}"
  f="$DEST/$destname"
  have=0
  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
    if sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
@@ -55,29 +60,29 @@ for entry in "${FILES[@]}"; do
    fi
  fi
  if (( have )); then
-    echo "OK        $name"
+    echo "OK        $destname"
    continue
  fi
  if (( VERIFY_ONLY )); then
-    echo "MISSING   $name"
+    echo "MISSING   $destname"
    status=1
    continue
  fi
-  echo "FETCHING  $name"
+  echo "FETCHING  $destname (from $repo)"
  tmp="$f.part"
-  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  curl -fsSL --fail-with-body -o "$tmp" "$url"
  mv "$tmp" "$f"
  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
-    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    echo "SIZE FAIL $destname ($(stat -c%s "$f") != $size)"
    status=1
    continue
  fi
  if ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
-    echo "SHA FAIL  $name"
+    echo "SHA FAIL  $destname"
    status=1
    continue
  fi
-  echo "VERIFIED  $name"
+  echo "VERIFIED  $destname"
 done
 if (( status == 0 )); then
  echo "DOWNLOAD_DONE $DEST"
@@ -1,79 +1,64 @@
 # frank-small — llama.cpp (master, CUDA sm_86) serving
-# ornith-ai/Ornith-1.5-9B-GGUF:Q8_0 on the RTX 3090, :7771.
+# ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF:IQ3_S **-mtp** on the RTX 3090,
 # :7771.
 #
-# 2026-10-05 (Frank): the payload moved off bartowski/MiMo-V2.6 (Q6_K) to
+# 2026-10-05 (Frank, evening): the payload moved to Swift-1.5 Qwen3.8-27B
-# ornith-ai/Ornith-1.5-9B Q8_0 (9,786,060,384 B, LFS oid 22086870...).
+# GSQ-RCO IQ3_S WITH the MTP head in the file, each conversation capped at
-# Header-verified facts that re-price the shape: qwen35 hybrid like MiMo, but
+# 131,072 tokens, KV at FP8 class, vision on. Header-verified facts that
-# 33 blocks (not 32) with full_attention_interval 4 => NINE full-attention
+# re-price the shape (read from the Hub file 2026-10-05, HTTP range + struct
-# layers (MiMo had 8), 16 Q heads / 4 KV heads, k/v length 256, native window
+# parse): qwen35 hybrid like MiMo/Ornith but bigger — block_count 65 (64 real
-# 262144, dense (no expert keys), nextn_predict_layers=1 (an MTP head is IN
+# layers + the blk.64 MTP head), full_attention_interval 4 => SIXTEEN
-# the file — capability only; no spec decode ships, same law as before).
+# full-attention layers (MiMo had 8, Ornith 9), 24 Q heads / 4 KV heads,
 # k/v length 256, native window 262144, dense, nextn_predict_layers=1 with
 # blk.64.nextn.{enorm,hnorm,shared_head_norm} (the -mtp file's head tensors
 # are Q6_K/F32 per the publisher RECIPE, "MTP_OK" verified).
 #
-# THE KV CONTRACT STAYS f16 K + f16 V — NO KV QUANTIZATION (Frank, 2026-10-05,
+# THE KV CONTRACT IS NOW q8_0 K + q8_0 V — Frank's order for THIS swap
-# overruling a written q8_0-V proposal). The arithmetic that makes this hard:
+# ("k/v at FP8", 2026-10-05), superseding the same-day f16/f16 rule, which
-#   KV rate 9 attn layers x 4 heads x 256 x 2 B = 18 KiB/token K, same again
+# was written for the Ornith 9B fit ("no further quantization anywhere on
-#   for V at f16 => 36 KiB/token (MiMo was ~24.5 measured: one fewer
+# this endpoint"). llama.cpp has no true fp8 KV dtype; q8_0 is the 8-bit
-#   full-attn layer AND one fewer KV head).
+# class (1.0625 B/elem => 8.5 effective bits) and the only FP8-class lever
-#   Weights +1,900 MiB (9,334 vs Q6_K's 7,432). Measured free at the Q6_K
+# this runtime moves. Arithmetic that makes the order load-bearing:
-#   4096/2048 shape was 2,279 MiB; Q8_0 at f16/f16 does not fit that shape —
+#   KV rate = 16 attn layers x 4 heads x 256 x 1.0625 B, K and V again
-#   the deficit is made up the way Frank ordered: UBATCH FIRST (2048 -> 1024,
+#   => 34 KiB/token (f16 would be 64 KiB/token — halogen's 27B card quotes
-#   the measured 616 MiB rung), then POOL STEPS, then measure. Guess-and-check
+#   the same f16 number; at f16 the 131,072 slot alone would cost 8.0 GiB).
 #   on the live card is the accepted method for llama.cpp sizing here: boot,
 #   read nvidia-smi free, step the pool (12,288-token increments, ~432 MiB
 #   each at the 36 KiB/token rate) until the 2-3 GiB floor holds. Compute
 #   buffers allocate at graph capture during BOOT, so a failure is loud and
 #   fast, not a surprise at the first long prompt.
 #
-# Previous shape history (MiMo era, for the rollback trail): 2026-10-03 the
+# Why ONE slot at 131,072 is the maximum (Frank's cap per conversation):
-# rung went Q5_K_M -> Q6_K with the pool HELD at 401,408; the freed third
+#   card 24,576 MiB. Weights 11,561 MiB (12,120,016,896 B, the IQ3_S-mtp
-# slot funded exact f16 V first; the ~3.5 GiB still free funded the +875 MiB
+#   file; the GSQ "IQ3_S" tier is a per-tensor ALLOCATION mix — histogram
-# of weights.
+#   IQ2_XS 144 / IQ3_XXS 96 / Q6_K 78 / F32 360 / BF16 96 / Q3_K_M 39 ...
 #   all stock types, stock llama.cpp loads it). Projector ~1,350 MiB
 #   (927,607,488 B F16 file at the measured ~1.45x card law). Compute
 #   buffers at ubatch 1024 ~1.6 GiB. CUDA ctx ~500 MiB. DeltaNet state
 #   ~150 MiB/slot (48 linear layers x 48 value heads x 128x128 x 4 B).
 #   MTP draft context ~0.3-0.5 GiB (one nextn layer over the slot window;
 #   the boot ledger is the truth). Non-KV ~= 15.5 GiB. One q8_0 slot =
 #   4,352 MiB => free ~= 4.5 GiB. A SECOND slot wants +4,352 MiB => free
 #   ~0.2 GiB, far below the 2-3 GiB floor. "As many 128k contexts as fit"
 #   = 1. MEASURE nvidia-smi free at boot (quote the free field): under
 #   ~1.5 GiB or a boot failure, step the pool down in 12,288-token
 #   increments (~410 MiB each at this rate). Compute buffers allocate at
 #   graph capture during BOOT, so a failure is loud and fast.
 #
-# 2026-10-03 (Frank): TWO streams instead of three, each one WIDE — 200,704
+# Payload history (for the rollback trail): the Ornith-1.5-9B Q8_0 swap of
-# tokens per conversation, above the 200,700 the harness compacts at — and the
+# earlier on 2026-10-05 (commits 77decb5..380aafa) was committed but never
-# KV cache is now FULLY UNQUANTIZED (f16 K, f16 V). The two moves are the same
+# deployed — the live service was still MiMo-V2.6 Q6_K (2 x 200,704, f16
-# move: llama.cpp divides --ctx-size across --parallel slots, so the slot given
+# KV) when Swift cut over. Rollback payloads on disk: Ornith-Q8/ and
-# up is exactly the VRAM that pays for unquantized V. No further quantization
+# MiMo-Q6K/ (one MODEL_FILE + shape edit back; see the runbook).
 # anywhere on this endpoint.
 #
 # Three measurements on this card, this runtime, price the terms:
 #   2 x 262,144, no projector: 20,190 MiB used / 4,386 free
 #     KV = 524,288 tok x 24.5 KiB = 12,800 MiB => non-KV (weights 6,557 +
 #     DeltaNet state ~25/slot + compute buffers + CUDA graphs) = 7,390 MiB
 #   3 x 196,608, projector on: 22,972 MiB used / 1,153 free
 #     KV = 14,113 MiB => non-KV WITH the vision tower = 8,859 MiB
 #   3 x 180,224, projector on: 21,684 MiB used / 2,441 free (the 2026-10-02
 #     shape) => non-KV = 8,748 MiB
 # Two slots give back one DeltaNet state (~25 MiB). Priced at f16 K+V
 # (32 KiB/token) this pool is 12,544 MiB, so the prediction is
 # 12,544 + ~8,800 = ~20.9 GiB used, ~3.1 GiB free — the band the 2026-10-02
 # shape hit, bought with precision instead of a slot. The 1024 ubatch
 # re-prices the compute buffers upward, so MEASURE after any change: if the
 # boot lands under 2 GiB free, step the pool down in 12,288-token
 # (384 MiB at this rate) increments.
 #
 # KV arithmetic: attention KV exists only in the 8 full-attention layers
 # (4 KV heads x 256 head_dim x 2 B) = 16 KiB/token for K and the same again for
 # V => 32 KiB/token, so 12,288 tokens (= 6144/slot) is one 384 MiB step — that
 # is the ladder to walk if this moves again. Two native windows (2 x 262,144)
 # are NOT reachable with full-precision V: 524,288 x 32 KiB = 16.1 GiB of KV
 # against a card with ~15.7 GiB of KV room once weights, the vision tower, the
 # DeltaNet states and the graphs are loaded. 200,704 per stream is what full
 # precision buys; q8_0 on V would buy 262,144 instead.
 #
 # The asymmetric-KV contract this endpoint was created for (Frank, 2026-10-01
 # night: f16 K + q8_0 V) is now spent — it existed to make a third slot fit,
 # and the third slot is gone. The flags stay explicit so the choice is visible
 # in the argv either way.
 LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server
-MODEL_FILE=/home/dank/models/Ornith-Q8/Ornith-1.5-9B-Q8_0.gguf
+MODEL_FILE=/home/dank/models/Swift15-IQ3S/Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf
-# Vision: the Ornith Q8_0 GGUF ships no `v.*` tensors either (dense-9B byte
+# Vision: no Swift repo ships a projector (ukisai's release is
-# arithmetic: 9.79 GB ≈ 9B params x ~1.06 B/param leaves no room for a tower),
+# language-model-only by their own README), so the projector comes from the
-# so the projector file IS the vision capability (mmproj-Ornith-1.5-9B-BF16
+# SHARED BASE — unsloth's mmproj-F16.gguf built from Qwen/Qwen3.8-27B,
-# .gguf, 921,704,672 B, LFS oid 626f9f90..., fetched by
+# stored as mmproj-Qwen3.8-27B-F16.gguf (927,607,488 B, LFS sha256
-# scripts/download-frank-small.sh). Empty string = text-only on purpose.
+# cbb841a9..., fetched by scripts/download-frank-small.sh). The substitution
-MMPROJ_FILE=/home/dank/models/Ornith-Q8/mmproj-Ornith-1.5-9B-BF16.gguf
+# is proven, not assumed: Swift-1.5's vision stack (tower + merger) was
 # hash-compared against the base on 2026-10-05 — 333/333 visual tensor names
 # and 6/6 sampled tensors byte-identical — so the base projector IS the
 # model's own projector (Swift's RL/OPD post-training touched text only).
 # Empty string = text-only on purpose.
 MMPROJ_FILE=/home/dank/models/Swift15-IQ3S/mmproj-Qwen3.8-27B-F16.gguf
 # Fleet endpoint: same port and alias as every frank-small before it, so the
 # harness models.yml id and the homepage probe do not move.
@@ -81,56 +66,38 @@ ALIAS=frank-small
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7771
-# ONE conversation at the NATIVE 262,144 window (Frank, 2026-10-05). The
+# ONE conversation at Frank's 128k cap: --ctx-size is the POOL llama.cpp
-# two-slot shape died under Ornith and the NO-QUANTIZE law, by arithmetic:
+# divides across slots; 131072 / 1 = 131,072 per conversation. The cap is a
-# the KV rate is the ARCHITECTURE's, not the quant's — 9 full-attn layers x
+# product decision (Swift's own benchmark protocol ran at 131,072), and a
-# 4 KV heads x 256 dim x 2 B = 36 KiB/token at f16 K+V (MiMo measured 24.5),
+# second 131,072 slot does not fit the 2-3 GiB floor (arithmetic above).
-# so 2 x 200,704 wants ~14.1 GiB of KV on top of +1,900 MiB of Q8 weights
+CTX_SIZE=131072
 # against 2,279 MiB measured free; ubatch 1024 claws back 616, and no
 # pool step fits BOTH the 2-3 GiB floor and the harness compaction
 # invariant (slot >= 191,148). One native slot fits with room; the window
 # per conversation RISES 200,704 -> 262,144. The cost: a second concurrent
 # request queues behind the released slot (subagent concurrency 2 -> 1).
 CTX_SIZE=262144
 PARALLEL=1
-# Compute-buffer geometry. This endpoint passed neither flag until 2026-10-03,
+# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves);
-# so it ran on llama.cpp's own defaults (2048/512); the pair is now 4096/2048.
+# on this CUDA 3090 the curve climbs monotonically to 2048, but Frank's
-# Both are tunables because the pair is a measurement, not a law — and the
+# standing law for a resize is LOWER UBATCH FIRST (the measured 616 MiB rung
-# measurement is CARD-SPECIFIC. On the 2x R9700 layer-split sibling the ubatch
+# on the 9B; the 27B's buffers are larger, so the claw-back is worth more).
-# curve peaks at 512 and every step up costs GiB of compute buffer for no
+# The pair is a measurement, not a law: if boot lands well above the floor,
-# speed (see frank-medium-batch-ubatch). On THIS CUDA 3090 the curve climbs
+# ubatch 2048 is the first thing to A/B for prefill (loud failure at graph
-# monotonically to 2048: 710 -> 1188 t/s prefill from ubatch 1024 to 2048 on
+# capture if it does not fit).
 # the 35B MoE sweep, 1944 -> 2423 t/s on the Ornith 8k sweep, with decode flat
 # to under 1 t/s on every rung. Ubatch is a prefill knob; decode never moves.
 #
 # The logical batch is free — two independent sweeps agree that at a fixed
 # ubatch, 2048/4096/8192 give identical prefill speed AND identical peak VRAM,
 # because the compute buffers are sized by the ubatch alone. Doubling it costs
 # nothing and buys nothing; it is here because it is the value the pair was
 # measured at.
 #
 # THE RISK IS ALL IN THE UBATCH. The 1024 -> 2048 step cost ~1,244 MiB of peak
 # graph VRAM on the 35B MoE (7,330 -> 8,574 MiB); this 9B hybrid has far
 # smaller compute tensors, so the delta should be well under that, but the
 # predecessor Q5_K_M shape recorded "ubatch 2048 failed the scratch boot" at
 # ~3,571 MiB free. The Q8_0 rung ships UBATCH 1024 (Frank, 2026-10-05: lower
 # ubatch FIRST — the measured 616 MiB step is the first VRAM the Q8 weights
 # claw back, before any pool move). Compute buffers are allocated at graph
 # capture during BOOT, so a failure is loud and fast, not a surprise at the
 # first long prompt. MEASURE nvidia-smi free immediately after the deploy:
 # under ~1.5 GiB, or a boot failure, step the POOL down next (12,288-token
 # increments, ~432 MiB each at the Ornith 36 KiB/token f16 KV rate).
 BATCH_SIZE=4096
 UBATCH_SIZE=1024
-# The KV contract (2026-10-05, Frank): f16 K + f16 V — NO KV quantization.
+# The KV contract for this swap (Frank, 2026-10-05): q8_0 K + q8_0 V —
-# The q8_0-V arithmetic was written and OVERRULED. The Q8 weights eat the
+# llama.cpp's FP8 class. The flags stay explicit so the choice is visible in
-# VRAM that funded f16 V on 2026-10-03, so the remedy ladder is UBATCH FIRST
+# the argv either way, and --flash-attn stays pinned on in the serve script:
-# (2048 -> 1024, the measured 616 MiB rung), then POOL STEPS — and
+# it is required for quantized V (the build carries
-# guess-and-check on the live card is the accepted method: boot, read
+# GGML_CUDA_FA_ALL_QUANTS=ON since 2026-09-17, so q8_0 KV is a real path,
-# nvidia-smi, step down until the floor holds. --flash-attn stays pinned on
+# not a silent f16 fallback).
-# in the serve script: it is the fast path for f16 KV.
+CACHE_TYPE_K=q8_0
-CACHE_TYPE_K=f16
+CACHE_TYPE_V=q8_0
-CACHE_TYPE_V=f16
+
 # Speculative decoding: the MTP head is IN the -mtp file (blk.64.nextn.*,
 # nextn_predict_layers=1), so draft-mtp needs no second model and no second
 # KV pool — the head shares the main model's weights. Depth 3 matches the
 # fleet's radiance MTP depth; measure draft acceptance from the response
 # timings object and retune here if the head earns more.
 SPEC_TYPE=draft-mtp
 SPEC_DRAFT_N_MAX=3
 NGL=99
@@ -1,20 +1,23 @@
 #!/usr/bin/env bash
-# Serve ornith-ai/Ornith-1.5-9B-GGUF (Q8_0) as frank-small on
+# Serve ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF (IQ3_S **-mtp**) as
-# Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
+# frank-small on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA
 # sm_86).
 #
-# 2026-10-01 night (Frank): swapped from SGLang + INT8-W8A16-AutoRound
+# 2026-10-05 evening (Frank): a 27B on the 24 GiB card — ONE 131,072-token
-# safetensors to the llama.cpp GGUF path, because llama.cpp is the only
+# conversation (Frank's cap), KV at FP8 class (q8_0 both halves), the MTP
-# runtime on this host that moves K and V independently — the asymmetric
+# head that lives IN the -mtp file drafting 3, vision on. The q8_0 KV
-# f16-K + q8_0-V contract that endpoint was built for. 2026-10-03 (Frank):
+# contract supersedes the same-day f16/f16 rule, which was written for the
-# the pair is now f16/f16 — two wide slots freed the VRAM, so the values are
+# Ornith 9B fit; the per-tensor flags stay explicit either way.
 # unquantized too. The per-tensor flags stay explicit either way.
 #
-# Architecture: qwen3_5 hybrid — 32 layers, 8 full-attention + 24 Gated
+# Architecture: qwen35 hybrid — 64 real layers, 16 full-attention + 48
-# DeltaNet, native 262144 window. KV lives only in the 8 attention layers;
+# Gated DeltaNet (full_attention_interval 4), native 262144 window, plus
-# the DeltaNet state is per-slot fixed size. Vision comes from the repo's
+# the blk.64 MTP head (nextn_predict_layers=1). KV lives only in the 16
-# separate projector (the Q6_K GGUF carries no `v.*` tensors at all), so
+# attention layers (34 KiB/token at q8_0); the DeltaNet state is per-slot
-# --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is the documented
+# fixed size. Vision comes from the projector file (the GGUF carries no
-# text-only opt-out.
+# vision tensors; unsloth's mmproj built from the shared Qwen3.8-27B base —
 # proven byte-identical to Swift's own tower, see download-frank-small.sh),
 # so --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is the
 # documented text-only opt-out.
 #
 # Sampling: llama.cpp takes it per request (the fleet harness sends its
 # recipe from models.yml); greedy is the harness default.
@@ -33,31 +36,37 @@ fi
 # --- defaults (used when the env file does not set them) -------------------
 : "${LLAMA_BIN:=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
-: "${MODEL_FILE:=/home/dank/models/Ornith-Q8/Ornith-1.5-9B-Q8_0.gguf}"
+: "${MODEL_FILE:=/home/dank/models/Swift15-IQ3S/Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf}"
 : "${ALIAS:=frank-small}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7771}"
-# ctx-size is the POOL llama.cpp divides across slots: 401408 / 2 gives
+# ctx-size is the POOL llama.cpp divides across slots: 131072 / 1 gives
-# 200,704 per slot — two independent conversations, each one wide enough to
+# 131,072 per conversation — Frank's 128k cap. A second 131,072 slot does
-# hold a whole frank-primary-sized prompt in a single shot (the native 262,144
+# not fit the 2-3 GiB floor on the 24 GiB card with the 27B payload
-# no longer fits twice on a 24 GiB card once the vision tower is loaded;
+# (frank-small.env carries the arithmetic); "as many 128k contexts as fit"
-# frank-small.env carries the three measurements that set this number).
+# = 1.
-: "${CTX_SIZE:=401408}"
+: "${CTX_SIZE:=131072}"
-: "${PARALLEL:=2}"
+: "${PARALLEL:=1}"
-# Compute-buffer geometry. Before 2026-10-03 this endpoint never passed either
+# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves),
-# flag, so it ran on llama.cpp's own defaults (2048/512). The pair is 4096/2048
+# and Frank's standing law for a resize is LOWER UBATCH FIRST; the shipped
-# since 2026-10-05: on this CUDA card the prefill curve climbs all the way to
+# pair is 4096/1024. If boot lands well above the 2-3 GiB floor, ubatch
-# ubatch 2048 (unlike the R9700 layer-split sibling, where 512 is the peak),
+# 2048 is the first A/B for prefill (a miss fails loud at graph capture).
-# and the logical batch is VRAM- and throughput-neutral. frank-small.env
+# The logical batch is VRAM- and throughput-neutral at a fixed ubatch.
 # carries the sweep tables and the VRAM risk note.
 : "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=2048}"
+: "${UBATCH_SIZE:=1024}"
-: "${CACHE_TYPE_K:=f16}"
+# KV at FP8 class = q8_0 on llama.cpp (1.0625 B/elem => 8.5 effective bits;
-: "${CACHE_TYPE_V:=f16}"
+# there is no true fp8 KV dtype). Explicit either way so the contract is
 # visible in the argv.
 : "${CACHE_TYPE_K:=q8_0}"
 : "${CACHE_TYPE_V:=q8_0}"
 # Speculative decoding: draft-mtp loads the MTP head that is IN the -mtp
 # GGUF; the flags move as a set.
 : "${SPEC_TYPE:=draft-mtp}"
 : "${SPEC_DRAFT_N_MAX:=3}"
 : "${NGL:=99}"
 # Vision projector. No colon on purpose: an explicitly empty value means
 # "text-only, on purpose" and must NOT be replaced by the default path.
-: "${MMPROJ_FILE=/home/dank/models/Ornith-Q8/mmproj-Ornith-1.5-9B-BF16.gguf}"
+: "${MMPROJ_FILE=/home/dank/models/Swift15-IQ3S/mmproj-Qwen3.8-27B-F16.gguf}"
 # Fail loudly rather than half-loading: a missing file is a download that
 # has not finished, and Restart=always would turn it into a crash loop that
@@ -71,14 +80,21 @@ if [[ -n "$MMPROJ_FILE" ]]; then
    EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
 fi
 # The MTP head ships IN the -mtp GGUF (blk.64.nextn.*,
 # nextn_predict_layers=1): draft-mtp loads it against the main model's own
 # weights — no --model-draft and no second KV pool. Speculative flags move
 # as a set: a no-spec baseline must strip BOTH of these, not just one.
 EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
 # The UVM device node is created lazily by nvidia-modprobe on the first CUDA
 # open after boot; make sure it exists before the server needs it.
 nvidia-modprobe -u -c=0 2>/dev/null || true
-# Flash attention stays pinned on: it is the fast path for f16 KV, and it is
+# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
-# the only path that could serve quantized V at all — so if CACHE_TYPE_V is
+# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
-# ever set back to q8_0 this flag must not be lost, or the build falls back
+# the q8_0 KV contract in frank-small.env is a real path — without this
-# silently and the KV contract in frank-small.env becomes a lie.
+# flag or that build option the engine silently falls back and the
 # contract becomes a lie.
 ARGS=(
    --model "$MODEL_FILE"
    --alias "$ALIAS"
@@ -1,7 +1,7 @@
 ---
 type: reference
-title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6 Q6_K, 2 x 200704, f16 KV"
+title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6 Q6_K, 2 x 200704, f16 KV (RETIRED)"
-description: "The active frank-small runbook — llama.cpp master (CUDA sm_86) serving bartowski's MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF on the RTX 3090: two conversations of 200,704 tokens with fully unquantized f16 KV, vision through the bundled projector, and the boot-pair arithmetic that prices every pool change."
+description: "The RETIRED frank-small runbook (2026-10-03 → 10-05 evening, rollback target) — llama.cpp master (CUDA sm_86) serving bartowski's MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF on the RTX 3090: two conversations of 200,704 tokens with fully unquantized f16 KV, vision through the bundled projector, and the boot-pair arithmetic that prices every pool change. Superseded by the Swift-1.5-27B IQ3_S-mtp swap."
 tags:
  - llm
  - frank-small
@@ -14,13 +14,15 @@ timestamp: 2026-10-03
 # frank-small — llama.cpp MiMo Q6_K, 2 × 200,704, f16 KV
-> **ACTIVE since 2026-10-03.** The third shape Frank bought in two days on
+> **RETIRED 2026-10-05 (evening).** Superseded by
-> this card: the freed third slot paid for **exact KV values**
+> [[frank-small-llama-swift15-iq3s-mtp-131072]] — the Swift-1.5-27B GSQ-RCO
-> ([[frank-small-llama-mimo-q5km]] documents that step and remains the
+> IQ3_S-mtp swap. Two facts for the trail: (1) the Ornith-1.5-9B Q8_0 swap
-> rollback), and the ~3.5 GiB still free then paid for the **Q6_K quant rung**
+> committed earlier the same day (77decb5..380aafa) was **never deployed** —
-> — the same file this endpoint served on 2026-10-01, come back as an
+> THIS MiMo shape was the live payload when Swift cut over; (2) the KV
-> upgrade rather than a compromise. The pool held at 401,408 (2 × 200,704);
+> contract Frank set here (f16 K + f16 V, "no further quantization on this
-> only the weights moved up.
+> endpoint") was superseded the same evening by the Swift swap's explicit
 > q8_0/q8_0 order. This page stays as the rollback and the measured
 > boot-pair record. Payload on disk: `/home/dank/models/MiMo-Q6K/`.
 ## Identity
@@ -0,0 +1,147 @@
 ---
 type: reference
 title: "frank-small on Dankclaw — llama.cpp, Swift-1.5-27B GSQ-RCO IQ3_S-mtp, 1 x 131072, q8_0 KV"
 description: "The active frank-small runbook (2026-10-05 evening) — llama.cpp master (CUDA sm_86) serving ukisai's Swift-1.5-Qwen3.8-27B GSQ-RCO IQ3_S WITH the in-file MTP head on the RTX 3090: one 131,072-token conversation at Frank's 128k cap, q8_0 K + q8_0 V (llama.cpp's FP8 class), vision through the base-model projector (proven byte-identical tower), and the arithmetic that makes one slot the maximum."
 tags:
  - llm
  - frank-small
  - dankclaw
  - llama-cpp
  - gguf
  - rtx3090
  - qwen35
  - mtp
  - vision
 timestamp: 2026-10-05
 ---
 # frank-small — llama.cpp Swift-1.5-27B IQ3_S-mtp, 1 × 131,072, q8_0 KV
 > **ACTIVE since 2026-10-05 (evening).** A 27B replaces the 9B on the same
 > 24 GiB card, bought by Frank's explicit contract for this swap: context
 > capped at 128k per conversation, KV at FP8 class, vision on, "as many of
 > those contexts as you can" — the answer turned out to be **one**, by
 > arithmetic, not by choice (see *Sizing*). This supersedes BOTH earlier
 > 2026-10-05 payloads: the live MiMo-V2.6 Q6_K shape
 > ([[frank-small-llama-mimo-q6k-2x200704]], now retired) and the Ornith-1.5-9B
 > Q8_0 swap that was committed (77decb5..380aafa) but never deployed.
 ## Identity
 | | |
 |---|---|
 | Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
 | Endpoint | `http://192.168.0.143:7771` — alias `frank-small`, `/v1` OpenAI surface |
 | Runtime | llama.cpp master (0.5.0-dev, commit 5fc4f3c), CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` (`GGML_CUDA_FA_ALL_QUANTS=ON` — required for quantized V) |
 | Unit | root systemd `frank-small.service` ([`scripts/frank-small.service`](../../scripts/frank-small.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
 | Serve script | [`scripts/serve-frank-small.sh`](../../scripts/serve-frank-small.sh), tunables [`scripts/frank-small.env`](../../scripts/frank-small.env) |
 | Weights | `/home/dank/models/Swift15-IQ3S/Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf` — ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF **IQ3_S -mtp**, 12,120,016,896 B, sha256 `9aecf1cd…` (qwen35 hybrid: 64 real layers, 16 full-attention + 48 Gated DeltaNet, native 262,144; **MTP head in the file**: blk.64.nextn.*, `nextn_predict_layers=1`) |
 | Projector | `…/Swift15-IQ3S/mmproj-Qwen3.8-27B-F16.gguf` — unsloth/Qwen3.8-27B-GGUF `mmproj-F16.gguf`, 927,607,488 B, sha256 `cbb841a9…` (27-layer ViT, 1152 hidden, `qwen3vl_merger` → 5120) |
 | Fetch | [`scripts/download-frank-small.sh`](../../scripts/download-frank-small.sh) — `-f`, byte count + LFS sha256 verified per file; both verified on the host before deploy |
 | Seam | `tests/frank_small_unit.bats` (argv assertions against a stub engine: payload path, slot geometry, KV pair, MTP pair, batch pair, projector opt-out) |
 ## Why a base-model projector is legal here
 No Swift repo ships a projector — ukisai's GSQ-RCO release is
 language-model-only by their own README ("A Swift 27B vision projector has
 not been verified for this release"). The substitution is proven, not
 assumed: Swift-1.5 is an RL/OPD post-train of `Qwen/Qwen3.8-27B`, and its
 vision stack was hash-compared against the base on 2026-10-05 over HTTP
 range reads of the two BF16 repos — **333/333 visual tensor names match and
 6/6 sampled tensors are byte-identical** (merger `linear_fc1`/`linear_fc2`,
 blocks 0/13/26 attn/mlp). Swift's post-training touched text only, so the
 base projector IS the model's own projector. The mmproj header matches
 Swift's `vision_config` field-for-field (depth 27, hidden 1152, patch 16,
 merge 2, ffn 4304, 16 heads, out 5120).
 ## Shape and KV contract
 `--ctx-size 131072 --parallel 1` — ONE conversation at Frank's 128k cap
 (the GSQ publisher's own benchmark protocol ran at 131,072). `CTX_SIZE`
 stays divisible by `PARALLEL`; the seam enforces it.
 **KV at FP8 class: `q8_0` K + `q8_0` V (Frank, 2026-10-05 evening).**
 llama.cpp has no true fp8 KV dtype; q8_0 is the 8-bit class (1.0625 B/elem
 ⇒ 8.5 effective bits) and the only FP8-class lever this runtime moves. The
 order supersedes the same-day f16/f16 rule, which was written for the
 Ornith 9B fit — at 64 KiB/token f16 the single 131,072 slot alone would
 cost 8.0 GiB of KV. At q8_0 the rate is **34 KiB/token** (16 attn layers ×
 4 KV heads × 256 dim × 1.0625 B, K and again V). `--flash-attn on` is
 pinned: quantized V requires it, and the build's
 `GGML_CUDA_FA_ALL_QUANTS=ON` (2026-09-17) makes the path real rather than a
 silent f16 fallback.
 **One slot is the maximum, by arithmetic.** Card 24,576 MiB; weights
 11,561 MiB; projector ~1,350 MiB (927,607,488 B at the measured ~1.45× card
 law); compute buffers at ubatch 1024 ~1.6 GiB; CUDA ~500 MiB; DeltaNet
 state ~150 MiB/slot (48 linear layers × 48 value heads × 128×128 × 4 B); MTP
 draft context ~0.3–0.5 GiB. Non-KV ≈ 15.5 GiB. One q8_0 slot = 4,352 MiB ⇒
 free ≈ 4.5 GiB. A second slot wants +4,352 MiB ⇒ ~0.2 GiB free, far under
 the 2–3 GiB floor. The pool is capped by the 128k law, not by the card —
 leftover headroom is honest slack, not an undersized pool.
 ## MTP (speculative decoding)
 The MTP head ships **in the file** — the `-mtp` GGUF carries
 `blk.64.nextn.{enorm,hnorm,shared_head_norm}` + the head layer's tensors at
 Q6_K/F32 (publisher RECIPE: "MTP_OK verified"),
 `qwen35.nextn_predict_layers=1`. `--spec-type draft-mtp` loads it against
 the main model's own weights — no `--model-draft`, no second draft KV pool
 (an MTP head can NEVER load as a separate draft model on this stack; two
 context-backed drafters cannot share a process). Depth 3
 (`--spec-draft-n-max 3`) matches the fleet's radiance MTP depth. The flags
 move as a set: a no-spec baseline must strip BOTH. Acceptance reads from
 the response `timings` object (`draft_n`, `draft_n_accepted`) — measure on
 real traffic, never on a synthetic repeat (greedy-text acceptance lies).
 ## Vision contract
 `--mmproj` is the entire vision path (the GGUF carries no vision tensors).
 Probes: a solid-color PNG **≥ 256 px** answers its exact color — a tiny
 probe undersupplies image tokens and the model invents ("video"); the
 legacy top-level `images` field is IGNORED, only the OpenAI `image_url`
 content part reaches the model. Same traps the MiMo eras documented.
 ## Client contract
 - Reasoning in `reasoning_content` (`--reasoning-format deepseek`); thinking
  tokens COUNT against `max_tokens`.
 - The embedded chat template gates `reasoning_effort` to
  `{xhigh, medium, low}` — anything else is a raise/HTTP 500, so
  `models.yml` `compat.reasoningEffortMap` folds the harness levels onto
  that set (mandatory, same as every Qwen3.8-template endpoint).
 - Sampling rides the wire (`extraBody`): Swift's publisher recipe temp 1.0
  / top_p 0.95 / top_k 20 / min_p 0 (the card was evaluated at exactly
  this recipe at ctx 131,072).
 - Swift 1.5 is a reasoning-efficient post-train: −58.5 % median thinking
  tokens vs base at equal-or-better accuracy. Token budgets go further;
  the 32,768 `maxTokens` is unchanged from the MiMo era and still clears
  the compaction invariant (91,750 + 32,768 ≤ 131,072).
 - The GSQ-RCO "IQ3_S" tier is a **per-tensor allocation mix** (IQ2_XS 144 /
  IQ3_XXS 96 / Q6_K 78 / F32 360 / BF16 96 / Q3_K_M 39 …), all stock quant
  types — stock llama.cpp loads it. Publisher KLD vs Swift BF16: 0.0513
  (development), held-out 0.024–0.052 per domain; the MTP packaging is NOT
  separately evaluated by the publisher.
 ## Rollback
 Two payloads on disk, both fetchable by the same script's git-history
 recipes:
 1. **MiMo-V2.6 Q6_K 2×200,704 f16 KV** — the payload that was LIVE until
   this cutover: `/home/dank/models/MiMo-Q6K/`; revert
   [[frank-small-llama-mimo-q6k-2x200704]] (env trio at commit e5893dc or
   380aafa's parent), one `MODEL_FILE` + shape edit.
 2. **Ornith-1.5-9B Q8_0** — the committed-but-never-deployed swap:
   `/home/dank/models/Ornith-Q8/`; revert to 380aafa's tree (ONE native
   262,144 slot, f16/f16 KV).
 ## Trail
 - The Ornith-1.5-9B Q8_0 swap (77decb5 → 380aafa, earlier 2026-10-05) was
  committed and pushed but its deploy never completed (blocked at the
  whole-tree docs gate); the live service was still MiMo Q6_K when Swift
  cut over. Both Ornith payloads were verified on disk before Swift.
 - Swift header facts (block_count 65, interval 4, KV geometry, nextn) were
  read from the Hub file by HTTP range + struct parse — no download
  needed — before the payload was fetched.
@@ -1,25 +1,24 @@
 # LLM Fleet — Knowledge Base
-Documentation for the LLM inference fleet on the two hosts (2026-10-05):
+**Dankserver runs the radiance engine (StillDeadcode) on frank-primary**
-**Dankserver runs FreeToken (PR #587 ROCm) on frank-primary** (the local
+(digest-pinned `stilldeadcode/radiance` image, engine as entrypoint,
-image `frank-primary-freetoken:pr587-77a7201` built on the host from
+driving `podman run` directly, root systemd unit, :7770): the OFFICIAL
-`scripts/docker/frank-primary-freetoken.Dockerfile` — FreeToken at PR head
+`Qwen/Qwen3.8-Flash-Next-FP8` checkpoint at NATIVE block-FP8 precision
-`77a7201` in the PR's own ROCm 7.14 base image, torch 2.11.0+rocm7.14 —
+(converted no-recipe to `qwen4exp-fp8-native.rad` — see the runbook for
-driving `podman run` directly, serving orcarouter
+the descriptor-staging law), on BOTH R9700s TP=2 (`--tp-wire wht6`),
-/Qwen3.8-Flash-Next-Uncensored-FP8 on ONE R9700, :7770: all experts in
+experts in a 40 GiB/rank pinned host pool + VRAM cache (ratio 0.82), the
-pinned host RAM behind a GPU LRU (`--moe-cache-rate 0.10`), KV pool
+47.7 GiB PLE table pinned to host RAM, fp8 KV, MTP spec decode depth 3,
-`--num-tokens 299968` (multiple-of-64 law of the qsa_sparse backend), 2
+2 × 200,000 first boot. Rollback tags frank-primary-freetoken-pr587-20261005
-decode slots, parsers AUTO (thinking → `reasoning_content` natively;
+and frank-primary-vllm-shapeG-20261003),
-measured 17.1 tok/s decode, TP=2 exists but does not boot on ROCm — see the
+and Dankclaw runs two endpoints — **halogen container (frank-large, port 7770:
 runbook). Rollback tag frank-primary-vllm-shapeG-20261003),
 and Dankclaw
 runs two endpoints — **halogen container (frank-large, port 7770:
 ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
 /halogen-qwen3.8-27b, a 33.4 GiB .hgn checkpoint that loads in no other
 engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
 lives on frank-small)** and
-**llama.cpp CUDA (frank-small, port 7771: MiMo-V2.6-Distill-Qwen-9B Q6_K
+**llama.cpp CUDA (frank-small, port 7771: ukisai Swift-1.5-Qwen3.8-27B
-GGUF, 2 x 200,704, fully unquantized f16 KV, on the RTX 3090)**.
+GSQ-RCO IQ3_S-mtp GGUF — 27B with the MTP head IN the file, ONE 131,072
 conversation at Frank's 128k cap, q8_0 K + q8_0 V (FP8 class), vision via
 the base-model projector, on the RTX 3090)**.
 > 2026-08-22: the former `vllmScripts` repository is merged into the
 > serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -33,12 +32,12 @@ graph LR
    subgraph Dankserver[AMD Dankserver - Ubuntu 26.04]
        A[2x Radeon AI PRO R9700
 gfx1201, 32 GiB each]
-        B2[FreeToken PR587 ROCm, local image
+        B2[radiance StillDeadcode, digest-pinned image
 podman run, frank-primary, port 7770]
-        C[Qwen3.8-Flash-Next FP8
+        C[Qwen3.8-Flash-Next FP8 OFFICIAL, native block-FP8
-1x R9700, experts in pinned host RAM + GPU LRU 10%,
+2x R9700 TP=2 wht6, experts 40 GiB/rank pinned pool + VRAM,
-KV pool 299,968 tokens, 2 slots, parsers AUTO,
+PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 3,
-TP=2 untested on ROCm; second R9700 idle]
+2 x 200000, froggeric v22.5]
    end
    subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
@@ -50,8 +49,8 @@ frank-large, port 7770]
 1 x 200700 ctx, d2 KV, text-only]
        G[llama.cpp master CUDA sm_86
 frank-small, port 7771]
-        H[MiMo-V2.6 Q5_K_M GGUF
+        H[Swift-1.5-27B GSQ-RCO IQ3_S-mtp GGUF
-2 x 200704 ctx, f16 K + f16 V, flash-attn, mmproj vision]
+1 x 131072 ctx, q8_0 K + q8_0 V, draft-mtp depth 3, mmproj vision]
        E2 --> E --> F
        E2 --> G --> H
    end
@@ -81,22 +80,52 @@ frank-small, port 7771]
  No server-side sampler — the thinking recipe rides the wire in models.yml.
  Rollback: revert the swap commit; the ROCmFPX payload + fork build +
  `build-rocmfpx.sh` pin all remain on disk
- [[frank-small-llama-mimo-q6k-2x200704]] — **ACTIVE since 2026-10-03**
+- [[frank-small-llama-swift15-iq3s-mtp-131072]] — **ACTIVE since 2026-10-05
-  (root systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
+  (evening)** (root systemd unit, port 7771): llama.cpp master (CUDA sm_86)
-  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on the RTX 3090, TWO
+  serving ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF **IQ3_S-mtp** (27B, the
-  slots of 200,704 tokens (pool 401,408) with FULLY UNQUANTIZED KV (f16 K +
+  checkpoint's own MTP head IN the file — `--spec-type draft-mtp`, depth 3)
-  f16 V) and the mmproj for vision — 94.4 tok/s decode (the rung costs a
+  on the RTX 3090, ONE 131,072-token conversation at Frank's 128k cap (a
-  measured -10 % vs Q5_K_M: weight-bandwidth-bound), batch 4096 / ubatch 2048
+  second 131,072 slot misses the 2-3 GiB floor) with q8_0 K + q8_0 V (FP8
-  since 2026-10-05 (+4.6 % prefill, measured), 2,279 MiB of the card
+  class, 34 KiB/token vs 64 at f16) and vision via unsloth's
-  left free. Probe-size trap (>=256 px images or the model says "video")
+  mmproj-Qwen3.8-27B-F16.gguf — no Swift projector ships, and the base
-  documented. Rollback: one MODEL_FILE line back to Q5_K_M
+  vision stack was hash-proven byte-identical to Swift's (6/6 tensors).
  The GSQ-RCO "IQ3_S" tier is a per-tensor allocation mix of stock types.
  Replaces BOTH same-day payloads: the live MiMo Q6_K shape (below) and
  the Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but
  never deployed
 - [[frank-small-llama-mimo-q6k-2x200704]] — **RETIRED 2026-10-05 (evening;
  rollback target; payload on disk)**: llama.cpp master (CUDA sm_86)
  serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K**, TWO slots of
  200,704 with FULLY UNQUANTIZED KV (f16 K + f16 V) and the mmproj for
  vision — 94.4 tok/s decode (the rung costs a measured -10 % vs Q5_K_M:
  weight-bandwidth-bound), batch 4096 / ubatch 2048 since 2026-10-05
  (+4.6 % prefill, measured), 2,279 MiB of the card left free. This was
  the LIVE payload when Swift cut over. The ubatch A/B table and the
  probe-size trap (>=256 px images or the model says "video") live inside
 - [[frank-small-llama-mimo-q5km]] — **RETIRED 2026-10-02 → 10-03 (rollback
  target; payload on disk; owns the measured boot-pair table)**: the same
  runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
  104.8 tok/s, 3,571 MiB true free
- [[frank-primary-freetoken-pr587]] — **ACTIVE since 2026-10-05**
+- [[frank-primary-radiance-qwen38-native-fp8]] — **ACTIVE since 2026-10-05**
-  (root systemd unit, port 7770): FreeToken at PR #587 head `77a7201`
+  (root systemd unit, port 7770): the StillDeadcode **radiance** engine
-  (branch rocm714_torch214) built on the host into the local image
+  (C++/HIP, libr4d gfx1201 kernels, digest-pinned 287 MiB image, engine as
  the image entrypoint) serving the OFFICIAL `Qwen/Qwen3.8-Flash-Next-FP8`
  checkpoint at NATIVE precision — converted no-recipe to
  `qwen4exp-fp8-native.rad` (~161 GiB) after dropping `quantization_config`
  in a staging dir (the plugin descriptor-selection law: nothing claims
  `(qwen4exp, "fp8")`, the shipped plugin claims `(qwen4exp, "")`;
  per-weight encodings are data, not the descriptor) — on BOTH R9700s
  **TP=2** (`--tp-wire wht6` fused all-reduce), experts in a 40 GiB/rank
  pinned host pool + VRAM cache (ratio 0.82 = the measured 99 %-hit point),
  the 47.7 GiB PLE n-gram table PINNED to host RAM (`--ngram-placement
  ram`), fp8 KV 7.32 KiB/token, MTP spec decode depth 3, 2 × 200,000 first
  boot, froggeric v22.5 template, vision + tools + `reasoning_content` auto,
  `/metrics` scraped. Rollback: git tag
  `frank-primary-freetoken-pr587-20261005` (FreeToken image + orcarouter
  payload on disk); older rollback `frank-primary-vllm-shapeG-20261003`
 - [[frank-primary-freetoken-pr587]] — **RETIRED 2026-10-05 (same day;
  rollback target)**: FreeToken at PR #587 head `77a7201` (branch
  rocm714_torch214) built on the host into the local image
  `frank-primary-freetoken:pr587-77a7201` (Dockerfile: digest-pinned
  rocm/pytorch rocm7.14 torch-2.11.0 base, libhiprtc dev-symlink law,
  curated --no-deps install per docs/install_amd.md), serving the SAME
@@ -105,8 +134,9 @@ frank-small, port 7771]
  (multiple-of-64 law), 2 slots, parsers AUTO (thinking →
  `reasoning_content`). Measured: boot ~2.3 min, decode 17.1 tok/s
  single-stream, vision "Red" probe passes, native tool_calls; TP=2 does
-  not boot on ROCm (pynccl JIT) — second card idle. Rollback: git tag
+  not boot on ROCm (pynccl JIT) — second card idle. Superseded by the
-  frank-primary-vllm-shapeG-20261003
+  radiance swap for TP=2; the image + payload stay on disk until the
  rollback window closes
 - [[dankserver-qwen38-fp8-expert-offload-tuning]] — **RETIRED 2026-10-05
  (rollback target; the vLLM trio + env lore live in the tag)**: the
  tcclaviger RDNA4
@@ -181,11 +181,20 @@ The LLM inference fleet on the two hosts. Full index:
 ### Deployment and Topology
 - [[deployment]] — unified deploy and update process for both servers
- [[frank-primary-freetoken-pr587]] — the ACTIVE frank-primary runbook
+- [[frank-primary-radiance-qwen38-native-fp8]] — the ACTIVE frank-primary
-  (2026-10-05): FreeToken built at PR #587 head `77a7201` on the host into
+  runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
  287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
  `Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert
  after the descriptor-staging drop) on BOTH R9700s TP=2 `wht6`, experts
  in a 40 GiB/rank pinned host pool, PLE 47.7 GiB pinned to RAM, fp8 KV
  7.32 KiB/token, MTP depth 3, 2 × 200,000 first boot; rollback is the
  FreeToken one-card era (tag `frank-primary-freetoken-pr587-20261005`),
  then the tcclaviger shape-G era (tag `frank-primary-vllm-shapeG-20261003`)
 - [[frank-primary-freetoken-pr587]] — RETIRED 2026-10-05 (superseded the
  same day by the radiance cutover above; kept as the rollback runbook):
  FreeToken built at PR #587 head `77a7201` on the host into
  `frank-primary-freetoken:pr587-77a7201`, ONE R9700 with experts in pinned
-  host RAM behind a 0.10 GPU LRU, 2 slots, parsers AUTO; rollback is the
+  host RAM behind a 0.10 GPU LRU, 2 slots, parsers AUTO
  tcclaviger shape-G era (tag `frank-primary-vllm-shapeG-20261003`)
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
 - [[frank-primary-r9v-radiance-qwen38-q8]] — RETIRED 2026-09-30 (two swaps
  back; rollback target): the R9V radiance-vLLM fork (docker-pinned bundle
@@ -359,14 +368,20 @@ The LLM inference fleet on the two hosts. Full index:
 - [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
 - [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
 - [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
- [[frank-small-llama-mimo-q6k-2x200704]] — the ACTIVE frank-small runbook
+- [[frank-small-llama-swift15-iq3s-mtp-131072]] — the ACTIVE frank-small
-  since 2026-10-03: llama.cpp master (CUDA sm_86) serving
+  runbook since 2026-10-05 (evening): llama.cpp master (CUDA sm_86) serving
-  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on Dankclaw's RTX 3090,
+  ukisai's Swift-1.5-Qwen3.8-27B GSQ-RCO IQ3_S WITH the in-file MTP head on
-  port 7771 — TWO conversations at 200,704 tokens each with fully
+  the RTX 3090 — ONE 131,072-token conversation at Frank's 128k cap, q8_0
-  unquantized f16 KV and mmproj vision; 94.4 tok/s decode (the Q6_K rung
+  K + q8_0 V, vision through the base-model projector; the arithmetic that
-  costs a measured −10 % vs Q5_K_M — weight-bandwidth), batch 4096 / ubatch
+  makes one slot the maximum
-  2048 (+4.6 % prefill, measured 2026-10-05), 2,279 MiB free, and
+- [[frank-small-llama-mimo-q6k-2x200704]] — RETIRED 2026-10-05 (evening,
-  the ≥256 px image-probe trap (tiny images answer "video")
+  superseded by swift15): llama.cpp master serving
  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** — TWO conversations at
  200,704 tokens each with fully unquantized f16 KV and mmproj vision;
  94.4 tok/s decode (the Q6_K rung costs a measured −10 % vs Q5_K_M —
  weight-bandwidth), batch 4096 / ubatch 2048 (+4.6 % prefill, measured
  2026-10-05), 2,279 MiB free, and the ≥256 px image-probe trap (tiny
  images answer "video")
 - [[frank-small-llama-mimo-q5km]] — RETIRED 2026-10-02 → 10-03 (rollback
  target, payload on disk; owns the measured boot-pair table): same runtime
  and shape one rung lower — 104.8 tok/s, 3,571 MiB true free
@@ -1,10 +1,10 @@
 #!/usr/bin/env bats
 # Seam: the frank-small llama.cpp root unit on dankclaw.
-# frank-small is the RTX 3090 endpoint; since 2026-10-05 it serves
+# frank-small is the RTX 3090 endpoint; since 2026-10-05 evening it serves
-# ornith-ai/Ornith-1.5-9B-GGUF:Q8_0 via a CUDA (sm_86) build of
+# ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF:IQ3_S-mtp (27B, MTP head in the
-# llama.cpp master, chosen because it is the only runtime here that moves K and
+# file) via a CUDA (sm_86) build of llama.cpp master: ONE 131,072-token
-# V independently. Since 2026-10-03 it runs TWO unquantized contexts of
+# conversation at Frank's 128k cap, q8_0 K + q8_0 V (FP8 class), the
-# 200,704 tokens each and carries a vision projector, and the card must keep
+# in-file MTP head drafting 3, and a vision projector. The card must keep
 # 2-3 GiB free. This seam runs the serve script against a stub engine and
 # asserts on the argv it actually execs, so what is pinned is the
 # command the operator gets.
@@ -16,9 +16,9 @@ SERVE="$REPO_ROOT/scripts/serve-frank-small.sh"
 ENV_FILE="$REPO_ROOT/scripts/frank-small.env"
 REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
 ENV_LARGE="$REPO_ROOT/scripts/frank-large.env"
-TARGET_STUB="Ornith-1.5-9B-Q8_0.gguf"
+TARGET_STUB="Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf"
 # Same basename as the shipped projector: the test env repoints it into $STAGE.
-MM_STUB="mmproj-Ornith-1.5-9B-BF16.gguf"
+MM_STUB="mmproj-Qwen3.8-27B-F16.gguf"
 setup() {
    STAGE="$(mktemp -d)"
@@ -81,9 +81,9 @@ argv_value() {
    grep -q 'FRANK_SMALL_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-small.env' "$UNIT"
 }
-@test "the shipped env points at the llama.cpp runtime and the Ornith-1.5-9B Q8_0 GGUF" {
+@test "the shipped env points at the llama.cpp runtime and the Swift-1.5-27B IQ3_S-mtp GGUF" {
    grep -qE '^LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server$' "$ENV_FILE"
-    grep -qE "^MODEL_FILE=/home/dank/models/Ornith-Q8/$TARGET_STUB$" "$ENV_FILE"
+    grep -qE "^MODEL_FILE=/home/dank/models/Swift15-IQ3S/$TARGET_STUB$" "$ENV_FILE"
 }
@test "the deployer gates on exactly the payload paths the env file ships" {
@@ -100,7 +100,7 @@ argv_value() {
    done
 }
-@test "the shipped env serves the fleet shape: one native 262144-token context" {
+@test "the shipped env serves the fleet shape: one conversation at the 128k cap" {
    serve_argv
    [ "$status" -eq 0 ]
    [ "$(argv_value --alias)" = "frank-small" ]
@@ -110,54 +110,50 @@ argv_value() {
    [ "$(argv_value --n-gpu-layers)" -gt 0 ]
    # llama.cpp divides --ctx-size across slots and a mamba hybrid cannot
    # share one cache, so the observable contract is the per-slot window.
-    # 2026-10-05 (Frank): NO KV quantization forces ONE slot at the NATIVE
+    # 2026-10-05 evening (Frank): the 27B Swift-1.5 swap caps each
-    # window — Ornith's 9 full-attn layers price f16 KV at 36 KiB/token, and
+    # conversation at 131,072, and ONE slot is the maximum — a second
-    # two 200,704 slots cannot fit the 2-3 GiB floor on 24 GiB with Q8
+    # 131,072 slot does not fit the 2-3 GiB floor (KV budget test below).
    # weights. The window per conversation ROSE 200,704 -> 262,144.
    [ "$(argv_value --parallel)" -eq 1 ]
    ctx="$(argv_value --ctx-size)"
    par="$(argv_value --parallel)"
    [ $((ctx % par)) -eq 0 ]
-    [ $((ctx / par)) -eq 262144 ]
+    [ $((ctx / par)) -eq 131072 ]
 }
-@test "the KV contract is unquantized both halves, flash-attn on" {
+@test "the KV contract is q8_0 both halves (FP8 class), flash-attn on" {
    serve_argv
    [ "$status" -eq 0 ]
-    [ "$(argv_value --cache-type-k)" = "f16" ]
+    # Frank, 2026-10-05 evening: "k/v at FP8" for the Swift-1.5 swap —
-    # Frank, 2026-10-05: NO KV quantization on this endpoint. The Q8_0 rung
+    # llama.cpp's FP8 class is q8_0 (1.0625 B/elem => 8.5 effective bits;
-    # funds itself by UBATCH first (2048 -> 1024), then pool steps — a q8_0-V
+    # there is no true fp8 KV dtype). This SUPERSEDES the same-day f16/f16
-    # proposal was written and overruled. The flag pair is what makes this
+    # rule written for the Ornith 9B fit. The flag pair is what makes this
    # observable — a build without the FA path silently falls back, so both
    # must appear in the argv the supervisor runs.
-    [ "$(argv_value --cache-type-v)" = "f16" ]
+    [ "$(argv_value --cache-type-k)" = "q8_0" ]
    [ "$(argv_value --cache-type-v)" = "q8_0" ]
    [ "$(argv_value --flash-attn)" = "on" ]
 }
-@test "the KV budget leaves 2-3 GiB of the 3090 free" {
+@test "the KV budget fits the 2-3 GiB floor and forbids a second 128k slot" {
-    # Rate, from the Ornith GGUF header (read from the Hub file, 2026-10-05):
+    # Rate, from the Swift-1.5 IQ3_S-mtp GGUF header (read from the Hub
-    # NINE full-attention layers (block_count 33, full_attention_interval 4)
+    # file, 2026-10-05): SIXTEEN full-attention layers (block_count 65 = 64
-    # x 4 kv heads x 256 dim x 2 B = 18 KiB/token for K and the same for V
+    # real layers + the blk.64 MTP head, full_attention_interval 4) x 4 kv
-    # => 36 KiB/token (budgeted 37 with slack; MiMo measured 24.5 — one
+    # heads x 256 dim x 1.0625 B = 17 KiB/token for K and the same for V
-    # fewer layer AND one fewer KV head). The non-KV half: Q8_0 weights
+    # => 34 KiB/token at q8_0 (budgeted 37 with slack; f16 would be 64).
-    # 9,334 MiB (9,786,060,384 B) + vision tower ~1.3 GiB (mmproj 879 MiB at
+    # The non-KV half: IQ3_S-mtp weights 11,561 MiB (12,120,016,896 B) +
-    # the measured ~1.5x cost law) + DeltaNet state + compute buffers +
+    # vision tower ~1.35 GiB (927,607,488 B F16 mmproj at the measured
-    # CUDA graphs => 11 GiB budgeted. Two asserts, both about the output
+    # ~1.45x cost law) + DeltaNet states + compute buffers at ubatch 1024 +
-    # the operator sees: the pool must still fit with 2 GiB spare, and it
+    # CUDA graphs + the MTP draft context => 17 GiB budgeted. Two asserts,
-    # must be large enough that 5 GiB spare is NOT reachable (an undersized
+    # both about the output the operator sees: the pool must still fit with
-    # pool is as wrong as an oversized one).
+    # 2 GiB spare, and a SECOND 131,072 slot must not fit — Frank's "as
    # many 128k contexts as fit" has the answer 1 on this card, and the
    # pool itself is capped by the 128k law, not by the card.
    # Arithmetic, not a boot log: change CTX_SIZE and this bites before the
-    # deploy does. At Ornith's rate the two-200704-slot pool (14.1 GiB KV)
+    # deploy does.
    # FAILS the first assert — which is exactly why the shipped shape is
    # ONE native slot.
    ctx="$(sed -nE 's/^CTX_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")"
    kv_gib=$((ctx * 37 / 1048576))
-    [ $((kv_gib + 11 + 2)) -le 24 ]
+    [ $((kv_gib + 17 + 2)) -le 24 ]
-    [ $((kv_gib + 11 + 5)) -gt 24 ]
+    [ $((2 * kv_gib + 17)) -gt 24 ]
    # And the ceiling that sets the per-slot window: two NATIVE windows cannot
    # be paid for in full precision on this card.
    native=$((2 * 262144))
    [ $((native * 37 / 1048576 + 11)) -gt 24 ]
 }
@test "the compute-buffer pair follows the env file and stays self-consistent" {
@@ -187,16 +183,22 @@ argv_value() {
    argv_value --ubatch-size | grep -qE '^[0-9]+$'
 }
-@test "no speculative draft ships by default" {
+@test "the in-file MTP head ships as draft-mtp at depth 3" {
-    # The qwen35 GGUF has no MTP tensors, and the cross-model DFlash draft
+    # The -mtp GGUF carries the head (blk.64.nextn.*, nextn_predict_layers
-    # is a measured decision the harness makes later, not a default.
+    # =1): draft-mtp loads it against the main model's own weights — no
-    ! grep -q -- '--model-draft' "$SERVE"
+    # --model-draft and no second draft KV pool. Speculative flags move as
-    ! grep -q -- '--spec-type' "$SERVE"
+    # a set; both must appear in the argv the supervisor runs, and the env
-    ! grep -q '^DRAFT' "$ENV_FILE"
+    # must carry the same contract the argv executes.
    serve_argv
    [ "$status" -eq 0 ]
    [ "$(argv_value --spec-type)" = "draft-mtp" ]
    [ "$(argv_value --spec-draft-n-max)" = "3" ]
    grep -qE '^SPEC_TYPE=draft-mtp$' "$ENV_FILE"
    grep -qE '^SPEC_DRAFT_N_MAX=3$' "$ENV_FILE"
 }
@test "vision ships by default and MMPROJ_FILE empty opts out of the image path" {
-    # The Q6_K GGUF carries no `v.*` tensors, so the projector argument IS
+    # The Swift GGUF carries no vision tensors, so the projector argument IS
    # the vision capability: no flag, no eyes.
    serve_argv
    [ "$status" -eq 0 ]