frank-large: Q4_K_XL payload + 4 x 163840 sessions (f16 KV)

Frank's call. Payload: ajgazin's abliterated Swift-1.5 UD-Q8_K_XL (29.3 GiB,
SWIFT OPEN LICENSE) -> unsloth/Qwen3.8-27B-GGUF UD-Q4_K_XL (16.35 GiB,
sha256 3f227079...b01e, vanilla/Apache-2.0). NOTE: the endpoint is no longer
uncensored — that was one of the reasons for the Swift payload on 2026-09-28.

New fetcher scripts/download-frank-large-q4.sh (no auth header: repo is public
and ungated; size + sha256 verified after transfer). The speculative drafter
and its fetcher are unchanged.

Shape: --sessions 4 --context 163840 with unquantized KV. gufo exposes no KV
quantization lever (execution_policy.hpp pins kv_cache_storage to kFp16), so
f16 is inherent.

KV arithmetic: f16 KV = 64 KiB/token (16 full-attention layers x 4 KV heads x
256 head_dim x 2 x 2B; the other 48 layers are linear-attention with a
per-sequence fp32 state). 4 x 163840 = 655,360 tokens = 40.0 GiB KV, each
session exactly 10 GiB. Budget ~62 GiB of the 114 GiB GTT.

Seam retargeted (FETCH -> the q4 script, TARGET_STUB, context/sessions pins).

AGENTS.md

@@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
 | Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **bf16 KV** (`--kv-cache-dtype auto`), **no speculative decode**, MNBT 2048, MoE gather 32×64, **2 seqs × 262144 ctx (the model's full native window)** at KV_BYTES 7,300,000,000/rank (bf16 measures 13,839 B/token), with **`--expert-precise-memory`** — the fitted 5.25 GiB overhead instead of the 7.60 GiB default, which hands the compile/graph-capture headroom to the expert arena (~7,278 slots vs 6,248) — util 0.98, and the client contract wired: `--reasoning-parser qwen3` (thinking in `reasoning`, never inline), `--enable-auto-tool-choice --tool-call-parser qwen3_xml`, default chat-template kwargs `enable_thinking` + `preserve_thinking`. Capacity history: five full-length contexts boot but leave ~0 bytes free at runtime (a real request with `max_tokens=261149` killed the worker: `Tried to allocate 50.00 MiB ... 0 bytes free`); six never fit (6×241664 asked 10.70 GiB vs 9.18 GiB free). DCP cannot help here — it requires TP > the model's 2 KV heads, i.e. four GPUs. fp8+MTP4 shape restorable from git tag `frank-primary-fp8-mtp-20261001` | 7770 |
-| Dankclaw | `frank-large` | gufo `gufo-org/gufo` v0.1.1, the Strix Halo engine, BUILT FROM SOURCE (root systemd unit; links the host ROCm 7.2.4) | Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (27B dense, ABLITERATED/uncensored, 29.3 GiB, SWIFT OPEN LICENSE 1.0) with the z-lab DFlash2 Q4_K_M drafter (adaptive block policy, cap 7) on the gfx1151 iGPU (Strix Halo LPDDR, 114 GiB GTT), 262144 ctx single session, greedy, gufo's compiled-in Qwen renderer (NO Qwen-Sharp), text only | 7770 |
+| Dankclaw | `frank-large` | gufo `gufo-org/gufo` v0.1.1, the Strix Halo engine, BUILT FROM SOURCE (root systemd unit; links the host ROCm 7.2.4) | `unsloth/Qwen3.8-27B-GGUF` **UD-Q4_K_XL** (27B dense, **vanilla — no longer abliterated**, 16.35 GiB, Apache-2.0) with the z-lab DFlash2 Q4_K_M drafter (adaptive block policy, cap 7) on the gfx1151 iGPU (Strix Halo LPDDR, 114 GiB GTT), **4 sessions x 163840 ctx, unquantized (f16) KV** — gufo has no KV-quant lever — greedy, gufo's compiled-in Qwen renderer (NO Qwen-Sharp), text only. Swapped 2026-10-01 from ajgazin's abliterated Swift-1.5 UD-Q8_K_XL (29.3 GiB): ~13 GiB less weights is what funds four f16 sessions, at the cost of the uncensored behaviour | 7770 |
 | Dankclaw | `frank-small` | SGLang in a self-contained python3.12 venv (root systemd unit; the venv carries its own CUDA 13 stack — see the runbook) | letechlead MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound (qwen3_5 hybrid, 9B dense, 12.0 GiB, INT8 W8A16 AutoRound, in-checkpoint vision tower, NO MTP weights shipped so no spec decode) on the RTX 3090, fp8 KV, pinned 606,208-token pool / 2 slots of 262,144 (native window; 2026-10-01 reslice from 4 x 151,552 for compaction headroom), bf16 mamba states, eager prefill, 83.9 tok/s decode | 7771 |
 
 Retired endpoints (`frank-npu`, and every prior runtime) are in
@@ -437,8 +437,13 @@ Details in the runbook.
 Verified at deploy (2026-10-01): `/v1/models` reports `frank-small`;
 chat returns content plus `reasoning_content` (SGLang's qwen3 parser uses
 llama.cpp's field name); a red-square image answers "Red"
-(`image_tokens: 64`); 4 concurrent 54k-token requests all answer; a
-170,019-token prompt prefills in 50.5 s (~3,370 t/s eager). Decode
+(`image_tokens: 64`). At the 4 x 151,552 shape: 4 concurrent 54k-token
+requests answered; a 170,019-token prompt prefilled in 50.5 s (~3,370 t/s
+eager). At the 2 x 262,144 shape (reslice, same day): boot flags read
+`context_len=262144 max_running_requests=2 max_total_num_tokens=606208`;
+a 200,000-token prompt — well past frank-primary's 183,501 compaction
+trigger — prefilled and completed in ONE request (97.3 s wall, ~2,056
+tok/s eager); two concurrent 50k-token requests both answered. Decode
 **83.9 tok/s**.
 Runbook: `serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md`.
 Rollback: [[frank-small-gemma4-12b]] (payload and binary intact on disk).