docs: the 2026-09-17 frank-medium shape -- 36:29, 327680 pool, vision

AGENTS.md topology row and section, and the runbook: the pool is 327680
positions (163840 per conversation over two slots), the split is 36:29 -- one
layer past the 34:31 balance at this context, on purpose, for headroom on the
drafter card (fit 29.19/26.04, measured 28.40/26.66 GiB at load, 3.46/5.20
headroom) -- and vision is on via the explicit --mmproj flag, because this
build has no autoload (the first deploy came up text-only and 500'd image
requests). The rope caveat is written down where the cap lives: the GGUF has
no scaling keys, so the top 65k of the pool extrapolates on the base rope.

serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md

@@ -1,7 +1,7 @@
 ---
 type: configuration
 title: frank-medium llama.cpp Runbook (DFlash2 drafter)
-description: The ACTIVE frank-medium deployment since 2026-09-16 — mainline llama.cpp on Dankserver serving ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP at UD-Q8_K_XL with the z-lab Qwen3.8-27B-DFlash2 Q8_0 drafter, as ONE layer-split process across both Radeon AI PRO R9700 cards at a 35:30 split, 262144 ctx over two slots.
+description: The ACTIVE frank-medium deployment since 2026-09-16 — mainline llama.cpp on Dankserver serving ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP at UD-Q8_K_XL with the z-lab Qwen3.8-27B-DFlash2 Q8_0 drafter, as ONE layer-split process across both Radeon AI PRO R9700 cards at a 36:29 split, a 327680-position KV pool (163840 per conversation over two slots), vision via the BF16 mmproj.
 tags:
   - frank-medium
   - dankserver
@@ -12,7 +12,7 @@ tags:
   - gfx1201
   - r9700
   - layer-split
-timestamp: 2026-09-16
+timestamp: 2026-09-17
 ---
 
 # frank-medium — llama.cpp (DFlash2 drafter)
@@ -144,9 +144,9 @@ Deployed flags:
 
 | Flag | Value | Why |
 |---|---|---|
-| `--split-mode` / `--device` / `--tensor-split` | `layer` / `ROCm0,ROCm1` / `35,30` | both cards, one model, no collective; the split is skewed because ROCm1 carries the drafter, and it is one LAYER of granularity — 0.43 GiB on a DeltaNet boundary, 1.43 GiB on an attention one |
+| `--split-mode` / `--device` / `--tensor-split` | `layer` / `ROCm0,ROCm1` / `36,29` | both cards, one model, no collective; the split is skewed because ROCm1 carries the drafter, and it is one LAYER of granularity — 0.43 GiB on a DeltaNet boundary, 1.43 GiB on an attention one. Since 2026-09-17 it is one layer PAST the balance point at the 327680 pool, on purpose: the move bought headroom on the drafter card (see below) |
 | `--gpu-layers` / `--fit` | `999` / `off` | everything on the cards; auto-fit must not resize a context we set |
-| `--ctx-size` / `--parallel` | `262144` / `2` | the model's whole window over two slots of 131072; `--parallel` **divides** the context |
+| `--ctx-size` / `--parallel` | `327680` / `2` | the pool over two slots of 163840; `--parallel` **divides** the context. 1.25x the trained 262144 window, and the GGUF has no rope scaling — see the 2026-09-17 section |
 | `--batch-size` / `--ubatch-size` | `6144` / `512` | measured: ubatch 512 is the prefill optimum *and* the cheapest point on the curve; the logical batch is throughput- and VRAM-neutral. See [[frank-medium-batch-ubatch]] |
 | `--ctx-checkpoints` / `--cache-ram` | `0` / `8192` | no per-slot KV snapshots (the build's default); the RAM-side prompt cache is capped at 8 GiB |
 | `--flash-attn` / cache types | `on` / `f16` K and V | an agentic checkpoint is the wrong place to spend KV precision |
@@ -154,7 +154,7 @@ Deployed flags:
 | `--spec-draft-device` | `ROCm1` | the drafter rides the card that ends up holding the target's last layers |
 | `--temp` / `--top-p` / `--top-k` | `0.7` / `0.95` / `20` | the values the vLLM predecessor served on this endpoint |
 | `--reasoning-format` / `--reasoning-budget` | `deepseek` / `-1` | trace into `reasoning_content`, unlimited |
-| `--no-mmproj` | — | text only |
+| `--mmproj` | `mmproj-BF16.gguf` (888 MiB, beside the target) | vision enabled since 2026-09-17; this build has NO autoload, so the flag is explicit and the serve script refuses to start without the file |
 
 ## The DSpark excursion (2026-09-16, tried and reverted)
 
@@ -260,6 +260,51 @@ larger, so the question is closed rather than pending.
 > 39:26 itself was only *loaded* at the deployed batch, not exercised), and it
 > is **not** an OOM: no amount of headroom would prevent it.
 
+## 2026-09-17: the pool grows to 327680, the split moves to 36:29, vision comes on
+
+The window grew from 262144 to a **327680-position pool** (163840 per
+conversation over the two slots), the split moved one layer to ROCm0
+(35:30 → 36:29), and the **BF16 mmproj** (888 MiB) came on.
+
+**Why 36:29, not the balance.** At this context the fit model puts the balance
+point at 34:31 (0.11 GiB gap). The operator moved a layer to ROCm0 to buy
+headroom on ROCm1 — the card that carries the drafter, its KV, the output
+layers and the heavier compute buffers — accepting a 3.15 GiB gap by
+construction:
+
+| Card | Fit | Measured at load | Headroom |
+|---|---:|---:|---:|
+| ROCm0 (36 layers + mmproj) | 29.19 GiB | 28.40 GiB | 3.46 GiB |
+| ROCm1 (29 layers + drafter) | 26.04 GiB | 26.66 GiB | 5.20 GiB |
+
+Both clear the 1.5 GiB floor with room; the fit lands within 0.8 GiB on both
+cards, which is the accuracy it has always had (at the 35:30 point it
+reproduced ROCm1 to 0.00 and ROCm0 to 0.86). The seam's slack is the 3.15 GiB
+gap rounded up, with the trade written in its comment.
+
+**The rope caveat.** The GGUF carries no `context_length` and no rope-scaling
+keys (freq_base 1e7, `dimension_sections [11,11,10,0]`); the checkpoint's
+trained window is 262144. Positions beyond that — the top 65k of the pool —
+extrapolate on the base rope. It runs, and 1.25x is a modest ask of a rope,
+but it is untrained territory, and the seam's ctx cap encodes 327680 with
+that note.
+
+**Vision: this build has no mmproj autoload.** The first 327680/36:29 deploy
+came up healthy and *text-only*: image requests answered 500 "image input is
+not supported — hint: … provide the mmproj" and the load log had no mmproj
+line, so `llama-server` fb27a525 does not pick up `mmproj*.gguf` from the
+model directory. The serve script now passes `--mmproj` explicitly (the
+frank-small pattern), refuses to start without the file when vision is on, and
+the deploy handler's preflight checks for it. Verified end-to-end after the
+fix: a generated 64x32 test image was described from its pixels — with
+thinking on the model reasoned about "the three horizontal bars" (its reading
+of the crude glyph) and with thinking off answered the letter question in one
+token.
+
+The e2e crawl spec flaked once on the first 327680 deploy (1 failed / 31
+passed) and passed 32/32 on the redeploy; the service itself was healthy
+throughout (health gate green, `NRestarts=0`).
+
 ## Verified serving behaviour (2026-09-16)
 
 - Load: `model loaded` ~25 s after start with a warm page cache, then
@@ -458,13 +503,13 @@ produced wrong numbers on this fleet:
 
 - **It separates devices differently from `llama-server`.** `/` splits devices
   *within one configuration*, `,` splits *sweep values* — so this service's
-  `--tensor-split 35,30` is `-ts 35/30` to llama-bench, and the comma form
+  `--tensor-split 36,29` is `-ts 36/29` to llama-bench, and the comma form
   silently benches each card on its own instead. Likewise `--device
   ROCm0,ROCm1` → `-dev ROCm0/ROCm1`.
 - **There is no `-c`.** The context is a consequence of the test shape:
   `n_ctx = n_prompt + n_gen + n_depth`. No bench can therefore prove that a
   configuration's activation buffers fit beside the *deployed* KV — that needs
-  the live unit at 262144 across two slots.
+  the live unit at 327680 across two slots.
 - **A test that cannot allocate calls `exit(1)`**, taking the whole process with
   it, so the harness runs one invocation per cell and a failure costs one TSV
   line instead of the rest of the sweep.