From 4e8ea24b2ff7a7ab244ffbc57296fa751eb1f1e3 Mon Sep 17 00:00:00 2001
From: Frank <frank@dankware>
Date: Tue, 25 Aug 2026 18:40:32 -0700
Subject: [PATCH] Record complete 32K ubatch benchmark findings

---
 docs/LLM/ubatch-benchmark-large.md | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/docs/LLM/ubatch-benchmark-large.md b/docs/LLM/ubatch-benchmark-large.md
index d3b9b89..6fac208 100644
--- a/docs/LLM/ubatch-benchmark-large.md
+++ b/docs/LLM/ubatch-benchmark-large.md
@@ -16,6 +16,10 @@ This benchmark sweeps the active `frank-default-large` model on Dankclaw using
 scratch ports and a 32K-context workload. The production service is stopped
 before the sweep and restarted by the benchmark's exit handler.
 
+The 2026-09-01 run used `ctx=32768` and `batch=32768`. It completed through
+`ubatch=3072`; `ubatch=4096` failed during initialization with a ROCm
+allocation error, so 4096, 5120, and 6144 were not run.
+
 ## Sweep
 
 The default points are `256, 512, 768, 1024, 1280, 1536, 2048, 3072, 4096,
@@ -39,19 +43,19 @@ interrupted; the GPU is shared with live workloads.
 
 Record results below after the sweep, including failures or OOM points.
 
-| ubatch | prefill | generation | notes |
+| ubatch | cold prefill | generation runs 1/2/3 | notes |
 | ---: | ---: | ---: | --- |
-| 256 | — | — | pending |
-| 512 | — | — | pending |
-| 768 | — | — | pending |
-| 1024 | — | — | pending |
-| 1280 | — | — | pending |
-| 1536 | — | — | pending |
-| 2048 | — | — | pending |
-| 3072 | — | — | pending |
-| 4096 | — | — | pending |
-| 5120 | — | — | pending |
-| 6144 | — | — | pending |
+| 256 | 214.73 tok/s | 17.68 / 17.15 / 17.12 tok/s | completed |
+| 512 | 213.47 tok/s | 17.91 / 17.29 / 17.72 tok/s | completed |
+| 768 | 215.60 tok/s | 17.96 / 18.17 / 17.98 tok/s | completed |
+| 1024 | 222.20 tok/s | 18.07 / 18.10 / 18.12 tok/s | completed |
+| 1280 | 219.13 tok/s | 17.47 / 17.69 / 17.49 tok/s | completed |
+| 1536 | 220.52 tok/s | 16.85 / 16.86 / 16.86 tok/s | completed |
+| 2048 | 60.97 tok/s | 6.24 / 4.94 / 11.19 tok/s | severe slowdown |
+| 3072 | 28.50 tok/s | 1.45 / 5.03 / 4.44 tok/s | severe slowdown |
+| 4096 | — | — | initialization OOM: rs-cache allocation |
+| 5120 | — | — | not reached |
+| 6144 | — | — | not reached |
 
 ## See also