From 11f37dafd8e804971f7dc92b8914e4b1c4ad93b2 Mon Sep 17 00:00:00 2001
From: Frank <frank@dankware>
Date: Mon, 17 Aug 2026 02:05:04 -0700
Subject: [PATCH] frank-default-large: split 0.68,0.32 = 45 blocks on 3090
 (Q6_K phase 2)

Max for the Q6_K + q8_0/f16 KV config: 45 blocks (0-44), 11 FA layers
(3..43), KV 8624 MiB, ~0.7 GiB free. 46 layers -> ~0.4 GiB margin; 48
layers (f>=0.7197) adds FA block 47 -> KV 9408 -> OOM by ~0.85 GiB.
Draft model measured on ROCm0 only (no worker impact). Phase 1 (0.53)
verified: acceptance 0.833, KV 6272/6272, flash_attn auto, healthy.
---
 scripts/frank-default-large.env | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/scripts/frank-default-large.env b/scripts/frank-default-large.env
index e944255..875e62c 100644
--- a/scripts/frank-default-large.env
+++ b/scripts/frank-default-large.env
@@ -25,13 +25,19 @@ LISTEN_PORT=7770
 RPC_SERVERS=192.168.0.235:50052
 
 # Model split fractions, one per device in order [RPC0, iGPU].
-# 0.53,0.47 = 34 of 65 blocks on the 3090 (blocks 0-33). Phase-1 value
-# for the Q6_K swap (2026-08-17); the split will be re-maximized after
-# measuring the draft-model placement on the worker. With the Q6_K model
-# (layer pool 20334 MB) + q8_0/f16 KV (784 MiB per FA layer) the worker
-# frees ~4.7 GiB vs the old model at this split; the 0.53 boundary was
-# provably safe under the old model and is comfortably safe here.
-TENSOR_SPLIT=0.53,0.47
+# 0.68,0.32 = 45 of 65 blocks on the 3090 (blocks 0-44) - the maximum
+# for the Q6_K + q8_0/f16 KV config, measured 2026-08-17 phase 2:
+#  - KV per FA layer = 784 MiB (K 1088 B/token + V 2048 B/token @262144)
+#  - weights(f) = 20334 MB layer pool x f (GGUF scan, exact); embd on
+#    the head side (RPC0 buffer observed 10429.88 MiB at f=0.53)
+#  - KV-per-device count = round(f x 66) layers -> 45 layers = 11 FA
+#    (3..43) -> 8624 MiB. 46 layers (f>=0.7045) still 11 FA but margin
+#    drops to ~0.4 GiB; 48 layers (f>=0.7197) adds FA block 47 -> KV
+#    9408 -> OOM by ~0.85 GiB.
+# Predicted: 13430 (weights) + 8624 (KV) + ~530 (RS) + 189 (compute) =
+# ~22773 MiB -> ~0.7 GiB free. Draft model runs on ROCm0 only (measured
+# phase 1: 1718 MiB, no worker impact).
+TENSOR_SPLIT=0.68,0.32
 
 # Context pool. 262144 total = 1 parallel slot x 262144 per slot
 # (n_ctx_seq = n_ctx / n_seq_max, llama-context.cpp:293).