frank-default-large: cap CTX_SIZE at 131072 — 262144 OOMs SVM pool (cudaMalloc 103176 MiB failed with fully GPU-resident 87 GiB model)

AGENTS.md

@@ -27,7 +27,7 @@ host.
 | Target | Runtime | Service | Model | Port |
 |--------|---------|---------|-------|------|
 | Dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
-| Dankclaw | charlie12345/ROCmFPX llama.cpp (build-qwen38) | frank-default-large | agentionai Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87 GiB Q4_0_ROCMFP4_FAST, PLE joined table in v2 GGUF), ngram-mod spec (MTP head on disk, incompatible with this build), 262K context / 1 slot, reasoning on | 7770 |
+| Dankclaw | charlie12345/ROCmFPX llama.cpp (build-qwen38) | frank-default-large | agentionai Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87 GiB Q4_0_ROCMFP4_FAST, PLE joined table in v2 GGUF), ngram-mod spec (MTP head on disk, incompatible with this build), 131K context / 1 slot, reasoning on | 7770 |
 
 History: the 2026-08-17 redo retired the pipeline-parallel split and the
 FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
@@ -293,8 +293,9 @@ VRAM-resident, no sidecar, no `--ngram-on-disk`. The model's own MTP draft head
 quant matched to target) is on disk but does **not** load on this binary
 (loader requires a full per-layer `blk.<il>.hc_attn_norm.weight` set; the head
 carries only `blk.48`) — speculative decoding runs ngram-mod only. Vision
-tower (f16) ships as mmproj/. It runs 262144 context / 1
-slot, b 4096 / ub 4096, t 16 / tb 16, f16 KV cache, `--fit off` (auto-fit
+tower (f16) ships as mmproj/. It runs 131072 context / 1
+slot (capped: the GPU-resident 87 GiB model + 262144 f16 KV + compute
+buffer OOMs the SVM pool — verified live, cudaMalloc 103176 MiB), b 4096 / ub 4096, t 16 / tb 16, f16 KV cache, `--fit off` (auto-fit
 segfaults on qwen4exp hyper-connection config), `--flash-attn off` (required:
 fa on aborts graph build on the QSA indexer path — F32 `ggml_fill` on the F16
 kq mask, ggml.c:5371), and reasoning on (Sharp

scripts/frank-default-large.env

@@ -54,12 +54,17 @@ ALIAS=frank-default-large
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7770
 
-# Serving: 262144-token context on ONE slot (parallel 1), batch 4096 /
+# Serving: 131072-token context on ONE slot (parallel 1), batch 4096 /
 # ubatch 4096, 16+16 threads, f16 KV, 16 context checkpoints, 8 GiB
 # prompt cache. --fit off: auto-fit segfaults on this model's
 # hyper-connection/indexer config; -ngl 999 puts every layer on the
 # SVM-backed GPU pool.
-CTX_SIZE=262144
+#
+# CTX_SIZE is capped at 131072: the fully GPU-resident 87 GiB model plus a
+# 262144-token f16 KV cache and the worst-case compute buffer exceed the
+# Strix Halo SVM pool — verified live (cudaMalloc 103176 MiB OOM at
+# 262144). 131072 is the proven STRIX-family value on this box.
+CTX_SIZE=131072
 PARALLEL=1
 BATCH_SIZE=4096
 UBATCH_SIZE=4096

scripts/serve-frank-large.sh

@@ -45,7 +45,7 @@ fi
 : "${ALIAS:=frank-default-large}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7770}"
-: "${CTX_SIZE:=262144}"
+: "${CTX_SIZE:=131072}"
 : "${PARALLEL:=1}"
 : "${BATCH_SIZE:=4096}"
 : "${UBATCH_SIZE:=4096}"

serverAppConfigs-docs/LLM/agentionai-fast-imatrix.md

@@ -89,7 +89,7 @@ llama-server \
   --host 0.0.0.0 --port 7770 --jinja \
   --device ROCm0 --gpu-layers 999 \
   --split-mode none --fit off --flash-attn off \
-  --ctx-size 262144 --parallel 1 \
+  --ctx-size 131072 --parallel 1 \
   --batch-size 4096 --ubatch-size 4096 \
   --cache-type-k f16 --cache-type-v f16 \
   --threads 16 --threads-batch 16 \
@@ -119,8 +119,12 @@ Key choices and deviations from the CIRU deployment:
   mask and aborts graph build (`GGML_ASSERT(a->type == GGML_TYPE_F32)`,
   ggml.c:5371, backtrace through `build_attn_qsa`). Verified live on this
   v2 main — loads and serves fine with fa off.
-- Context kept at 262144 × 1 slot (same as the CIRU deployment; the author's
-  quick-start is 32768 but the full window is fine on SVM).
+- Context capped at 131072 × 1 slot. The 262144 window does NOT fit: the
+  fully GPU-resident 87 GiB model plus a 262144-token f16 KV cache plus the
+  worst-case compute buffer exceeds the Strix Halo SVM pool (verified live:
+  `cudaMalloc 103176 MiB failed: out of memory` on first graph alloc at
+  262144). 131072 is the proven STRIX-family value on this box. (The author's
+  quick-start is 32768 — the full 131K is comfortable.)
 - MTP is disabled (ngram-mod only) — see the MTP section below.
 
 ## MTP draft head — INCOMPATIBLE on build-qwen38