From 5d632bbaea8f036fc954b9d96f3585bb41c5bbf1 Mon Sep 17 00:00:00 2001 From: Frank Date: Sat, 26 Sep 2026 20:49:04 -0700 Subject: [PATCH] frank-primary: pin the drafter to ROCm1 (the last-layers card) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The shared MTP head borrows the main model output head, which a -ts 1,1 layer split places on ROCm1. With tensor overrides set, n_copies==1 in the draft's scheduler, and a borrowed output.weight in a ROCm1 buffer aborts the draft graph (ggml-backend.cpp:941) when the draft context is on ROCm0 — the unit crash-looped 4x at ~12 s per attempt during the 2026-09-26 cutover. Probed live on the host: ROCm0 aborts identically, ROCm1 loads and serves. --- scripts/frank-primary.env | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/scripts/frank-primary.env b/scripts/frank-primary.env index e8bdec2..a3531e3 100644 --- a/scripts/frank-primary.env +++ b/scripts/frank-primary.env @@ -62,9 +62,15 @@ NGL=999 OT_OVERRIDES=exps=CPU,per_layer_token_embd.weight=CPU TENSOR_SPLIT=1,1 -# The drafter lives on the first R9700 (the card the load log shows holding -# the shared embedding borrow path). -SPEC_DRAFT_DEVICE=ROCm0 +# The drafter MUST sit on the SECOND R9700 (ROCm1): with -ts 1,1 the main +# model's output head lands on the last-layers card, and the shared MTP head +# BORROWS it. --override-tensor sets has_tensor_overrides(), which pins +# n_copies==1 in the draft's scheduler — a borrowed output.weight sitting in +# a ROCm1 buffer then aborts the draft graph with ggml-backend.cpp:941 +# "pre-allocated tensor (output.weight) in a buffer (ROCm1) that cannot run +# the operation" when the draft is elsewhere (verified live 2026-09-26: the +# ROCm0 placement crash-looped 4x at ~12 s, ROCm1 loaded and served). +SPEC_DRAFT_DEVICE=ROCm1 # Batch geometry: logical batch 8192 with ubatch 512 — the measured optimum # class on this fleet's ROCm stack (a bigger logical batch is free; compute