frank-medium: llama.cpp layer split across both R9700s, DFlash2 drafting

The vllm-radiance replica pair could not use both cards for one model: TP=2
needs a cross-card collective and the second R9700 cannot dispatch one. The
same wall blocks vLLM's pipeline parallelism, whose stage handoff is an RCCL
send/recv of the hidden state (distributed/parallel_state.py) -- a fact worth
writing down, because the obvious reading of the ask ('use PP to dodge the
atomics problem') is exactly backwards.

llama.cpp's layer split needs no collective at all: layers are distributed and
the boundary activation is a per-backend copy, once per forward pass. One
process, both cards, one model. The runtime is mainline llama.cpp at fb27a525
(the ref is pinned by DFlash2 AND qwen35 MTP; the ROCmFPX fork this host had
before has neither), built for gfx1201 against the private ROCm 7.14 toolchain
the earlier build already used, and bundled next to the binary.

Server: ajgazin Swift-Qwen3.8-27B-Uncensored UD-Q8_K_XL with the model's MTP
head in the file, plus the z-lab DFlash2 Q8_0 block-diffusion drafter at depth
7 (its block is 8, so 7 is the ceiling llama.cpp allows). Text only: vision
stays on frank-small.

Retired into the trash rather than deleted: the three vLLM quadlets, the
radiance entrypoint/healthcheck, the routing proxy and its tests, and the two
bench scripts that drove 'vllm bench serve' inside the retired image. The
deploy handler now stops them, waits for 7770 to be released, installs the
ROOT unit and refuses to start anything when the runtime or payload is
missing. The planner maps the scripts/ trio to frank-medium only, and the
scoping seam asserts the retired service dir is gone rather than merely
unmapped.

scripts/deploy-remote.sh

@@ -1100,62 +1100,72 @@ deploy_llm_small() {
     wait_for_health frank-small http://127.0.0.1:7771/health 900
 }
 
-# deploy_llm_medium — frank-medium runs as a rootless Quadlet since the
-# 2026-09-15 runtime swap to the vllm-radiance MXFP4 stack. It began the day
-# as a root llama.cpp unit (the ROCmFPX/Q8_0_ROCMFPX_AGENT deployment from
-# the morning GPU swap); that root unit is retired here first — stop,
-# disable, move the installed unit into the home trash. Same failure mode as
-# frank-large's retirement: this host's `systemctl disable` has been
-# observed to delete the unit file itself (2026-08-26 incident), so both
-# outcomes are tolerated.
-retire_root_llm_medium() {
-    local trash="$HOME/.AGENTTRASH/20260915-rocmfpx-retire/host-units"
-    run_privileged systemctl stop frank-medium.service 2>/dev/null || true
-    run_privileged systemctl disable frank-medium.service 2>/dev/null || true
-    if [[ -e /etc/systemd/system/frank-medium.service || -L /etc/systemd/system/frank-medium.service ]]; then
-        mkdir -p "$trash"
-        run_privileged mv -- /etc/systemd/system/frank-medium.service \
-            "$trash/frank-medium.service"
-        echo "retired root unit frank-medium.service -> $trash"
-    fi
-    run_privileged systemctl daemon-reload
+# deploy_llm_medium — frank-medium is mainline llama.cpp on the two R9700s as a
+# ROOT unit since the 2026-09-16 cutover: the runtime is a native gfx1201 HIP
+# binary, so there is no image to install and no generated unit to dodge. It
+# replaces the vllm-radiance replica pair + routing proxy, which held this
+# endpoint for a day and could never use both cards for one model (TP=2 needs
+# an RCCL collective, and the second card cannot dispatch one).
+#
+# Layer split in ONE process is what the successor does instead: layers are
+# distributed across the cards and only the activation at the boundary crosses
+# PCIe, once per forward pass. That is the "pipeline parallelism" the endpoint
+# wanted, with no collective anywhere in it.
+LLM_MEDIUM_BIN="${LLM_MEDIUM_BIN:-/home/dank/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714/bin/llama-server}"
+LLM_MEDIUM_MODEL="${LLM_MEDIUM_MODEL:-/home/dank/models/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf}"
+LLM_MEDIUM_DRAFT="${LLM_MEDIUM_DRAFT:-/home/dank/models/Qwen3.8-27B-DFlash2-GGUF/Qwen3.8-27B-DFlash2-Q8_0.gguf}"
+
+# The three vLLM user units are retired HERE, not left to rot: the proxy
+# published 7770 on the host network, so the llama-server below cannot bind
+# until they are stopped and the port is released. Never `rm` — the unit files
+# move into the home trash; the images stay in local storage.
+retire_vllm_medium_quadlets() {
+    local trash="$HOME/.AGENTTRASH/20260916-vllm-radiance-retire" unit
+    systemctl --user stop frank-medium-a.service frank-medium-b.service \
+        frank-medium-proxy.service 2>/dev/null || true
+    for unit in frank-medium-a frank-medium-b frank-medium-proxy frank-medium; do
+        if [[ -e "$quadlet_unit_dir/$unit.container" ]]; then
+            mkdir -p "$trash"
+            mv -- "$quadlet_unit_dir/$unit.container" "$trash/$unit.container"
+            echo "retired quadlet $unit.container -> $trash"
+        fi
+    done
+    systemctl --user daemon-reload
 }
 
 deploy_llm_medium() {
-    retire_root_llm_medium
-    # Pull once, then run the localhost/-aliased tag: this host's quadlet
-    # path cannot resolve registry-prefixed refs from local storage at
-    # create time (same as frank-large on dankclaw). Pull=never keeps boots
-    # off the registry.
-    podman pull docker.io/stilldeadcode/vllm-radiance:0.9.3 >/dev/null 2>&1 || true
-    podman tag docker.io/stilldeadcode/vllm-radiance:0.9.3 localhost/radiance-vllm:0.9.3
-    # The compile cache must exist before the quadlet mounts resolve; the
-    # one-time setup (setup-mxfp4.sh / PREPARE_ONLY) normally creates it.
-    mkdir -p /home/dank/.radiance-cache-w4a8-093-gdnm-nqft-fp8s-gnq/{vllm,inductor,triton,aiter}
-    # Three units, not one. TP=2 across both R9700s cannot launch a collective
-    # on this host — RCCL's kernels need PCIe atomics and the second card's
-    # chipset path advertises AtomicOpsCap: Routing- — so the endpoint is two
-    # single-GPU replicas behind a round-robin proxy. frank-medium.container
-    # stays in the repo as the TP=2 target for a host whose second card sits
-    # on a CPU root port; it is deliberately NOT installed here.
-    install_quadlet frank-medium-a dankserver/frank-medium/quadlet frank-medium-a
-    install_quadlet frank-medium-b dankserver/frank-medium/quadlet frank-medium-b
-    install_quadlet frank-medium-proxy dankserver/frank-medium/quadlet frank-medium-proxy
-    # Drop a stale TP=2 unit from an earlier deploy of this service, or the
-    # generator keeps trying to start a container that can never become
-    # healthy. Never `rm`: it moves into the home trash.
-    if [[ -e "$quadlet_unit_dir/frank-medium.container" ]]; then
-        local trash="$HOME/.AGENTTRASH/20260916-frank-medium-tp2"
-        mkdir -p "$trash"
-        systemctl --user stop frank-medium.service 2>/dev/null || true
-        mv -- "$quadlet_unit_dir/frank-medium.container" "$trash/frank-medium.container"
-        systemctl --user daemon-reload
-        echo "retired the TP=2 quadlet -> $trash (see the runbook for why)"
+    # Refuse rather than restart-loop. A unit whose runtime or payload is
+    # missing never becomes healthy, and a green deploy would leave the
+    # endpoint dark with nothing in the deploy log to say why.
+    local missing=0
+    if [[ ! -x "$LLM_MEDIUM_BIN" ]]; then
+        echo "ERROR: llama-server absent at $LLM_MEDIUM_BIN" >&2
+        echo "frank-medium: build it ON the host (gfx1201): bash scripts/build-llama-rdna4.sh" >&2
+        missing=1
     fi
-    # The proxy waits on the replicas, but a restart of all three is what a
-    # deploy does: the replicas reload the model, the proxy reconnects.
-    systemctl --user restart frank-medium-a.service frank-medium-b.service frank-medium-proxy.service
-    wait_for_health frank-medium http://127.0.0.1:7770/health 1800
+    if [[ ! -f "$LLM_MEDIUM_MODEL" ]]; then
+        echo "ERROR: target model absent at $LLM_MEDIUM_MODEL" >&2
+        echo "frank-medium: fetch it: bash scripts/download-frank-medium-gguf.sh" >&2
+        missing=1
+    fi
+    if [[ ! -f "$LLM_MEDIUM_DRAFT" ]]; then
+        echo "ERROR: DFlash2 drafter absent at $LLM_MEDIUM_DRAFT" >&2
+        echo "frank-medium: fetch it: bash scripts/download-frank-medium-gguf.sh" >&2
+        missing=1
+    fi
+    [[ "$missing" -eq 0 ]] || return 1
+
+    retire_vllm_medium_quadlets
+    # The retired proxy held 7770 on the host network; pasta releases it at
+    # container teardown and a stopped engine lingers in TIME_WAIT just past
+    # that. Same gate as frank-large's cutover (2026-09-16 addressed-in-use
+    # failure).
+    wait_port_free 7770 || return 1
+    relink_unit frank-medium
+    run_privileged systemctl daemon-reload
+    run_privileged systemctl enable frank-medium.service
+    run_privileged systemctl restart frank-medium.service
+    wait_for_health frank-medium http://127.0.0.1:7770/health 900
 }
 
 wait_for_health() {