frank-primary: wire --decode-context-parallel-size (opt-in)

DCP is passed only when FRANK_PRIMARY_DCP is set, so the deployed shape is
unchanged. Verified in this fork: DCP reuses the TP ranks when PCP is
disabled (tp % dcp == 0), so TP=2 + DCP=2 fits the two R9700s with no extra
ranks; the flash_attn backend carries DCP support, the 36 linear-attention
layers of this hybrid do not — which is what the A/B window tests.
This commit is contained in:
Frank
2026-10-01 13:05:16 -07:00
parent 667ec2987c
commit 148a5f2575
2 changed files with 5 additions and 0 deletions
+1
View File
@@ -111,6 +111,7 @@ podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
"$FRANK_PRIMARY_IMAGE" /models \
--served-model-name "$FRANK_PRIMARY_SERVED_NAME" --port 7770 \
--tensor-parallel-size 2 \
${FRANK_PRIMARY_DCP:+--decode-context-parallel-size "$FRANK_PRIMARY_DCP"} \
--max-model-len "$FRANK_PRIMARY_MAXLEN" \
--max-num-seqs "$FRANK_PRIMARY_SEQS" \
${FRANK_PRIMARY_MNBT:+--max-num-batched-tokens "$FRANK_PRIMARY_MNBT"} \