docs(frank-primary): parallelism survey — DCP rejected on 2 GPUs (GQA constraint)

--decode-context-parallel-size 2 fails VllmConfig validation at boot:
'Decode context parallelism for GQA/MQA requires --tensor-parallel-size (2) to
be greater than the model's total number of KV heads (2)'. DCP shards KV along
the sequence dimension AFTER KV heads spread across TP ranks, so it needs
TP > num_kv_heads; this model has 2 KV heads, so TP >= 4 — four GPUs, not two.
Recorded with the rest of the survey (PP/DP/EP/multi-node verdicts for this
box), the container-injection requirement for /home/dank/mtp_measure.py
(it imports /bench_iter.py by absolute path and must be copied in), and the
re-measured deployed capacity (1,079,582 tokens / 4.12x).
This commit is contained in:
Frank
2026-10-01 13:17:45 -07:00
parent 148a5f2575
commit 6bbcaf3999
@@ -320,6 +320,60 @@ only available while the container lives).
Rollback to the 2026-09-29 fp8+MTP4 shape: git tag
`frank-primary-fp8-mtp-20261001` (env trio + seam re-pinned there).
## Parallelism options on this box (2026-10-01 survey)
Surveyed from the running image's own CLI surface. Verdicts are specific to
dankserver: 2x R9700 (gfx1201), a ~172.8 GiB expert-offload MoE, and a model
with only 2 KV heads.
| option | flag | verdict on this box |
|---|---|---|
| TP | `--tensor-parallel-size` | **DEPLOYED, =2.** All-reduce dispatch shows `R4D_AR` then `PYNCCL` |
| DCP | `--decode-context-parallel-size` | **REJECTED at TP=2** — see below |
| PP | `--pipeline-parallel-size` | supported; untried (stage handoff plus a TP-keyed expert plan) |
| DP | `--data-parallel-size` (+`-rank`, `-start-rank`) | impossible: every replica holds a full model copy (172.8 GiB vs 31.86 GiB/card) |
| EP | `--enable-expert-parallel` + `--all2all-backend` (+`--enable-eplb`, `--enable-dbo`, `--enable-elastic-ep`) | supported (`naive`, `allgather_reducescatter`, `deepep_high_throughput`, `deepep_low_latency`, `pplx`, `mori`, `nixl`); would replace the fork's tiered-offload LOCAL gather with a per-layer PCIe all-to-all |
| multi-node | `--nnodes`, `--node-rank`, `--master-addr/port`, `--distributed-executor-backend` | supported; the actual path to more cards |
### DCP is structurally unavailable here — do not re-test on 2 GPUs
`--decode-context-parallel-size 2` fails `VllmConfig` validation at boot:
```
Decode context parallelism for GQA/MQA requires `--tensor-parallel-size` (2) to be
greater than the model's total number of KV heads (2). Increase
`--tensor-parallel-size` or set `--decode-context-parallel-size 1`.
```
DCP shards the KV along the *sequence* dimension after KV heads have been spread
across the TP ranks, so it requires **TP > num_kv_heads**. This model has 2 KV
heads (24 Q heads, GQA 12:1), so TP must be >= 4 — four GPUs, not two. Related
fork facts: DCP *reuses* the TP ranks rather than multiplying them (it is not
`TP x DCP`), and with PCP enabled the valid DCP sizes are `{1, pcp, tp x pcp}`
(`vllm/config/parallel.py`). The KV memory a DCP win would have bought is real —
each rank would hold half the sequence's KV — but it is unreachable without two
more cards.
### Measuring decode: the harness lives INSIDE the container
`/home/dank/mtp_measure.py` imports `/bench_iter.py` by absolute path, so it must
run in the engine's namespace and the files must be copied in first:
```
podman cp /home/dank/mtp_measure.py frank-primary-qwen:/
podman cp /home/dank/bench_iter.py frank-primary-qwen:/
podman exec frank-primary-qwen python3 /mtp_measure.py
```
Without the copy the exec dies with `can't open file '/mtp_measure.py': [Errno 2]
No such file or directory` — measured 2026-10-01, which silently cost a whole
arm's numbers. A window script must do the copy, not assume the path exists.
### The deployed shape, measured again (2026-10-01 A/B window)
A clean production-environment boot of the deployed config (4 x 262144 fp8, MNBT
2048 in that window) reported **`GPU KV cache size: 1,079,582 tokens, Maximum
concurrency for 262,144 tokens per request: 4.12x`** — the same figures as the
deployed MNBT-1024 shape, i.e. MNBT does not move the pool, and the capacity is
reproducible across boots.
## See Also
- [[dankserver-frontier-moe-feasibility]] — the sibling feasibility page