Prefill is the priority for this workload. Switch from decode-max
(ncmoe 32 / ubatch 1024) to prefill-max:
- n-cpu-moe 35, ubatch 3072: prefill ~1458 t/s (top tier), gen ~45.4 t/s,
~1.2 GiB headroom at the 800k pool.
- Gen is independent of ubatch; ub 3072 reaches best prefill with no
decode cost. Docs synced (benchmark Chosen State, deployment, specs).
Frank
01eb957f90