The harness is in the repo now, so the runbook says how to run it and what the
tool will not tell you:
- / splits devices within one configuration and , splits sweep values, so a
legitimate --tensor-split 36,29 would have benched one card at a time.
- there is no -c: n_ctx = n_prompt + n_gen + n_depth, so no bench can decide
whether a configuration's buffers fit beside the deployed 262144 KV.
- a test that cannot allocate exit(1)s the process, hence one invocation per
cell.
- a shell's memlock limit (8 MiB here) is not the unit's (infinity), which
fails the load for the wrong reason.
- no --spec-* and no --parallel: decode figures are drafterless and slots are
never exercised; -d is what puts KV in front of the measurement at all.
Frank
262ac53c14