docs: the ubatch optimum, the rebalance, and the live atomics chain

New page [[frank-medium-batch-ubatch]]: the sweep's method (one invocation per
cell, a VRAM sampler per cell, 64k prefill measured AT 64k depth), the full
batch x ubatch matrix, the depth-0 control, the deployed-shape VRAM table, and
the limitations — including the one that cost me two runs, that amdgpu does not
return ~26 GiB at once, so a reload after a kill fails with "unable to allocate
ROCm0 buffer" and reads exactly like a configuration that does not fit.

AGENTS.md: the batch lesson now carries the measured curve (the peak at 512, the
10 % loss at 2048, the irrelevant logical batch, the invariant decode) instead
of the "~3.3 vs ~1.7 GiB" figure, which was measured at the bench's shape; at
the deployed shape the asymmetry is 2.41 against 1.30. Fleet row and the
frank-medium section now say 35:30 and batch 6144x512.

Runbook: deployed flags, the split history with the ubatch-512 table, the fault
note, and the verified VRAM (6.44 / 6.90 idle, 6.23 / 6.64 under two concurrent
requests).

Dual-R9700-Cooperation: the live capability table (card 0 ReqEn+ against card 1
ReqEn-, x1 effective link on card 1) and an honest label on the M.2_2 riser
premise — it is ASUS's spec sheet, not a measurement, and CPU-attached does not
imply Routing+ on this board (00:08.3 is CPU-side and Routing-).

log.md: the round trip recorded, newest first.
This commit is contained in:
Frank committed 2026-09-17 00:27:59 -07:00
1 parent a11fae07d8
commit 6472d706be
6 files changed
+275 -32

No files matched your search

+5
View File
@@ -191,6 +191,11 @@ The LLM inference fleet on the two hosts. Full index:
- [[frank-large-benchmark]] — backend/spec benchmark behind the old head
- [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results
- [[frank-small-benchmark]] — the small-model benchmark suite
- [[frank-medium-batch-ubatch]] — the 2026-09-16 batch/ubatch sweep on the 2x
R9700 layer split: prefill PEAKS at ubatch 512 and falls on both sides, the
logical batch buys nothing measurable, decode never moves, and the ubatch's
uneven VRAM give-back (1.30 GiB on card 0 against 2.41 GiB on card 1) moved
the tensor split from 36:29 to 35:30
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
- [[qwen3.8-strix-optimizations]] — Strix Halo tuning notes