docs: the ubatch optimum, the rebalance, and the live atomics chain
New page [[frank-medium-batch-ubatch]]: the sweep's method (one invocation per cell, a VRAM sampler per cell, 64k prefill measured AT 64k depth), the full batch x ubatch matrix, the depth-0 control, the deployed-shape VRAM table, and the limitations — including the one that cost me two runs, that amdgpu does not return ~26 GiB at once, so a reload after a kill fails with "unable to allocate ROCm0 buffer" and reads exactly like a configuration that does not fit. AGENTS.md: the batch lesson now carries the measured curve (the peak at 512, the 10 % loss at 2048, the irrelevant logical batch, the invariant decode) instead of the "~3.3 vs ~1.7 GiB" figure, which was measured at the bench's shape; at the deployed shape the asymmetry is 2.41 against 1.30. Fleet row and the frank-medium section now say 35:30 and batch 6144x512. Runbook: deployed flags, the split history with the ubatch-512 table, the fault note, and the verified VRAM (6.44 / 6.90 idle, 6.23 / 6.64 under two concurrent requests). Dual-R9700-Cooperation: the live capability table (card 0 ReqEn+ against card 1 ReqEn-, x1 effective link on card 1) and an honest label on the M.2_2 riser premise — it is ASUS's spec sheet, not a measurement, and CPU-attached does not imply Routing+ on this board (00:08.3 is CPU-side and Routing-). log.md: the round trip recorded, newest first.
This commit is contained in:
1 parent
a11fae07d8
commit
6472d706be
6 files changed
+275
-32
No files matched your search
@@ -191,6 +191,11 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
- [[frank-large-benchmark]] — backend/spec benchmark behind the old head
|
||||
- [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results
|
||||
- [[frank-small-benchmark]] — the small-model benchmark suite
|
||||
- [[frank-medium-batch-ubatch]] — the 2026-09-16 batch/ubatch sweep on the 2x
|
||||
R9700 layer split: prefill PEAKS at ubatch 512 and falls on both sides, the
|
||||
logical batch buys nothing measurable, decode never moves, and the ubatch's
|
||||
uneven VRAM give-back (1.30 GiB on card 0 against 2.41 GiB on card 1) moved
|
||||
the tensor split from 36:29 to 35:30
|
||||
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
|
||||
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
|
||||
- [[qwen3.8-strix-optimizations]] — Strix Halo tuning notes
|
||||
|
||||
Reference in new issue
Block a user