docs(frank-small): record the batch doubling and scope the ubatch law to its backend

The AGENTS.md lesson "`--ubatch-size` has an OPTIMUM on this stack" was load-
bearing and over-general: it was measured on the 2x R9700 layer split, and the
same knob on the CUDA 3090 behaves the opposite way (curve climbs monotonically
to 2048). That difference is now the reason frank-small ships ubatch 2048, so
the lesson names the backend it was measured on and points at the counterpoint.

Adds the 2026-10-05 boot-pair A/B to the runbook (+4.6 % prefill at 8,928
tokens for 616 MiB, decode flat, the sub-ubatch prompt as the control), marks
the superseded 21,276/2,849 VRAM row, and logs the day. The old "ubatch 2048
failed the scratch boot" line stays visible as what it was: a predecessor-shape
observation, not a law.
This commit is contained in:
Frank committed 2026-10-05 15:53:06 -07:00
1 parent e5893dc30d
commit 67141c8c02
3 files changed
+107 -7

No files matched your search

+36
View File
@@ -1,5 +1,41 @@
# Bundle Change Log
## 2026-10-05
- **frank-small: compute-buffer pair doubled to 4096/2048, and the prefill win
measured rather than assumed.** Frank asked for the batch and ubatch doubled.
The blocker was on record: the Q5_K_M predecessor documented **"ubatch 2048
failed the scratch boot"**, and this shape carries 616 MiB less free space
than that predecessor did.
- The A/B used the service itself as the bench (one card, one server — no
scratch port): the old rung was re-booted through a transient
`FRANK_SMALL_ENV_FILE` systemd drop-in, and it reproduced the 2026-10-03
boot **byte-identically** (21,276 used / 2,849 free), which is what makes
ubatch the only variable. Repo never saw a fake revert commit; scaffolding
staged to `~/.AGENTTRASH/20261005-frank-small-ubatch-ab/`.
- **At 8,928 prompt tokens: prefill 3,972/4,077 -> 4,207/4,212 t/s, +4.6 %,
for 616 MiB (free 2,849 -> 2,279).** Decode flat (91.9–93.0 vs 93.8–94.3 —
noise). The control that proves it is geometry and not drift: a 279-token
prompt sits below *both* micro-batches, so it cannot see the knob, and it
did not move (2,676 vs 2,658–2,811).
- **The predecessor's boot failure did not reproduce** — that was a shape
observation, not a law; this shape boots 2048 clean. And the fleet lesson
"**`--ubatch-size` has an optimum**" is now scoped to what it actually is:
true on the 2x R9700 layer split (peak 512, −10 % by 2048), false on the
CUDA 3090 where the curve climbs monotonically to 2048. AGENTS.md corrected
— the curve is backend-specific and must never be carried across backends.
- The seam stopped hard-coding a rung: `serve_argv` now carries `BATCH_SIZE`
and `UBATCH_SIZE` the way it already carried `CTX_SIZE`, and the pair test
asserts the argv follows the shipped env plus the `ubatch <= batch`
invariant. Proven live both ways — retuning the env to 8192/4096 keeps it
green, inverting to 2048/4096 turns it red. (The old assertions only ever
exercised the serve-script *defaults*, because the test env never carried
these two keys.)
- `4096/1536` is the documented middle rung; 2,279 MiB free sits at the bottom
edge of the 2–3 GiB floor. Deployed `e5893dc`, e2e gate PASSED (2 mini specs
flaky-then-passed once the sleeping FrankBook-Pro woke mid-run — WoL took a
second burst and ~90 s). Docs:
[[frank-small-llama-mimo-q6k-2x200704]] (*Compute-buffer geometry*).
## 2026-10-03
- **frank-primary: shape F then shape G — MNBT 2048, the CUDA-graph plan term
pinned to the measured capture, KV forced to 400,700 tokens, and UTIL 1.0