docs(frank-small): record the batch doubling and scope the ubatch law to its backend
The AGENTS.md lesson "`--ubatch-size` has an OPTIMUM on this stack" was load- bearing and over-general: it was measured on the 2x R9700 layer split, and the same knob on the CUDA 3090 behaves the opposite way (curve climbs monotonically to 2048). That difference is now the reason frank-small ships ubatch 2048, so the lesson names the backend it was measured on and points at the counterpoint. Adds the 2026-10-05 boot-pair A/B to the runbook (+4.6 % prefill at 8,928 tokens for 616 MiB, decode flat, the sub-ubatch prompt as the control), marks the superseded 21,276/2,849 VRAM row, and logs the day. The old "ubatch 2048 failed the scratch boot" line stays visible as what it was: a predecessor-shape observation, not a law.
This commit is contained in:
1 parent
e5893dc30d
commit
67141c8c02
3 files changed
+107
-7
No files matched your search
@@ -1,5 +1,41 @@
|
||||
# Bundle Change Log
|
||||
|
||||
## 2026-10-05
|
||||
- **frank-small: compute-buffer pair doubled to 4096/2048, and the prefill win
|
||||
measured rather than assumed.** Frank asked for the batch and ubatch doubled.
|
||||
The blocker was on record: the Q5_K_M predecessor documented **"ubatch 2048
|
||||
failed the scratch boot"**, and this shape carries 616 MiB less free space
|
||||
than that predecessor did.
|
||||
- The A/B used the service itself as the bench (one card, one server — no
|
||||
scratch port): the old rung was re-booted through a transient
|
||||
`FRANK_SMALL_ENV_FILE` systemd drop-in, and it reproduced the 2026-10-03
|
||||
boot **byte-identically** (21,276 used / 2,849 free), which is what makes
|
||||
ubatch the only variable. Repo never saw a fake revert commit; scaffolding
|
||||
staged to `~/.AGENTTRASH/20261005-frank-small-ubatch-ab/`.
|
||||
- **At 8,928 prompt tokens: prefill 3,972/4,077 -> 4,207/4,212 t/s, +4.6 %,
|
||||
for 616 MiB (free 2,849 -> 2,279).** Decode flat (91.9–93.0 vs 93.8–94.3 —
|
||||
noise). The control that proves it is geometry and not drift: a 279-token
|
||||
prompt sits below *both* micro-batches, so it cannot see the knob, and it
|
||||
did not move (2,676 vs 2,658–2,811).
|
||||
- **The predecessor's boot failure did not reproduce** — that was a shape
|
||||
observation, not a law; this shape boots 2048 clean. And the fleet lesson
|
||||
"**`--ubatch-size` has an optimum**" is now scoped to what it actually is:
|
||||
true on the 2x R9700 layer split (peak 512, −10 % by 2048), false on the
|
||||
CUDA 3090 where the curve climbs monotonically to 2048. AGENTS.md corrected
|
||||
— the curve is backend-specific and must never be carried across backends.
|
||||
- The seam stopped hard-coding a rung: `serve_argv` now carries `BATCH_SIZE`
|
||||
and `UBATCH_SIZE` the way it already carried `CTX_SIZE`, and the pair test
|
||||
asserts the argv follows the shipped env plus the `ubatch <= batch`
|
||||
invariant. Proven live both ways — retuning the env to 8192/4096 keeps it
|
||||
green, inverting to 2048/4096 turns it red. (The old assertions only ever
|
||||
exercised the serve-script *defaults*, because the test env never carried
|
||||
these two keys.)
|
||||
- `4096/1536` is the documented middle rung; 2,279 MiB free sits at the bottom
|
||||
edge of the 2–3 GiB floor. Deployed `e5893dc`, e2e gate PASSED (2 mini specs
|
||||
flaky-then-passed once the sleeping FrankBook-Pro woke mid-run — WoL took a
|
||||
second burst and ~90 s). Docs:
|
||||
[[frank-small-llama-mimo-q6k-2x200704]] (*Compute-buffer geometry*).
|
||||
|
||||
## 2026-10-03
|
||||
- **frank-primary: shape F then shape G — MNBT 2048, the CUDA-graph plan term
|
||||
pinned to the measured capture, KV forced to 400,700 tokens, and UTIL 1.0
|
||||
|
||||
Reference in new issue
Block a user