docs: the max-length sweep tabled; frank-small.env NGL/FLASH_ATTN restored

- runbook: identity now 4x262144, audio verified, throughput table with
  the OOM and negative-result rows, deployed flags updated to reality.
- AGENTS.md frank-small paragraph matches the deployed shape.
- log.md: the sweep entry.
- scripts/frank-small.env: my earlier line-range edit had clobbered
  NGL=999 and FLASH_ATTN=on out of the file; the seams caught it. The
  live service was unaffected because the script's defaults match the
  declared values exactly, but the file contract is restored.
Suite: ALL TESTS PASSED (321), vault 125 pages 0 unresolved.
This commit is contained in:
Frank committed 2026-09-17 04:56:59 -07:00
1 parent 24eaba1078
commit 17450dd172
4 files changed
+102 -34

No files matched your search

+28
View File
@@ -820,3 +820,31 @@ timestamp: 2026-09-14
set` error is `--fit`'s own probe — normal, self-recovers seconds later.
- Runbook: [[frank-small-gemma4-12b]] (active);
[[frank-small-qwen38-gsq-rco]] retired 2026-09-17 as rollback target.
## 2026-09-17 (later) — frank-small: four full windows, max-length throughput sweep
- **Slot ceiling measured:** 1 full-ctx slot idles 11,996 MiB, 4 idle
22,346 MiB -> 3,450 MiB/slot, ~8,546 fixed; a 5th projects past the
24,576 MiB card. Deployed `CTX_SIZE=1048576` / `PARALLEL=4` (each
slot the FULL 262,144 window; explicit CTX bypasses `--fit` on
purpose — fit would divide the pool at parallel>1). The bats seam now
pins the division and the budget inequality.
- **Audio verified:** JFK WAV via `input_audio` -> verbatim transcript.
- **Batch sweep at max length** (salted API probes, pp ~254k /
tg @ ~253k ctx): ubatch 512 -> pp 912–924, tg 90–204;
**ubatch 1024 -> pp 962–965, tg 226–302 (deployed)**;
ubatch 2048 -> OOM in the MTP draft's pp compute buffer (~3.3 GiB
vs 2.2 GiB headroom).
- **Negative results kept:** draft n-max 4 + poll 100 decoded WORSE at
250k (230–257) — longer verification rounds re-read more long-ctx
attention than the accepted token earns. No CUDA-graph flags exist in
this build, so launch-overhead tuning is off the table.
- **Concurrency caveat:** a 254k prefill beside a decode drops it
~240 -> ~100–135 t/s; one engine, no per-slot QoS.
- **Housekeeping during the sweep:** the earlier live service was
discovered to be an orphaned manual boot holding 7771 past the unit
stop (kill -9'd); the sweep now refuses to measure a port it does not
own. Peer frank-medium WIP was stash-held around the deploy and
self-committed as `23f0978`/`74dda30` — nothing lost.
- **Live after deploy:** `n_slots = 4, n_ctx_slot = 262144`, idle
23,300 MiB, e2e gate 32 passed.