docs: the max-length sweep tabled; frank-small.env NGL/FLASH_ATTN restored
- runbook: identity now 4x262144, audio verified, throughput table with the OOM and negative-result rows, deployed flags updated to reality. - AGENTS.md frank-small paragraph matches the deployed shape. - log.md: the sweep entry. - scripts/frank-small.env: my earlier line-range edit had clobbered NGL=999 and FLASH_ATTN=on out of the file; the seams caught it. The live service was unaffected because the script's defaults match the declared values exactly, but the file contract is restored. Suite: ALL TESTS PASSED (321), vault 125 pages 0 unresolved.
This commit is contained in:
1 parent
24eaba1078
commit
17450dd172
4 files changed
+102
-34
No files matched your search
@@ -820,3 +820,31 @@ timestamp: 2026-09-14
|
||||
set` error is `--fit`'s own probe — normal, self-recovers seconds later.
|
||||
- Runbook: [[frank-small-gemma4-12b]] (active);
|
||||
[[frank-small-qwen38-gsq-rco]] retired 2026-09-17 as rollback target.
|
||||
|
||||
## 2026-09-17 (later) — frank-small: four full windows, max-length throughput sweep
|
||||
|
||||
- **Slot ceiling measured:** 1 full-ctx slot idles 11,996 MiB, 4 idle
|
||||
22,346 MiB -> 3,450 MiB/slot, ~8,546 fixed; a 5th projects past the
|
||||
24,576 MiB card. Deployed `CTX_SIZE=1048576` / `PARALLEL=4` (each
|
||||
slot the FULL 262,144 window; explicit CTX bypasses `--fit` on
|
||||
purpose — fit would divide the pool at parallel>1). The bats seam now
|
||||
pins the division and the budget inequality.
|
||||
- **Audio verified:** JFK WAV via `input_audio` -> verbatim transcript.
|
||||
- **Batch sweep at max length** (salted API probes, pp ~254k /
|
||||
tg @ ~253k ctx): ubatch 512 -> pp 912–924, tg 90–204;
|
||||
**ubatch 1024 -> pp 962–965, tg 226–302 (deployed)**;
|
||||
ubatch 2048 -> OOM in the MTP draft's pp compute buffer (~3.3 GiB
|
||||
vs 2.2 GiB headroom).
|
||||
- **Negative results kept:** draft n-max 4 + poll 100 decoded WORSE at
|
||||
250k (230–257) — longer verification rounds re-read more long-ctx
|
||||
attention than the accepted token earns. No CUDA-graph flags exist in
|
||||
this build, so launch-overhead tuning is off the table.
|
||||
- **Concurrency caveat:** a 254k prefill beside a decode drops it
|
||||
~240 -> ~100–135 t/s; one engine, no per-slot QoS.
|
||||
- **Housekeeping during the sweep:** the earlier live service was
|
||||
discovered to be an orphaned manual boot holding 7771 past the unit
|
||||
stop (kill -9'd); the sweep now refuses to measure a port it does not
|
||||
own. Peer frank-medium WIP was stash-held around the deploy and
|
||||
self-committed as `23f0978`/`74dda30` — nothing lost.
|
||||
- **Live after deploy:** `n_slots = 4, n_ctx_slot = 262144`, idle
|
||||
23,300 MiB, e2e gate 32 passed.
|
||||
Reference in new issue
Block a user