docs(frank-archon): work-log entry and the 4-way capacity probe
Probe on the live service: four concurrent 196,960-token chats, 4/4 exact marker retrieval, 787,842 tokens in 238.4 s (3,305 t/s aggregate prefill = the single-stream ceiling, so the four streams shared one pipeline and the KV pool never ran short); prefix-cache repeat answered in 1.0 s with cache_n=43008. Recorded with the caveat that 8 generated tokens do not make a decode rate.
This commit is contained in:
@@ -385,6 +385,33 @@ Oct 13 19:21:08 dankserver systemd[1]: Started frank-archon.service
|
||||
the boot, i.e. the root `ExecStartPre` hook did the write the service process
|
||||
(User=dank) cannot.
|
||||
|
||||
### Capacity probe (2026-10-13, live service)
|
||||
|
||||
`scripts/probe-frank-archon-kv.py` now reads the slot count, window and KV dtype
|
||||
from the deployed env on the host, so the probe cannot drift from the service:
|
||||
it printed `shape: 4 x 200,000 tokens, kv fp8` and fired **four concurrent chats
|
||||
of 196,960 prompt tokens each** (646,299 chars, 98.5 % of the window), each
|
||||
carrying a marker whose first word is unique.
|
||||
|
||||
| Stream | prompt | wall | prefill | retrieval |
|
||||
|---|---|---|---|---|
|
||||
| B1 | 196,961 | 238.3 s | 1,423 t/s | exact, HIT |
|
||||
| B2 | 196,960 | 98.0 s | 2,043 t/s | exact, HIT |
|
||||
| B3 | 196,961 | 201.1 s | 993 t/s | exact, HIT |
|
||||
| B4 | 196,960 | 149.9 s | 1,327 t/s | exact, HIT |
|
||||
| **total** | **787,842** | **238.4 s** | **3,305 t/s aggregate** | 4/4 |
|
||||
|
||||
All four answered their own marker; nothing was preempted, refused, or truncated
|
||||
(`finish=stop`, `cache_n=0`, so no prefix cache padded the result). Aggregate
|
||||
prefill equals the single-stream rate measured in the same probe (3,397 t/s at
|
||||
43.5k), i.e. the four streams share one prefill pipeline at the slab-limited
|
||||
ceiling and the engine never ran short of KV. Prefix cache intact: the same
|
||||
43.5k prompt repeated answered in 1.0 s with `cache_n=43008`.
|
||||
|
||||
The per-stream `decode` column is NOT a decode rate here — 8 generated tokens
|
||||
make it first-token dominated (the bench law: generate ≥ 128 tokens to read a
|
||||
rate). This probe answers one question only: does the endpoint owe four
|
||||
full-height conversations at once, and answer each one exactly. It does.
|
||||
|
||||
## Gates (measured at cutover)
|
||||
|
||||
|
||||
@@ -1,5 +1,44 @@
|
||||
# Bundle Change Log
|
||||
|
||||
## 2026-10-13
|
||||
- **frank-archon outage: a kernel knob, not the model.** The 2026-10-12 23:17
|
||||
power failure on dankserver reset `/sys/module/ttm/parameters/pages_limit`
|
||||
from the manual 200 GiB raise back to its default (half of RAM = 125,988 MiB),
|
||||
below the ~147 GiB pinned-RAM ledger the engine needs; the serve script's
|
||||
preflight named the refusal and exited 1, and `Restart=always` re-ran it
|
||||
**6,919 times in ~20 h** — `activating (auto-restart)`, exit 1 in 50 ms, no
|
||||
container, no engine log. Law adopted: **a sysfs raise is a countdown to the
|
||||
next reboot**, so the gate raises the knob itself. New
|
||||
`scripts/ttm-preflight.sh` is sourced by the serve script (verify, raise,
|
||||
re-verify, then refuse by name) and is also the unit's
|
||||
`ExecStartPre=+…--raise` root hook — `+` because the service is `User=dank`
|
||||
and the knob is root-owned 0644, which is what the first attempt (a plain
|
||||
write inside the serve script) got wrong. `FRANK_ARCHON_TTM_TARGET_MIB=204800`
|
||||
records the host ceiling.
|
||||
- **Shape: 4 × 200,000 with fp8 KV (Frank's call).** `--max-num-seqs 4`,
|
||||
`--kv-cache-dtype fp8`, pool and slab untouched (6,912 / 22,208 MiB = the
|
||||
measured 29,120/card ceiling): the pool must back `seqs × max-model-len`, and
|
||||
at bf16 four slots cost 11,679 MiB, which would cut the resident expert slab
|
||||
to 17,441 (−33 %) — fp8 costs 6,601 and leaves it alone. 4 × (1 + spec 2) = 12
|
||||
fused-all-reduce rows, inside the 25-row bound where `wht6` is bit-exact.
|
||||
`models.yml` needed no change: the per-request window is unchanged, only the
|
||||
concurrency.
|
||||
- **Deployed at `bae78e7`, gates green.** Boot ledger: `kv cache 6.75 GiB …
|
||||
backs 4 sequences all at 200000 tokens at once` — 843,276 tokens against the
|
||||
sizing law's 843,566 (0.03 %), sampler at 12 rows, weights filled to 147 KiB
|
||||
of slack, cards ~2 GiB free, KV lending 5.87 GiB to the slab. Live probe
|
||||
(now reading the shape from the deployed env): four concurrent 196,960-token
|
||||
chats, **4/4 exact marker retrieval**, 787,842 tokens in 238.4 s = 3,305 t/s
|
||||
aggregate prefill, no preemption; prefix cache verified (repeat answered in
|
||||
1.0 s, `cache_n=43008`). Playwright e2e 32 passed + 1 flaky (the known Plex
|
||||
homepage retry); archon bats seam 21/21 including three new ttm-gate tests.
|
||||
- **Power-failure casualties, NOT fixed (outside the archon window):**
|
||||
`mnt-md0.mount` failed at boot, so `/mnt/md0` is an empty mountpoint (the
|
||||
raid6 array itself is healthy, 7/7; fstab has `nofail` so boot continued) —
|
||||
media libraries on it read empty. `frank-job-digest.service` (frankbook
|
||||
job-hunt digest) is in `list-units --state=failed`.
|
||||
|
||||
|
||||
## 2026-10-10
|
||||
- **Fleet service rename: the StarCraft names became the SERVICE names.**
|
||||
The 2026-10-09 wire alias moved onto every repo artifact: units, serve
|
||||
|
||||
Reference in New Issue
Block a user