docs(frank-archon): record the crash signature — clean-looking exit, silent engine #147

Merged
dank merged 1 commits from docs/frank-archon-crash-signature into main 2026-11-04 13:19:50 -08:00
+15
View File
@@ -525,6 +525,21 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
- `--prefix-cache-dir` creation is **not recursive** — the serve script `mkdir -p`s the path or
the disk tier silently never opens. The tier does survive a payload swap when arch and
tokenizer match (it recovered 900,176 blocks on the 2026-10-15 swap; `cache_n` proves a hit).
- **A radiance crash reads as a clean exit unless you know where to look, and the engine never
says why.** The unit's `ExecStart` ends in `exec podman wait`, so `systemctl show -p
ExecMainStatus` reports **0** after a container death — that is `podman wait`'s own status, not
the engine's — and `Result=success` beside `NRestarts=1` is the *normal* fingerprint of a
crash. The container's real code is the bare digit `podman wait` prints to stdout
(`serve-frank-archon.sh[NNNN]: 1`). The engine's fatal path writes **nothing** to its log: no
stderr line, no message before the exit and no core. The only copy of the reason is the error
the client received — `the engine stopped on an error and is not serving (<errno>)`, where the
parenthesis is the failing call's errno text (`invalid argument` = EINVAL, 2026-11-04). So
grep the *client* logs before concluding the journal is empty: omp keeps the provider error in
`~/.omp/logs/omp.<date>.<pid>.log` (`agent turn ended with provider error`), and OpenWebUI
shows it as the chat's error toast. Seen once (2026-11-04, restart counter 1 after 3d20h up):
three concurrent triple-digit-thousand-token agent turns prefilled at once, prefill rate
collapsed, all in-flight streams failed at the same instant, then a silent teardown — kernel
logged only the KFD SVM workqueue churning, with no OOM, GPU hang, MCE or swap pressure.
### Dead ends — do not re-try