frank-primary: drop --expert-precise-memory (boot crash loop with bf16)

NRestarts climbed to 3, each attempt ~6.7 min then 'Deactivated successfully'
(API server drains after an EngineCore fatal error; Restart=always turned the
silent death into a visible loop). The arena is device-resident, so the budget
decides it: weights 6.28 + pool 6.80 (bf16 2 x 262144) + graphs 0.55 + arena
16.65 (precise overhead 5.25) + non-torch 1.45 = 31.73 of 31.86 GiB -> ~0.13
GiB free at boot, and graph capture is what allocates transiently.

Dropping the flag restores the 7.60 GiB default overhead: arena 14.31 GiB,
total ~29.4 GiB, ~2.4 GiB free. bf16 2 x 262144 (Frank's KV change) stays.
The serve script keeps the opt-in flag; the seam now asserts it is ABSENT.
This commit is contained in:
Frank committed 2026-10-01 14:16:47 -07:00
1 parent db54fa3af1
commit d3d180fd3f
3 files changed
+48 -9

No files matched your search

+32
View File
@@ -663,6 +663,38 @@
- `/tests/caddy_quadlet.bats` (new) + caddyfile/deployment_flow/pihole seams — quadlet boot persistence, self-healing, network, env, and sysctl coverage.
- `/serverAppConfigs-docs/Infrastructure/Caddy.md`, `/serverAppConfigs-docs/Infrastructure/Pi-hole.md`, `/serverAppConfigs-docs/Dankclaw/Dankclaw Stack.md`, `/serverAppConfigs-docs/Dankclaw/Gitea.md`, `/serverAppConfigs-docs/Architecture/Deployment Flow.md` — quadlet runtime facts, the 2026-08-26 incident, the sysctl-conflict failure mode, canonical Caddy ownership.
## 2026-10-01 — frank-primary: --expert-precise-memory + bf16 = boot crash loop
**Reverted.** The two changes Frank asked for were applied together and the
result crash-looped: `NRestarts` climbed 1 -> 2 -> 3, each attempt running
~6.7 minutes then logging `Deactivated successfully` (vLLM's API server drains
and exits 0 after an EngineCore fatal error, so `podman wait` returns 0; the
new `Restart=always` turned a silent death into a visible loop).
The budget arithmetic explains it without needing the error text, because the
arena is DEVICE-resident (the fp8 3-window shape's plan predicts 28.27 GiB and
that is exactly what `rocm-smi` measured):
| line | bf16 2 x 262144 + precise | fp8 3 x 262144 |
|---|---|---|
| non-expert weights | 6.28 | 6.28 |
| pinned pool | 6.80 | 5.68 |
| CUDA graphs | 0.55 | 0.55 |
| arena | 16.65 (precise: overhead 5.25) | 14.31 (default: overhead 7.60) |
| non-torch | ~1.45 | ~1.45 |
| **total** | **31.73 of 31.86 GiB** | 28.27 (measured) |
| free at boot | **~0.13 GiB** | 3.59 |
`--expert-precise-memory` hands the 2.35 GiB compile/graph-capture headroom to
the arena, and bf16 simultaneously adds 1.08 GiB of pinned pool; together they
put graph capture ~0.13 GiB from the ceiling, where every transient allocation
dies. So the flag is dropped and bf16 2 x 262144 ships with the default
overhead: arena 14.31 GiB, ~2.4 GiB free at boot.
**Lesson:** the registry's "compile headroom" is not slack to be reclaimed by
default — it is the budget for the phase that allocates last. Reclaim it only
by paying for it elsewhere (smaller pool, fewer sessions, or a shorter context).
## 2026-10-01 — frank-primary: 2 bf16 windows + --expert-precise-memory
Frank's final shape for the night. Two changes at once, both measured in