frank-primary: drop --expert-precise-memory (boot crash loop with bf16)
NRestarts climbed to 3, each attempt ~6.7 min then 'Deactivated successfully' (API server drains after an EngineCore fatal error; Restart=always turned the silent death into a visible loop). The arena is device-resident, so the budget decides it: weights 6.28 + pool 6.80 (bf16 2 x 262144) + graphs 0.55 + arena 16.65 (precise overhead 5.25) + non-torch 1.45 = 31.73 of 31.86 GiB -> ~0.13 GiB free at boot, and graph capture is what allocates transiently. Dropping the flag restores the 7.60 GiB default overhead: arena 14.31 GiB, total ~29.4 GiB, ~2.4 GiB free. bf16 2 x 262144 (Frank's KV change) stays. The serve script keeps the opt-in flag; the seam now asserts it is ABSENT.
This commit is contained in:
1 parent
db54fa3af1
commit
d3d180fd3f
3 files changed
+48
-9
No files matched your search
@@ -663,6 +663,38 @@
|
||||
- `/tests/caddy_quadlet.bats` (new) + caddyfile/deployment_flow/pihole seams — quadlet boot persistence, self-healing, network, env, and sysctl coverage.
|
||||
- `/serverAppConfigs-docs/Infrastructure/Caddy.md`, `/serverAppConfigs-docs/Infrastructure/Pi-hole.md`, `/serverAppConfigs-docs/Dankclaw/Dankclaw Stack.md`, `/serverAppConfigs-docs/Dankclaw/Gitea.md`, `/serverAppConfigs-docs/Architecture/Deployment Flow.md` — quadlet runtime facts, the 2026-08-26 incident, the sysctl-conflict failure mode, canonical Caddy ownership.
|
||||
|
||||
## 2026-10-01 — frank-primary: --expert-precise-memory + bf16 = boot crash loop
|
||||
|
||||
**Reverted.** The two changes Frank asked for were applied together and the
|
||||
result crash-looped: `NRestarts` climbed 1 -> 2 -> 3, each attempt running
|
||||
~6.7 minutes then logging `Deactivated successfully` (vLLM's API server drains
|
||||
and exits 0 after an EngineCore fatal error, so `podman wait` returns 0; the
|
||||
new `Restart=always` turned a silent death into a visible loop).
|
||||
|
||||
The budget arithmetic explains it without needing the error text, because the
|
||||
arena is DEVICE-resident (the fp8 3-window shape's plan predicts 28.27 GiB and
|
||||
that is exactly what `rocm-smi` measured):
|
||||
|
||||
| line | bf16 2 x 262144 + precise | fp8 3 x 262144 |
|
||||
|---|---|---|
|
||||
| non-expert weights | 6.28 | 6.28 |
|
||||
| pinned pool | 6.80 | 5.68 |
|
||||
| CUDA graphs | 0.55 | 0.55 |
|
||||
| arena | 16.65 (precise: overhead 5.25) | 14.31 (default: overhead 7.60) |
|
||||
| non-torch | ~1.45 | ~1.45 |
|
||||
| **total** | **31.73 of 31.86 GiB** | 28.27 (measured) |
|
||||
| free at boot | **~0.13 GiB** | 3.59 |
|
||||
|
||||
`--expert-precise-memory` hands the 2.35 GiB compile/graph-capture headroom to
|
||||
the arena, and bf16 simultaneously adds 1.08 GiB of pinned pool; together they
|
||||
put graph capture ~0.13 GiB from the ceiling, where every transient allocation
|
||||
dies. So the flag is dropped and bf16 2 x 262144 ships with the default
|
||||
overhead: arena 14.31 GiB, ~2.4 GiB free at boot.
|
||||
|
||||
**Lesson:** the registry's "compile headroom" is not slack to be reclaimed by
|
||||
default — it is the budget for the phase that allocates last. Reclaim it only
|
||||
by paying for it elsewhere (smaller pool, fewer sessions, or a shorter context).
|
||||
|
||||
## 2026-10-01 — frank-primary: 2 bf16 windows + --expert-precise-memory
|
||||
|
||||
Frank's final shape for the night. Two changes at once, both measured in
|
||||
|
||||
Reference in new issue
Block a user