frank-dragoon boot-looped 88 times tonight (chat dead, GPU lines healthy). The NPU lane's child dlopens libxrt_core.so.2 from a third prefix — /opt/xilinx/lib/<multiarch>/ — that our mounts never filled. Each distro XRT library now mounts three times from the same REAL file; multiarch derives from FRANK_DRAGOON_XRT_DIR's basename (the bats fixture proves the derivation).
Signature (recognise it next time)
halogen-npu: No such library '/opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2'
halogen npu: the NPU engine exited before it was ready (rc=1; ...)
followed by exit-1 in ~3 s every RestartSec. A dead lane is not survivable on this engine — the supervisor takes the whole container down; the GPU/serve_api lines look healthy beside it, which is how it reads like an unrelated fault.
Why now
The lane ran fine 2026-10-19 with byte-identical mounts (the child found XRT another way); the host's libxrt2/libxrt-npu2 were reconfigured 2026-10-20 12:35 (dpkg.log) — the only mover. Proof before commit: production-shaped scratch boot + three extra mounts → full boot ledger (62.9 GiB weights, 11.0 GiB KV @401408, no clamp), npu_api: qwen3-embedding-0.6b … buckets u0…u9, /health ok at the 262,144 ceiling.
Change / verification
scripts/serve-frank-dragoon.sh: third mount per lib, comment carries the signature.
scripts/frank-dragoon.env + vault page: mount law now "three prefixes per library".
Seam: "each XRT library three times" (+ fixture renamed to a multiarch-shaped dir so the derivation is observable from output).
./tests/run.sh: 490/490 green.
Landing this and re-deploying retires tonight's scratch instance (/tmp/dragoon-npu-test.sh, auto-exiting) — the scratch env/container are removed in the same session and the deploy runs the sanctioned unit path with full gates.
## What
frank-dragoon boot-looped 88 times tonight (chat dead, GPU lines healthy). The NPU lane's child `dlopen`s `libxrt_core.so.2` from a **third prefix** — `/opt/xilinx/lib/<multiarch>/` — that our mounts never filled. Each distro XRT library now mounts three times from the same REAL file; multiarch derives from `FRANK_DRAGOON_XRT_DIR`'s basename (the bats fixture proves the derivation).
## Signature (recognise it next time)
```
halogen-npu: No such library '/opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2'
halogen npu: the NPU engine exited before it was ready (rc=1; ...)
```
followed by exit-1 in ~3 s every `RestartSec`. **A dead lane is not survivable on this engine** — the supervisor takes the whole container down; the GPU/serve_api lines look healthy beside it, which is how it reads like an unrelated fault.
## Why now
The lane ran fine 2026-10-19 with byte-identical mounts (the child found XRT another way); the host's `libxrt2`/`libxrt-npu2` were reconfigured **2026-10-20 12:35** (dpkg.log) — the only mover. Proof before commit: production-shaped scratch boot + three extra mounts → full boot ledger (`62.9 GiB weights, 11.0 GiB KV @401408, no clamp`), `npu_api: qwen3-embedding-0.6b … buckets u0…u9`, `/health ok` at the 262,144 ceiling.
## Change / verification
- `scripts/serve-frank-dragoon.sh`: third mount per lib, comment carries the signature.
- `scripts/frank-dragoon.env` + vault page: mount law now "three prefixes per library".
- Seam: "each XRT library three times" (+ fixture renamed to a multiarch-shaped dir so the derivation is observable from output).
- `./tests/run.sh`: 490/490 green.
Landing this and re-deploying retires tonight's scratch instance (`/tmp/dragoon-npu-test.sh`, auto-exiting) — the scratch env/container are removed in the same session and the deploy runs the sanctioned unit path with full gates.
Follows #65 (ceiling 262,144 / 200k window), already merged; branch carries both.
---
cwd: /home/blank/.omp/wt/wt-20261020-125416-6d1fc1f · session: 01a15d93-efb8-7108-b94d-4fabb14597ef
The lane's child dlopens libxrt_core.so.2 from /opt/xilinx/lib/<multiarch>/
as well; our mounts filled only /opt/xilinx/xrt/lib and the install path.
Since the host's libxrt2 was reconfigured 2026-10-20 the child dies naming
the missing prefix ('No such library
/opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2') and the engine
supervisor takes the WHOLE container down on a dead lane — measured
2026-10-21: frank-dragoon boot-looped 88 times with chat dead while the
GPU lines looked healthy.
Each XRT library now mounts THREE times from the same real file (never a
symlink); the multiarch name derives from FRANK_DRAGOON_XRT_DIR's basename
so the bats fixture proves the derivation. Proven end-to-end before commit:
production-shaped scratch boot + the three extra mounts answers the full
boot ledger, npu_api buckets u0..u9, and /health ok at the 262144 ceiling.
A dead lane is not survivable on this engine — arm it only with all three
prefixes (page law updated).
dank
merged commit ccb805cf0b into main2026-10-21 02:25:42 -07:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
What
frank-dragoon boot-looped 88 times tonight (chat dead, GPU lines healthy). The NPU lane's child
dlopenslibxrt_core.so.2from a third prefix —/opt/xilinx/lib/<multiarch>/— that our mounts never filled. Each distro XRT library now mounts three times from the same REAL file; multiarch derives fromFRANK_DRAGOON_XRT_DIR's basename (the bats fixture proves the derivation).Signature (recognise it next time)
followed by exit-1 in ~3 s every
RestartSec. A dead lane is not survivable on this engine — the supervisor takes the whole container down; the GPU/serve_api lines look healthy beside it, which is how it reads like an unrelated fault.Why now
The lane ran fine 2026-10-19 with byte-identical mounts (the child found XRT another way); the host's
libxrt2/libxrt-npu2were reconfigured 2026-10-20 12:35 (dpkg.log) — the only mover. Proof before commit: production-shaped scratch boot + three extra mounts → full boot ledger (62.9 GiB weights, 11.0 GiB KV @401408, no clamp),npu_api: qwen3-embedding-0.6b … buckets u0…u9,/health okat the 262,144 ceiling.Change / verification
scripts/serve-frank-dragoon.sh: third mount per lib, comment carries the signature.scripts/frank-dragoon.env+ vault page: mount law now "three prefixes per library"../tests/run.sh: 490/490 green.Landing this and re-deploying retires tonight's scratch instance (
/tmp/dragoon-npu-test.sh, auto-exiting) — the scratch env/container are removed in the same session and the deploy runs the sanctioned unit path with full gates.Follows #65 (ceiling 262,144 / 200k window), already merged; branch carries both.
cwd: /home/blank/.omp/wt/wt-20261020-125416-6d1fc1f · session: 01a15d93-efb8-7108-b94d-4fabb14597ef
The lane's child dlopens libxrt_core.so.2 from /opt/xilinx/lib/<multiarch>/ as well; our mounts filled only /opt/xilinx/xrt/lib and the install path. Since the host's libxrt2 was reconfigured 2026-10-20 the child dies naming the missing prefix ('No such library /opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2') and the engine supervisor takes the WHOLE container down on a dead lane — measured 2026-10-21: frank-dragoon boot-looped 88 times with chat dead while the GPU lines looked healthy. Each XRT library now mounts THREE times from the same real file (never a symlink); the multiarch name derives from FRANK_DRAGOON_XRT_DIR's basename so the bats fixture proves the derivation. Proven end-to-end before commit: production-shaped scratch boot + the three extra mounts answers the full boot ledger, npu_api buckets u0..u9, and /health ok at the 262144 ceiling. A dead lane is not survivable on this engine — arm it only with all three prefixes (page law updated).