fix(open-notebook): the database probe has to be exec-form JSON
The service's first-ever deploy failed at its own gate:
=== Deploying open-notebook at 9ffa999 ===
… pulled both pinned images …
FAIL: open-notebook-surrealdb did not become healthy in 90s (status: starting)
Cause, measured on the host. podman 5.7 wraps a bare HealthCmd string as CMD-SHELL — --health-cmd "/surreal is-ready" is stored as {"Test":["CMD-SHELL","/surreal is-ready"]} — and the pinned surrealdb:v2@sha256:e5d7387e… image is distroless: podman exec … ls answers exec: "ls": executable file not found in $PATH, so there is no /bin/sh to
run the probe. Every interval exited 1 in ~30 ms with empty output while /surreal is-ready itself printed OK and the server logged Started web server on 0.0.0.0:8000. That mismatch — a probe that always fails
beside a healthy database — is what the unit comment asserted could not happen
("Argument-only HealthCmd, like every unit on this host, because the quadlet
generator splits it on spaces"): the generator does not split it, podman shells
it.
Fix.HealthCmd=["/surreal","is-ready"]. Quadlet quotes it through the
generator (--health-cmd "[\"/surreal\",\"is-ready\"]"), podman stores ["CMD","/surreal","is-ready"], and a throwaway container of the pinned digest
on the same host went healthy on that form (manual podman healthcheck run
exit 0). This is the shell-free answer for a distroless image whose healthcheck
is a binary in the image — loki's units still can't declare one (its health
command lives in a purpose-built image), but here the image already ships the
probe and only the invocation shape was wrong.
Seam. The old assertion forbade any quote character in any HealthCmd and
pinned HealthCmd=/surreal is-ready — it blessed the defect. Re-pinned to the
contract that matters: the database probe must be runnable with no shell; the API
probe (its image does ship curl and a shell) must name /health on the container
port. bats tests/open_notebook.bats -f "database probe" → ok. The card seam
(stays hidden until the service is deployed) still fails by design: the portal
flip lands in the follow-up PR with the guard chain.
Not in this PR: the vhost, the guard chain, the single-origin API_URL, the
homepage card. Those are the second PR; this one only makes the service
deployable.
# fix(open-notebook): the database probe has to be exec-form JSON
The service's first-ever deploy failed at its own gate:
```
=== Deploying open-notebook at 9ffa999 ===
… pulled both pinned images …
FAIL: open-notebook-surrealdb did not become healthy in 90s (status: starting)
```
**Cause, measured on the host.** podman 5.7 wraps a bare `HealthCmd` string as
`CMD-SHELL` — `--health-cmd "/surreal is-ready"` is stored as
`{"Test":["CMD-SHELL","/surreal is-ready"]}` — and the pinned
`surrealdb:v2@sha256:e5d7387e…` image is distroless: `podman exec … ls` answers
`exec: "ls": executable file not found in $PATH`, so there is no `/bin/sh` to
run the probe. Every interval exited 1 in ~30 ms with empty output while
`/surreal is-ready` itself printed `OK` and the server logged
`Started web server on 0.0.0.0:8000`. That mismatch — a probe that always fails
beside a healthy database — is what the unit comment asserted could not happen
("Argument-only HealthCmd, like every unit on this host, because the quadlet
generator splits it on spaces"): the generator does not split it, podman shells
it.
**Fix.** `HealthCmd=["/surreal","is-ready"]`. Quadlet quotes it through the
generator (`--health-cmd "[\"/surreal\",\"is-ready\"]"`), podman stores
`["CMD","/surreal","is-ready"]`, and a throwaway container of the pinned digest
on the same host went **healthy** on that form (manual `podman healthcheck run`
exit 0). This is the shell-free answer for a distroless image whose healthcheck
is a binary in the image — [[loki]]'s units still can't declare one (its health
command lives in a purpose-built image), but here the image already ships the
probe and only the invocation shape was wrong.
**Seam.** The old assertion forbade any quote character in any `HealthCmd` and
pinned `HealthCmd=/surreal is-ready` — it blessed the defect. Re-pinned to the
contract that matters: the database probe must be runnable with no shell; the API
probe (its image does ship curl and a shell) must name `/health` on the container
port. `bats tests/open_notebook.bats -f "database probe"` → ok. The card seam
(`stays hidden until the service is deployed`) still fails by design: the portal
flip lands in the follow-up PR with the guard chain.
**Not in this PR:** the vhost, the guard chain, the single-origin `API_URL`, the
homepage card. Those are the second PR; this one only makes the service
deployable.
---
cwd: /home/blank/.omp/wt/wt-20261019-020309-6d1fc1f · session: 01a15366-7601-7175-bb69-c62186c3f8b5
podman wraps a bare HealthCmd as CMD-SHELL (measured on Dankclaw's podman 5.7:
--health-cmd "/surreal is-ready" stores {"Test":["CMD-SHELL","/surreal is-ready"]}),
and the pinned SurrealDB v2 image is distroless — no /bin/sh, not even `ls`.
So the probe could never run: every 30 s interval exited 1 in ~30 ms with empty
output while the database was listening and answering, and open-notebook's first
deploy died at the gate ("did not become healthy in 90s"). The hand-check
`podman exec … /surreal is-ready` printed OK, which is why the old comment
believed the probe worked.
The JSON array survives the Quadlet generator (--health-cmd "[\"/surreal\",\"is-ready\"]")
and stores ["CMD","/surreal","is-ready"], which execs directly: a throwaway
instance of the pinned digest went healthy on that form. The seam is re-pinned
to the shell-free contract; the old assertion forbade quotes in any HealthCmd,
which is what pinned the broken shape.
dank
merged commit ee6d8610b3 into main2026-10-19 16:07:54 -07:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
fix(open-notebook): the database probe has to be exec-form JSON
The service's first-ever deploy failed at its own gate:
Cause, measured on the host. podman 5.7 wraps a bare
HealthCmdstring asCMD-SHELL—--health-cmd "/surreal is-ready"is stored as{"Test":["CMD-SHELL","/surreal is-ready"]}— and the pinnedsurrealdb:v2@sha256:e5d7387e…image is distroless:podman exec … lsanswersexec: "ls": executable file not found in $PATH, so there is no/bin/shtorun the probe. Every interval exited 1 in ~30 ms with empty output while
/surreal is-readyitself printedOKand the server loggedStarted web server on 0.0.0.0:8000. That mismatch — a probe that always failsbeside a healthy database — is what the unit comment asserted could not happen
("Argument-only HealthCmd, like every unit on this host, because the quadlet
generator splits it on spaces"): the generator does not split it, podman shells
it.
Fix.
HealthCmd=["/surreal","is-ready"]. Quadlet quotes it through thegenerator (
--health-cmd "[\"/surreal\",\"is-ready\"]"), podman stores["CMD","/surreal","is-ready"], and a throwaway container of the pinned digeston the same host went healthy on that form (manual
podman healthcheck runexit 0). This is the shell-free answer for a distroless image whose healthcheck
is a binary in the image — loki's units still can't declare one (its health
command lives in a purpose-built image), but here the image already ships the
probe and only the invocation shape was wrong.
Seam. The old assertion forbade any quote character in any
HealthCmdandpinned
HealthCmd=/surreal is-ready— it blessed the defect. Re-pinned to thecontract that matters: the database probe must be runnable with no shell; the API
probe (its image does ship curl and a shell) must name
/healthon the containerport.
bats tests/open_notebook.bats -f "database probe"→ ok. The card seam(
stays hidden until the service is deployed) still fails by design: the portalflip lands in the follow-up PR with the guard chain.
Not in this PR: the vhost, the guard chain, the single-origin
API_URL, thehomepage card. Those are the second PR; this one only makes the service
deployable.
cwd: /home/blank/.omp/wt/wt-20261019-020309-6d1fc1f · session: 01a15366-7601-7175-bb69-c62186c3f8b5
podman wraps a bare HealthCmd as CMD-SHELL (measured on Dankclaw's podman 5.7: --health-cmd "/surreal is-ready" stores {"Test":["CMD-SHELL","/surreal is-ready"]}), and the pinned SurrealDB v2 image is distroless — no /bin/sh, not even `ls`. So the probe could never run: every 30 s interval exited 1 in ~30 ms with empty output while the database was listening and answering, and open-notebook's first deploy died at the gate ("did not become healthy in 90s"). The hand-check `podman exec … /surreal is-ready` printed OK, which is why the old comment believed the probe worked. The JSON array survives the Quadlet generator (--health-cmd "[\"/surreal\",\"is-ready\"]") and stores ["CMD","/surreal","is-ready"], which execs directly: a throwaway instance of the pinned digest went healthy on that form. The seam is re-pinned to the shell-free contract; the old assertion forbade quotes in any HealthCmd, which is what pinned the broken shape.