--- name: rootless-quadlet-log-retrieval description: "Retrieve logs for a rootless Podman Quadlet service on the Dank fleet — which command covers which time window, and the sudo/per-user-storage trap" --- # Rootless Quadlet Log Retrieval (Dank fleet) Applies to any service deployed as a **rootless** Quadlet `.container` unit (frank-large on Dankclaw is the reference case). Rootless storage is per-user, so the deploy user matters more than the container name. ## Step 0 — establish ownership before choosing a command ```bash # Which manager owns the unit, and which user runs it? systemctl --user status .service # works => rootless user unit loginctl show-user -p Linger # Linger=yes => starts at boot ``` If `systemctl --user` knows the unit, **every** subsequent command runs as that user with NO `sudo`. ## Step 1 — pick the log source by time window | Need | Command | |---|---| | Current instance, tail | `podman logs --tail 200 ` | | Current instance, follow | `podman logs -f ` | | Across restarts/recreates | `journalctl --user -u .service -n 200 --no-pager` | | Time-bounded | `journalctl --user -u .service --since "14:30"` | | Crash/restart signal only | `journalctl --user -u .service -p warning` | | Kernel-side evidence (OOM, SVM) | `sudo journalctl -k --since ...` | Quadlet units commonly run `--replace --rm`. Consequence: **`podman logs` is destroyed when the container is recreated**; the journal captured `podman run`'s stdout and is the only durable record. When the question is "what happened before the last restart", go straight to `journalctl --user`. ## Step 2 — the traps - `sudo podman logs ` → `Error: no such container`. Root's store does not contain rootless containers. Same for `sudo podman ps`. - A system-level `journalctl -u ` returns nothing for a user unit — the `-u` must be paired with `--user`. - During a first-start weight download, health checks report unhealthy **by design**. Do not read that as a crash; check whether the payload file is still growing. - Vendor verbose switches (e.g. `HALOGEN_VERBOSE=1`, `HALOGEN_DMALLOC_LOG=1`) are a **config change**: they go through git and `deploy.sh`, never a live env poke on the host. ## Step 3 — if you cannot reach the host No SSH transport is configured on the `blank` dev box. Do not probe `dank@`/`frank@` and hope. Ask Frank for a key or alias, and label any answer derived only from repo files as **not live-verified**.