agent: repoint roles to frank-medium/dankclaw-3090; sync niri tweaks, memories, managed skills
- config.yml: default/commit -> frank-medium (dankserver); task/tiny/smol/vision -> frank-small (dankclaw-3090); vision xhigh - models.yml: add llamacpp (dankclaw-3090) provider; frank-large preserve_thinking + context notes - niri: focus-ring/border/shadow layout, hot-corners off, re-add noctalia.kdl include; blankpad eDP-1 scale 1.0 - memories: flush rollouts; retire frank_large + marreta_authentik skills; add auth-chain/kv-pool/quadlet-log skills - managed-skills: 40+ new fleet runbook skills - agent/skills: import built-in skill set
This commit is contained in:
1 parent
83c9c92b3f
commit
135bb2194d
103 files changed
+5333
-170
No files matched your search
@@ -0,0 +1,91 @@
|
||||
---
|
||||
name: fleet-container-ownership-audit
|
||||
description: "Audit a rootless-Podman fleet for containers that will not survive a reboot: owner-label detection, the stale-container and inspect-field traps, and the migration shape that fixes them."
|
||||
---
|
||||
|
||||
# Finding the containers that will not survive a reboot
|
||||
|
||||
Use when auditing a Podman fleet for boot persistence, when a service is
|
||||
"up" but stopped working at some past reboot, or when a compose-managed stack is
|
||||
being converted to Quadlet units. This is the *detection* half; the conversion
|
||||
checklist is separate.
|
||||
|
||||
## The one-line test
|
||||
|
||||
Every running container on a boot-persistent host should carry a
|
||||
`PODMAN_SYSTEMD_UNIT` label. Anything without one is compose-only or bare, and
|
||||
rootless Podman does not restart bare containers after the explicit stop that
|
||||
shutdown performs — so it will be missing tomorrow morning.
|
||||
|
||||
```bash
|
||||
for c in $(podman ps --format '{{.Names}}'); do
|
||||
printf '%-26s %s\n' "$c" "$(podman inspect $c --format '{{index .Config.Labels "PODMAN_SYSTEMD_UNIT"}}')"
|
||||
done
|
||||
```
|
||||
|
||||
Compare the result against the host's installed units
|
||||
(`ls ~/.config/containers/systemd/*.container`) and the repo's
|
||||
`<host>/<service>/quadlet/`. Three outcomes, three meanings:
|
||||
|
||||
- **label present** → the unit owns it; boot persistence comes from `[Install]
|
||||
WantedBy=default.target` plus `loginctl show-user <user> -p Linger` = yes.
|
||||
- **label absent, compose service exists** → the classic gap: it dies at reboot.
|
||||
Also check the publish address while you are there; compose forms often bind
|
||||
the wildcard, which puts an unauthenticated UI/API on every interface.
|
||||
- **label present AND a compose service with the same name** → a second owner.
|
||||
The runtime is fine, but a stray `podman compose up` can create a duplicate
|
||||
that fights the unit for the container name and the port. Remove the compose
|
||||
service and leave an explanatory comment; that is the repo's convention.
|
||||
|
||||
## Traps that make this hard to see
|
||||
|
||||
- **Editing a compose service does not replace a running container.**
|
||||
`restart: unless-stopped` will happily keep an old container — old image, old
|
||||
device list, old mounts — alive forever. One instance ran 9,146 restart
|
||||
attempts on stale config before anyone looked. If the container predates the
|
||||
edit, recreate it explicitly; do not debug the service.
|
||||
- **`podman inspect --format '{{.HostConfig.Devices}}'` is not a device check.**
|
||||
It can read empty on a container that demonstrably has the devices. Read the
|
||||
real argv with `{{.Config.CreateCommand}}` and, decisively, probe from inside:
|
||||
`podman exec <c> python3 -c 'import torch; print(torch.cuda.is_available())'`.
|
||||
- **A monitoring target's failure string can prove a fix.** A `/metrics` scrape
|
||||
that flips from `connection refused` to `unsupported Content-Type "text/html"`
|
||||
means the service came up and only the scrape is wrong. Read the error, not
|
||||
just up/down — and never "fix" a red probe by deleting the job, because an
|
||||
absent scrape and a healthy one look identical.
|
||||
- **A hand-written `~/.config/systemd/user/<svc>.service` that shells out to
|
||||
`podman compose up` collides by name with the Quadlet the generator produces
|
||||
for `<svc>.container`.** Retire it to a dated trash directory before
|
||||
installing the unit.
|
||||
- **`is-enabled` saying `generated` is the healthy state** for Quadlet units;
|
||||
`systemctl --user enable` fails on them.
|
||||
|
||||
## Migration shape that works
|
||||
|
||||
1. Unit file in the repo at `<host>/<service>/quadlet/<service>.container`:
|
||||
`Image=` with the `localhost/` prefix for locally built images, `Pull=never`,
|
||||
explicit `PublishPort` (loopback unless the service is the documented LAN
|
||||
exception), `AddDevice=`/`GroupAdd=keep-groups` where a GPU is needed,
|
||||
`Restart=always`, `[Install] WantedBy=default.target`.
|
||||
2. A healthcheck **as a file**, never inline: the podman 5.x v1 generator
|
||||
splits `HealthCmd` on spaces, so `python3 -c "..."` becomes a quoting puzzle
|
||||
that fails silently.
|
||||
3. `deploy_<svc>()` in the remote handler: install the unit, retire any
|
||||
compose-era unit into `~/.AGENTTRASH/`, `podman rm -f <container>` so the
|
||||
unit's own can take the name, daemon-reload, restart, wait for health.
|
||||
4. Delete the compose service (one owner per container name) and leave a comment
|
||||
naming the unit, so the next reader does not re-add it.
|
||||
5. Planner: add a path rule (`<host>/<dir>/**` -> service) **and** a
|
||||
`service_is_allowed` entry, or the shared dispatcher rejects the service
|
||||
before anything runs.
|
||||
6. Migrate the seams: tests that read the compose file for devices, command
|
||||
flags or groups must read the unit instead — and resolve file paths from the
|
||||
tooling's own declarations, never from `find ... | head -1`.
|
||||
|
||||
## When the last service leaves a compose file
|
||||
|
||||
The repo's compose-validity seam requires at least one service, so a file whose
|
||||
services have all migrated fails its own test. Either keep the file as
|
||||
comments and relax that seam deliberately, or delete the file and fix the
|
||||
planner's compose-diff path — decide with the owner, because the deploy tooling
|
||||
reads it.
|
||||
Reference in new issue
Block a user