agent: repoint roles to frank-medium/dankclaw-3090; sync niri tweaks, memories, managed skills
- config.yml: default/commit -> frank-medium (dankserver); task/tiny/smol/vision -> frank-small (dankclaw-3090); vision xhigh - models.yml: add llamacpp (dankclaw-3090) provider; frank-large preserve_thinking + context notes - niri: focus-ring/border/shadow layout, hot-corners off, re-add noctalia.kdl include; blankpad eDP-1 scale 1.0 - memories: flush rollouts; retire frank_large + marreta_authentik skills; add auth-chain/kv-pool/quadlet-log skills - managed-skills: 40+ new fleet runbook skills - agent/skills: import built-in skill set
This commit is contained in:
1 parent
83c9c92b3f
commit
135bb2194d
103 files changed
+5333
-170
No files matched your search
@@ -0,0 +1,88 @@
|
||||
---
|
||||
name: published-port-cutover-gate
|
||||
description: "When replacing or restarting a service that publishes a TCP port on this rootless-podman/pasta fleet, gate on the port actually being free first — otherwise pasta's bind fails at container create and the failure hides behind Restart=on-failure."
|
||||
---
|
||||
|
||||
# Published-port cutover gate (rootless podman / pasta)
|
||||
|
||||
On hosts where **rootless podman publishes ports through pasta**, the published
|
||||
port is bound in userspace at **container CREATE**, not when the app binds. A
|
||||
predecessor process that has just been stopped can still hold that port —
|
||||
listener, accepted connection, or TIME_WAIT residue from the connections it
|
||||
served — so a replacement started immediately after `systemctl stop` dies with:
|
||||
|
||||
```
|
||||
pasta: ... bind: Address already in use
|
||||
conmon ... exit status 126
|
||||
```
|
||||
|
||||
and because the unit is usually `Restart=on-failure`/`RestartSec=10`, it
|
||||
**self-heals seconds-to-minutes later**. That is the trap: the deploy looks
|
||||
like it failed, then looks fine, and the real defect (no port gate) stays
|
||||
invisible because the retry masks it.
|
||||
|
||||
Observed on Dankclaw 2026-09-16 cutting `frank-large` from a root unit
|
||||
(ik_llama.cpp) to a rootless Quadlet (halogen): `systemd` logged
|
||||
"Deactivated successfully", and pasta tried to bind 7770 about a second later
|
||||
and reported "Address already in use". Only the quadlet's own restart policy
|
||||
recovered it.
|
||||
|
||||
## Procedure
|
||||
|
||||
Before starting the replacement:
|
||||
|
||||
1. **Retire the old unit and wait for it to be genuinely inactive.**
|
||||
`systemctl stop <old>` (root) or `systemctl --user stop <old>`.
|
||||
2. **Stop the NEW unit too, if this is a redeploy.**
|
||||
`systemctl --user stop <new>.service || true`
|
||||
|
||||
Without this the gate deadlocks against itself: the currently-running
|
||||
replacement is the thing holding the port. This mistake was made and caught
|
||||
in the same session — a gate placed before the stop would have failed every
|
||||
subsequent deploy of an already-running service.
|
||||
3. **Gate on the port being empty, covering every socket state:**
|
||||
|
||||
```bash
|
||||
wait_port_free() {
|
||||
local port="$1" limit="${2:-90}" i
|
||||
for ((i = 1; i <= limit; i++)); do
|
||||
if [[ -z "$(ss -Htan "sport = :$port" 2>/dev/null)" ]]; then
|
||||
return 0
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
echo "FAIL: port $port is still held after ${limit}s:" >&2
|
||||
ss -tan "sport = :$port" >&2 || true
|
||||
return 1
|
||||
}
|
||||
```
|
||||
|
||||
`sport` matches both the listener and the server side of accepted
|
||||
connections, so this also catches TIME_WAIT residue that a plain
|
||||
`ss -ltn` check misses. Fail loudly with the socket table attached; a deploy
|
||||
that starts a doomed container has not deployed anything.
|
||||
4. **Then** install/pull and start, and wait on the service's health endpoint.
|
||||
|
||||
## Verifying the gate has teeth
|
||||
|
||||
- Re-run the same deploy: it must pass with the service already running (that
|
||||
is the idempotency case the self-deadlock broke).
|
||||
- Reproduce the failure once on purpose if the service is not user-facing:
|
||||
stop the container and start the replacement with the gate bypassed, and
|
||||
confirm `Address already in use` / exit 126.
|
||||
- Assert it in the repo's seam tests: e.g. `grep -q 'wait_port_free <port>'`
|
||||
and `grep -q '^wait_port_free() {'` in the deploy handler, so a later
|
||||
refactor cannot quietly drop the gate.
|
||||
|
||||
## Related traps
|
||||
|
||||
- `systemctl disable` on some of these hosts **deletes the unit file** rather
|
||||
than just the symlink, so a "move the unit to trash" step must tolerate a
|
||||
file that is already gone — and should say so, or "no message" carries no
|
||||
evidence.
|
||||
- Never `rm`; move retired units into trash (`$HOME/.AGENTTRASH/<date>-<topic>/`).
|
||||
- A rootless Quadlet's generated unit **cannot** be `systemctl --user enable`d;
|
||||
boot persistence comes from `[Install] WantedBy=default.target` in the
|
||||
`.container`. To keep a masked/failed quadlet from being regenerated, move
|
||||
the `.container` out of `~/.config/containers/systemd/` and `daemon-reload` —
|
||||
the generator's source is the lever, not `mask`.
|
||||
Reference in new issue
Block a user