---
name: rotate-fleet-service-host-secret
description: "Rotate a Dank-fleet rootless-quadlet service's host-side secret (e.g. open-notebook OPEN_NOTEBOOK_PASSWORD) safely: confirm the deployer's secret seeder is idempotent first so a future deploy won't clobber the new value, change ONLY the target line while holding encryption keys byte-identical, then restart the user unit and prove the rotation (new token 200, old token 401, no token 401). Not for git/Gitea tokens or LLM endpoint secrets."
---

# Rotate a fleet service's host-side secret safely

Host secrets live OUTSIDE the repo, in `/home/dank/Containers/<svc>/env` (mode 600). The repo only holds the knobs (`scripts/<svc>.env`, secrets declared EMPTY). Changing a secret = edit the host file, never the repo.

## 1. First: prove the deployer won't clobber it (MANDATORY)
`deploy-remote.sh` seeds some secrets from `dankclaw/.env`. Confirm the seed fn is **idempotent** before editing, or a later `deploy.sh` overwrites your value. In `scripts/deploy-remote.sh`, `seed_<svc>_env` must look like open-notebook's:

```bash
for var in A B C D; do grep -q "^${var}=." "$dest" 2>/dev/null || missing=1; done
[[ "$missing" == 0 ]] && return 0          # all present → leaves the file untouched
if [[ ! -e "$dest" ]]; then ...seed from dankclaw/.env...; fi
```

If it returns-early on all-present, editing `$dest` directly is deploy-safe. If it seeds unconditionally, edit BOTH the host file and `dankclaw/.env` instead.

## 2. Read, then rewrite only the target line
`read ssh://<host>/home/dank/Containers/<svc>/env` (never print secrets to chat — they leak into the transcript). Then `write ssh://...` overwriting with **all** lines, changing only the one secret. **Hold encryption keys byte-identical** (e.g. `OPEN_NOTEBOOK_ENCRYPTION_KEY`): a DB-encrypted credential store is unreadable if that key changes. Then `chmod 600`.
- ssh:// `read` emits NO edit tag, so the `edit` tool can't target it — use `write` (full overwrite) with the exact values from the read.

## 3. Restart the right unit
Rootless quadlets are USER units: `ssh <host> systemctl --user restart <svc>.service`. (systemctl --user works over plain ssh here — linger + bus are set up.) Restart only the app unit; DB password unchanged means the DB unit needs no restart.

## 4. Prove the rotation behaviorally (this is the smoke test)
Hit a protected API route three ways, expecting **new=200 / old=401 / none=401**. Pass the bearer via a mode-600 curl config + `-K` (never `-H "Bearer …"` — argv leak):

```bash
ssh <host> 'cfg=$(mktemp); chmod 600 "$cfg"
printf "header = \"Authorization: Bearer %s\"\n" "<new-or-old>" > "$cfg"
curl -s -o /dev/null -w "%{http_code}\n" -K "$cfg" http://127.0.0.1:5056/api/<protected>'
```

## Gotchas that bit
- **Health is on the API loopback port, not the web port.** open-notebook: `/health` answers on **127.0.0.1:5056** (the FastAPI), NOT `:8502` (Next.js, which 404s `/health`). Probing the wrong port reads as "service failed" when it's up and `healthy`. Excluded/auth-free paths (`/`, `/health`, `/docs`, `/openapi.json`, `/redoc`) — everything else needs the bearer.
- **`OPEN_NOTEBOOK_PASSWORD` unset = auth OFF** for anything that can reach the port (other containers dial it by name on `dankclaw_default`), so it's mandatory, not cosmetic; deployer refuses to start on blank.
- Browser/clients keep the old secret in localStorage → they 401 on next call and re-prompt. Expected.
- **Harness command-parser gotcha:** adjacent/interleaved single+double quotes in one bash line (`'...'"$x"'...'`) crash `pi-natives:command: unterminated quote`. Build a curl-config `header=` line with the proven single-quoted printf (`printf 'header = "Authorization: token %s"\n' "$tok"`) and put long bodies in a file via the `write` tool + `jq --rawfile`, not a giant inline `$(printf …)`.