# Homelab service triage

Use this when checking service state across Dankclaw/Dankserver from the workstation or another controller.

1. Start from the repo and trust current repo state over memory:
   - cd /home/dank/Projects/serverAppConfigs
   - git status --short
   - find . -path ./.git -prune -o -name '*.container' -print | sort

2. Identify host and unit style:
   - Dankclaw: 192.168.0.143
   - Dankserver: 192.168.0.235
   - Most services are rootless Quadlet user units: systemctl --user <action> <name>.service
   - Some model services may be system units or custom scripts; inspect repo before acting.

3. Inventory a host:
   - ssh HOST 'systemctl --user list-units --type=service; podman ps -a --format "{{.Names}} {{.Status}} nets=[{{.Networks}}]"'
   - For a suspected unit: ssh HOST 'systemctl --user status UNIT.service --no-pager; journalctl --user -u UNIT.service -n 120 --no-pager'

4. For port/listener issues:
   - ssh HOST 'ss -lntp 2>/dev/null | tr -s " "'
   - Match published ports to Quadlet PublishPort and podman ps networks.

5. For deploy issues:
   - Review scripts/deploy.sh behavior in current repo.
   - Confirm target checkout is clean or intentionally stashed by deploy workflow.
   - Do not hand-edit generated systemd output; fix Quadlet/source files.
   - After Quadlet changes run daemon reload via deployer or systemctl --user daemon-reload, then inspect generated unit with systemctl --user cat.

6. Do not commit secrets. Machine-local env files are the source for Caddy/Porkbun, app passwords, API keys, OAuth creds, phone numbers, sessions, and live generated state.