---
name: sftp-client-unreachable-triage
description: "Diagnose \"SFTP/SSH from box A to box B doesn't work\" when box A is unreachable from the fleet: exonerate the server with a log+key census, then discriminate sleep vs firewall vs name-resolution vs missing key from outside."
---

# Triage a client-side SFTP/SSH failure (client box unreachable)

Use when Frank reports "FileZilla/SFTP/scp from <client> to <server> isn't working" and the
client box is not reachable from the fleet. Produces a ranked, evidence-backed verdict
without access to the client. **Read-only probes only**; no restarts, no config writes.

## Order of operations (each step can end the investigation)

### 1. Server-side exoneration — gather before theorising
```bash
ssh -o BatchMode=yes <server> 'sudo -n sshd -T | grep -Ei "^(port|listenaddress|pubkeyauth|passwordauth|kbdinteractive|allowusers|subsystem|authorizedkeysfile)"'
ssh -o BatchMode=yes <server> 'ls -l /usr/lib/openssh/sftp-server; sudo -n ls -l /etc/ssh/ssh_host_*key.pub'   # host-key mtime = known-hosts churn check
ssh -o BatchMode=yes <server> 'sudo -n iptables -S | head; sudo -n nft list ruleset | head -20'               # INPUT ACCEPT = no firewall
```
Host keys unchanged for years ⇒ FileZilla cannot be tripping a host-key warning.

### 2. Censuses that turn "maybe" into "never" (the payload of this skill)
```bash
# keys trusted by the server, by comment
ssh <server> 'ssh-keygen -lf ~/.ssh/authorized_keys'

# every IP that ever authenticated, and with which key
ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-90 days" \
  | grep -oE "Accepted publickey for [a-z]+ from [0-9.]+" | awk "{print \$NF}" | sort | uniq -c'
ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-90 days" \
  | grep -oE "ED25519 SHA256:[A-Za-z0-9+/=]+" | sort | uniq -c'

# did the client IP EVER appear, anywhere?
ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-45 days" | grep -c "<client-ip>"'
ssh <server> 'sudo -n zgrep -h "<client-ip>" /var/log/auth.log* | grep -v sudo | tail'
```
Count 0 across journal **and** rotated logs ⇒ no TCP ever established ⇒ the failure is
DNS, client egress, or a sleeping host — **not** credentials. Say so explicitly, because
"permission denied" and "connection timed out" have different fixes.

### 3. Is the client even at the IP everyone believes? (DHCP drift)
```bash
avahi-resolve -n <Client>.local    # mDNS truth
ip neigh | grep -E '\.(old|new) '  # FAILED vs REACHABLE
nmap -sn -n 192.168.0.0/24         # works unprivileged; arp-scan needs root
```
macOS **Private Wi-Fi Address** rotation changes the MAC ⇒ new DHCP lease ⇒ every
hardcoded IP for that box rots. Check both old and new addresses for mentions in
`/var/log/auth.log*` and `journalctl`.

### 4. Discriminate sleep vs firewall (ARP/mDNS alive, everything else dead)
- ARP replies + live mDNS but **zero** ICMP and **zero** TCP on a spread of ports
  (22, 80, 445, 548, 5900, 8080, 5000, 7000, 3283) from **multiple** hosts including a
  wired one: probe from a wired box on the same L2 — that also rules out Wi-Fi AP isolation.
- Same result ⇒ either the host is asleep (Bonjour sleep proxy / AP answers ARP+mDNS on its
  behalf) **or** a host packet filter (macOS "Block all incoming", LuLu/Little Snitch,
  ProtonVPN kill switch) is dropping it.
- **Do not overclaim which.** State both, and note the built-in macOS firewall only blocks
  *inbound* — so if Frank is actively using that Mac, a built-in-firewall verdict cannot
  explain an *outbound* SFTP failure; a third-party filter/VPN or DNS is then the cause.

### 5. Name resolution (the silent failure)
```bash
getent hosts <server> <server>.local <server>.<domain>
dig +short <server> @<pihole-ip>
```
Traps seen on this fleet:
- Bare `dankserver` resolves **nowhere** — no Pi-hole custom record, gateway DNS refuses
  client queries. Only mDNS `<host>.local`, which may advertise a **link-local IPv6 only**.
- `dankserver.franksdank.cloud` resolves to **Caddy (192.168.0.143)**, not the SSH box —
  a wildcard vhost. Never suggest a `*.franksdank.cloud` name for SSH/SFTP.
- **FileZilla ignores `~/.ssh/config`** — `Host dankserver` aliases do not apply; the Host
  field needs a literal IP that resolves.

### 6. Fix recipe handed to the user (client side)
```bash
ping -c2 <server-ip> && nc -vz <server-ip> 22
ls ~/.ssh/id_ed25519.pub || ssh-keygen -t ed25519 -C "<client-name>"
ssh-copy-id -i ~/.ssh/id_ed25519.pub dank@<server-ip>
sftp -i ~/.ssh/id_ed25519 dank@<server-ip>
```
FileZilla: Protocol **SFTP**, Host literal IP, Port 22, Logon Type **Key file** →
`/Users/dank/.ssh/id_ed25519` (or Normal + password). The "Encryption" dropdown is FTP-only
noise for SFTP sites.

### 7. Report collateral you proved, don't guess it
Dead client IP ⇒ grep the fleet for hardcoded references and *observe* the breakage:
```bash
grep -rn --exclude-dir=.git "<old-ip>" ~/Projects/serverAppConfig | wc -l
curl -sk -o /dev/null -w '%{http_code}\n' --resolve "radarr.franksdank.cloud:443:<caddy-ip>" https://radarr.franksdank.cloud/
```
(`-H Host:` alone gives `000` — curl sends the IP as TLS SNI and the handshake fails. Use
`--resolve` to keep SNI correct before concluding anything about a vhost.)

## Laws
- Zero packets at the server beats any credential theory; log census first, key census second.
- ARP/mDNS answering is **not** proof a host is awake; the AP or a sleep proxy can answer.
- Probe from a wired host before blaming Wi-Fi isolation.
- A rotating private Wi-Fi MAC will re-break every hardcoded IP: recommend a DHCP
  reservation as the fix, not 40 file edits.