live: heartbeat sync 2026-10-29T16:32:39-07:00 (Refs #11)

agent/config.yml

@@ -96,7 +96,7 @@ compaction:
   experimentalContextManagement: false
 interruptMode: immediate
 defaultThinkingLevel: xhigh
-hideThinkingBlock: false
+hideThinkingBlock: true
 proseOnlyThinking: false
 terminal:
   showProgress: true

agent/managed-skills/sftp-client-unreachable-triage/SKILL.md

@@ -0,0 +1,100 @@
+---
+name: sftp-client-unreachable-triage
+description: "Diagnose \"SFTP/SSH from box A to box B doesn't work\" when box A is unreachable from the fleet: exonerate the server with a log+key census, then discriminate sleep vs firewall vs name-resolution vs missing key from outside."
+---
+
+# Triage a client-side SFTP/SSH failure (client box unreachable)
+
+Use when Frank reports "FileZilla/SFTP/scp from <client> to <server> isn't working" and the
+client box is not reachable from the fleet. Produces a ranked, evidence-backed verdict
+without access to the client. **Read-only probes only**; no restarts, no config writes.
+
+## Order of operations (each step can end the investigation)
+
+### 1. Server-side exoneration — gather before theorising
+```bash
+ssh -o BatchMode=yes <server> 'sudo -n sshd -T | grep -Ei "^(port|listenaddress|pubkeyauth|passwordauth|kbdinteractive|allowusers|subsystem|authorizedkeysfile)"'
+ssh -o BatchMode=yes <server> 'ls -l /usr/lib/openssh/sftp-server; sudo -n ls -l /etc/ssh/ssh_host_*key.pub'   # host-key mtime = known-hosts churn check
+ssh -o BatchMode=yes <server> 'sudo -n iptables -S | head; sudo -n nft list ruleset | head -20'               # INPUT ACCEPT = no firewall
+```
+Host keys unchanged for years ⇒ FileZilla cannot be tripping a host-key warning.
+
+### 2. Censuses that turn "maybe" into "never" (the payload of this skill)
+```bash
+# keys trusted by the server, by comment
+ssh <server> 'ssh-keygen -lf ~/.ssh/authorized_keys'
+
+# every IP that ever authenticated, and with which key
+ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-90 days" \
+  | grep -oE "Accepted publickey for [a-z]+ from [0-9.]+" | awk "{print \$NF}" | sort | uniq -c'
+ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-90 days" \
+  | grep -oE "ED25519 SHA256:[A-Za-z0-9+/=]+" | sort | uniq -c'
+
+# did the client IP EVER appear, anywhere?
+ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-45 days" | grep -c "<client-ip>"'
+ssh <server> 'sudo -n zgrep -h "<client-ip>" /var/log/auth.log* | grep -v sudo | tail'
+```
+Count 0 across journal **and** rotated logs ⇒ no TCP ever established ⇒ the failure is
+DNS, client egress, or a sleeping host — **not** credentials. Say so explicitly, because
+"permission denied" and "connection timed out" have different fixes.
+
+### 3. Is the client even at the IP everyone believes? (DHCP drift)
+```bash
+avahi-resolve -n <Client>.local    # mDNS truth
+ip neigh | grep -E '\.(old|new) '  # FAILED vs REACHABLE
+nmap -sn -n 192.168.0.0/24         # works unprivileged; arp-scan needs root
+```
+macOS **Private Wi-Fi Address** rotation changes the MAC ⇒ new DHCP lease ⇒ every
+hardcoded IP for that box rots. Check both old and new addresses for mentions in
+`/var/log/auth.log*` and `journalctl`.
+
+### 4. Discriminate sleep vs firewall (ARP/mDNS alive, everything else dead)
+- ARP replies + live mDNS but **zero** ICMP and **zero** TCP on a spread of ports
+  (22, 80, 445, 548, 5900, 8080, 5000, 7000, 3283) from **multiple** hosts including a
+  wired one: probe from a wired box on the same L2 — that also rules out Wi-Fi AP isolation.
+- Same result ⇒ either the host is asleep (Bonjour sleep proxy / AP answers ARP+mDNS on its
+  behalf) **or** a host packet filter (macOS "Block all incoming", LuLu/Little Snitch,
+  ProtonVPN kill switch) is dropping it.
+- **Do not overclaim which.** State both, and note the built-in macOS firewall only blocks
+  *inbound* — so if Frank is actively using that Mac, a built-in-firewall verdict cannot
+  explain an *outbound* SFTP failure; a third-party filter/VPN or DNS is then the cause.
+
+### 5. Name resolution (the silent failure)
+```bash
+getent hosts <server> <server>.local <server>.<domain>
+dig +short <server> @<pihole-ip>
+```
+Traps seen on this fleet:
+- Bare `dankserver` resolves **nowhere** — no Pi-hole custom record, gateway DNS refuses
+  client queries. Only mDNS `<host>.local`, which may advertise a **link-local IPv6 only**.
+- `dankserver.franksdank.cloud` resolves to **Caddy (192.168.0.143)**, not the SSH box —
+  a wildcard vhost. Never suggest a `*.franksdank.cloud` name for SSH/SFTP.
+- **FileZilla ignores `~/.ssh/config`** — `Host dankserver` aliases do not apply; the Host
+  field needs a literal IP that resolves.
+
+### 6. Fix recipe handed to the user (client side)
+```bash
+ping -c2 <server-ip> && nc -vz <server-ip> 22
+ls ~/.ssh/id_ed25519.pub || ssh-keygen -t ed25519 -C "<client-name>"
+ssh-copy-id -i ~/.ssh/id_ed25519.pub dank@<server-ip>
+sftp -i ~/.ssh/id_ed25519 dank@<server-ip>
+```
+FileZilla: Protocol **SFTP**, Host literal IP, Port 22, Logon Type **Key file** →
+`/Users/dank/.ssh/id_ed25519` (or Normal + password). The "Encryption" dropdown is FTP-only
+noise for SFTP sites.
+
+### 7. Report collateral you proved, don't guess it
+Dead client IP ⇒ grep the fleet for hardcoded references and *observe* the breakage:
+```bash
+grep -rn --exclude-dir=.git "<old-ip>" ~/Projects/serverAppConfig | wc -l
+curl -sk -o /dev/null -w '%{http_code}\n' --resolve "radarr.franksdank.cloud:443:<caddy-ip>" https://radarr.franksdank.cloud/
+```
+(`-H Host:` alone gives `000` — curl sends the IP as TLS SNI and the handshake fails. Use
+`--resolve` to keep SNI correct before concluding anything about a vhost.)
+
+## Laws
+- Zero packets at the server beats any credential theory; log census first, key census second.
+- ARP/mDNS answering is **not** proof a host is awake; the AP or a sleep proxy can answer.
+- Probe from a wired host before blaming Wi-Fi isolation.
+- A rotating private Wi-Fi MAC will re-break every hardcoded IP: recommend a DHCP
+  reservation as the fix, not 40 file edits.

agent/memories/sharpshooter/omp-32c11n2wnglul/state.json

@@ -1,6 +1,6 @@
 {
 	"v": 1,
-	"lastConsolidatedAt": 1793308587833,
+	"lastConsolidatedAt": 1793314235443,
 	"lastResult": {
 		"at": 1791599865775,
 		"sessions": 1,

agent/memories/sharpshooter/wt-20261027-142335-6d1fc1f-1hpiehxnq73s0/architecture.md

@@ -1,3 +1,3 @@
 ## Structural Standards
 
-Quant quality must hold regardless of implementation choice.
+Quant quality must hold regardless of implementation choice.

agent/memories/sharpshooter/wt-20261027-142335-6d1fc1f-1hpiehxnq73s0/product.md

@@ -2,4 +2,4 @@
 
 ## Behavioral Updates
 
-Hourly updates on a deterministic timer.
+Merge any fixes immediately.

agent/memories/sharpshooter/wt-20261027-142335-6d1fc1f-1hpiehxnq73s0/state.json

@@ -1,8 +1,8 @@
 {
 	"v": 1,
-	"lastConsolidatedAt": 1793312933825,
+	"lastConsolidatedAt": 1793316653836,
 	"lastResult": {
-		"at": 1793312601006,
+		"at": 1793315963987,
 		"sessions": 1,
 		"deltas": 1,
 		"model": "frank-zealot"

agent/memories/sharpshooter/wt-20261029-155036-7c8a99d-1ohxrda9bajjs/state.json

@@ -0,0 +1,10 @@
+{
+	"v": 1,
+	"lastConsolidatedAt": 1793315256444,
+	"lastResult": {
+		"at": 1793314927653,
+		"sessions": 1,
+		"deltas": 1,
+		"model": "frank-zealot"
+	}
+}