Files
command-center/agent/managed-skills/fleet-status-daily-report/SKILL.md
T

3.3 KiB

name, description
name description
fleet-status-daily-report Answer "status update / what happened yesterday" for the dank fleet: git-window record on origin/main plus a live per-host census (endpoints, units, RAM, DNS), with the reboot-attribution check that stops false incident causes.

Fleet status report (Dankclaw + Dankserver)

Use for "what's the state of the fleet", "what happened yesterday", or a post-incident sweep. Read-only. For one service's live state, prefer fleet-service-runtime-census; this is the fleet-wide sweep.

1. The day's record (repo is the ledger)

git fetch origin --quiet
git log --no-merges --since="YYYY-MM-DD 00:00" --until="YYYY-MM-DD+1 00:00" \
        --date=format:'%H:%M' --pretty='%ad %h %s' origin/main
git log --merges  --since=... --pretty='%ad %h %s' --date=format:'%H:%M' origin/main   # PR numbers

Merge lines carry the PR number — that is the review trail; the non-merge log is what actually changed. Group commits into streams (endpoint names lead the subject) rather than listing 20 subjects.

2. Live census (parallel: one call per host + one endpoint sweep)

Endpoint sweep from any LAN box — /health is honest on all four (archon binds only after full load):

for u in 192.168.0.143:7770 192.168.0.143:7771 192.168.0.235:7770 192.168.0.143:7772; do
  printf "%s %s\n" "$u" "$(curl -s -m 12 -w ' [%{http_code}]' http://$u/health | head -c 160)"; done
curl -s http://192.168.0.143:7770/v1/models    # dragoon lists EVERY engine it serves, incl. NPU-lane models

Per host: uptime -s; uptime -p; systemctl show <llm units> -p ActiveEnterTimestamp -p NRestarts -p DropInPaths (sudo for the root LLM units, --user for the quadlets); list-units --failed; free -h; journalctl -k --since "<evening>" -p err; the host's own engine log tail.

3. Attribution rules (where reports go wrong)

  • Never call a host reboot from a container's start timestamp. Check uptime -s first; if uptime spans the event, the trigger was a container restart — find it with journalctl --user -u <name>.service --since … --until … + podman inspect <c> --format '{{.State.StartedAt}}'. (This exact mistake mislabelled a 2026-10-19 Pi-hole incident as boot-caused; the law's remedy belongs after any restart, which is a different action item.)
  • podman stats is a 28-container wall — sort/limit it; RSS of the LLM tenants is what matters.
  • This podman's inspect template has no .Config.PodName, no .Config.GroupAdd, and .HostConfig.Devices can read [] even when a quadlet AddDevice= worked. Prove GPU use from the engine's own log (ggml_vulkan: Found 1 Vulkan devices: … Radeon 8060S (RADV GFX1151)), not from inspect.
  • Kernel lines: count occurrences (journalctl -k --since '7 days' | grep -c "NETDEV WATCHDOG") before calling a one-off a trend.
  • Failed units: systemctl show <u> -p ExecMainExitTimestamp -p ExecMainStatus --value separates "broken since boot" from "broke yesterday" — and a nightly failure (e.g. logrotate) is a live risk, not noise.

4. Report shape

Stream table (what landed) → live-state table (what is true now) → corrections to my own earlier record → open items that need Frank's decision, each with the evidence line and the one action. Flag anything that would touch host config outside the repo as a decision, never a silent fix.