Frank
33eae71d33
All ~173 managed skills read in full and merged, one polished reference page per skill-cluster: - LLM cross-cutting (17 pages): GGUF Compatibility, Llama Cpp Serving Sizing, Llama Cpp Multi-Device Splits, PCIe Link and Atomics, RDNA4 Container Serving, RDNA4 MoE Expert Offload VRAM Budget, vLLM Boot Forensics, vLLM Porting and Compile Triage, LLM Bench Windows, LLM Performance Diagnosis, Model Swap and Rollback, Tcclaviger Fork Ops, Heterogeneous Split Triage, FreeToken Builds + deltas folded into the retired runbooks (GLM DFlash2 port laws, Lucebox dual-R9700, R9V qualification pipeline, FLM pack registration, ik hybrid envelope, frank-medium split rebalance, llama.cpp GGUF preflight, batch-ubatch split section, bench-window D-state/stale-read notes, second-endpoint bring-up). - Architecture (3): Stale Signal Audit, Quadlet Lifecycle, Deploy State Triage. - Operations (6 + index): Remote Job Runner, Payload and Disk Reclaim, Container Host Reachability, Fleet Ops Playbook, Fleet Audit and Posture, SSH and Remote Access. - Infrastructure: Observability Provisioning + Homepage Status Probes; updates to Authentik, Anubis, LAN-DNS (per-client DoH bypass, Porkbun zone ops), Swap Configuration, OpenWebUI, Hermes (pre-existing page preserved and extended). - FrankBook-Pro: ARR-Stack filled (API split, schema-first probing, auth-posture, auth-config trap); qBittorrent E2E verification. - Desktop/Agent/Research: Noctalia and Niri, OMP Harness, Agent Docs and Edits, Command Center Dotfiles, PICO-8 Asset Standards. - Indexes: LLM Cross-Cutting References section, Operations/ and Agent/ index pages, root-index Skill Distillation section, AGENTS.md pointer. - log.md carries the wave plan. check-vault-links.py 0 unresolved (185 pages); ./tests/run.sh 366/366.
159 lines
7.8 KiB
Markdown
159 lines
7.8 KiB
Markdown
---
|
|
type: reference
|
|
title: Homepage Status Probes
|
|
description: The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage procedure, local render check, deploy, and the e2e gate interactions (asleep Mac mini, raw-IP matrix).
|
|
tags: [homepage, status, probes, e2e, homepage-status]
|
|
timestamp: 2026-10-05T00:00:00-07:00
|
|
---
|
|
|
|
# Homepage Status Probes
|
|
|
|
The homepage is a static site —
|
|
`shared/homepage/{index.html,admin.html,app.js,style.css,services.json,docs.html}`
|
|
— served from `/srv/homepage` on **dankclaw** only (`deploy_homepage` in
|
|
`scripts/deploy-remote.sh` refuses other targets). A checker
|
|
(`scripts/homepage-status.py`, run by the `homepage-status.timer` unit) probes
|
|
each card's target and writes `/srv/homepage/status.json` (mode **0644** so
|
|
Caddy can serve it — losing the mode 403s the page).
|
|
|
|
## The checker's vantage and its three false-down shapes
|
|
|
|
The checker probes TCP/HTTP **from the host netns**. It only has three states
|
|
— `wip` | `up` | `down`; there is no "unknown", so an unprobeable service can
|
|
never render honestly and must be dropped or re-pointed, not "fixed".
|
|
|
|
1. **TLS-only app probed with `method:"http"`.** Plex Media Server serves TLS
|
|
only on `32400`; plain-HTTP GET → TLS alert → curl exit 52, code `000` →
|
|
card says down while PMS answers `200` over HTTPS. Fix:
|
|
`"method":"tcp"` (connect is the honest liveness signal; the public URL
|
|
stays HTTP because Caddy re-keys).
|
|
2. **Retired port.** The Hermes card probed `127.0.0.1:8090` after the
|
|
compose→quadlet move moved the dashboard to `9119` and the webui to
|
|
`8787`; nothing listened on 8090, so a healthy stack read `down`.
|
|
3. **Unpublished port / wrong netns.** `gitea-postgres` (`db:5432`, net
|
|
`systemd-gitea`), `searxng-valkey` (`valkey:6379`, net `systemd-searxng`)
|
|
and `alloy` (OTLP 12345) publish **no** ports. Container DNS aliases are
|
|
netns-scoped, so from the host these are unreachable *by construction* —
|
|
no probe value makes them honest. Drop those cards.
|
|
|
|
A host-vantage probe can only reach a listener **published to the host**
|
|
(rootlessport/pasta); unpublished ports are invisible from the host netns.
|
|
See [[Container Host Reachability]] for the netns rules.
|
|
|
|
## Triage procedure
|
|
|
|
1. Read the card's probe target in `shared/homepage/services.json`; note
|
|
method (`tcp` vs `http`).
|
|
2. Does that port have a **host-visible listener**?
|
|
`podman ps -a --format '{{.Names}}|{{.Status}}'`, then per container
|
|
`podman inspect <c> --format '{{json .NetworkSettings.Networks}}
|
|
{{.HostConfig.NetworkMode}}'` and `podman port <c>` (empty output =
|
|
published nothing → structurally unprobeable from the host).
|
|
3. Probe from the **correct vantage**: from a bridge-net container the host
|
|
is its gateway IP (`/proc/net/route`, e.g. `10.89.1.1` on
|
|
`dankclaw_default`) — *not* `host.containers.internal` (`169.254.1.2`),
|
|
which belongs to the default `podman` net (`10.88.0.0/16`). Probe with
|
|
bash `/dev/tcp` (no `python3`/`ip` in these images); escape loop vars so
|
|
they expand **inside** `podman exec bash -c`.
|
|
4. Classify: real outage (container exited — read `State.ExitCode`: 137 =
|
|
SIGKILL, 143 = SIGTERM/manual stop; `OOMKilled=false` rules out the OOM
|
|
killer) vs false down (running + unreachable target).
|
|
5. Fix: re-point or re-method for HTTP/TLS mismatches; **drop** cards whose
|
|
only exposure is an internal netns; never invent a probe target.
|
|
6. Verify by re-running the checker path, not by eyeballing: after deploy,
|
|
re-read `/srv/homepage/status.json` and diff the `down` list
|
|
before/after.
|
|
|
|
Removing a card's `probe` key without a code change crashes the checker
|
|
(`svc["probe"]` KeyError) — dropping the whole entry is the only zero-code
|
|
fix. The `description` field is consumed only by `app.js` `descHTML()`, so
|
|
adding a `details` object (role/spec/inputs/use) is schema-safe:
|
|
`tests/homepage_config.bats` checks only name/category/section/probe.
|
|
|
|
## Local render check (before every deploy)
|
|
|
|
The page fetches `services.json` + `status.json` at load; `file://` is
|
|
CORS-blocked, so serve over HTTP:
|
|
|
|
```bash
|
|
cd shared/homepage
|
|
python3 -c "import json; s=json.load(open('services.json')); \
|
|
json.dump({'services':[{'name':x['name'],'state':'up'} for x in s]}, open('status.json','w'), indent=2)"
|
|
python3 -m http.server 8899 --bind 127.0.0.1
|
|
```
|
|
|
|
Then in the browser: load `http://127.0.0.1:8899/index.html`, **wait ~1.5 s**
|
|
(panels render after the async fetch; checking immediately returns 0 panels),
|
|
extract via `tab.evaluate` — LLM cards render `.desc-role`/`.desc-spec`/
|
|
`.desc-inputs`/`.desc-use` from the card's `details` object, other cards a
|
|
plain `.desc`; `getComputedStyle(el).color` confirms tones. Screenshot for
|
|
visual proof. Expected local noise: `deployed.js` 404s (baked in at deploy
|
|
time) — harmless.
|
|
|
|
The scratch `status.json` is untracked and NOT git-ignored — move it to
|
|
`shared/homepage/.AGENTTRASH/` after the check (repo policy: never `rm`).
|
|
|
|
CSS palette lives in `style.css` `:root` (`--fg` bright green, `--dim` dim,
|
|
`--head` pale, `--link` cyan, `--wip` yellow, `--down` red). Add new tones as
|
|
`--*` vars and include any new `.desc-*` class in the `html[data-bare="1"]`
|
|
reset so `?bare=1` strips it to black.
|
|
|
|
## Deploy
|
|
|
|
```
|
|
./scripts/deploy.sh dankclaw --service homepage --skip-e2e
|
|
```
|
|
|
|
Copies the assets to `/srv/homepage`, regenerates `deployed.js` (git commit
|
|
datetime), restarts the `homepage-status` timer. Verify live:
|
|
`ssh dank@192.168.0.143 'grep … /srv/homepage/…'`.
|
|
|
|
**`--skip-e2e` is justified for homepage-only changes when the Mac mini is
|
|
asleep**: the e2e admin crawl (`tests/e2e/specs/11-admin.spec.ts:24`) walks
|
|
Prowlarr at `192.168.0.112:8989` on FrankBook-Pro; an asleep mini (ARP STALE,
|
|
no remote wake) times out that test — a pre-existing environmental failure,
|
|
not a regression. All non-Mac-mini e2e tests pass.
|
|
|
|
## e2e raw-IP *arr failures — probe from the browser's box, not a peer
|
|
|
|
When `11-admin` fails with `ERR_CONNECTION_TIMED_OUT`, the instinct is "the
|
|
Mac is asleep". Sometimes true; not always. Playwright runs on the dev box,
|
|
so a healthy probe from dankclaw proves nothing about the **browser's** path:
|
|
|
|
```bash
|
|
# 1. From the box that RUNS the browser:
|
|
curl -s -o /dev/null -w '%{http_code}\n' --max-time 7 http://192.168.0.112:8989/
|
|
# 2. From a LAN peer:
|
|
ssh dankclaw 'for p in 8989 7878 9696; do curl -s -o /dev/null -w "$p %{http_code}\n" \
|
|
--max-time 6 http://192.168.0.112:$p/; done'
|
|
# 3. Liveness (ICMP, not TCP):
|
|
ping -c 2 -W 3 192.168.0.112
|
|
```
|
|
|
|
| peer probe | dev-box probe | ping | meaning |
|
|
|---|---|---|---|
|
|
| 200 | 200 | ok | transient — re-run the spec |
|
|
| timeout | timeout | fails | Mac asleep; wake, ports return in seconds |
|
|
| **200** | **rc=28** | **ok** | **path problem, not sleep** — ICMP allowed, TCP blocked. Waking cannot help |
|
|
|
|
Third row: confirm the Mac is innocent through the dankclaw jump
|
|
(`pgrep -fl socat`; `netstat -an | grep -c 8989`; loopback
|
|
`curl http://127.0.0.1:8989/` → 200; `/usr/libexec/ApplicationFirewall/socketfilterfw
|
|
--getglobalstate` → disabled). Forwarders up + loopback 200 ⇒ the block is
|
|
between the dev box and the Mac.
|
|
|
|
Don't hold a deploy hostage to a known-environmental red gate: verify the
|
|
deployed service **directly** (its own ledger/health/real completion), state
|
|
the gate was skipped or partially red and why (`--skip-e2e` prints the
|
|
warning, so the skip is recorded), and report the residual gap plainly
|
|
("29/30 e2e pass; 11-admin fails on Sonarr's raw IP from the dev box; peer
|
|
probe returns 200; deploy verified by other means"). Do not weaken or skip
|
|
the spec — it guards a real SSO-plus-RBAC contract.
|
|
|
|
## Related
|
|
|
|
- [[Container Host Reachability]] — the netns rules behind the vantage
|
|
- [[Observability Provisioning]] — the monitoring stack reading the same fleet
|
|
- [[Deploy State Triage]] — service-scoped deploys and the e2e gate
|
|
- [[Testing Policy]] — the seam/e2e contract this page's checks live in
|