docs(vault): fleet-wide fact re-verification pass (67 pages)
The docs-fact census wave re-read the live hosts and re-keyed every page it could: runtime/payload identities, ports, sizes, ledger figures, unit state, and the retirement banners that drifted as the fleet moved. The wave was interrupted mid-run, so this commit carries only what passed the integrity gates: every touched page parses (front matter or H1+status convention intact, balanced fences, no truncated tables, trailing newline), the docs_completeness and link-integrity seams are green, and bats is 384/384. Numbers that this session proved on live hosts are keyed to the measured value, not the page's earlier claim.
This commit is contained in:
1 parent
466fd477d8
commit
e7f0d93c86
67 files changed
+2619
-1152
No files matched your search
@@ -19,64 +19,82 @@ the Hermes agent in-process from the shared `HERMES_HOME`, so it is the *same*
|
||||
agent — same memory, skills, and sessions — as the [[Hermes]] gateway and the
|
||||
[[Hermes-Dashboard]].
|
||||
|
||||
**Host:** Dankclaw — **Port:** `0.0.0.0:8787` (all interfaces) — **Ingress:** SSH tunnel for the phone app; direct LAN access also works
|
||||
**Host:** Dankclaw — **Port:** `0.0.0.0:8787` (all interfaces; `ss -ltnp`
|
||||
2026-10-18 shows the webui listening there in the host netns) — **Ingress:** SSH
|
||||
tunnel for the phone app; direct LAN access also works (re-measured 2026-10-18
|
||||
from another LAN host: `/health` → 200, `/` → 302) — **Vhost:**
|
||||
`hermes.franksdank.cloud`, **exempt by construction** — no Anubis, no Authentik;
|
||||
`HERMES_WEBUI_PASSWORD` is the only gate
|
||||
|
||||
## Facts
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Image | `docker.io/nousresearch/hermes-agent:latest` (same as the gateway) |
|
||||
| Command | `python3 /webui/bootstrap.py --foreground` |
|
||||
| Source checkout | `/home/dank/Containers/hermes/webui` on Dankclaw, bind-mounted at `/webui` |
|
||||
| Port | `8787`, bound to `0.0.0.0` in the **host** network namespace (`Network=host`) — LAN-reachable, not loopback-only |
|
||||
| Auth | `HERMES_WEBUI_PASSWORD` (host `.env`; Hermex signs in with it) |
|
||||
| Image | `localhost/hermes-agent:latest` with `Pull=never` (`dankclaw/hermes/quadlet/hermes-webui.container:35-36`) — the same image as the gateway and dashboard. The deployer keeps that alias fresh by tagging the upstream name into local storage (`podman tag docker.io/nousresearch/hermes-agent:latest localhost/hermes-agent:latest`, `scripts/deploy-remote.sh:1056`); the installed image is the 2026-09-24 build, revision `749220e` (`podman image inspect` 2026-10-18). |
|
||||
| Command | `python3 /webui/bootstrap.py --foreground` (`hermes-webui.container:48`) |
|
||||
| Source checkout | `/home/dank/Containers/hermes/webui` on Dankclaw, bind-mounted at `/webui` (`hermes-webui.container:47`; live dir `drwxrwxr-x dank dank`) |
|
||||
| Port | `8787`, bound to `0.0.0.0` in the **host** network namespace (`Network=host` + `HERMES_WEBUI_HOST=0.0.0.0`, `hermes-webui.container:37-40`) — LAN-reachable, not loopback-only |
|
||||
| Auth | `HERMES_WEBUI_PASSWORD` (from `/home/dank/serverAppConfigs/dankclaw/.env`, the unit's only `EnvironmentFile`; Hermex signs in with it) |
|
||||
| Data | bind mount `/home/dank/Containers/hermes/data` → `/data`, shared with the gateway and dashboard; anonymous volume at `/opt/data` |
|
||||
| Health | `HealthCmd=` in the Quadlet unit probes `http://127.0.0.1:8787/health` from inside the unit (works because the webui runs in the host netns); the deployer probes the same path from the host |
|
||||
| Health | `HealthCmd=` in the Quadlet unit probes `http://127.0.0.1:8787/health` from inside the unit (`hermes-webui.container:53`; works because the webui runs in the host netns). Container state 2026-10-18: **healthy**. There is no host-side probe in the deployer — the contract is the seam `tests/hermes_webui.bats`, which reads the live unit and pins the Caddyfile upstream |
|
||||
|
||||
## Why a container, not a host process
|
||||
|
||||
The gateway's state files are owned by the container-mapped uid (`109999`,
|
||||
from rootless Podman's `1 → 100000/65536` subuid range). A fresh rootless
|
||||
from rootless Podman's `1 → 100000/65536` subuid range — `/etc/subuid` →
|
||||
`dank:100000:65536`, and the agent's runtime uid is 10000, so 100000 + 9999 =
|
||||
109999; `ls -ld /home/dank/Containers/hermes/data` → uid 109999, 2026-10-18). A fresh rootless
|
||||
namespace on Dankclaw maps identically (verified with `podman unshare
|
||||
cat /proc/self/uid_map`), so a second container using the same image reads
|
||||
and writes the shared volume with exactly the ownership the gateway already
|
||||
uses. A host-side process running as `dank` (uid 1000) could not read the
|
||||
gateway's `0600` state files.
|
||||
gateway's `0600` state files (a non-privileged read of the data dir is
|
||||
`EACCES`, checked 2026-10-18).
|
||||
|
||||
## How it works
|
||||
|
||||
- `bootstrap.py` creates `/webui/.venv` on first boot, pip-installs the WebUI
|
||||
and agent requirements, then **execs** `server.py` as one foreground process
|
||||
(`--foreground`). systemd-free: Podman supervises it via
|
||||
`restart: unless-stopped`.
|
||||
- `HERMES_WEBUI_SKIP_ONBOARDING=1` keeps the first-run wizard off; a profile
|
||||
(`--foreground`). systemd-free inside the container: supervision is the unit's
|
||||
`Restart=always` / `RestartSec=5` (`hermes-webui.container:60-61`) — the
|
||||
compose-era `restart: unless-stopped` is retired with `dankclaw/compose.yaml`.
|
||||
- `HERMES_WEBUI_SKIP_ONBOARDING=1` keeps the first-run wizard off (`:45`; live
|
||||
env confirms it, `podman inspect hermes-webui` 2026-10-18); a profile
|
||||
already exists in `/data` from the gateway.
|
||||
- The healthcheck `start_period` is 600 s because the very first boot spends
|
||||
- The healthcheck `start_period` is 600 s
|
||||
(`HealthStartPeriod=600s`, `:57`) because the very first boot spends
|
||||
minutes pip-installing into a fresh venv.
|
||||
|
||||
## Remote access (Hermex on iOS — persistent SSH tunnel)
|
||||
|
||||
Only port `22` is forwarded to Dankclaw at the router, so the phone reaches the
|
||||
Only port `22` is forwarded to Dankclaw at the router *(unverified 2026-10-18:
|
||||
router configuration is not readable from this host; `sshd -T` confirms sshd
|
||||
itself listens on port 22)*, so the phone reaches the
|
||||
webui through an SSH **local forward** — no extra router ports, no third-party
|
||||
relay (no Tailscale, no Cloudflare). Implemented 2026-08-31. The tunnel is an
|
||||
*access path* for the phone, **not** a containment boundary: the server binds
|
||||
`0.0.0.0:8787`, so any host that can route to `192.168.0.143` can reach the API
|
||||
directly and needs only `HERMES_WEBUI_PASSWORD` to get in (verified 2026-09-02:
|
||||
an unauthenticated `GET /health` from another LAN machine returned HTTP 200).
|
||||
an unauthenticated `GET /health` from another LAN machine returned HTTP 200;
|
||||
re-measured from a different LAN host on 2026-10-18: `/health` → 200, `/` → 302).
|
||||
See [[Container Network Namespaces]] for why the webui is reachable this way and
|
||||
the sibling dashboard on `9119` is not.
|
||||
|
||||
- Dedicated `tunnel` account on Dankclaw (`/usr/sbin/nologin` shell, password
|
||||
locked). Authentication is key-only; the account can do nothing except
|
||||
- Dedicated `tunnel` account on Dankclaw (`getent passwd tunnel` → uid 994,
|
||||
shell `/usr/sbin/nologin`, 2026-10-18; password locked). Authentication is key-only
|
||||
*(unverified 2026-10-18: `/home/tunnel/.ssh/authorized_keys` is not readable
|
||||
without `sudo`)*; the account can do nothing except
|
||||
authenticate — `ForceCommand /usr/sbin/nologin` kills any session channel
|
||||
while the transport stays up for port forwarding.
|
||||
- sshd drop-in `/etc/ssh/sshd_config.d/99-tunnel.conf` scopes
|
||||
`AllowTcpForwarding yes` to that user only (plus `PermitTTY no`,
|
||||
`X11Forwarding no`).
|
||||
- sshd drop-in `/etc/ssh/sshd_config.d/99-tunnel.conf` (exists, 118 B, 2026-08-31)
|
||||
scopes `AllowTcpForwarding yes` to that user only, plus `PermitTTY no`,
|
||||
`X11Forwarding no` and `ForceCommand /usr/sbin/nologin` — all four lines are
|
||||
present under `Match User tunnel` (`cat` 2026-10-18).
|
||||
- Key options on the authorized key pin the blast radius:
|
||||
`command="/usr/sbin/nologin",no-pty,restrict,port-forwarding,permitopen="localhost:8787"`
|
||||
— a leaked tunnel key can only forward to `localhost:8787`, never act as a
|
||||
general SOCKS/SSH proxy.
|
||||
general SOCKS/SSH proxy *(unverified 2026-10-18: the authorized_keys file
|
||||
needs `sudo` to read)*.
|
||||
- On the phone (client side): create a key on-device (private key never
|
||||
leaves the device), hand the **public** key to the admin who appends it to
|
||||
`/home/tunnel/.ssh/authorized_keys`, then connect from the SSH client:
|
||||
@@ -104,10 +122,16 @@ graph LR
|
||||
./scripts/deploy.sh dankclaw --service hermes-webui
|
||||
```
|
||||
|
||||
`deploy.sh` copies `dankclaw/hermes/quadlet/*.container` into the quadlet unit
|
||||
directory and restarts `hermes.service`, `hermes-dashboard.service` and
|
||||
`deploy_hermes` (`scripts/deploy-remote.sh:1034-1071`; any of
|
||||
`--service hermes|hermes-dashboard|hermes-webui` routes to it,
|
||||
`scripts/deploy-remote.sh:1594`) copies `chrome-devtools-protocol.container`
|
||||
plus the three trio units into the quadlet unit directory, starts the CDP daemon
|
||||
only if inactive (it is never restarted on deploy, so in-flight browser
|
||||
sessions survive), re-tags the image alias, then restarts
|
||||
`hermes.service`, `hermes-dashboard.service` and
|
||||
`hermes-webui.service` in the **user** manager of `dank` (linger must be on for
|
||||
them to come up at boot), so gateway/dashboard/webui restarts stay isolated from
|
||||
them to come up at boot — `loginctl show-user dank -p Linger` → `yes`,
|
||||
2026-10-18), so gateway/dashboard/webui restarts stay isolated from
|
||||
inference deploys.
|
||||
Before first deploy, set a strong `HERMES_WEBUI_PASSWORD` in the live
|
||||
`dankclaw/.env`.
|
||||
@@ -118,21 +142,35 @@ Before first deploy, set a strong `HERMES_WEBUI_PASSWORD` in the live
|
||||
the repo root. Do not flip it to `:ro`.
|
||||
- The webui binds `0.0.0.0:8787`, and **two consumers depend on that**: the SSH
|
||||
tunnel (which dials the host's `localhost`) *and* Caddy, which fronts
|
||||
`hermes.franksdank.cloud` with `reverse_proxy 192.168.0.143:8787` — a host
|
||||
address rather than a container name, because this unit is host-networked and
|
||||
there is no port forwarder to bridge the gap (`dankclaw/Caddyfile` records the
|
||||
same constraint). So the SSH tunnel is an access path for the phone, not a
|
||||
`hermes.franksdank.cloud` with **`reverse_proxy host.containers.internal:8787`**
|
||||
(`dankclaw/Caddyfile:186-188`) — netavark's host address (169.254.1.2), not
|
||||
the host's LAN IP. That distinction is load-bearing: this unit is
|
||||
host-networked and publishes nothing, so netavark DNATs nothing, and
|
||||
`192.168.0.143:8787` is REFUSED from Caddy's bridge netns while
|
||||
`host.containers.internal:8787` answers (measured inside the Caddy container
|
||||
2026-09-16, recorded at `dankclaw/Caddyfile:178-185` and pinned by the seam
|
||||
`tests/hermes_webui.bats:52-64`, which fails any Caddyfile that reverts to the
|
||||
LAN IP). The dashboard's `9119` answers on the LAN IP precisely because it IS
|
||||
published. So the SSH tunnel is an access path for the phone, not a
|
||||
security boundary: any LAN host that can route to `192.168.0.143` reaches the
|
||||
API and needs only `HERMES_WEBUI_PASSWORD`. The webui can run agent commands,
|
||||
so treat that password as a credential with command-execution blast radius.
|
||||
The vhost itself carries **no** guard chain — no Anubis challenge, no
|
||||
Authentik gate (measured 2026-10-18 from another LAN host:
|
||||
`https://hermes.franksdank.cloud/` → 302 → `/login?next=/` and `/health` → 200,
|
||||
with no authentik or bot-challenge bytes in the body); it is private only
|
||||
because no public DNS record exists for the name
|
||||
(`dankclaw/pihole/addn-hosts:39`, LAN-only).
|
||||
To make loopback-only binding true again you must set
|
||||
`HERMES_WEBUI_HOST=127.0.0.1` **and** give Caddy another way in (a
|
||||
host-gateway route or a bridge-network unit) — that is an exposure change to
|
||||
decide deliberately, not a silent fix. It was tried on 2026-09-16 as a silent
|
||||
fix and broke the vhost with a 502 while the whole e2e suite stayed green;
|
||||
`tests/e2e/specs/17-hermes-vhost.spec.ts` now asserts that vhost reaches its
|
||||
app through the guard chain, so a dead upstream fails a gate instead of
|
||||
hiding.
|
||||
`tests/e2e/specs/17-hermes-vhost.spec.ts` now asserts the vhost reaches its
|
||||
app (no 5xx, no Caddy error page, no gate page), so a dead upstream fails a
|
||||
gate instead of hiding. Its file comment still names the
|
||||
`192.168.0.143:8787` upstream and calls this "the only guarded vhost" — both
|
||||
are stale against `dankclaw/Caddyfile:186-188`.
|
||||
- The webui and the gateway share one data root (`/data` is a bind mount of
|
||||
`/home/dank/Containers/hermes/data`). A config change made on one side takes
|
||||
effect when the other restarts (same caveat as the dashboard).
|
||||
|
||||
Reference in new issue
Block a user