The vault was 155 pages / 31,210 lines, most of it measurement ledgers for payloads that no longer run. A throughput figure for a runtime nobody operates is not a record: AI moves faster than the vault, so the number is wrong before anyone reads it, and a wrong number in a runbook gets believed. Per the new purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed. Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest): - the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM, the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two "Running Qwen3.6 …" pages; - the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag) and the multi-device/x4-pair/hybrid/offload experiment pages; - every vLLM page — no vLLM endpoint exists on this fleet; - `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff, self-hosted app census, the TTS survey) that were obsolete by construction; - `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the page that owns the thing it governs, never to a diary. Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras tables keep the window, payload, runtime, reason and rollback pointer; the rate columns are blanked. The live shapes stay verbatim — those numbers are what the deployer and `models.yml` consume.
333 lines
19 KiB
Markdown
333 lines
19 KiB
Markdown
---
|
|
type: reference
|
|
description: "Hermes is the self-improving AI agent from Nous Research."
|
|
tags:
|
|
- dankclaw
|
|
- hermes
|
|
timestamp: 2026-09-15
|
|
---
|
|
|
|
# Hermes
|
|
|
|
Hermes is the self-improving AI agent from Nous Research. It runs in a terminal, Telegram, Discord, Slack, WhatsApp, or Signal, and keeps a persistent learning loop: it creates skills from experience, searches past conversations, and builds a model of the user across sessions.
|
|
|
|
**Host:** Dankclaw — the gateway itself publishes **no port** (it runs in the
|
|
host network namespace; `podman inspect systemd-hermes` → `net=host`, no port
|
|
bindings, 2026-10-18). Note the **container name drift**: the unit is
|
|
`hermes.service` but the generated start line is
|
|
`podman run --name systemd-%N …` (`systemctl --user cat hermes.service`), so
|
|
`podman ps` shows the gateway as **`systemd-hermes`** — the repo unit sets no
|
|
`ContainerName=` key even though its header comment says "container name:
|
|
hermes". The user-facing web surface is [[Hermes-WebUI]] on `:8787`, reached
|
|
`hermes.franksdank.cloud` — internal-only by DNS absence (no public Porkbun A
|
|
record; the name resolves from the Pi-hole hosts file,
|
|
`dankclaw/pihole/addn-hosts:39`) — **Docs:**
|
|
https://hermes-agent.nousresearch.com/docs
|
|
|
|
**Signal is not carrying traffic today (measured 2026-10-18).** The gateway
|
|
boots and stays up, but its own log says
|
|
`Platform 'signal' is explicitly disabled by platforms.signal.enabled: false in
|
|
config.yaml, so the credentials found in the environment (SIGNAL_HTTP_URL) will
|
|
NOT start its adapter` (`journalctl --user -u hermes` 2026-10-18), and the host
|
|
daemon it would dial is not running either: `systemctl --user
|
|
is-enabled signal-cli-daemon` → `disabled`, `is-active` → `inactive`, and
|
|
nothing listens on `:8080` (`ss -ltn`). Everything below still describes the
|
|
designed wiring; the remedy for both halves is in the log line
|
|
(`platforms.signal.enabled: true`) plus enabling the daemon unit.
|
|
|
|
**Persistent state:** `/home/dank/Containers/hermes/data` on Dankclaw, bind-mounted
|
|
at `/data` in all three Hermes containers (gateway, webui, dashboard —
|
|
`hermes.container:36`, `hermes-webui.container:46`
|
|
`hermes-dashboard.container:24`). Each
|
|
container additionally gets an anonymous volume at `/opt/data`, which is
|
|
re-created empty on every container recreation — anything that lives only there
|
|
is not durable. The tracked, sanitized configuration baseline is
|
|
[`dankclaw/hermes/config.yaml.example`](../../dankclaw/hermes/config.yaml.example).
|
|
|
|
## Facts (from research, primary source)
|
|
|
|
| Item | Value |
|
|
|------|-------|
|
|
| Image | `localhost/hermes-agent:latest`, built from `NousResearch/hermes-agent` (official Dockerfile in repo); `Pull=never` in all three units (`hermes.container:28-29`, `hermes-webui.container:35-36`, `hermes-dashboard.container:19-20`), so the tag never moves at runtime. Live image: built 2026-09-24, OCI revision label `749220ef0007f8d87bd1531f1c24b0fe93816385` (`podman image inspect localhost/hermes-agent:latest` 2026-10-18) — that build is what is running. |
|
|
| Upstream latest | agent v0.21.4 / CalVer `v2026.9.21` (2026-09-21); hermes-webui v0.52.113 (2026-08-14). Pin the CalVer tag, not `:latest`. *(unverified 2026-10-18: the local image records only a git revision, and the webui version endpoint answers 401 without a session, so neither number can be re-derived from the host.)* |
|
|
| License | MIT |
|
|
| Model backends | Nous Portal, OpenRouter, OpenAI, or any custom endpoint |
|
|
| Interfaces | TUI, Telegram, Discord, Slack, WhatsApp, Signal |
|
|
| Memory | FTS5 session search, Honcho user modeling |
|
|
| Skills | self-created, self-improving during use |
|
|
|
|
## Configuration
|
|
|
|
- Point Hermes at the local model endpoint or an external provider through its provider config.
|
|
- The gateway runs `hermes gateway run --force -v` in the **host** network
|
|
namespace (`Exec=` + `Network=host`, `dankclaw/hermes/quadlet/hermes.container:27,37`) and reaches the Signal daemon at
|
|
`http://127.0.0.1:8080` (`SIGNAL_HTTP_URL`, `hermes.container:33`). That loopback hop works only because the gateway is in
|
|
the host netns — see [[Container Network Namespaces]]. The daemon itself is a
|
|
**host** user unit (`~/.config/systemd/user/signal-cli-daemon.service`), not a
|
|
container. **Today it is `disabled` + `inactive` and nothing is bound to
|
|
`:8080`** (`systemctl --user is-enabled/is-active`, `ss -ltn`, both
|
|
2026-10-18), and the gateway's own config gate (`platforms.signal.enabled:
|
|
false`) keeps the adapter off even if the daemon came back — see the dated
|
|
note at the top of this page. When the daemon does run it binds
|
|
`0.0.0.0:8080`, i.e. the Signal HTTP API is reachable from the whole LAN.
|
|
- The data volume persists skills, memory, and session history.
|
|
- Hermes has two named local providers (the `dankclaw-large` name is confirmed
|
|
live from the gateway log — `resolve_provider_client: named custom provider
|
|
'dankclaw-large' has no resolvable api_key`, `journalctl --user -u hermes`
|
|
2026-10-18; the base URLs below are the repo record, since
|
|
`/data/config.yaml` is not readable without `sudo`):
|
|
`dankclaw-large` reaches `http://192.168.0.143:7770/v1` and exposes
|
|
`frank-dragoon` — verified from the host with `curl
|
|
http://127.0.0.1:7770/v1/models` → a single served id (2026-10-18; write
|
|
`127.0.0.1`, not `localhost`, because that port is a rootless pasta publish
|
|
and is IPv4-only)
|
|
`dankserver-small` reaches `http://192.168.0.235:7770/v1` and, **as of
|
|
2026-10-18, that endpoint serves only `frank-archon`** (`GET
|
|
http://192.168.0.235:7770/v1/models` → `[{"id":"frank-archon"}]`). The name
|
|
this page used to carry for it — `frank-medium`, renamed from `frank-small` on
|
|
2026-09-15 — is retired-era: nothing on that endpoint answers to it today, so
|
|
if the live `config.yaml` still names `frank-medium` the provider is dead
|
|
weight and requests will 404 on a nonexistent model. Diff the live config
|
|
before trusting either name. Which payload sits behind each port, and the
|
|
served id it answers to, is on [[frank-dragoon]] and [[frank-archon]].
|
|
- The live `config.yaml` contains secrets and user state and is not committed;
|
|
the tracked example is the sanitized backup of its non-secret settings.
|
|
|
|
## Volumes
|
|
|
|
| Container path | Host path (live) | Backing |
|
|
|----------------|------------------|---------|
|
|
| `/data` | `/home/dank/Containers/hermes/data` | bind mount (durable; shared by all three containers) |
|
|
| `/webui` | `/home/dank/Containers/hermes/webui` | bind mount (webui only — the `bootstrap.py` venv lives here) |
|
|
| `/opt/data` | anonymous volume | image `VOLUME`; fresh empty volume per recreation |
|
|
|
|
## Gotchas
|
|
|
|
- Hermes is under active development; pin the image tag for reproducibility. The units already pin by absence of pulling (`Pull=never`) and by building `localhost/hermes-agent:latest` locally — a rebuild, not a pull, is what changes the code.
|
|
- Model choice drives quality; the local `frank-dragoon` (Dankclaw, 7770) is the intended backend.
|
|
- Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth. The gateway itself publishes no port today; the surfaces that are exposed are the webui (`hermes.` vhost, exempt by construction because the app enforces its own password — see [[Hermes-WebUI]]) and the dashboard (`hermesadmin.` vhost → `192.168.0.143:9119`, app-enforced basic auth — see [[Hermes-Dashboard]]).
|
|
|
|
## Browser automation (Chrome DevTools Protocol daemon)
|
|
|
|
The agent's browser tool is an MCP client (`mcp_tool.py` in the image) that
|
|
spawns `npx -y chrome-devtools-mcp@latest --browserUrl=http://127.0.0.1:9222`
|
|
per session; the browser itself lives in a dedicated daemon container, the
|
|
digest-pinned headless-shell unit `chrome-devtools-protocol.container` (host
|
|
network, loopback-only bind on 127.0.0.1:9222 — `ss -ltnp` 2026-10-18 shows
|
|
`127.0.0.1:9222 headless-shell` and nothing on `0.0.0.0`; image
|
|
`docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5…`
|
|
`dankclaw/hermes/quadlet/chrome-devtools-protocol.container:28`). Both consumers — the Signal
|
|
gateway and the webui server — run in the host network namespace (`podman
|
|
inspect` → `net=host` for both, 2026-10-18), so they can
|
|
dial the daemon on loopback; nothing on the LAN can reach the CDP endpoint.
|
|
`GET http://127.0.0.1:9222/json/version` answers **200** today with
|
|
`Browser: Chrome/151.0.7922.109`, `Protocol-Version: 1.3` (2026-10-18).
|
|
The wiring lives in the live `/data/config.yaml` under `mcp_servers.chrome-devtools`
|
|
(command `npx`, args `-y chrome-devtools-mcp@latest --browserUrl=http://127.0.0.1:9222`) —
|
|
that file is mode `drwx------` uid 109999, so reading it needs `sudo` (a
|
|
non-privileged read is `EACCES`, checked 2026-10-18).
|
|
The npx package cache lives in the container layer, so a recreation (not just a
|
|
restart) pays one first-call npm download; subsequent calls are cached.
|
|
|
|
## Config-home drift ("cannot connect" to models that answer curl)
|
|
|
|
`podman exec <container> printenv HERMES_HOME` for each hermes container. If
|
|
the webui reads the image default (`/opt/data`) while the gateway reads
|
|
`/data`, the webui runs the image-default template config (`provider: auto` →
|
|
openrouter.ai with no key = connect failure).
|
|
|
|
The recreation trap is the same one the volumes table shows: image `VOLUME`
|
|
points get a fresh anonymous volume on every container recreation
|
|
(`podman run --replace --rm`) — edits inside such a mount are lost on
|
|
restart/redeploy. Compare volume ids before/after a restart to confirm.
|
|
|
|
Fix: pin `Environment=HERMES_HOME=/data` in the quadlet unit (shared state
|
|
root), deploy service-scoped. Verify with the app's own code, not guesses:
|
|
|
|
```python
|
|
from api import config as C; C.resolve_model_provider("<model-id>")
|
|
# -> (model_id, provider, base_url); bare ids map to the owning provider by
|
|
# scanning the providers: dict models allow-list; @provider:model forces one.
|
|
hermes_cli.providers.resolve_user_provider(name, cfg["providers"])
|
|
# -> ProviderDef with real base_url; accepts api/url/base_url keys.
|
|
```
|
|
|
|
Named providers (`providers:` dict) are a catalog + connection source;
|
|
`custom:<slug>` connections resolve only via `custom_providers[]` entries. If
|
|
neither holds the provider name, the request has no endpoint and fails to
|
|
connect. Probe the endpoint with the **exact served model id** — the retired
|
|
FLM/NPU endpoints routed only by exact registry tag and answered HTTP 200
|
|
with `{"error":"Failed to load <tag> model!"}` for unknown tags; status-code
|
|
checks miss this class entirely.
|
|
|
|
## Live config edits
|
|
|
|
Live file: `/home/dank/Containers/hermes/data/config.yaml` (dir `drwx------`
|
|
uid 109999 — `ls -ld /home/dank/Containers/hermes/data` → `drwx------ 29
|
|
109999 109999` 2026-10-18; a non-privileged read returns `EACCES`, so all
|
|
reads/edits need `sudo`). The repo file
|
|
`dankclaw/hermes/config.yaml.example` is a sanitized backup and **can lag
|
|
live — always diff the live file first; never treat the example as truth**.
|
|
|
|
1. **Backup first:** `sudo cp -a …/config.yaml …/config.yaml.bak-YYYYMMDD`
|
|
(in-dir `.AGENTTRASH/` also exists for removals *(unverified 2026-10-18:
|
|
inside the `0700` data dir, unreadable without `sudo`)*).
|
|
2. **Apply** with surgical `sudo sed -i` or full-file write; YAML comments in
|
|
the live file are tool-generated — preserve them.
|
|
3. **Sweep stale references** across the whole data dir:
|
|
`sudo grep -rln "<old-id>" /home/dank/Containers/hermes/data
|
|
--include="*.yaml" --include="*.json"`. Session files/MEMORY.md are
|
|
historical — leave them.
|
|
4. **Derived caches:** `webui/models_cache.json` is keyed by a config
|
|
fingerprint (`_source_fingerprint.config_yaml` mtime/size); after edits it
|
|
is stale. Let it regenerate lazily or trash it — never hand-edit.
|
|
5. **Restart the model-consuming units** (they parse config at startup):
|
|
`systemctl --user restart hermes-webui.service hermes.service`.
|
|
`hermes-dashboard.service` is unaffected by routing changes.
|
|
6. **Verify behaviorally**: one tiny completion per endpoint with the exact
|
|
served id — POST `/v1/chat/completions` and read the body, never just the
|
|
status code (endpoint names rotate; see the LLM knowledge base).
|
|
7. **Sync the repo backup** (`dankclaw/hermes/config.yaml.example`) to the
|
|
live shape, redacting secrets (`password_hash`, phone account stay
|
|
live-only), then `./tests/run.sh` before commit.
|
|
|
|
`provider: custom` + `base_url` entries route by base_url and send the model
|
|
id verbatim; named providers resolve via their `api:` URL. **Renaming a
|
|
served alias anywhere (serve scripts, env ALIAS) MUST propagate to this
|
|
config in the same change** or Hermes requests 404 on nonexistent models —
|
|
this exact gap caused the 2026-09-05 outage.
|
|
|
|
## Auxiliary routing against local backends
|
|
|
|
When a Hermes feature that calls an auxiliary task (e.g. title generation)
|
|
fails against a local endpoint, the usual cause is a **request shape the
|
|
local runtime does not implement**, not a routing mistake.
|
|
|
|
Diagnose with the real call path — run the actual function inside the
|
|
container with the app's own constants (the probe cannot drift from
|
|
production):
|
|
|
|
```bash
|
|
base64 -w0 /tmp/probe.py | ssh dank@192.168.0.143 \
|
|
'base64 -d > /tmp/probe.py && systemctl --user restart hermes.service && sleep 20 && \
|
|
podman cp /tmp/probe.py systemd-hermes:/tmp/probe.py && \
|
|
podman exec -i systemd-hermes /opt/hermes/.venv/bin/python /tmp/probe.py'
|
|
```
|
|
|
|
Copy the file AFTER any restart (`--replace --rm` discards the container
|
|
layer). The CLI at `/opt/hermes/hermes` uses system python and dies
|
|
`No module named 'yaml'`; use `/opt/hermes/.venv/bin/python`.
|
|
|
|
```python
|
|
import sys; sys.path.insert(0, "/opt/hermes")
|
|
from agent.title_generator import _TITLE_PROMPT_TEMPLATE, _TITLE_RESPONSE_FORMAT, _clean_title, _extract_title_text
|
|
from agent.auxiliary_client import call_llm
|
|
resp = call_llm(task="title_generation", messages=[...], max_tokens=64,
|
|
temperature=0.3, extra_body={"response_format": _TITLE_RESPONSE_FORMAT})
|
|
```
|
|
|
|
What actually breaks, and the fix:
|
|
|
|
- **`response_format` (json_object / json_schema) is a hard 400 on the local
|
|
backends this task was written against** — what each endpoint accepts on the
|
|
wire is on [[frank-dragoon]] / [[frank-zealot]] / [[frank-archon]]. Config
|
|
cannot override it: `call_llm` merges the task's
|
|
`extra_body` first and then `update()`s the caller's, so the hardcoded one
|
|
wins. Only temperature and max_tokens have reactive retry paths.
|
|
- **Fix: move that one task to the Responses transport**, which never
|
|
forwards `response_format`:
|
|
|
|
```yaml
|
|
auxiliary:
|
|
title_generation:
|
|
provider: dankserver-small # names track the live config; diff first
|
|
model: frank-zealot
|
|
api_mode: codex_responses # real per-task key
|
|
extra_body:
|
|
reasoning:
|
|
effort: none # mandatory, see below
|
|
```
|
|
|
|
Before copying this: which id is served on which port is endpoint state —
|
|
[[frank-zealot]] and [[frank-archon]]. The provider/model
|
|
pairing above is the shape that was landed in 2026; re-derive it from the live
|
|
config before reusing it.
|
|
|
|
- **Disable thinking for any cap-limited task.** The adapter omits reasoning
|
|
only when config says `reasoning.enabled: false`. The
|
|
`reasoning_effort: none` *shorthand* produces exactly that and leaves the
|
|
server default (thinking ON) — the 64-token title budget gets spent on the
|
|
think: empty content, `finish_reason: length`. The explicit
|
|
`extra_body.reasoning.effort: none` instead sends
|
|
`{"effort":"none","summary":"auto"}`, which the local `/v1/responses`
|
|
honors.
|
|
- Both local runtimes also answer `/v1/responses` and `/v1/messages` — no
|
|
proxy needed for the alternate transport.
|
|
|
|
Verify and land: re-run the probe with **no** `api_mode`/reasoning kwargs —
|
|
it must pass from config alone (that is the production path); keep the live
|
|
file and the repo example in step; `./tests/run.sh`, commit, push — a Hermes
|
|
config change is docs+example only, no service deploy needed.
|
|
|
|
Gotchas: `auxiliary.<task>.enabled: false` is checked BEFORE
|
|
`apply_instant_title`, so disabling a broken title task also removes the
|
|
derived title — it does not degrade gracefully the way a failure does. A
|
|
local call can also fail at the model level: a 200 with an error-text body,
|
|
or a chatty non-title answer. Always read the raw content, not the status.
|
|
|
|
## Webui password rotation ("Hermex")
|
|
|
|
Auth contract (verified live 2026-08-31, re-checked 2026-10-18): `POST /api/auth/login` with JSON
|
|
`{"password":"…"}` → 200 + signed HttpOnly `hermes_session` cookie (24 h TTL)
|
|
on match, 401 on miss. No username field. `/health` answers 200
|
|
unauthenticated (`curl http://127.0.0.1:8787/health` → 200 with
|
|
`{"status":"ok", …}` today) and `/` answers 302 (`curl -o /dev/null -w
|
|
%{http_code} http://127.0.0.1:8787/` → 302 today) — **never verify auth
|
|
those**; probe
|
|
login with a deliberately wrong password (expect 401) and the real one
|
|
(expect 200). Details on the webui surface: [[Hermes-WebUI]].
|
|
|
|
1. Run the whole probe as a heredoc'd bash script
|
|
(`ssh -o BatchMode=yes dankclaw 'bash -s'`) — the host's zsh mangles
|
|
inline `${…}` even inside single-quoted ssh args; never interpolate curl
|
|
`-w %{http_code}` inline.
|
|
2. Update BOTH env files with
|
|
`sed -i "s/^HERMES_WEBUI_PASSWORD=.*/HERMES_WEBUI_PASSWORD=<new>/"`:
|
|
- `/home/dank/serverAppConfigs/.env` — the repo-root checkout's env file
|
|
(exists, `-rw------- dank dank`, 2026-10-18) but **not** loaded by this
|
|
unit: keep it in sync only so other consumers do not drift.
|
|
- `/home/dank/serverAppConfigs/dankclaw/.env` — **the effective one, for
|
|
both `[Container]` and `[Service]`**: `systemctl --user show hermes-webui
|
|
-p EnvironmentFiles` reports exactly
|
|
`/home/dank/serverAppConfigs/dankclaw/.env` (2026-10-18;
|
|
`dankclaw/hermes/quadlet/hermes-webui.container:38,62`). That is what the
|
|
container reads at create; rotating only the root file leaves the running
|
|
container serving the old secret.
|
|
3. `systemctl --user restart hermes-webui.service` (unit recreation re-reads
|
|
env; no deploy needed if the unit file itself didn't change).
|
|
4. Verify wrong-pw → 401, right-pw → 200, from host loopback **and** a LAN
|
|
client at `http://192.168.0.143:8787` (webui publishes on the LAN
|
|
interface since commit `4443897`, 2026-08-31 "publish webui on LAN interface
|
|
instead of loopback" — live bind `0.0.0.0:8787`, `ss -ltnp` 2026-10-18;
|
|
off-network ingress is SSH tunnel only —
|
|
user `tunnel`, port **22** (`sshd -T` → `port 22`, 2026-10-18), key-only auth
|
|
- (unverified 2026-10-18: the per-user `Match` block needs `sudo` to read)*
|
|
forward remote end
|
|
localhost:8787).
|
|
|
|
Gotchas: `sudo podman exec` fails exit 125 here — root's namespace does not
|
|
see rootless containers; run `podman exec hermes-webui printenv
|
|
HERMES_WEBUI_PASSWORD` unprivileged (the ssh user owns the env files, mode
|
|
600). If rotating because a secret leaked into transcripts/logs, rotate in
|
|
place via a script that fetches the live value from the running container
|
|
(`printenv`) instead of transcribing it.
|
|
|
|
## See Also
|
|
|
|
- [[Hermes-Dashboard]] — the LAN browser management UI
|
|
- [[Hermes-WebUI]] — the Hermex server (phone-facing API, SSH-tunneled)
|
|
- [[Dankclaw Stack]]
|
|
- [[OpenWebUI]] — the chat UI that pairs with Hermes
|
|
- Knowledge base: [[SearXNG]] (Hermes as the search agent)
|
|
- [[Container Network Namespaces]] — why the gateway and webui can dial loopback services and the dashboard cannot
|