Files
serverAppConfigs/serverAppConfigs-docs/Dankclaw/Hermes.md
T
Frank 87721ab0fe rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)
Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:

- Service trio: scripts/{frank-small.env,frank-small.service,
  serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
  frank-secondary-*.md (9 files); the FreeToken runbook stays
  freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
  LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
  3090, and the Dankserver service that became frank-medium) are
  preserved verbatim — they describe what the service was called
  at the time.
2026-10-06 15:37:13 -07:00

14 KiB

type, description, tags, timestamp
type description tags timestamp
reference Hermes is the self-improving AI agent from Nous Research.
dankclaw
hermes
2026-09-15

Hermes

Hermes is the self-improving AI agent from Nous Research. It runs in a terminal, Telegram, Discord, Slack, WhatsApp, or Signal, and keeps a persistent learning loop: it creates skills from experience, searches past conversations, and builds a model of the user across sessions.

Host: Dankclaw — the gateway itself publishes no port (it runs in the host network namespace and dials the Signal daemon at 127.0.0.1:8080); the user-facing web surface is Hermes-WebUI on :8787, reached via hermes.franksdank.cloud — internal-only by DNS absence (no public Porkbun A record) — Docs: https://hermes-agent.nousresearch.com/docs

Persistent state: /home/dank/Containers/hermes/data on Dankclaw, bind-mounted at /data in all three Hermes containers (gateway, webui, dashboard). Each container additionally gets an anonymous volume at /opt/data, which is re-created empty on every container recreation — anything that lives only there is not durable. The tracked, sanitized configuration baseline is dankclaw/hermes/config.yaml.example.

Facts (from research, primary source)

Item Value
Image built from NousResearch/hermes-agent (official Dockerfile in repo)
Upstream latest agent v0.21.4 / CalVer v2026.9.21 (2026-09-21); hermes-webui v0.52.113 (2026-08-14). Pin the CalVer tag, not :latest.
License MIT
Model backends Nous Portal, OpenRouter, OpenAI, or any custom endpoint
Interfaces TUI, Telegram, Discord, Slack, WhatsApp, Signal
Memory FTS5 session search, Honcho user modeling
Skills self-created, self-improving during use

Configuration

  • Point Hermes at the local model endpoint or an external provider through its provider config.
  • The gateway runs hermes gateway run --force -v in the host network namespace (Network=host in its Quadlet unit) and reaches the Signal daemon at http://127.0.0.1:8080. That loopback hop works only because the gateway is in the host netns — see Container Network Namespaces. The daemon itself is a host user unit (signal-cli-daemon.service), not a container, and binds 0.0.0.0:8080, so the Signal HTTP API is reachable from the whole LAN.
  • The data volume persists skills, memory, and session history.
  • Hermes has two named local providers: dankclaw-large reaches http://192.168.0.143:7770/v1 and exposes frank-large; dankserver-small reaches http://192.168.0.235:7770/v1 and exposes frank-medium (renamed from frank-small on 2026-09-15).
  • The live config.yaml contains secrets and user state and is not committed; the tracked example is the sanitized backup of its non-secret settings.

Volumes

Container path Host path (live) Backing
/data /home/dank/Containers/hermes/data bind mount (durable; shared by all three containers)
/webui /home/dank/Containers/hermes/webui bind mount (webui only — the bootstrap.py venv lives here)
/opt/data anonymous volume image VOLUME; fresh empty volume per recreation

Gotchas

  • Hermes is under active development; pin the image tag for reproducibility.
  • Model choice drives quality; the local frank-large (Dankclaw, 7770) is the intended backend.
  • Do not expose the gateway without auth — put it behind Caddy with basic auth.

Browser automation (Chrome DevTools Protocol daemon)

The agent's browser tool is an MCP client (mcp_tool.py in the image) that spawns npx -y chrome-devtools-mcp@latest --browserUrl=http://127.0.0.1:9222 per session; the browser itself lives in a dedicated daemon container, the digest-pinned headless-shell unit chrome-devtools-protocol.container (host network, loopback-only bind on 127.0.0.1:9222). Both consumers — the Signal gateway and the webui server — run in the host network namespace, so they can dial the daemon on loopback; nothing on the LAN can reach the CDP endpoint. The wiring lives in the live /data/config.yaml under mcp_servers.chrome-devtools (command npx, args -y chrome-devtools-mcp@latest --browserUrl=http://127.0.0.1:9222). The npx package cache lives in the container layer, so a recreation (not just a restart) pays one first-call npm download; subsequent calls are cached.

Config-home drift ("cannot connect" to models that answer curl)

podman exec <container> printenv HERMES_HOME for each hermes container. If the webui reads the image default (/opt/data) while the gateway reads /data, the webui runs the image-default template config (provider: auto → openrouter.ai with no key = connect failure).

The recreation trap is the same one the volumes table shows: image VOLUME points get a fresh anonymous volume on every container recreation (podman run --replace --rm) — edits inside such a mount are lost on restart/redeploy. Compare volume ids before/after a restart to confirm.

Fix: pin Environment=HERMES_HOME=/data in the quadlet unit (shared state root), deploy service-scoped. Verify with the app's own code, not guesses:

from api import config as C; C.resolve_model_provider("<model-id>")
# -> (model_id, provider, base_url); bare ids map to the owning provider by
#    scanning the providers: dict models allow-list; @provider:model forces one.
hermes_cli.providers.resolve_user_provider(name, cfg["providers"])
# -> ProviderDef with real base_url; accepts api/url/base_url keys.

Named providers (providers: dict) are a catalog + connection source; custom:<slug> connections resolve only via custom_providers[] entries. If neither holds the provider name, the request has no endpoint and fails to connect. Probe the endpoint with the exact served model id — the retired FLM/NPU endpoints routed only by exact registry tag and answered HTTP 200 with {"error":"Failed to load <tag> model!"} for unknown tags; status-code checks miss this class entirely.

Live config edits

Live file: /home/dank/Containers/hermes/data/config.yaml (dir drwx------ uid 109999; all reads/edits need sudo). The repo file dankclaw/hermes/config.yaml.example is a sanitized backup and can lag live — always diff the live file first; never treat the example as truth.

  1. Backup first: sudo cp -a …/config.yaml …/config.yaml.bak-YYYYMMDD (in-dir .AGENTTRASH/ also exists for removals).
  2. Apply with surgical sudo sed -i or full-file write; YAML comments in the live file are tool-generated — preserve them.
  3. Sweep stale references across the whole data dir: sudo grep -rln "<old-id>" /home/dank/Containers/hermes/data --include="*.yaml" --include="*.json". Session files/MEMORY.md are historical — leave them.
  4. Derived caches: webui/models_cache.json is keyed by a config fingerprint (_source_fingerprint.config_yaml mtime/size); after edits it is stale. Let it regenerate lazily or trash it — never hand-edit.
  5. Restart the model-consuming units (they parse config at startup): systemctl --user restart hermes-webui.service hermes.service. hermes-dashboard.service is unaffected by routing changes.
  6. Verify behaviorally: one tiny completion per endpoint with the exact served id — POST /v1/chat/completions and read the body, never just the status code (endpoint names rotate; see the LLM knowledge base).
  7. Sync the repo backup (dankclaw/hermes/config.yaml.example) to the live shape, redacting secrets (password_hash, phone account stay live-only), then ./tests/run.sh before commit.

provider: custom + base_url entries route by base_url and send the model id verbatim; named providers resolve via their api: URL. Renaming a served alias anywhere (serve scripts, env ALIAS) MUST propagate to this config in the same change or Hermes requests 404 on nonexistent models — this exact gap caused the 2026-09-05 outage.

Auxiliary routing against local backends

When a Hermes feature that calls an auxiliary task (e.g. title generation) fails against a local endpoint, the usual cause is a request shape the local runtime does not implement, not a routing mistake.

Diagnose with the real call path — run the actual function inside the container with the app's own constants (the probe cannot drift from production):

base64 -w0 /tmp/probe.py | ssh dank@192.168.0.143 \
  'base64 -d > /tmp/probe.py && systemctl --user restart hermes.service && sleep 20 && \
   podman cp /tmp/probe.py systemd-hermes:/tmp/probe.py && \
   podman exec -i systemd-hermes /opt/hermes/.venv/bin/python /tmp/probe.py'

Copy the file AFTER any restart (--replace --rm discards the container layer). The CLI at /opt/hermes/hermes uses system python and dies No module named 'yaml'; use /opt/hermes/.venv/bin/python.

import sys; sys.path.insert(0, "/opt/hermes")
from agent.title_generator import _TITLE_PROMPT_TEMPLATE, _TITLE_RESPONSE_FORMAT, _clean_title, _extract_title_text
from agent.auxiliary_client import call_llm
resp = call_llm(task="title_generation", messages=[...], max_tokens=64,
                temperature=0.3, extra_body={"response_format": _TITLE_RESPONSE_FORMAT})

What actually breaks, and the fix:

  • response_format (json_object / json_schema) is a hard 400 on every local backend here — neither FreeToken nor halogen implements constrained decoding. Config cannot override it: call_llm merges the task's extra_body first and then update()s the caller's, so the hardcoded one wins. Only temperature and max_tokens have reactive retry paths.

  • Fix: move that one task to the Responses transport, which never forwards response_format:

    auxiliary:
      title_generation:
        provider: dankserver-small        # names track the live config; diff first
        model: frank-secondary
        api_mode: codex_responses        # real per-task key
        extra_body:
          reasoning:
            effort: none                 # mandatory, see below
    
  • Disable thinking for any cap-limited task. The adapter omits reasoning only when config says reasoning.enabled: false. The reasoning_effort: none shorthand produces exactly that and leaves the server default (thinking ON) — the 64-token title budget gets spent on the think: empty content, finish_reason: length. The explicit extra_body.reasoning.effort: none instead sends {"effort":"none","summary":"auto"}, which the local /v1/responses honors.

  • Both local runtimes also answer /v1/responses and /v1/messages — no proxy needed for the alternate transport.

Verify and land: re-run the probe with no api_mode/reasoning kwargs — it must pass from config alone (that is the production path); keep the live file and the repo example in step; ./tests/run.sh, commit, push — a Hermes config change is docs+example only, no service deploy needed.

Gotchas: auxiliary.<task>.enabled: false is checked BEFORE apply_instant_title, so disabling a broken title task also removes the derived title — it does not degrade gracefully the way a failure does. A local call can also fail at the model level: a 200 with an error-text body, or a chatty non-title answer. Always read the raw content, not the status.

Webui password rotation ("Hermex")

Auth contract (verified live 2026-08-31): POST /api/auth/login with JSON {"password":"…"} → 200 + signed HttpOnly hermes_session cookie (24 h TTL) on match, 401 on miss. No username field. /health answers 200 unauthenticated and / answers 302 — never verify auth via those; probe login with a deliberately wrong password (expect 401) and the real one (expect 200). Details on the webui surface: Hermes-WebUI.

  1. Run the whole probe as a heredoc'd bash script (ssh -o BatchMode=yes dankclaw 'bash -s') — the host's zsh mangles inline ${…} even inside single-quoted ssh args; never interpolate curl -w %{http_code} inline.
  2. Update BOTH env files with sed -i "s/^HERMES_WEBUI_PASSWORD=.*/HERMES_WEBUI_PASSWORD=<new>/":
    • /home/dank/serverAppConfigs/.env ([Service] EnvironmentFile)
    • /home/dank/serverAppConfigs/dankclaw/.env — the effective one: the quadlet's [Container] EnvironmentFile= is what the container reads at create; rotating only the root file leaves the running container serving the old secret.
  3. systemctl --user restart hermes-webui.service (unit recreation re-reads env; no deploy needed if the unit file itself didn't change).
  4. Verify wrong-pw → 401, right-pw → 200, from host loopback and a LAN client at http://192.168.0.143:8787 (webui publishes on the LAN interface since commit 4443897; off-network ingress is SSH tunnel only — user tunnel, port 22, key-only auth, forward remote end localhost:8787).

Gotchas: sudo podman exec fails exit 125 here — root's namespace does not see rootless containers; run podman exec hermes-webui printenv HERMES_WEBUI_PASSWORD unprivileged (the ssh user owns the env files, mode 600). If rotating because a secret leaked into transcripts/logs, rotate in place via a script that fetches the live value from the running container (printenv) instead of transcribing it.

See Also