Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:
- Service trio: scripts/{frank-small.env,frank-small.service,
serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
frank-secondary-*.md (9 files); the FreeToken runbook stays
freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
3090, and the Dankserver service that became frank-medium) are
preserved verbatim — they describe what the service was called
at the time.
14 KiB
type, description, tags, timestamp
| type | description | tags | timestamp | ||
|---|---|---|---|---|---|
| reference | Hermes is the self-improving AI agent from Nous Research. |
|
2026-09-15 |
Hermes
Hermes is the self-improving AI agent from Nous Research. It runs in a terminal, Telegram, Discord, Slack, WhatsApp, or Signal, and keeps a persistent learning loop: it creates skills from experience, searches past conversations, and builds a model of the user across sessions.
Host: Dankclaw — the gateway itself publishes no port (it runs in the
host network namespace and dials the Signal daemon at 127.0.0.1:8080); the
user-facing web surface is Hermes-WebUI on :8787, reached via
hermes.franksdank.cloud — internal-only by DNS absence (no public Porkbun A
record) — Docs: https://hermes-agent.nousresearch.com/docs
Persistent state: /home/dank/Containers/hermes/data on Dankclaw, bind-mounted
at /data in all three Hermes containers (gateway, webui, dashboard). Each
container additionally gets an anonymous volume at /opt/data, which is
re-created empty on every container recreation — anything that lives only there
is not durable. The tracked, sanitized configuration baseline is
dankclaw/hermes/config.yaml.example.
Facts (from research, primary source)
| Item | Value |
|---|---|
| Image | built from NousResearch/hermes-agent (official Dockerfile in repo) |
| Upstream latest | agent v0.21.4 / CalVer v2026.9.21 (2026-09-21); hermes-webui v0.52.113 (2026-08-14). Pin the CalVer tag, not :latest. |
| License | MIT |
| Model backends | Nous Portal, OpenRouter, OpenAI, or any custom endpoint |
| Interfaces | TUI, Telegram, Discord, Slack, WhatsApp, Signal |
| Memory | FTS5 session search, Honcho user modeling |
| Skills | self-created, self-improving during use |
Configuration
- Point Hermes at the local model endpoint or an external provider through its provider config.
- The gateway runs
hermes gateway run --force -vin the host network namespace (Network=hostin its Quadlet unit) and reaches the Signal daemon athttp://127.0.0.1:8080. That loopback hop works only because the gateway is in the host netns — see Container Network Namespaces. The daemon itself is a host user unit (signal-cli-daemon.service), not a container, and binds0.0.0.0:8080, so the Signal HTTP API is reachable from the whole LAN. - The data volume persists skills, memory, and session history.
- Hermes has two named local providers:
dankclaw-largereacheshttp://192.168.0.143:7770/v1and exposesfrank-large;dankserver-smallreacheshttp://192.168.0.235:7770/v1and exposesfrank-medium(renamed fromfrank-smallon 2026-09-15). - The live
config.yamlcontains secrets and user state and is not committed; the tracked example is the sanitized backup of its non-secret settings.
Volumes
| Container path | Host path (live) | Backing |
|---|---|---|
/data |
/home/dank/Containers/hermes/data |
bind mount (durable; shared by all three containers) |
/webui |
/home/dank/Containers/hermes/webui |
bind mount (webui only — the bootstrap.py venv lives here) |
/opt/data |
anonymous volume | image VOLUME; fresh empty volume per recreation |
Gotchas
- Hermes is under active development; pin the image tag for reproducibility.
- Model choice drives quality; the local
frank-large(Dankclaw, 7770) is the intended backend. - Do not expose the gateway without auth — put it behind Caddy with basic auth.
Browser automation (Chrome DevTools Protocol daemon)
The agent's browser tool is an MCP client (mcp_tool.py in the image) that
spawns npx -y chrome-devtools-mcp@latest --browserUrl=http://127.0.0.1:9222
per session; the browser itself lives in a dedicated daemon container, the
digest-pinned headless-shell unit chrome-devtools-protocol.container (host
network, loopback-only bind on 127.0.0.1:9222). Both consumers — the Signal
gateway and the webui server — run in the host network namespace, so they can
dial the daemon on loopback; nothing on the LAN can reach the CDP endpoint.
The wiring lives in the live /data/config.yaml under mcp_servers.chrome-devtools
(command npx, args -y chrome-devtools-mcp@latest --browserUrl=http://127.0.0.1:9222).
The npx package cache lives in the container layer, so a recreation (not just a
restart) pays one first-call npm download; subsequent calls are cached.
Config-home drift ("cannot connect" to models that answer curl)
podman exec <container> printenv HERMES_HOME for each hermes container. If
the webui reads the image default (/opt/data) while the gateway reads
/data, the webui runs the image-default template config (provider: auto →
openrouter.ai with no key = connect failure).
The recreation trap is the same one the volumes table shows: image VOLUME
points get a fresh anonymous volume on every container recreation
(podman run --replace --rm) — edits inside such a mount are lost on
restart/redeploy. Compare volume ids before/after a restart to confirm.
Fix: pin Environment=HERMES_HOME=/data in the quadlet unit (shared state
root), deploy service-scoped. Verify with the app's own code, not guesses:
from api import config as C; C.resolve_model_provider("<model-id>")
# -> (model_id, provider, base_url); bare ids map to the owning provider by
# scanning the providers: dict models allow-list; @provider:model forces one.
hermes_cli.providers.resolve_user_provider(name, cfg["providers"])
# -> ProviderDef with real base_url; accepts api/url/base_url keys.
Named providers (providers: dict) are a catalog + connection source;
custom:<slug> connections resolve only via custom_providers[] entries. If
neither holds the provider name, the request has no endpoint and fails to
connect. Probe the endpoint with the exact served model id — the retired
FLM/NPU endpoints routed only by exact registry tag and answered HTTP 200
with {"error":"Failed to load <tag> model!"} for unknown tags; status-code
checks miss this class entirely.
Live config edits
Live file: /home/dank/Containers/hermes/data/config.yaml (dir drwx------
uid 109999; all reads/edits need sudo). The repo file
dankclaw/hermes/config.yaml.example is a sanitized backup and can lag
live — always diff the live file first; never treat the example as truth.
- Backup first:
sudo cp -a …/config.yaml …/config.yaml.bak-YYYYMMDD(in-dir.AGENTTRASH/also exists for removals). - Apply with surgical
sudo sed -ior full-file write; YAML comments in the live file are tool-generated — preserve them. - Sweep stale references across the whole data dir:
sudo grep -rln "<old-id>" /home/dank/Containers/hermes/data --include="*.yaml" --include="*.json". Session files/MEMORY.md are historical — leave them. - Derived caches:
webui/models_cache.jsonis keyed by a config fingerprint (_source_fingerprint.config_yamlmtime/size); after edits it is stale. Let it regenerate lazily or trash it — never hand-edit. - Restart the model-consuming units (they parse config at startup):
systemctl --user restart hermes-webui.service hermes.service.hermes-dashboard.serviceis unaffected by routing changes. - Verify behaviorally: one tiny completion per endpoint with the exact
served id — POST
/v1/chat/completionsand read the body, never just the status code (endpoint names rotate; see the LLM knowledge base). - Sync the repo backup (
dankclaw/hermes/config.yaml.example) to the live shape, redacting secrets (password_hash, phone account stay live-only), then./tests/run.shbefore commit.
provider: custom + base_url entries route by base_url and send the model
id verbatim; named providers resolve via their api: URL. Renaming a
served alias anywhere (serve scripts, env ALIAS) MUST propagate to this
config in the same change or Hermes requests 404 on nonexistent models —
this exact gap caused the 2026-09-05 outage.
Auxiliary routing against local backends
When a Hermes feature that calls an auxiliary task (e.g. title generation) fails against a local endpoint, the usual cause is a request shape the local runtime does not implement, not a routing mistake.
Diagnose with the real call path — run the actual function inside the container with the app's own constants (the probe cannot drift from production):
base64 -w0 /tmp/probe.py | ssh dank@192.168.0.143 \
'base64 -d > /tmp/probe.py && systemctl --user restart hermes.service && sleep 20 && \
podman cp /tmp/probe.py systemd-hermes:/tmp/probe.py && \
podman exec -i systemd-hermes /opt/hermes/.venv/bin/python /tmp/probe.py'
Copy the file AFTER any restart (--replace --rm discards the container
layer). The CLI at /opt/hermes/hermes uses system python and dies
No module named 'yaml'; use /opt/hermes/.venv/bin/python.
import sys; sys.path.insert(0, "/opt/hermes")
from agent.title_generator import _TITLE_PROMPT_TEMPLATE, _TITLE_RESPONSE_FORMAT, _clean_title, _extract_title_text
from agent.auxiliary_client import call_llm
resp = call_llm(task="title_generation", messages=[...], max_tokens=64,
temperature=0.3, extra_body={"response_format": _TITLE_RESPONSE_FORMAT})
What actually breaks, and the fix:
-
response_format(json_object / json_schema) is a hard 400 on every local backend here — neither FreeToken nor halogen implements constrained decoding. Config cannot override it:call_llmmerges the task'sextra_bodyfirst and thenupdate()s the caller's, so the hardcoded one wins. Only temperature and max_tokens have reactive retry paths. -
Fix: move that one task to the Responses transport, which never forwards
response_format:auxiliary: title_generation: provider: dankserver-small # names track the live config; diff first model: frank-secondary api_mode: codex_responses # real per-task key extra_body: reasoning: effort: none # mandatory, see below -
Disable thinking for any cap-limited task. The adapter omits reasoning only when config says
reasoning.enabled: false. Thereasoning_effort: noneshorthand produces exactly that and leaves the server default (thinking ON) — the 64-token title budget gets spent on the think: empty content,finish_reason: length. The explicitextra_body.reasoning.effort: noneinstead sends{"effort":"none","summary":"auto"}, which the local/v1/responseshonors. -
Both local runtimes also answer
/v1/responsesand/v1/messages— no proxy needed for the alternate transport.
Verify and land: re-run the probe with no api_mode/reasoning kwargs —
it must pass from config alone (that is the production path); keep the live
file and the repo example in step; ./tests/run.sh, commit, push — a Hermes
config change is docs+example only, no service deploy needed.
Gotchas: auxiliary.<task>.enabled: false is checked BEFORE
apply_instant_title, so disabling a broken title task also removes the
derived title — it does not degrade gracefully the way a failure does. A
local call can also fail at the model level: a 200 with an error-text body,
or a chatty non-title answer. Always read the raw content, not the status.
Webui password rotation ("Hermex")
Auth contract (verified live 2026-08-31): POST /api/auth/login with JSON
{"password":"…"} → 200 + signed HttpOnly hermes_session cookie (24 h TTL)
on match, 401 on miss. No username field. /health answers 200
unauthenticated and / answers 302 — never verify auth via those; probe
login with a deliberately wrong password (expect 401) and the real one
(expect 200). Details on the webui surface: Hermes-WebUI.
- Run the whole probe as a heredoc'd bash script
(
ssh -o BatchMode=yes dankclaw 'bash -s') — the host's zsh mangles inline${…}even inside single-quoted ssh args; never interpolate curl-w %{http_code}inline. - Update BOTH env files with
sed -i "s/^HERMES_WEBUI_PASSWORD=.*/HERMES_WEBUI_PASSWORD=<new>/":/home/dank/serverAppConfigs/.env([Service] EnvironmentFile)/home/dank/serverAppConfigs/dankclaw/.env— the effective one: the quadlet's[Container] EnvironmentFile=is what the container reads at create; rotating only the root file leaves the running container serving the old secret.
systemctl --user restart hermes-webui.service(unit recreation re-reads env; no deploy needed if the unit file itself didn't change).- Verify wrong-pw → 401, right-pw → 200, from host loopback and a LAN
client at
http://192.168.0.143:8787(webui publishes on the LAN interface since commit4443897; off-network ingress is SSH tunnel only — usertunnel, port 22, key-only auth, forward remote end localhost:8787).
Gotchas: sudo podman exec fails exit 125 here — root's namespace does not
see rootless containers; run podman exec hermes-webui printenv HERMES_WEBUI_PASSWORD unprivileged (the ssh user owns the env files, mode
600). If rotating because a secret leaked into transcripts/logs, rotate in
place via a script that fetches the live value from the running container
(printenv) instead of transcribing it.
See Also
- Hermes-Dashboard — the LAN browser management UI
- Hermes-WebUI — the Hermex server (phone-facing API, SSH-tunneled)
- Dankclaw Stack
- OpenWebUI — the chat UI that pairs with Hermes
- Knowledge base: SearXNG (Hermes as the search agent)
- Container Network Namespaces — why the gateway and webui can dial loopback services and the dashboard cannot