88 lines
4.0 KiB
Markdown
88 lines
4.0 KiB
Markdown
---
|
|
name: hermes-local-aux-routing
|
|
description: "Fix a broken Hermes auxiliary route (title generation, compression, vision, etc.) when the target is a local OpenAI-compatible backend — diagnose by probing the real call path in-container, then repoint the transport"
|
|
---
|
|
|
|
# Hermes auxiliary routing against local backends
|
|
|
|
Use when a Hermes feature that calls an auxiliary task fails against a local
|
|
endpoint on this fleet (Frank-small on Dankserver :7770, frank-large on Dankclaw
|
|
:7770). The usual cause is a request shape the local runtime does not implement,
|
|
not a routing mistake.
|
|
|
|
## Diagnose with the real call path
|
|
|
|
Do not guess from the config. Run the real function inside the container — it
|
|
uses the caller's actual kwargs, the config-driven provider resolution and the
|
|
real response parsing:
|
|
|
|
```bash
|
|
base64 -w0 /tmp/probe.py | ssh dank@192.168.0.143 \
|
|
'base64 -d > /tmp/probe.py && systemctl --user restart hermes.service && sleep 20 && \
|
|
podman cp /tmp/probe.py systemd-hermes:/tmp/probe.py && \
|
|
podman exec -i systemd-hermes /opt/hermes/.venv/bin/python /tmp/probe.py'
|
|
```
|
|
|
|
Copy the file AFTER any restart (`--replace --rm` discards the container layer),
|
|
and import the app's own constants so the probe cannot drift from production:
|
|
|
|
```python
|
|
import sys; sys.path.insert(0, "/opt/hermes")
|
|
from agent.title_generator import _TITLE_PROMPT_TEMPLATE, _TITLE_RESPONSE_FORMAT, _clean_title, _extract_title_text
|
|
from agent.auxiliary_client import call_llm
|
|
resp = call_llm(task="title_generation", messages=[...], max_tokens=64,
|
|
temperature=0.3, extra_body={"response_format": _TITLE_RESPONSE_FORMAT})
|
|
```
|
|
|
|
Pass `api_mode=` / extra `extra_body` keys to A/B a fix before touching config.
|
|
(The CLI at `/opt/hermes/hermes` uses the system python and dies on
|
|
`No module named 'yaml'`; use `/opt/hermes/.venv/bin/python`.)
|
|
|
|
## What actually breaks, and the fix
|
|
|
|
- **`response_format` (json_object / json_schema) is a hard 400** on every local
|
|
backend here — neither FreeToken nor halogen implements constrained decoding.
|
|
Config cannot override it: `call_llm` merges the task's `extra_body` first and
|
|
then `update()`s the caller's, so the hardcoded one wins. Only temperature and
|
|
max_tokens have reactive retry paths.
|
|
- **Fix: move that one task to the Responses transport**, which never forwards
|
|
`response_format`:
|
|
```yaml
|
|
auxiliary:
|
|
title_generation:
|
|
provider: dankserver-small
|
|
model: frank-small
|
|
api_mode: codex_responses # real per-task key
|
|
extra_body:
|
|
reasoning:
|
|
effort: none # mandatory, see below
|
|
```
|
|
- **Disable thinking for any cap-limited task.** The adapter omits reasoning
|
|
only when the config says `reasoning.enabled: false`; note that the
|
|
`reasoning_effort: none` shorthand produces exactly that, which leaves the
|
|
server default (thinking ON) and the 64-token title budget is spent on the
|
|
think — empty content, `finish_reason: length`. The explicit
|
|
`extra_body.reasoning.effort: none` instead sends
|
|
`{"effort":"none","summary":"auto"}`, which the local /v1/responses honors.
|
|
- Both local runtimes also answer `/v1/responses` and `/v1/messages`, so no
|
|
proxy is needed to use the alternate transport.
|
|
|
|
## Verify and land it
|
|
|
|
1. Re-run the probe with **no** `api_mode`/reasoning kwargs — it must pass from
|
|
config alone; that is the production path.
|
|
2. Keep the live file (`/home/dank/Containers/hermes/data/config.yaml`, owned by
|
|
uid 109999, needs sudo) and the repo example
|
|
`dankclaw/hermes/config.yaml.example` in step; the example is a sanitized
|
|
backup and lags otherwise.
|
|
3. `./tests/run.sh`, commit, push — a Hermes config change is docs+example only,
|
|
no service deploy needed.
|
|
|
|
## Gotchas
|
|
|
|
- `auxiliary.<task>.enabled: false` is checked BEFORE `apply_instant_title`, so
|
|
disabling a broken title task also removes the derived title — it does not
|
|
degrade gracefully the way a failure does.
|
|
- A local call can also fail at the model level: a 200 with an error-text body,
|
|
or a chatty non-title answer. Always read the raw content, not the status.
|