---
name: hermes-local-aux-routing
description: "Fix a broken Hermes auxiliary route (title generation, compression, vision, etc.) when the target is a local OpenAI-compatible backend — diagnose by probing the real call path in-container, then repoint the transport"
---

# Hermes auxiliary routing against local backends

Use when a Hermes feature that calls an auxiliary task fails against a local
endpoint on this fleet (Frank-small on Dankserver :7770, frank-large on Dankclaw
:7770). The usual cause is a request shape the local runtime does not implement,
not a routing mistake.

## Diagnose with the real call path

Do not guess from the config. Run the real function inside the container — it
uses the caller's actual kwargs, the config-driven provider resolution and the
real response parsing:

```bash
base64 -w0 /tmp/probe.py | ssh dank@192.168.0.143 \
  'base64 -d > /tmp/probe.py && systemctl --user restart hermes.service && sleep 20 && \
   podman cp /tmp/probe.py systemd-hermes:/tmp/probe.py && \
   podman exec -i systemd-hermes /opt/hermes/.venv/bin/python /tmp/probe.py'
```

Copy the file AFTER any restart (`--replace --rm` discards the container layer),
and import the app's own constants so the probe cannot drift from production:

```python
import sys; sys.path.insert(0, "/opt/hermes")
from agent.title_generator import _TITLE_PROMPT_TEMPLATE, _TITLE_RESPONSE_FORMAT, _clean_title, _extract_title_text
from agent.auxiliary_client import call_llm
resp = call_llm(task="title_generation", messages=[...], max_tokens=64,
                temperature=0.3, extra_body={"response_format": _TITLE_RESPONSE_FORMAT})
```

Pass `api_mode=` / extra `extra_body` keys to A/B a fix before touching config.
(The CLI at `/opt/hermes/hermes` uses the system python and dies on
`No module named 'yaml'`; use `/opt/hermes/.venv/bin/python`.)

## What actually breaks, and the fix

- **`response_format` (json_object / json_schema) is a hard 400** on every local
  backend here — neither FreeToken nor halogen implements constrained decoding.
  Config cannot override it: `call_llm` merges the task's `extra_body` first and
  then `update()`s the caller's, so the hardcoded one wins. Only temperature and
  max_tokens have reactive retry paths.
- **Fix: move that one task to the Responses transport**, which never forwards
  `response_format`:
  ```yaml
  auxiliary:
    title_generation:
      provider: dankserver-small
      model: frank-small
      api_mode: codex_responses        # real per-task key
      extra_body:
        reasoning:
          effort: none                 # mandatory, see below
  ```
- **Disable thinking for any cap-limited task.** The adapter omits reasoning
  only when the config says `reasoning.enabled: false`; note that the
  `reasoning_effort: none` shorthand produces exactly that, which leaves the
  server default (thinking ON) and the 64-token title budget is spent on the
  think — empty content, `finish_reason: length`. The explicit
  `extra_body.reasoning.effort: none` instead sends
  `{"effort":"none","summary":"auto"}`, which the local /v1/responses honors.
- Both local runtimes also answer `/v1/responses` and `/v1/messages`, so no
  proxy is needed to use the alternate transport.

## Verify and land it

1. Re-run the probe with **no** `api_mode`/reasoning kwargs — it must pass from
   config alone; that is the production path.
2. Keep the live file (`/home/dank/Containers/hermes/data/config.yaml`, owned by
   uid 109999, needs sudo) and the repo example
   `dankclaw/hermes/config.yaml.example` in step; the example is a sanitized
   backup and lags otherwise.
3. `./tests/run.sh`, commit, push — a Hermes config change is docs+example only,
   no service deploy needed.

## Gotchas

- `auxiliary.<task>.enabled: false` is checked BEFORE `apply_instant_title`, so
  disabling a broken title task also removes the derived title — it does not
  degrade gracefully the way a failure does.
- A local call can also fail at the model level: a 200 with an error-text body,
  or a chatty non-title answer. Always read the raw content, not the status.