---
name: fleet-second-llm-endpoint
description: "Add a second LLM endpoint to a host that already serves one on the Dank fleet (new port, unit shape, co-tenancy budget, harness provider/roles, monitoring, cutover, verification) — with the port-bind race, the reasoning_effort template trap, and the tools/--jinja trap that silently break subagents."
---

# Adding a second LLM endpoint to a Dank fleet host

Use when a host must serve two models at once (e.g. an iGPU runtime plus a
second card), or when one endpoint is moving and a new one takes the freed
hardware. Rebuilt from the 2026-09-16 Dankclaw split (halogen `frank-large` on
:7770 + llama.cpp `frank-small` on :7771).

## 0. Decide the shape before touching hardware

- **One provider per HOST:PORT** in `~/.omp/agent/models.yml` (on the box where
the harness runs, not on the servers). A second endpoint is a new provider
block, not a second model in the existing one.
- **Read the budget as an inequality, not a wish.** For a unified-memory runtime
the engine's own fit check is the arbiter (`budget = host − pinned weights −
reserve`; `need = KV + arena + vision`); for a discrete card, weight the KV by
counting only the full-attention layers — and remember `--parallel N` DIVIDES
`--ctx-size`, it does not multiply.
- **A co-tenant is a budget line, not a footnote.** Free host RAM by changing
the POOL or the ARENA (the reserve is a fit-check term, never an allocation),
and drop optional charges (a vision tower is ~2 GiB) rather than hoping.

## 1. Repo first, and retire in the same commit

1. New service files: `scripts/<svc>.env` (documented tunables), `.service`
   (root unit for a native binary; a Quadlet only for a container),
   `serve-<svc>.sh` (defaults + hard guards for missing artifacts).
2. Planner: add the service to `SERVICE_ORDER` AND the target's service set in
   `scripts/deploy-plan.py`, map its paths (including "the unit/env/serve trio
   selects only me"), add it to the inventories in `deploy.sh` and
   `deploy-remote.sh`, and the dispatch case.
3. Retire what the cutover obsoletes into `.AGENTTRASH/` with a suffix — and
   **check what pointed at it**: bench harnesses, a chat template only the old
   runtime read, and `scripts/test-changed.sh` mappings are the usual victims
   (a mapping to a test file retired earlier fails the pre-commit wrapper
   silently for weeks).
4. Seams: one bats file per service asserting the *invariants* (`pool == 2 ×
   ctx`, KV bytes ≤ card budget, port separation), not just constant values.
   A negative grep must strip comments (`grep -vE '^\s*#' file | grep -q ...`)
   or it will match the prose that explains the very thing you removed.

## 2. The port-bind race (rootless podman)

pasta binds the published port at container CREATE. A predecessor that has just
stopped still holds it, so a naive handler starts a doomed container:

```
Deactivated successfully          <- predecessor gone
pasta: Listen failed for HOST TCP port */PORT: Address already in use
exit 126                          <- only Restart=on-failure saves you
```

Order the handler: **retire the old manager's unit → stop THIS service's own
unit (no-op on a first cutover) → wait for every socket on the port to clear
(listener, accepted, TIME_WAIT) → install → start.** Without the second step a
redeploy deadlocks against its own gate.

## 3. Wire the surfaces, and refuse impossible signals

- Harness roles: point the small/fast + image roles at the new endpoint, and
  move the vision role with the vision tower (only one endpoint should own
  each capability).
- `shared/homepage/services.json`: a card per endpoint, probed from the
  monitor's own host (loopback is legal there; the LAN address otherwise).
- Prometheus: only add a job if the runtime actually exposes Prometheus text.
  Halogen publishes `/health`, `/cache` and `/v1/*` and nothing else — scraping
  it would be red by construction, so the job is REMOVED with the reason written
  down rather than left as a permanent red.

## 4. Cutover, then verify by driving it

```bash
./scripts/deploy.sh <target> --service <large> --service <small>   # order matters
```

Then, per endpoint: `/health`; `/v1/models` reports the alias; a chat completion
returns content AND `reasoning_content`; **a request carrying `tools` returns
`tool_calls`**; an image request is answered (or refused with a named reason);
the runtime's own speculation counters move; and the unit is `enabled` /
`generated` with linger on. Read the engine's boot ledger for a silent clamp.

Finally spend one real request through the harness per changed role — routing
labels lie, the servers' own journals do not: `journalctl -u <svc>` shows which
endpoint actually served it.

## Traps that look like model faults

| symptom | cause |
|---|---|
| `500 tools param requires --jinja flag` | the endpoint's runtime lacks the Jinja chat-template path, so every tool-using agent on it is broken. mainline llama.cpp defaults it on — pin it |
| `500 Jinja Exception: Unexpected reasoning effort <level>` | the model's own template has a narrower accepted set (Qwen3.8: xhigh/medium/low only) than the harness's levels; fold them in `compat.reasoningEffortMap` |
| the same request works by hand but a subagent dies | subagents run at a different thinking level — check both |
| the container dies instantly although the predecessor "stopped" | the port race above |