searxng: wire paid Kagi engine with deploy-time key rendering

The stock SearXNG image ships a native kagi engine; enabling it needs an
API key that must never enter git, and SearXNG does not expand env vars
in settings.yml (the live secret_key literally was '${SEARXNG_SECRET}'
until now — the app tolerated the placeholder string silently).

- settings.yml: kagi engine enabled (weight 2, kg shortcut, paging off so
  exactly one API credit per aggregated query, timeout 8.0 vs 5.0 scrapers
  so a slow paid query is not a wasted credit); api_key stays ${KAGI_API_KEY}.
- new scripts/searxng-render-settings.py: deploy-time renderer, allowlisted
  to KAGI_API_KEY/SEARXNG_SECRET/SEARXNG_BASE_URL, fails the deploy loudly
  on an unresolvable placeholder, idempotent on redeploys.
- deploy-remote.sh copy_searxng_config runs the renderer against
  /opt/searxng/.env after copying settings.yml into /opt/searxng/config.
- KAGI_API_KEY placeholder added to dankclaw/.env.example (real key staged
  into the host-side /opt/searxng/.env, mode 600, out of band).
- bats: engine wiring, renderer substitution, loud-failure, deployer seam.
- docs: SearXNG.md Kagi section + renderer law; Providers.md carve-out on
  the no-credentials policy (paid engine acceptable when bounded + ordered).

Verified live on dankclaw: direct API probe via the gluetun netns (200,
data.search[] shape), then an aggregated query returned 29/29 results
tagged kagi, zero unresponsive engines.
This commit is contained in:
Frank committed 2026-09-26 23:25:55 -07:00
1 parent 4992e5a010
commit 7a8d992fce
7 files changed
+198 -8

No files matched your search

+42 -5
View File
@@ -60,8 +60,8 @@ Deploy with `scripts/deploy-searxng-quadlet.sh dankclaw`. It copies the units, d
```yaml
use_default_settings: true
server:
secret_key: "${SEARXNG_SECRET}" # from .env
base_url: "${SEARXNG_BASE_URL}" # resolves to http://192.168.0.143:8081/
secret_key: "${SEARXNG_SECRET}" # rendered at deploy by scripts/searxng-render-settings.py
base_url: "${SEARXNG_BASE_URL}" # same renderer; SearXNG does NOT expand env vars itself
limiter: true
method: "POST"
@@ -90,8 +90,16 @@ engines:
- name: duckduckgo
disabled: false
weight: 2
paging: false
timeout: 5.0
- name: kagi
engine: kagi
categories: [general, web]
shortcut: kg
api_key: "${KAGI_API_KEY}" # rendered at deploy from /opt/searxng/.env
kagi_categ: search
disabled: false
weight: 2
paging: false # one API credit per aggregated query
timeout: 8.0
- name: google cse
disabled: true
- name: brave
@@ -168,10 +176,39 @@ root cause, the suspend-model table, the `engines:` config, and the
LAN by construction (no public route to the LAN IP). No second agent-only
instance is needed; a duplicate would double upstream request volume from the
same rate-sensitive VPN exit IP for zero benefit.
- The config file is root-owned (uid of the rootless podman namespace). Update it with a one-off `sudo cp`, then run `systemctl --user restart searxng.service` to reload.
- The config file is root-owned (uid of the rootless podman namespace). The
official path is `deploy.sh --service searxng`: it copies the repo settings
and renders the `${VAR}` placeholders via `scripts/searxng-render-settings.py`
from `/opt/searxng/.env`. A hotfix can still hand-copy with `sudo cp`, but
remember the renderer: an unrendered copy silently serves the literal
placeholders (before 2026-09-26 the live `secret_key` literally was
`"${SEARXNG_SECRET}"` and nobody noticed).
- `base_url` must match the public address or proxy rewrites break.
- Keep the valkey cache container on the same podman network as searxng.
## Kagi (paid API engine)
Added 2026-09-26 (user-ordered). The stock SearXNG image ships a native
`kagi` engine module (`searx/engines/kagi.py`); it authenticates with
`Authorization: Bearer <key>` against `POST https://kagi.com/api/v1/search`.
- **Enabled**: `weight: 2` alongside bing/duckduckgo, categories
`[general, web]`, shortcut `kg`, `kagi_categ: search`.
- **Cost control**: `paging: false` — exactly one API credit per aggregated
query; `timeout: 8.0` so a slow paid query is not wasted (scrapers use 5.0).
- **Key handling**: the repo file keeps `api_key: "${KAGI_API_KEY}"`;
`deploy-remote.sh` bakes the real key into the live settings.yml via
`scripts/searxng-render-settings.py`, reading `/opt/searxng/.env`
(dank-owned, mode 600). The key never enters git and never appears in any
process argv. SearXNG itself does NOT expand env vars in settings.yml —
without the renderer the engine fails `setup()` and is dropped at load.
- **Immune to the VPN-exit-IP problem**: Kagi authenticates, so it does not
care about the shared Gluetun exit IP reputation (see [[SearXNG Providers]]).
- **Verified live 2026-09-26**: direct API probe through the gluetun netns
(200, `data.search[]` shape), then a live aggregated query returned
29/29 results tagged `kagi`, zero unresponsive engines.
## See Also
- [[Dankclaw Stack]]