searxng: wire paid Kagi engine with deploy-time key rendering
The stock SearXNG image ships a native kagi engine; enabling it needs an
API key that must never enter git, and SearXNG does not expand env vars
in settings.yml (the live secret_key literally was '${SEARXNG_SECRET}'
until now — the app tolerated the placeholder string silently).
- settings.yml: kagi engine enabled (weight 2, kg shortcut, paging off so
exactly one API credit per aggregated query, timeout 8.0 vs 5.0 scrapers
so a slow paid query is not a wasted credit); api_key stays ${KAGI_API_KEY}.
- new scripts/searxng-render-settings.py: deploy-time renderer, allowlisted
to KAGI_API_KEY/SEARXNG_SECRET/SEARXNG_BASE_URL, fails the deploy loudly
on an unresolvable placeholder, idempotent on redeploys.
- deploy-remote.sh copy_searxng_config runs the renderer against
/opt/searxng/.env after copying settings.yml into /opt/searxng/config.
- KAGI_API_KEY placeholder added to dankclaw/.env.example (real key staged
into the host-side /opt/searxng/.env, mode 600, out of band).
- bats: engine wiring, renderer substitution, loud-failure, deployer seam.
- docs: SearXNG.md Kagi section + renderer law; Providers.md carve-out on
the no-credentials policy (paid engine acceptable when bounded + ordered).
Verified live on dankclaw: direct API probe via the gluetun netns (200,
data.search[] shape), then an aggregated query returned 29/29 results
tagged kagi, zero unresponsive engines.
This commit is contained in:
@@ -363,8 +363,15 @@ misleading).
|
||||
disable pagination for rate-sensitive engines.
|
||||
3. **Promote by evidence** — raise `weight` only for engines that return useful
|
||||
results consistently; weight is not a health check.
|
||||
4. **Avoid credentials** — do not add API-backed engines unless the provider is
|
||||
genuinely free, anonymous, and compatible with our privacy requirements.
|
||||
4. **Avoid credentials, with one carve-out** — do not add API-backed engines
|
||||
unless the provider is genuinely free, anonymous, and compatible with our
|
||||
privacy requirements. Carve-out (2026-09-26): a paid, user-ordered API
|
||||
engine (Kagi) is acceptable when the blast radius is bounded — the
|
||||
`kagi` engine runs with `paging: false` and weight 2, so it spends exactly
|
||||
one API credit per aggregated query, and its key never enters git
|
||||
(deploy-time rendering from `/opt/searxng/.env`; see [[SearXNG]]). It is
|
||||
also immune to the shared-VPN-exit-IP blocking this page is about: it
|
||||
authenticates, so reputation of the exit IP is irrelevant.
|
||||
5. **Verify first** — `/json` + `?engines=google` and the searx-checker to see
|
||||
exactly which engines are down and why. Confirm the running SearXNG version
|
||||
(check whether `parsers`/`rate_limit` exist in the pinned build).
|
||||
|
||||
@@ -60,8 +60,8 @@ Deploy with `scripts/deploy-searxng-quadlet.sh dankclaw`. It copies the units, d
|
||||
```yaml
|
||||
use_default_settings: true
|
||||
server:
|
||||
secret_key: "${SEARXNG_SECRET}" # from .env
|
||||
base_url: "${SEARXNG_BASE_URL}" # resolves to http://192.168.0.143:8081/
|
||||
secret_key: "${SEARXNG_SECRET}" # rendered at deploy by scripts/searxng-render-settings.py
|
||||
base_url: "${SEARXNG_BASE_URL}" # same renderer; SearXNG does NOT expand env vars itself
|
||||
limiter: true
|
||||
method: "POST"
|
||||
|
||||
@@ -90,8 +90,16 @@ engines:
|
||||
- name: duckduckgo
|
||||
disabled: false
|
||||
weight: 2
|
||||
paging: false
|
||||
timeout: 5.0
|
||||
- name: kagi
|
||||
engine: kagi
|
||||
categories: [general, web]
|
||||
shortcut: kg
|
||||
api_key: "${KAGI_API_KEY}" # rendered at deploy from /opt/searxng/.env
|
||||
kagi_categ: search
|
||||
disabled: false
|
||||
weight: 2
|
||||
paging: false # one API credit per aggregated query
|
||||
timeout: 8.0
|
||||
- name: google cse
|
||||
disabled: true
|
||||
- name: brave
|
||||
@@ -168,10 +176,39 @@ root cause, the suspend-model table, the `engines:` config, and the
|
||||
LAN by construction (no public route to the LAN IP). No second agent-only
|
||||
instance is needed; a duplicate would double upstream request volume from the
|
||||
same rate-sensitive VPN exit IP for zero benefit.
|
||||
- The config file is root-owned (uid of the rootless podman namespace). Update it with a one-off `sudo cp`, then run `systemctl --user restart searxng.service` to reload.
|
||||
- The config file is root-owned (uid of the rootless podman namespace). The
|
||||
official path is `deploy.sh --service searxng`: it copies the repo settings
|
||||
and renders the `${VAR}` placeholders via `scripts/searxng-render-settings.py`
|
||||
from `/opt/searxng/.env`. A hotfix can still hand-copy with `sudo cp`, but
|
||||
remember the renderer: an unrendered copy silently serves the literal
|
||||
placeholders (before 2026-09-26 the live `secret_key` literally was
|
||||
`"${SEARXNG_SECRET}"` and nobody noticed).
|
||||
- `base_url` must match the public address or proxy rewrites break.
|
||||
- Keep the valkey cache container on the same podman network as searxng.
|
||||
|
||||
|
||||
## Kagi (paid API engine)
|
||||
|
||||
Added 2026-09-26 (user-ordered). The stock SearXNG image ships a native
|
||||
`kagi` engine module (`searx/engines/kagi.py`); it authenticates with
|
||||
`Authorization: Bearer <key>` against `POST https://kagi.com/api/v1/search`.
|
||||
|
||||
- **Enabled**: `weight: 2` alongside bing/duckduckgo, categories
|
||||
`[general, web]`, shortcut `kg`, `kagi_categ: search`.
|
||||
- **Cost control**: `paging: false` — exactly one API credit per aggregated
|
||||
query; `timeout: 8.0` so a slow paid query is not wasted (scrapers use 5.0).
|
||||
- **Key handling**: the repo file keeps `api_key: "${KAGI_API_KEY}"`;
|
||||
`deploy-remote.sh` bakes the real key into the live settings.yml via
|
||||
`scripts/searxng-render-settings.py`, reading `/opt/searxng/.env`
|
||||
(dank-owned, mode 600). The key never enters git and never appears in any
|
||||
process argv. SearXNG itself does NOT expand env vars in settings.yml —
|
||||
without the renderer the engine fails `setup()` and is dropped at load.
|
||||
- **Immune to the VPN-exit-IP problem**: Kagi authenticates, so it does not
|
||||
care about the shared Gluetun exit IP reputation (see [[SearXNG Providers]]).
|
||||
- **Verified live 2026-09-26**: direct API probe through the gluetun netns
|
||||
(200, `data.search[]` shape), then a live aggregated query returned
|
||||
29/29 results tagged `kagi`, zero unresponsive engines.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[Dankclaw Stack]]
|
||||
|
||||
Reference in New Issue
Block a user