serverAppConfigs/serverAppConfigs-docs/Dankclaw/SearXNG.md

SearXNG

SearXNG is a privacy-respecting metasearch engine. It aggregates results from many upstream search engines into one result set and serves both an HTML UI and a JSON API for agents.

Host: Dankclaw — Hostname: search.franksdank.cloud (Caddy) — Docs: https://docs.searxng.org

Facts (from research, primary source)

Item	Value
Image	ghcr.io/searxng/searxng:latest (or DockerHub mirror)
Port	8081 (host, on Gluetun) → 8080 (container)
Config	/etc/searxng/settings.yml (bind mount)
Cache	Valkey (replaces Redis) — valkey/valkey:9-alpine
Formats	html, json (agent API)

VPN Routing (critical)

All SearXNG outbound traffic exits through the Gluetun VPN container. The searxng unit sets Network=gluetun.container, which makes it reuse gluetun's network namespace:

[Container]
ContainerName=searxng
Network=gluetun.container

Port publishing happens on the gluetun unit, not on searxng. See Gluetun (KB Pattern 2 is the one used here).

Boot Persistence (Quadlet)

The trio runs as quadlet units, not podman-compose, so it starts at boot. The unit files live in searxng/quadlet/ and install to ~/.config/containers/systemd/.

Unit file	Service	Role
searxng.network	searxng-network.service	DNS-enabled bridge, subnet pinned to 10.89.2.0/24
gluetun.container	gluetun.service	WireGuard tunnel; publishes 8081:8080
searxng-valkey.container	searxng-valkey.service	Valkey cache, alias valkey
searxng.container	searxng.service	Reuses gluetun's netns; publishes no port

gluetun and valkey join the searxng network. searxng reuses gluetun's netns and reaches valkey by name on that network. Each unit sets WantedBy=default.target. User linger is on, so the units start at boot.

Deploy with scripts/deploy-searxng-quadlet.sh dankclaw. It copies the units, daemon-reloads, enables them, and starts them in dependency order.

settings.yml Keys

use_default_settings: true
server:
  secret_key: "${SEARXNG_SECRET}"   # from .env
  base_url: "${SEARXNG_BASE_URL}"  # resolves to http://192.168.0.143:8081/
  limiter: true
  method: "POST"

search:
  default_lang: "en"
  formats:
    - html
    - json

valkey:
  url: "valkey://searxng-valkey:6379/0"

outgoing:
  request_timeout: 5.0
  max_request_timeout: 15.0
  retries: 1                     # avoid amplifying upstream 429/CAPTCHA responses

# Anonymous, bounded default aggregation. Disabled engines remain configured
# for explicit/on-demand selection; see SearXNG Providers for the policy.
engines:
  - name: bing
    disabled: false
    weight: 2
    paging: false
    timeout: 5.0
  - name: duckduckgo
    disabled: false
    weight: 2
    paging: false
    timeout: 5.0
  - name: google cse
    disabled: true
  - name: brave
    disabled: true
  - name: startpage
    disabled: true

Volumes

Container path	Host path (convention)
/etc/searxng	/opt/searxng/config
/etc/searxng/valkey	(valkey container data)

Healthcheck

SearXNG responds on /healthz with 200 when ready.

Trusted Group (limiter.toml)

The trusted group lives in shared/searxng/limiter.toml, not in settings.yml. The settings.yml file only turns the limiter on. The file deploys to /opt/searxng/config/limiter.toml, mounted at /etc/searxng/limiter.toml.

Two lists control access:

• pass_ip — an IP in this list gets unrestricted access and bypasses all bot detection.
• trusted_proxies — if the request IP is in this list, SearXNG reads the real client IP from X-Forwarded-For (set by Caddy).

The pass_ip list holds loopback (127.0.0.1/32, ::1), the whole homelab LAN (192.168.0.0/24), and the host's private podman/slirp ranges (10.88.0.0/16, 10.89.0.0/16). An IP in pass_ip skips every bot-detection check, including the rate limiter. deploy.sh copies the file into place on each deploy.

Why so many ranges: rootless podman strips the client IP on published-port hits. A request to the published port reaches the container with a slirp/bridge source (10.88.x/10.89.x), never the real client IP. Only Caddy's X-Forwarded-For restores it. So a local subagent that hits the port directly must present a pass_ip source, or it is treated as an untrusted client. All these ranges are reachable only from the host or the LAN, never the internet.

The rate limiter's JSON cap is API_MAX = 4 requests per IP per hour. The constant is hardcoded in Python (searx/botdetection/ip_limit.py). It cannot be raised from limiter.toml. pass_ip is the only bypass. This is what lets 4 parallel subagents poll the JSON API at full rate without a 429.

Reach SearXNG through Caddy (search.dankclaw.local) for browser use. A local subagent can hit the published port directly because its slirp source is now trusted. The searxng subnet 10.89.2.0/24 is in trusted_proxies, so Caddy's X-Forwarded-For is honored. The subnet is pinned in searxng.network, so it stays stable across reboots.

Unblocking Google & Search Providers

The current request policy allows a modest amount of VPN latency (4.0s per request, 12.0s maximum) while limiting retries to one. This can recover slow providers, but it cannot fix an upstream CAPTCHA or 429 caused by the shared VPN exit IP. SearXNG queries all ~50 upstream engines from one Gluetun VPN exit IP. The rate-sensitive engines (Google, DuckDuckGo) then answer with reCAPTCHA / 429. The default provider policy therefore favors anonymous Bing and DuckDuckGo, disables observed CAPTCHA/429 providers from normal aggregation, and disables pagination to reduce upstream request volume. SearXNG suspends offending engines for hours to days, which is what shows up as "Google went unavailable". This is upstream, not local.

Fixes (cheapest first): tune the outgoing block (request_timeout, retry_on_http_error, suspended_times), duplicate the google engine under a fresh name so a CAPTCHA ban on one leaves the other returning results, and — if Google still re-blocks — route the Google engines at a clean-IP proxy via per-engine proxies in settings.yml. See SearXNG Providers for the full root cause, the suspend-model table, the engines: config, and the /json verification recipe.

Gotchas

• The JSON API rate cap (4 per 3600s per IP) is hardcoded in Python, not settable in limiter.toml. The only bypass is pass_ip. Local subagents are covered because their slirp/loopback/LAN sources are in pass_ip.
• The limiter stays armed for untrusted sources (a spoofed public IP still 429s). Do not disable it. Widen only pass_ip for sources you trust on this host.
• The config file is root-owned (uid of the rootless podman namespace). Update it with a one-off sudo cp, then run systemctl --user restart searxng.service to reload.
• base_url must match the public address or proxy rewrites break.
• Keep the valkey cache container on the same podman network as searxng.

See Also

• Dankclaw Stack
• Gluetun — the VPN container
• SearXNG Providers — unblocking Google & other upstream engines (VPN exit IP, engine config, proxies, verification)
• Knowledge base: Gluetun, SearXNG, SearXNG Providers, Hermes