Frank
c390135d56
A direct hit on the published port has no X-Forwarded-For header, so the bot-detection cannot resolve a client and returns 429. SearXNG must be reached through Caddy (search.dankclaw.local), which sets X-Forwarded-For. Add the podman bridge subnets to trusted_proxies so Caddy's source is trusted for XFF extraction; note the CIDR is dynamic and should be pinned. Verified agentic JSON search returns 26-31 relevant results via this path.
92 lines
3.4 KiB
Markdown
92 lines
3.4 KiB
Markdown
# SearXNG
|
|
|
|
SearXNG is a privacy-respecting metasearch engine. It aggregates results from many upstream search engines into one result set and serves both an HTML UI and a JSON API for agents.
|
|
|
|
**Host:** Dankclaw — **Hostname:** `search.dankclaw.local` — **Docs:** https://docs.searxng.org
|
|
|
|
## Facts (from research, primary source)
|
|
|
|
| Item | Value |
|
|
|------|-------|
|
|
| Image | `ghcr.io/searxng/searxng:latest` (or DockerHub mirror) |
|
|
| Port | 8081 (host, on Gluetun) → 8080 (container) |
|
|
| Config | `/etc/searxng/settings.yml` (bind mount) |
|
|
| Cache | Valkey (replaces Redis) — `valkey/valkey:9-alpine` |
|
|
| Formats | `html`, `json` (agent API) |
|
|
|
|
## VPN Routing (critical)
|
|
|
|
All SearXNG outbound traffic exits through the Gluetun VPN container. The container attaches to Gluetun's network namespace:
|
|
|
|
```yaml
|
|
services:
|
|
searxng:
|
|
network_mode: "service:gluetun"
|
|
depends_on:
|
|
- gluetun
|
|
```
|
|
|
|
Port publishing happens on the **gluetun** service, not on searxng. See [[Gluetun]] and the knowledge base page [[VPN Routing for SearXNG]] (KB Pattern 2 is the one used here).
|
|
|
|
## settings.yml Keys
|
|
|
|
```yaml
|
|
use_default_settings: true
|
|
server:
|
|
secret_key: "${SEARXNG_SECRET}" # from .env
|
|
base_url: "http://search.dankclaw.local/"
|
|
limiter: true
|
|
method: "POST"
|
|
|
|
search:
|
|
default_lang: "en"
|
|
formats:
|
|
- html
|
|
- json
|
|
|
|
valkey:
|
|
url: "valkey://searxng-valkey:6379/0"
|
|
|
|
outgoing:
|
|
request_timeout: 3.0
|
|
max_request_timeout: 10.0
|
|
retries: 3
|
|
```
|
|
|
|
## Volumes
|
|
|
|
| Container path | Host path (convention) |
|
|
|----------------|------------------------|
|
|
| `/etc/searxng` | `/opt/searxng/config` |
|
|
| `/etc/searxng/valkey` | (valkey container data) |
|
|
|
|
## Healthcheck
|
|
|
|
SearXNG responds on `/healthz` with 200 when ready.
|
|
|
|
## Trusted Group (limiter.toml)
|
|
|
|
The trusted group lives in `shared/searxng/limiter.toml`, not in `settings.yml`. The `settings.yml` file only turns the limiter on. The file deploys to `/opt/searxng/config/limiter.toml`, mounted at `/etc/searxng/limiter.toml`.
|
|
|
|
Two lists control access:
|
|
|
|
- `pass_ip` — an IP in this list gets unrestricted access and bypasses all bot detection.
|
|
- `trusted_proxies` — if the request IP is in this list, SearXNG reads the real client IP from `X-Forwarded-For` (set by Caddy).
|
|
|
|
The whole homelab LAN `192.168.0.0/24` is in both lists. This lets local agents and browsers poll the JSON API without tripping the 4-per-hour limit. `deploy.sh` copies the file into place on each deploy.
|
|
|
|
Access SearXNG through Caddy (`search.dankclaw.local`). A direct request on the published port has no `X-Forwarded-For` header. The bot-detection cannot resolve the client and returns 429. The podman bridge subnet is in `trusted_proxies` so Caddy's request is trusted. The bridge CIDR is assigned dynamically by rootless podman. Pin the network (`podman network create` with a fixed `--subnet`) for a stable value.
|
|
|
|
## Gotchas
|
|
|
|
- The JSON API is rate-limited hard by the limiter (4 per 3600s per IP for untrusted). Add the agent's IP to `pass_ip` in `limiter.toml` if it polls.
|
|
- Reach SearXNG through Caddy, not the raw published port. A direct port hit has no `X-Forwarded-For` header and the bot-detection returns 429.
|
|
- `base_url` must match the public address or proxy rewrites break.
|
|
- Keep the valkey cache container on the same podman network as searxng.
|
|
|
|
## See Also
|
|
|
|
- [[Dankclaw Stack]]
|
|
- [[Gluetun]] — the VPN container
|
|
- Knowledge base: [[VPN Routing for SearXNG]], [[SearXNG Tuning]], [[Agentic Search Best Practices]]
|