Files
command-center/omp-docs/agentic-web-research/searxng-tuning.md
T
Frank fb36a1b0fd docs(web-research): the search egress re-read from the live boxes
Two pages described Dankclaw's search stack from memory, and one of them invented a port.

`searxng-tuning.md` claimed "`:8081` (UI) / `:8082` (JSON API)". Re-probed 2026-10-21: JSON is
served on **8081** (`/search?q=…&format=json` returns results) and **8082 is not SearXNG at
all** — it publishes container port 8085, which is Marreta's nginx, moved off 8080 because
SearXNG owns that port inside the shared namespace. Four containers print the same port list in
`podman ps` because they share gluetun's namespace, which is exactly the detail that gets
mistaken for "the JSON API is on another port". The policy-drift flag now carries today's
reading: `/config` still reports 262 engines and zero disabled, unchanged since 2026-10-19, so
the tuned engine policy is still not live — still an operator decision, still not silently
re-applied.

`vpn-routing.md` gave a mullvad-shaped gluetun example and no fleet shape at all, while the
bundle's own work log says the manual-WireGuard law lives "on concept pages". It lives here now,
read from the running container: `VPN_SERVICE_PROVIDER=custom` + `VPN_TYPE=wireguard` + the
single-peer `WIREGUARD_*` names (`protonvpn` and the plural `WIREGUARD_PEERS_*` family are both
wrong for this image), `FIREWALL_ENABLED_DISABLING_IT_SHOOTS_YOU_IN_YOUR_FOOT=on` — whose name
is the warning that disabling it leaks egress while every check keeps passing — and
`tun0` present. The mullvad block stays as the generic illustration, now labelled as one.

Refs #64
2026-10-21 02:19:23 -07:00

220 lines
6.5 KiB
Markdown

---
type: concept
title: SearXNG Tuning
description: Configuration decisions for a fast, private, useful self-hosted SearXNG instance.
tags:
- searxng
- search
- privacy
- configuration
- unverified
verified: 2026-10-21
timestamp: 2026-08-11
resource: https://docs.searxng.org
---
# SearXNG Tuning
SearXNG is a metasearch engine. It aggregates hundreds of upstream search
services into one result set (upstream `settings.yml` defines 388 engine
entries at master, probed 2026-10-19; the live fleet instance loads 262).
Configuration lives in `settings.yml`. The recommended deployment is Docker
Compose (or Podman Quadlet) with a Valkey container.
This page lists the settings that matter for speed, privacy, and useful results.
> **Fleet instance:** the harness `web/searxng` provider calls
> `192.168.0.143:8081` on dankclaw — HTML **and** JSON on that one port
> (`/search?q=…&format=json`, re-probed 2026-10-21). Port **8082 is not a second SearXNG
> API**: it publishes container port 8085, which is Marreta's nginx (moved off 8080 because
> SearXNG owns it in that namespace). Both ports show up against four container names because
> `searxng`, `searxng-valkey`, `gluetun` and the `marreta` pair share one network namespace.
> The pi-era [Pi Web Access](../pi/packages/pi-web-access.md) called the same host.
## Live instance (probed 2026-10-21)
User Quadlet stack on dankclaw: `searxng.container`
(`ghcr.io/searxng/searxng:latest`) + `searxng-valkey.container`
(`docker.io/valkey/valkey:9-alpine`) + `gluetun.container` (VPN egress; the other containers
join its network namespace, hence the shared port list) + `marreta`/`marreta_browser`.
`limiter.toml`: enabled, `pass_searxng_org: true`.
**Policy drift flag:** this page's tuned engine policy (google and bing disabled;
DuckDuckGo/Wikipedia/Google News enabled — verified via `/config` on 2026-08-16) is NOT live.
`/config` re-read 2026-10-21: 262 engines, **zero disabled**, google and bing included —
unchanged since 2026-10-19. Either the override was lost in the 2026-10 stack reinstatement or
the policy was deliberately dropped. Confirm with Frank before re-applying.
## Plan the Deployment
```mermaid
graph LR
A[Browser or Agent] --> B[SearXNG container]
B --> C[Valkey cache + limiter]
B --> D[Engine request pool]
D --> E[Google News]
D --> F[DuckDuckGo]
D --> G[Wikipedia]
```
Deploy with the official Docker Compose file. It ships a `searxng-valkey` container (`valkey/valkey:9-alpine`). Valkey replaced Redis as the default cache. Mount your config at `./core-config/:/etc/searxng`.
```yaml
services:
searxng:
image: docker.io/searxng/searxng:latest
ports:
- "8081:8080"
volumes:
- ./core-config/:/etc/searxng/:Z
depends_on:
- searxng-valkey
searxng-valkey:
image: docker.io/valkey/valkey:9-alpine
restart: always
```
## Core Settings
```yaml
use_default_settings: true
server:
secret_key: "change-this" # or $SEARXNG_SECRET
base_url: "http://192.168.0.143:8081/" # or $SEARXNG_BASE_URL
limiter: true
method: "POST"
search:
default_lang: "en"
default_locale: "en-US"
safe_search: 0
formats:
- html
- json
```
Change `secret_key`. Set `base_url` to the public address. Use `POST` for privacy. Enable the `json` format for the agent.
## Rate Limiting
Enable the limiter and point it at Valkey. Without a limiter, an upstream engine can block SearXNG's IP.
```yaml
server:
limiter: true
valkey:
url: valkey://searxng-valkey:6379/0
```
The limiter uses a sliding window per IP:
| Window | HTML limit | JSON/API limit |
|--------|-----------|----------------|
| 20 seconds | 15 | 4 per 3600 s |
| 600 seconds | 150 | 4 per 3600 s |
Suspicious IPs get a much lower budget (2 in 20 s, 10 in 600 s). The agent that hits the JSON endpoint repeatedly can trip the 4-per-hour API limit. Back off or make the agent part of a trusted group.
## Outgoing Pool
```yaml
outgoing:
request_timeout: 3.0
max_request_timeout: 10.0
pool_connections: 100
pool_maxsize: 20
enable_http2: true
keepalive_expiry: 5.0
retries: 3
```
A short `request_timeout` keeps results snappy. `enable_http2` multiplexes connections. More `retries` makes results more reliable but slower.
## Engine Selection
Default engine state (current SearXNG):
| Engine | Default state | Note |
|--------|---------------|------|
| DuckDuckGo | Enabled | Reliable general search |
| Wikipedia | Enabled | Encyclopedia lookups |
| Google News | Enabled | News results |
| Google (main) | Inactive | Cloudflare CAPTCHA often blocks it |
| Bing | Disabled | Blocked upstream |
Keep only the engines you need. Fewer engines means faster responses.
```yaml
use_default_settings:
engines:
keep_only:
- duckduckgo
- wikipedia
- google news
```
Make an engine private to one agent with a token:
```yaml
engines:
- name: arch wiki
engine: archlinux
tokens: ['my-secret-token']
```
## Useful Plugins
```yaml
plugins:
searx.plugins.unit_converter.SXNGPlugin:
active: true
searx.plugins.calculator.SXNGPlugin:
active: true
searx.plugins.self_info.SXNGPlugin:
active: true
```
The hostnames plugin filters or rewrites results. It needs a separate `hostnames:` block:
```yaml
hostnames:
remove:
- '(.*\.)?facebook.com$'
high_priority:
- '(.*\.)?wikipedia.org$'
```
## JSON API for the Agent
The agent uses the JSON format. Confirm that it returns structured data:
```bash
curl 'http://192.168.0.143:8081/search?q=test&format=json'
```
The response contains `results`, `query`, `number_of_results`, and `suggestions`. The `/config` endpoint lists engine availability.
## Verify the Instance
1. Open the HTML interface and run a test query.
2. Confirm that the `json` format returns data.
3. Confirm that the limiter page appears on rapid repeats.
## Citations
- SearXNG documentation: <https://docs.searxng.org>
- SearXNG outgoing settings: <https://docs.searxng.org/admin/settings/settings_outgoing.html>
- SearXNG engine settings: <https://docs.searxng.org/admin/settings/settings_engines.html>
- SearXNG Valkey settings: <https://docs.searxng.org/admin/settings/settings_valkey.html>
- SearXNG limiter: <https://docs.searxng.org/admin/searx.limiter.html>
- SearXNG plugins: <https://docs.searxng.org/admin/settings/settings_plugins.html>
- Docker Compose install: <https://docs.searxng.org/admin/installation-docker.html>
## See Also
- [VPN Routing for SearXNG](vpn-routing.md) — Route the outbound path through a VPN.
- [Agentic Search Best Practices](agentic-search-best-practices.md) — Use the JSON format well.