Files
command-center/omp-docs/agentic-web-research/searxng-tuning.md
T
Frank fb36a1b0fd docs(web-research): the search egress re-read from the live boxes
Two pages described Dankclaw's search stack from memory, and one of them invented a port.

`searxng-tuning.md` claimed "`:8081` (UI) / `:8082` (JSON API)". Re-probed 2026-10-21: JSON is
served on **8081** (`/search?q=…&format=json` returns results) and **8082 is not SearXNG at
all** — it publishes container port 8085, which is Marreta's nginx, moved off 8080 because
SearXNG owns that port inside the shared namespace. Four containers print the same port list in
`podman ps` because they share gluetun's namespace, which is exactly the detail that gets
mistaken for "the JSON API is on another port". The policy-drift flag now carries today's
reading: `/config` still reports 262 engines and zero disabled, unchanged since 2026-10-19, so
the tuned engine policy is still not live — still an operator decision, still not silently
re-applied.

`vpn-routing.md` gave a mullvad-shaped gluetun example and no fleet shape at all, while the
bundle's own work log says the manual-WireGuard law lives "on concept pages". It lives here now,
read from the running container: `VPN_SERVICE_PROVIDER=custom` + `VPN_TYPE=wireguard` + the
single-peer `WIREGUARD_*` names (`protonvpn` and the plural `WIREGUARD_PEERS_*` family are both
wrong for this image), `FIREWALL_ENABLED_DISABLING_IT_SHOOTS_YOU_IN_YOUR_FOOT=on` — whose name
is the warning that disabling it leaks egress while every check keeps passing — and
`tun0` present. The mullvad block stays as the generic illustration, now labelled as one.

Refs #64
2026-10-21 02:19:23 -07:00

6.5 KiB

type, title, description, tags, verified, timestamp, resource
type title description tags verified timestamp resource
concept SearXNG Tuning Configuration decisions for a fast, private, useful self-hosted SearXNG instance.
searxng
search
privacy
configuration
unverified
2026-10-21 2026-08-11 https://docs.searxng.org

SearXNG Tuning

SearXNG is a metasearch engine. It aggregates hundreds of upstream search services into one result set (upstream settings.yml defines 388 engine entries at master, probed 2026-10-19; the live fleet instance loads 262). Configuration lives in settings.yml. The recommended deployment is Docker Compose (or Podman Quadlet) with a Valkey container.

This page lists the settings that matter for speed, privacy, and useful results.

Fleet instance: the harness web/searxng provider calls 192.168.0.143:8081 on dankclaw — HTML and JSON on that one port (/search?q=…&format=json, re-probed 2026-10-21). Port 8082 is not a second SearXNG API: it publishes container port 8085, which is Marreta's nginx (moved off 8080 because SearXNG owns it in that namespace). Both ports show up against four container names because searxng, searxng-valkey, gluetun and the marreta pair share one network namespace. The pi-era Pi Web Access called the same host.

Live instance (probed 2026-10-21)

User Quadlet stack on dankclaw: searxng.container (ghcr.io/searxng/searxng:latest) + searxng-valkey.container (docker.io/valkey/valkey:9-alpine) + gluetun.container (VPN egress; the other containers join its network namespace, hence the shared port list) + marreta/marreta_browser. limiter.toml: enabled, pass_searxng_org: true.

Policy drift flag: this page's tuned engine policy (google and bing disabled; DuckDuckGo/Wikipedia/Google News enabled — verified via /config on 2026-08-16) is NOT live. /config re-read 2026-10-21: 262 engines, zero disabled, google and bing included — unchanged since 2026-10-19. Either the override was lost in the 2026-10 stack reinstatement or the policy was deliberately dropped. Confirm with Frank before re-applying.

Plan the Deployment

graph LR
    A[Browser or Agent] --> B[SearXNG container]
    B --> C[Valkey cache + limiter]
    B --> D[Engine request pool]
    D --> E[Google News]
    D --> F[DuckDuckGo]
    D --> G[Wikipedia]

Deploy with the official Docker Compose file. It ships a searxng-valkey container (valkey/valkey:9-alpine). Valkey replaced Redis as the default cache. Mount your config at ./core-config/:/etc/searxng.

services:
  searxng:
    image: docker.io/searxng/searxng:latest
    ports:
      - "8081:8080"
    volumes:
      - ./core-config/:/etc/searxng/:Z
    depends_on:
      - searxng-valkey

  searxng-valkey:
    image: docker.io/valkey/valkey:9-alpine
    restart: always

Core Settings

use_default_settings: true
server:
  secret_key: "change-this"   # or $SEARXNG_SECRET
  base_url: "http://192.168.0.143:8081/"   # or $SEARXNG_BASE_URL
  limiter: true
  method: "POST"

search:
  default_lang: "en"
  default_locale: "en-US"
  safe_search: 0
  formats:
    - html
    - json

Change secret_key. Set base_url to the public address. Use POST for privacy. Enable the json format for the agent.

Rate Limiting

Enable the limiter and point it at Valkey. Without a limiter, an upstream engine can block SearXNG's IP.

server:
  limiter: true

valkey:
  url: valkey://searxng-valkey:6379/0

The limiter uses a sliding window per IP:

Window HTML limit JSON/API limit
20 seconds 15 4 per 3600 s
600 seconds 150 4 per 3600 s

Suspicious IPs get a much lower budget (2 in 20 s, 10 in 600 s). The agent that hits the JSON endpoint repeatedly can trip the 4-per-hour API limit. Back off or make the agent part of a trusted group.

Outgoing Pool

outgoing:
  request_timeout: 3.0
  max_request_timeout: 10.0
  pool_connections: 100
  pool_maxsize: 20
  enable_http2: true
  keepalive_expiry: 5.0
  retries: 3

A short request_timeout keeps results snappy. enable_http2 multiplexes connections. More retries makes results more reliable but slower.

Engine Selection

Default engine state (current SearXNG):

Engine Default state Note
DuckDuckGo Enabled Reliable general search
Wikipedia Enabled Encyclopedia lookups
Google News Enabled News results
Google (main) Inactive Cloudflare CAPTCHA often blocks it
Bing Disabled Blocked upstream

Keep only the engines you need. Fewer engines means faster responses.

use_default_settings:
  engines:
    keep_only:
      - duckduckgo
      - wikipedia
      - google news

Make an engine private to one agent with a token:

engines:
  - name: arch wiki
    engine: archlinux
    tokens: ['my-secret-token']

Useful Plugins

plugins:
  searx.plugins.unit_converter.SXNGPlugin:
    active: true
  searx.plugins.calculator.SXNGPlugin:
    active: true
  searx.plugins.self_info.SXNGPlugin:
    active: true

The hostnames plugin filters or rewrites results. It needs a separate hostnames: block:

hostnames:
  remove:
    - '(.*\.)?facebook.com$'
  high_priority:
    - '(.*\.)?wikipedia.org$'

JSON API for the Agent

The agent uses the JSON format. Confirm that it returns structured data:

curl 'http://192.168.0.143:8081/search?q=test&format=json'

The response contains results, query, number_of_results, and suggestions. The /config endpoint lists engine availability.

Verify the Instance

  1. Open the HTML interface and run a test query.
  2. Confirm that the json format returns data.
  3. Confirm that the limiter page appears on rapid repeats.

Citations

See Also