docs: add SearXNG providers guide for unblocking Google behind Gluetun VPN
SearXNG queries all ~50 upstream engines from one Gluetun VPN exit IP, so rate-sensitive engines (Google, DuckDuckGo) answer with reCAPTCHA/429 and SearXNG suspends the offending engine for hours-to-days, showing up as "unavailable". Add docs/Dankclaw/SearXNG Providers.md covering: the suspension model, the two failure modes (CAPTCHA vs selector drift), engine config (settings.yml engines: list — duplicate google, timeout, retry_on_http_error, JSON feed), request tuning (outgoing block, UA rotation, suspended_times — correcting the old rate_limit/useragents blog myths), routing/proxies (per-engine proxies, dedicated 2nd Gluetun, dedicated IP), and verification (/json, /config, check.searx.space). Crosslink into SearXNG.md and index.md.
This commit is contained in:
1 parent
b457457bfe
commit
24efa98d56
3 files changed
+349
-2
No files matched your search
@@ -94,7 +94,22 @@ The rate limiter's JSON cap is `API_MAX = 4` requests per IP per hour. The const
|
||||
|
||||
Reach SearXNG through Caddy (`search.dankclaw.local`) for browser use. A local subagent can hit the published port directly because its slirp source is now trusted. The searxng subnet 10.89.2.0/24 is in `trusted_proxies`, so Caddy's `X-Forwarded-For` is honored. The subnet is pinned in `searxng.network`, so it stays stable across reboots.
|
||||
|
||||
## Gotchas
|
||||
## Unblocking Google & Search Providers
|
||||
|
||||
SearXNG queries all ~50 upstream engines from one Gluetun VPN exit IP. The
|
||||
rate-sensitive engines (Google, DuckDuckGo) then answer with reCAPTCHA / 429,
|
||||
and SearXNG **suspends** the offending engine for hours to days, which is what
|
||||
shows up as "Google went unavailable". This is upstream, not local.
|
||||
|
||||
Fixes (cheapest first): tune the `outgoing` block (`request_timeout`,
|
||||
`retry_on_http_error`, `suspended_times`), duplicate the `google` engine under a
|
||||
fresh `name` so a CAPTCHA ban on one leaves the other returning results, and —
|
||||
if Google still re-blocks — route the Google engines at a clean-IP proxy via
|
||||
per-engine `proxies` in `settings.yml`. See [[SearXNG Providers]] for the full
|
||||
root cause, the suspend-model table, the `engines:` config, and the
|
||||
`/json` verification recipe.
|
||||
|
||||
### Gotchas
|
||||
|
||||
- The JSON API rate cap (4 per 3600s per IP) is hardcoded in Python, not settable in `limiter.toml`. The only bypass is `pass_ip`. Local subagents are covered because their slirp/loopback/LAN sources are in `pass_ip`.
|
||||
- The limiter stays armed for untrusted sources (a spoofed public IP still 429s). Do not disable it. Widen only `pass_ip` for sources you trust on this host.
|
||||
@@ -106,4 +121,5 @@ Reach SearXNG through Caddy (`search.dankclaw.local`) for browser use. A local s
|
||||
|
||||
- [[Dankclaw Stack]]
|
||||
- [[Gluetun]] — the VPN container
|
||||
- Knowledge base: [[Gluetun]], [[SearXNG]], [[Hermes]]
|
||||
- [[SearXNG Providers]] — unblocking Google & other upstream engines (VPN exit IP, engine config, proxies, verification)
|
||||
- Knowledge base: [[Gluetun]], [[SearXNG]], [[SearXNG Providers]], [[Hermes]]
|
||||
Reference in new issue
Block a user