SearXNG: stop rate-limiting local subagents (pass_ip covers slirp/loopback)
Root cause of the 429s: the ip_limit rate limiter caps JSON (format!=html) requests at API_MAX=4 per IP per hour, and that constant is hardcoded in Python (searx/botdetection/ip_limit.py), not settable in limiter.toml. The only bypass is pass_ip (filter_request returns before ip_limit runs). Rootless podman strips the client IP on published-port hits, so the container sees a slirp/bridge source (10.88.x/10.89.x), never the real client - hence even a LAN-IP curl 429'd on /search. Expand pass_ip to loopback + the host's private podman/slirp ranges (plus the existing LAN) so local subagents present a trusted source and skip the rate limit entirely. All these ranges are reachable only from the host or LAN, never the internet.
This commit is contained in:
@@ -39,10 +39,24 @@ link_token = false
|
||||
[botdetection.ip_lists]
|
||||
|
||||
# ip_lists outranks every other method: an IP in pass_ip gets unrestricted
|
||||
# access and is not checked by user-agent or other bot detection.
|
||||
# The whole homelab LAN is trusted.
|
||||
# access and is not checked by user-agent, rate limiting, or other bot
|
||||
# detection. This is the ONLY bypass for the rate limiter (ip_limit): its JSON
|
||||
# cap (API_MAX = 4 requests/IP/hour) is hardcoded in Python and cannot be
|
||||
# raised from this file.
|
||||
#
|
||||
# Rootless podman strips the client IP on published-port hits: a request to the
|
||||
# published port reaches the container with a slirp/bridge source (10.88.x /
|
||||
# 10.89.x), never the real client IP. Only Caddy's X-Forwarded-For restores it.
|
||||
# So a local subagent that hits the port directly must present a pass_ip source
|
||||
# or it hits the 4/hour JSON cap. We trust loopback, the homelab LAN, and the
|
||||
# host's own private podman/slirp ranges; all are reachable only from the host
|
||||
# or LAN, never the internet.
|
||||
pass_ip = [
|
||||
'127.0.0.1/32',
|
||||
'::1',
|
||||
'192.168.0.0/24',
|
||||
'10.88.0.0/16',
|
||||
'10.89.0.0/16',
|
||||
]
|
||||
|
||||
# Blocklist (none).
|
||||
|
||||
@@ -40,6 +40,23 @@ assert "192.168.0.0/24" in proxies, f"LAN missing from trusted_proxies; got {pro
|
||||
PYEOF
|
||||
}
|
||||
|
||||
@test "limiter.toml pass_ip covers local + podman sources (loopback + slirp/bridge)" {
|
||||
# Rootless podman strips the client IP on published-port hits: the container
|
||||
# sees a slirp/bridge source (10.88.x, 10.89.x), never the real client. And
|
||||
# the JSON rate limit (API_MAX=4/hour, hardcoded) 429s any non-passed IP.
|
||||
# So local subagents must present a pass_ip source. Loopback + the host's
|
||||
# private podman ranges are trusted; they are reachable only from the host.
|
||||
python3 - "$REPO_ROOT/shared/searxng/limiter.toml" <<'PYEOF'
|
||||
import sys, tomllib
|
||||
with open(sys.argv[1], "rb") as fh:
|
||||
data = tomllib.load(fh)
|
||||
pass_ip = data.get("botdetection", {}).get("ip_lists", {}).get("pass_ip", [])
|
||||
need = ["127.0.0.1/32", "::1", "192.168.0.0/24", "10.88.0.0/16", "10.89.0.0/16"]
|
||||
missing = [n for n in need if n not in pass_ip]
|
||||
assert not missing, f"pass_ip missing {missing}; got {pass_ip}"
|
||||
PYEOF
|
||||
}
|
||||
|
||||
@test "deploy.sh copies limiter.toml into /opt/searxng/config" {
|
||||
grep -q "shared/searxng/limiter.toml" "$REPO_ROOT/scripts/deploy.sh" \
|
||||
|| { echo "deploy.sh does not copy shared/searxng/limiter.toml"; return 1; }
|
||||
|
||||
Reference in New Issue
Block a user