SearXNG: stop rate-limiting local subagents (pass_ip covers slirp/loopback)

Root cause of the 429s: the ip_limit rate limiter caps JSON (format!=html)
requests at API_MAX=4 per IP per hour, and that constant is hardcoded in
Python (searx/botdetection/ip_limit.py), not settable in limiter.toml. The
only bypass is pass_ip (filter_request returns before ip_limit runs).

Rootless podman strips the client IP on published-port hits, so the container
sees a slirp/bridge source (10.88.x/10.89.x), never the real client - hence
even a LAN-IP curl 429'd on /search. Expand pass_ip to loopback + the host's
private podman/slirp ranges (plus the existing LAN) so local subagents present
a trusted source and skip the rate limit entirely. All these ranges are
reachable only from the host or LAN, never the internet.
This commit is contained in:
Frank
2026-08-19 12:29:27 -07:00
parent 3d4604c19f
commit 4fa9fdf36e
2 changed files with 33 additions and 2 deletions
+16 -2
View File
@@ -39,10 +39,24 @@ link_token = false
[botdetection.ip_lists]
# ip_lists outranks every other method: an IP in pass_ip gets unrestricted
# access and is not checked by user-agent or other bot detection.
# The whole homelab LAN is trusted.
# access and is not checked by user-agent, rate limiting, or other bot
# detection. This is the ONLY bypass for the rate limiter (ip_limit): its JSON
# cap (API_MAX = 4 requests/IP/hour) is hardcoded in Python and cannot be
# raised from this file.
#
# Rootless podman strips the client IP on published-port hits: a request to the
# published port reaches the container with a slirp/bridge source (10.88.x /
# 10.89.x), never the real client IP. Only Caddy's X-Forwarded-For restores it.
# So a local subagent that hits the port directly must present a pass_ip source
# or it hits the 4/hour JSON cap. We trust loopback, the homelab LAN, and the
# host's own private podman/slirp ranges; all are reachable only from the host
# or LAN, never the internet.
pass_ip = [
'127.0.0.1/32',
'::1',
'192.168.0.0/24',
'10.88.0.0/16',
'10.89.0.0/16',
]
# Blocklist (none).
+17
View File
@@ -40,6 +40,23 @@ assert "192.168.0.0/24" in proxies, f"LAN missing from trusted_proxies; got {pro
PYEOF
}
@test "limiter.toml pass_ip covers local + podman sources (loopback + slirp/bridge)" {
# Rootless podman strips the client IP on published-port hits: the container
# sees a slirp/bridge source (10.88.x, 10.89.x), never the real client. And
# the JSON rate limit (API_MAX=4/hour, hardcoded) 429s any non-passed IP.
# So local subagents must present a pass_ip source. Loopback + the host's
# private podman ranges are trusted; they are reachable only from the host.
python3 - "$REPO_ROOT/shared/searxng/limiter.toml" <<'PYEOF'
import sys, tomllib
with open(sys.argv[1], "rb") as fh:
data = tomllib.load(fh)
pass_ip = data.get("botdetection", {}).get("ip_lists", {}).get("pass_ip", [])
need = ["127.0.0.1/32", "::1", "192.168.0.0/24", "10.88.0.0/16", "10.89.0.0/16"]
missing = [n for n in need if n not in pass_ip]
assert not missing, f"pass_ip missing {missing}; got {pass_ip}"
PYEOF
}
@test "deploy.sh copies limiter.toml into /opt/searxng/config" {
grep -q "shared/searxng/limiter.toml" "$REPO_ROOT/scripts/deploy.sh" \
|| { echo "deploy.sh does not copy shared/searxng/limiter.toml"; return 1; }