SearXNG: stop rate-limiting local subagents (pass_ip covers slirp/loopback)
Root cause of the 429s: the ip_limit rate limiter caps JSON (format!=html) requests at API_MAX=4 per IP per hour, and that constant is hardcoded in Python (searx/botdetection/ip_limit.py), not settable in limiter.toml. The only bypass is pass_ip (filter_request returns before ip_limit runs). Rootless podman strips the client IP on published-port hits, so the container sees a slirp/bridge source (10.88.x/10.89.x), never the real client - hence even a LAN-IP curl 429'd on /search. Expand pass_ip to loopback + the host's private podman/slirp ranges (plus the existing LAN) so local subagents present a trusted source and skip the rate limit entirely. All these ranges are reachable only from the host or LAN, never the internet.
This commit is contained in:
1 parent
3d4604c19f
commit
4fa9fdf36e
2 files changed
+33
-2
No files matched your search
@@ -39,10 +39,24 @@ link_token = false
|
|||||||
[botdetection.ip_lists]
|
[botdetection.ip_lists]
|
||||||
|
|
||||||
# ip_lists outranks every other method: an IP in pass_ip gets unrestricted
|
# ip_lists outranks every other method: an IP in pass_ip gets unrestricted
|
||||||
# access and is not checked by user-agent or other bot detection.
|
# access and is not checked by user-agent, rate limiting, or other bot
|
||||||
# The whole homelab LAN is trusted.
|
# detection. This is the ONLY bypass for the rate limiter (ip_limit): its JSON
|
||||||
|
# cap (API_MAX = 4 requests/IP/hour) is hardcoded in Python and cannot be
|
||||||
|
# raised from this file.
|
||||||
|
#
|
||||||
|
# Rootless podman strips the client IP on published-port hits: a request to the
|
||||||
|
# published port reaches the container with a slirp/bridge source (10.88.x /
|
||||||
|
# 10.89.x), never the real client IP. Only Caddy's X-Forwarded-For restores it.
|
||||||
|
# So a local subagent that hits the port directly must present a pass_ip source
|
||||||
|
# or it hits the 4/hour JSON cap. We trust loopback, the homelab LAN, and the
|
||||||
|
# host's own private podman/slirp ranges; all are reachable only from the host
|
||||||
|
# or LAN, never the internet.
|
||||||
pass_ip = [
|
pass_ip = [
|
||||||
|
'127.0.0.1/32',
|
||||||
|
'::1',
|
||||||
'192.168.0.0/24',
|
'192.168.0.0/24',
|
||||||
|
'10.88.0.0/16',
|
||||||
|
'10.89.0.0/16',
|
||||||
]
|
]
|
||||||
|
|
||||||
# Blocklist (none).
|
# Blocklist (none).
|
||||||
|
|||||||
@@ -40,6 +40,23 @@ assert "192.168.0.0/24" in proxies, f"LAN missing from trusted_proxies; got {pro
|
|||||||
PYEOF
|
PYEOF
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@test "limiter.toml pass_ip covers local + podman sources (loopback + slirp/bridge)" {
|
||||||
|
# Rootless podman strips the client IP on published-port hits: the container
|
||||||
|
# sees a slirp/bridge source (10.88.x, 10.89.x), never the real client. And
|
||||||
|
# the JSON rate limit (API_MAX=4/hour, hardcoded) 429s any non-passed IP.
|
||||||
|
# So local subagents must present a pass_ip source. Loopback + the host's
|
||||||
|
# private podman ranges are trusted; they are reachable only from the host.
|
||||||
|
python3 - "$REPO_ROOT/shared/searxng/limiter.toml" <<'PYEOF'
|
||||||
|
import sys, tomllib
|
||||||
|
with open(sys.argv[1], "rb") as fh:
|
||||||
|
data = tomllib.load(fh)
|
||||||
|
pass_ip = data.get("botdetection", {}).get("ip_lists", {}).get("pass_ip", [])
|
||||||
|
need = ["127.0.0.1/32", "::1", "192.168.0.0/24", "10.88.0.0/16", "10.89.0.0/16"]
|
||||||
|
missing = [n for n in need if n not in pass_ip]
|
||||||
|
assert not missing, f"pass_ip missing {missing}; got {pass_ip}"
|
||||||
|
PYEOF
|
||||||
|
}
|
||||||
|
|
||||||
@test "deploy.sh copies limiter.toml into /opt/searxng/config" {
|
@test "deploy.sh copies limiter.toml into /opt/searxng/config" {
|
||||||
grep -q "shared/searxng/limiter.toml" "$REPO_ROOT/scripts/deploy.sh" \
|
grep -q "shared/searxng/limiter.toml" "$REPO_ROOT/scripts/deploy.sh" \
|
||||||
|| { echo "deploy.sh does not copy shared/searxng/limiter.toml"; return 1; }
|
|| { echo "deploy.sh does not copy shared/searxng/limiter.toml"; return 1; }
|
||||||
|
|||||||
Reference in new issue
Block a user