Frank
0ae3650414
Bots and AI scrapers hit the SSO-fronted vhosts and eat Authentik auth work for 403s. Anubis verifies clients first: DENY for known AI scraper UAs, JS challenge for everyone else; verified traffic resumes the normal Authentik forward_auth chain on Caddy's internal listener (:8016). Chain: client -> Caddy (TLS) -> Anubis :8923 (verify) -> Caddy :8016 (auth + routing resumes; Host survives both hops) -> app. Reverse-proxy through Anubis, not forward_auth: at the pinned tag a CHALLENGE answers HTTP 200 + HTML, and forward_auth treats 200 as "authenticated" — the page would pass through unchallenged. Exempt by construction (never routed through Anubis): plex (plain HTTP legacy), auth.* (IdP login must be reachable ungated), gitea (git smart-HTTP clients cannot solve JS challenges), and every app with its own login. Pinned ghcr.io/TecharoHQ/anubis:v1.13.0 — the tag was source-verified before pinning: at this tag the policy parser is JSON-only with a minimal bots[] schema (no imports, no CEL expressions), flags are the config surface via flagenv (TARGET/POLICY_FNAME map to --target/ --policy-fname), and the image's exec-form HEALTHCHECK is inherited by the quadlet unit (dodges the v1 generator's HealthCmd space-split quirk). Rollback nuance: stopping anubis 502s everything behind the chain (the public listeners proxy through it); a true rollback is reverting the Caddyfile and redeploying, not just systemctl --user stop anubis.
16 lines
468 B
JSON
16 lines
468 B
JSON
{
|
|
"bots": [
|
|
{
|
|
"name": "deny AI scrapers",
|
|
"user_agent_regex": "(?i)^(GPTBot|ChatGPT-User|OAI-SearchBot|CCBot|anthropic-ai|ClaudeBot|Claude-Web|PerplexityBot|PerplexityUser|Google-Extended|Applebot-Extended|Bytespider|Amazonbot|meta-externalagent|DuckAssistBot|cohere-ai|YouBot|KimiBot)",
|
|
"action": "DENY"
|
|
},
|
|
{
|
|
"name": "challenge everything else",
|
|
"path_regex": ".",
|
|
"action": "CHALLENGE"
|
|
}
|
|
],
|
|
"dnsbl": false
|
|
}
|