--- name: authentik-outpost-verification description: "Verify authentik embedded-outpost and proxy-provider state on a live host by probing from inside the container network namespace; covers meta-header routing, rootless podman observation artifacts, and 2026.x schema quirks (renamed columns, ak shell swallowing piped stdin)." --- # Authentik outpost/provider verification (live host) ## When to use Guarded apps behind Anubis+authentik return an error page after authentication, or you need ground truth on outpost/provider state on a host running authentik 2026.x with podman quadlet units. ## Core insight: probe from inside the network namespace - Rootless podman gives each container its own netns; **host `ss -ltn` cannot see listeners inside other containers' namespaces**. A missing listener on the host is an observation artifact, not a fault — verify by dialing *from inside* the namespace. - Working probe (distroless images have no shell, so `podman exec sh` fails): ``` podman run --rm --network container: --entrypoint /bin/busybox docker.io/library/busybox:latest \ wget -q -T 4 -t 1 -O /dev/null -S "" 2>&1 | head -8 ``` - Quote URLs over ssh: the remote shell is zsh and bare `?` in query strings triggers globbing (`no matches found`). ## Meta-header routing truth - The outpost routes by **meta-header middleware**, not URL-path lookup. A request to `/outpost.goauthentik.io/login` **without** `X-authentik-meta-outpost/-provider/-app` headers is a path that genuinely does not exist → a 404 there is *correct behavior*, not a fault. - Probe the auth hop exactly as Caddy sends it: full meta-header set (`X-authentik-meta-outpost: `, `X-authentik-meta-provider: `, `X-authentik-meta-app: name=`). If a full-meta-header request still 404s, the outpost's URL space is not mounted in the answering process — that is the real failure point, and it means the transport is aimed at a URL space that doesn't exist there (embedded outposts mount their URL space at `authentik_host`; check the outpost row's `_config.authentik_host` before theorizing). - Boot transience: immediately after a restart the same probe can return 503 (configure task pending); poll until the answer changes before concluding. ## Schema quirks in 2026.x - `ak shell` is interactive-only; piped stdin is swallowed (prints import banner, discards output). For live DB truth use: ``` podman exec authentik-postgres psql -U authentik -d authentik -x -c "select * from authentik_outposts_outpost" ``` - Columns are renamed vs older versions: `Provider.objects.filter(type=...)` raises FieldError (type moved to subclass tables); the outpost `enabled` column may not exist under that name; provider pk is `oauth2provider_ptr_id`. **Query with `select *` and stop guessing column names.** - `podman inspect --format "{{.Config.NetMode}}"` errors on this podman version — the field is `.HostConfig.NetworkMode`. ## Decision fork when the auth hop 404s A) point the guarded apps' auth hop at where auth actually lives (the vhost matching `authentik_host`), or B) flip the outpost to standalone (own container, own URL space, new quadlet unit + `AUTHENTIK_TOKEN`). This is an architecture decision for the user — present both with blast radius, don't pick silently.