---
type: reference
description: "Authentik is the single sign-on (SSO) and OpenID Connect (OIDC) provider for the"
tags:
  - infrastructure
  - authentik
timestamp: 2026-09-16
---

# Authentik — the homelab's SSO / OIDC provider

Authentik is the single sign-on (SSO) and OpenID Connect (OIDC) provider for the
homelab. It protects the Caddy vhosts that do not carry their own
application-level authentication with its embedded proxy outpost. Apps that
have their own login (Grafana, Gitea, OpenWebUI, Jellyfin, the *arr stack,
Pi-hole) are not wrapped — they handle their own session.

- **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`)
- **Host:** dankclaw (`192.168.0.143`)
- **Name:** `auth.franksdank.cloud`
- **Stack:** 3 compose services — `authentik-postgres`, `authentik-server`,
  `authentik-worker` (see [Pi-hole](Pi-hole.md) for the LAN-DNS pattern)
- **LDAP outpost:** a 4th, quadlet-deployed unit (`authentik-outpost-ldap`)
  serves LDAP for Jellyfin — see the section below.

## Why Authentik

The comparison that chose it (against Authelia, Keycloak, Zitadel, Ory) is in
`.docaudit/research/auth.md`. In short: full OIDC with single-logout, a web GUI
for all configuration, an outpost that terminates auth for reverse-proxied
apps, and a footprint that fits on a 3-user homelab (~1–1.5 GB idle). Redis
was removed in 2025.10, so the stack is 3 services, not 4. (The integration
mechanism changed in 2026.x — see Architecture; the research file predates
the pin.)

## Architecture

```mermaid
graph LR
    U[Browser] -->|1. GET /search?q=x| A[Anubis :8923<br/>bot challenge]
    A -->|2. verified, dials internal listener| C[Caddy :8016<br/>auth decision]
    C -->|3. whole request, Host preserved| S[authentik-server :9000<br/>embedded outpost]
    S -->|4. no session: login flow served in place| U
    S -->|5. session valid: proxies on to internal_host| P[Caddy :8017<br/>post-auth routing]
    P -->|6. app response back up the chain| U
    S --> PG[(postgres)]
```

Two mechanisms were disproved live (2026-08-29/30) and one is the actual
contract:

- The old `/outpost.goauthentik.io/auth/caddy` forward-auth endpoint does not
  exist in the 2026.8 URL tree — every unauthenticated subrequest got
  authentik's own 404 page.
- `X-authentik-meta-outpost/-provider/-app` headers are IGNORED for routing
  in this build: a request carrying Caddy's full meta-header set still 404s,
  while the same request with only a matching `Host` answers. The embedded
  outpost routes **solely on the original `Host`**, matched against each
  attached proxy provider's `external_host`.
- Consequence: **every guarded host needs its own proxy provider** (exact
  `external_host`), and `:8016` is a single uniform `reverse_proxy` to
  `authentik-server:9000` that preserves `Host`. Unauthenticated → login flow
  served in place; authenticated → forwarded to the provider's
  `internal_host` (`http://caddy:8017`), which routes straight to the app.
  The chain terminates there — nothing dials back into Authentik from
  `:8017`.

## The three compose services

| Service | Role | Image |
|---|---|---|
| `authentik-postgres` | PostgreSQL 16 (users, flows, sessions, events) | `docker.io/library/postgres:16-alpine` |
| `authentik-server` | Core + admin API + the embedded Caddy outpost (port 9000) | `ghcr.io/goauthentik/server:2026.8.3` |
| `authentik-worker` | Background tasks (event processing, provider sync, flows) | `ghcr.io/goauthentik/server:2026.8.3` |

- **Postgres** is the only stateful store. Sessions, caching, and WebSockets
  moved to PostgreSQL in 2025.10 (Redis removed).
- **Server** and **worker** run the same image; the `command` selects the role
  (`server` / `worker`).
- The worker runs as `0:0` (rootless: container root maps to the host user that
  owns the data dir).
- Data persists under `/home/dank/Containers/authentik/{data,postgres}`
  (bind mounts, repo policy — no named volumes; moved from `/home/dank/authentik/`
  in the 2026-09-01 quadlet standardization).

## Caddy integration

Guarded vhosts reach Caddy's internal listener `:8016` (post-Anubis), which
proxies **every request whole** to the Authentik server with `Host`
preserved — there is no `forward_auth` and no per-vhost block:

```caddyfile
http://:8016 {
	reverse_proxy authentik-server:9000 {
		header_up X-Forwarded-Proto https
	}
}
```

The embedded outpost inside the server process decides by `Host`:
no session → the provider's `authentication_flow` is served in place;
session valid → the request is forwarded to the provider's `internal_host`
(`http://caddy:8017`), where per-vhost `handle` blocks reach the apps.

**Lockstep rule:** the Anubis guard set, the Caddy TLS edge, the `:8017`
routing table, and the Authentik proxy-provider list must be the same set of
hosts. A guarded host with no provider 404s at login time; a provider with no
DNS record breaks off-LAN clients.

Live provider inventory (verified 2026-09-02), all attached to outpost
`7e39655b-…` ("authentik Embedded Outpost"), all with
`internal_host = http://caddy:8017`, all three flows set, and each linked to
an Application (see the provisioning rules below):

| Provider pk | name | external_host | app slug |
|---|---|---|---|
| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
| 2 | search-proxy | `https://search.franksdank.cloud` | search |
| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
| 6 | files-proxy | `https://files.franksdank.cloud` | files |
| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |

Provider 3 (`hermes-proxy`) still exists but is dormant — its vhost was
decommissioned from Caddy on 2026-08-31. Provider 9 (`grafana`) and 12
(`Seerr OIDC`) are OAuth2 relying-party providers, not proxy providers; the
outpost does not deserialize them. Trailing slashes on `external_host` (the
runbook-recipe style) and bare origins both intercept correctly — the outpost
matches on hostname.

Two provisioning rules, each verified live by an outage:

1. **All three flows are required.** A ProxyProvider created via raw API/ORM
   with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
   null wedges the embedded outpost's config fetch: the Rust side rejects the
   whole config JSON (serde: `invalid type: null, expected a formatted UUID
   string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
   forever, and **everything behind the outpost dies at the next outpost
   reload** — including hosts whose providers were fine (verified 2026-09-02
   when three providers provisioned without any flows took down every guarded
   vhost after a server restart). Backfill all three flows (copy from pk 1:
   `default-authentication-flow`,
   `default-provider-authorization-implicit-consent` (the implicit-consent
   authorization flow, `default-invalidation-flow`) and the retry loop
   recovers on its next 3-second cycle. The silent killer: while the outpost
   process keeps its last good in-memory config, everything works — the
   outage only surfaces on restart/reload.
2. **Every proxy provider needs an attached Application.** The outpost
   resolves the post-login redirect through the provider's application; a
   provider with no app completes the login (authentik session established)
   but bounces to authentik's own user UI (`/if/user/#/library`) instead of
   the app. No error is logged anywhere — the SSO "works" and the app never
   appears. Create the Application with `provider_id` set (the UI does this
   automatically; raw API/ORM drives must do it explicitly).


## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)

The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
implementation diverges from upstream in ways that bit us; all verified live:

- **Application row required.** `OAuth2Provider.objects.create()` does not
  create the single-table-inheritance `Application` row; without it the flow
  executor can never complete an authorize request. Create both rows.
- **`grant_types` defaults to empty.** Until the provider's `grant_types`
  contains `"authorization_code"` (enum value with underscore), the token
  endpoint rejects every exchange with `invalid_request`.
- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
  `parse_token_request` see no `grant_type` → `unsupported_grant_type`;
  POST-body client secrets are not accepted — authenticate the confidential
  client with `Authorization: Basic base64(client_id:client_secret)`.
- **Provider resolution is by `client_id` query param** on the global
  `/application/o/authorize/` route; per-slug authorize paths do not exist in
  this build. Token/userinfo live at `/application/o/token/` and
  `/application/o/userinfo/?api=1`.
- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
  `PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
  `BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
  becomes the *expression source* instead of the context; every call raises a
  swallowed `PropertyMappingExpressionException` and the claim is skipped.
  Scope-grant filtering has the same class of bug: `ScopeMapping` has no
  provider FK (linkage lives in the `ProviderPropertyMapping` through model),
  so `filter(provider=...)` always yields an empty queryset and tokens asked
  to carry scopes get zero. Net effect: userinfo/introspection answers with
  `{sub}` only — email/groups/entitlements claims cannot be delivered by this
  build until the evaluator is patched or upgraded. Clients consuming this
  provider must key identity off `sub`.
- Scope mappings for this provider exist (`grafana-email-scope`,
  `grafana-profile-scope`, Python-syntax expressions) and will start emitting
  claims the moment the evaluator bug is fixed — no config change needed then.

## LDAP outpost (Jellyfin auth, live 2026-09-01)

A fourth unit, `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`
→ `authentik-outpost-ldap.service`, serves LDAP for Jellyfin's LDAP-Auth
plugin. The image is pinned by its **amd64 single-platform digest**
(`ghcr.io/goauthentik/ldap@sha256:fa33bc8f522113830e1662f4c034352f9eb08e0a7d3da2ddadad9a4b33b4cbf4`):
podman 5.7.0 refuses the
`ghcr.io/goauthentik/ldap:2026.8.0` tag because the tag resolves to a
*nested* manifest list (index → per-platform manifest lists) —
`unexpectedly received a manifest list instead of a manifest for a single
image`. Resolve once with `podman manifest inspect <image> | jq -r
'.manifests[] | select(.platform.architecture=="amd64") | .digest'` and pin
that digest.

- Ports **389** (ldap) and **636** (ldaps) publish on the LAN; Jellyfin on
  Dankserver binds `192.168.0.143:389`. The outpost token travels in
  `AUTHENTIK_LDAP_OUTPOST_TOKEN` from the host `.env` (git-ignored).

### Authentik-side objects (provisioned via `ak shell` ORM)

| Object | Value |
|---|---|
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud` |
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` |
| Service account | `svc-jellyfin-ldap` — passwordless/unusable; authenticates by token |
| Bind token | intent **`verification`**, non-expiring — its key is the LDAP bind password |
| RBAC role | `svc-jellyfin-ldap-search` carrying object-level permission `authentik_providers_ldap.search_full_directory` on the provider |
| Groups | `jellyfin_users`, `jellyfin_admins` |
| Application | `jellyfin` (slug `jellyfin`) |

Hard-won rules, each verified live:

- **Token intent is load-bearing.** A bind token with `intent=app_password`
  is rejected by the flow's password stage with `Invalid password`;
  `intent=verification` passes. The outpost's direct binder executes the
  provider's authorization flow, and that password stage only accepts
  verification-intent tokens as static passwords.
- **Search needs `search_full_directory`.** Without the RBAC grant the
  outpost returns only the bind user itself — a `(memberOf=…)` filter then
  matches zero entries and the plugin logs `Found no users matching X`.
  The permission is object-level on the provider; grant it via an RBAC role
  and retest.
- `ak shell` import path for the provider model in 2026.8:
  `from authentik.providers.ldap.models import LDAPProvider`.

## Configuration (secrets)

Two secrets live in the host `.env` (git-ignored; placeholders in
`.env.example`):

- `AUTHENTIK_PG_PASS` — the PostgreSQL password. Generate: `openssl rand -base64 32`.
- `AUTHENTIK_SECRET_KEY` — the server's signing key. Generate: `openssl rand -base64 48`.

## First-time setup

1. `./scripts/deploy.sh dankclaw` — pulls, starts the 3 services (postgres
   health-gated), reloads Caddy.
2. Open `https://auth.franksdank.cloud` — the initial setup wizard runs. It
   creates the superuser (email + password) and generates the DB schema.
3. For every guarded vhost, in the admin interface: **Directory →
   Applications** → add an application + a **Proxy** provider whose
   `external_host` is exactly the browser-facing origin
   (`https://<host>`), `internal_host = http://caddy:8017`, and **all three**
   flows set (see the provisioning pitfall above). Attach the provider to the
   embedded outpost.
4. For apps with native OIDC (Grafana, Gitea, OpenWebUI): add an **OAuth2/OpenID
   Connect** provider, create the app with the correct redirect URI, and paste
   the `client_id` / `client_secret` into the app's SSO settings.

## Operations

- **Health:** `https://auth.franksdank.cloud/-/health/` (the server healthcheck
  probes this).
- **Logs:** `podman logs authentik-server` / `podman logs authentik-worker`.
- **Backups:** `pg_dump` the `authentik-postgres` database (the single source of
  truth). The `/home/dank/Containers/authentik/data` dir holds media + static files.
- **Upgrades:** bump the image tag in the three units under
  `dankclaw/authentik/quadlet/`, run `./scripts/deploy.sh dankclaw`. Authentik
  migrates the schema on startup.

## Self-serve user-settings flow (password change)

Users change their own account details through the `default-user-settings-flow`
(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
target of the homepage footer's "account" link. It is a `stage_configuration`
designated flow, so it renders every bound `ConfigurableStage` for the logged-in
user.

Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
insertion order):

| order | stage | type | what it does |
|---|---|---|---|
| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |

**Password change is a prompt field, not a stage.** The `UserWriteStage`
writes whatever the preceding `PromptStage` collected: for the key `password`
it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
is a `password` field on the `default-user-settings` prompt
(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
`validate_password` runs `authenticate()` against the backends to check the
*current* password; it never sets a new one. (A `PasswordStage` was added on
2026-09-25 and removed the same day for exactly this reason.)

Two ORM gotchas: `Flow.stages.add()` fails on this through model
(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
and a prompt field is a `Prompt` row linked through `PromptStageField`
(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).

**A blank password must not set an empty password.** `set_password("")` would
make the account log in with no password, so a validation policy
(`default-user-settings-no-empty-password`, bound to the prompt via
`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
is `""` and allows it when set. Verified with the `PolicyEvaluator`
(empty → deny, set → allow, absent → allow) and end-to-end by driving
`UserWriteStageView.update_user` on a throwaway user (new password valid, old
invalid).

## Outpost claim cache (`proxysession`) — 403 after a group add

The embedded outpost does not re-evaluate group membership per request. Each
completed authorize writes one row to `authentik_providers_proxy_proxysession`
whose `session_data` JSON holds the claims (`groups` included) that get served
for that browser session until the row expires (observed TTLs 1–24 h per
row). A session authorized **before** a user was added to a group keeps
serving the old group list — Caddy-side RBAC header gates then 403 despite
correct membership, and no authentik event is generated (the 403 happens at
Caddy; authentik never sees the request).

Diagnosis + fix (verified live 2026-09-01):

```sql
-- who holds which cached groups (run in the authentik-postgres container)
SELECT session_data->'claims'->>'preferred_username' AS user,
       session_data->'claims'->>'groups'             AS cached_groups,
       expires
FROM authentik_providers_proxy_proxysession ORDER BY expires;
-- invalidate one user's cache; their authentik login session is untouched,
-- so the next visit re-authorizes silently and mints fresh claims
DELETE FROM authentik_providers_proxy_proxysession
WHERE session_data->'claims'->>'preferred_username' = 'akadmin';
```

The silent re-auth path was proven end-to-end: after deleting
`qa-e2e-admin`'s rows, the e2e admin spec ran against the stale storage
state and passed with no login prompt (2 tests, 2.9 s).

## Guard-chain flip recipe (adding a guarded vhost)

Move a public vhost from `reverse_proxy <app>:<port>` onto the chain:
client → TLS edge → Anubis :8923 → outpost decision → internal
`http://:8017` listener → Host-matched handler → real upstream.

**Order (never reorder):**

1. **Provider first.** Create the outpost ProxyProvider before touching
   routes, or authed traffic hits the outpost error page instead of the app.
   Fleet convention for `external_host` is the exact browser-facing origin
   (the inventory table carries both bare origins and trailing slashes;
   routing matches on hostname, verified 2026-09-02).
   `internal_host = http://caddy:8017`, all three flows set, and an
   Application attached — the two provisioning rules above.
2. **Route flip in ONE commit/deploy**: the vhost becomes
   `reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
   inside the `http://:8017` listener add `@<svc> host <vhost>` +
   `handle @<svc> { reverse_proxy <upstream> }`. Anubis guard set, Caddy TLS
   edge, `:8017` routing table, and provider list change together.
3. Deploy via the repo deploy flow (commit first; deploy pushes main itself).

**Verification (all three):**

- Unauthenticated probe from a client: `curl -s https://<vhost>/` → HTTP 200
  challenge page; grep the body for `challenge|not a bot|anubis` (~8
  markers on a live challenge). A 502 here instead of the challenge = chain
  mis-wired.
- Post-auth routing proof: grep the LIVE config inside the container —
  `podman exec caddy sh -c "grep -n -A3 <handler> /etc/caddy/Caddyfile"` —
  after the deploy prints its reload line; confirm the reload actually ran,
  don't trust exit 0.
- Outpost reachability: `podman exec authentik-server getent hosts caddy`
  resolves the compose DNS alias (10.89.1.x) that `internal_host` dials.

**Pitfalls:** probing `http://localhost:8017/` from the host returns HTTP
000 (connection refused) — the post-auth listener lives in the caddy
container's netns; probe inside it or grep the config. Non-browser clients
cannot solve the JS challenge — before flipping, audit service-to-service
traffic: ingestion/push paths must target internal addresses (`loki:3100`),
never a public hostname of a guarded service. Exempt-by-construction
categories stay unguarded: the IdP itself (`auth.`), git smart-HTTP
endpoints, services with their own complete auth — document the exemption
in the vhost comment.

## Driving the ORM (`ak shell`)

All authentik object work happens through the server container's ORM shell —
no repo change, no deploy (these providers are NOT repo-managed; capture new
provider+application pairs as a blueprint or setup script to avoid drift).

```bash
ssh dankclaw 'podman exec -i authentik-server ak shell' <<'EOF' | sed -n '/229 objects/,$p'
from authentik.core.models import User, Group
...
EOF
```

- The heredoc IS the program: `ak shell` reads all of stdin and execs it
  (heredoc-through-ssh quoting breaks; the raw-stdin pattern is the proven
  one).
- Bootstrap prints JSON log lines, then
  `### authentik shell (2026.8.3)` and `229 objects imported automatically…`
  — pipe through `sed -n '/229 objects/,$p'` to drop the noise.
- 229 models are pre-imported. Import paths (each wrong guess costs a failed
  exec): `User`, `Group`, `Application`, `ApplicationEntitlement`,
  `PropertyMapping` → `authentik.core.models`; `ProxyProvider` →
  `authentik.providers.proxy.models` (NOT `authentik.outposts.models`);
  `LDAPProvider` → `authentik.providers.ldap.models`;
  `authentik.core.models.application` does NOT exist as an import path; there
  is no `authentik.policies.group` module in this build.
- For live DB truth prefer the postgres container
  (`podman exec authentik-postgres psql -U authentik -d authentik -At -c
  "select * from …"`) — **query with `select *` and stop guessing column
  names**: 2026.x renamed columns (`Provider.objects.filter(type=…)` raises
  FieldError — `type` moved to subclass tables; the outpost `enabled` column
  may not exist under that name; provider pk is `oauth2provider_ptr_id`).

## User provisioning playbook (user-level account)

There is no `User.create_user` classmethod, and `is_superuser` is NOT a
constructor kwarg (it is a property derived from group membership) — passing
it raises `TypeError: User() got unexpected keyword arguments`:

```python
from authentik.core.models import User, Group
u = User(username="ridley", email="ridley@franksdank.cloud", name="Ridley", is_active=True)
u.set_password("...")        # password arrives on stdin only, never argv
u.save()
for g in ("Non-Admin Users", "jellyfin_users", "openwebui_users"):
    u.groups.add(Group.objects.get(name=g))
```

`path` defaults to `users` (regular). The access map (verified live
2026-10-04): there are NO per-app policy bindings on the proxy providers
(all use the implicit-consent "Authorize Application" flow), so guarded-vhost
access is decided entirely by GROUP MEMBERSHIP consumed downstream —

- `homepage-admins` → Caddy RBAC for homepage `/admin` (`X-Authentik-Groups`
  regex gate) and the homepage admin panel; regular users get only
  `category: user` panels from `services.json`.
- `openwebui_users` / `openwebui_admins` → ApplicationEntitlements
  "Open WebUI Users"/"Open WebUI Admins" (bound by group), consumed by the
  "Open WebUI roles" property mapping → `roles` claim. Verify live with
  `u.app_entitlements(app)`.
- `jellyfin_users` / `jellyfin_admins` → Jellyfin via the LDAP outpost
  (filter `(memberOf=cn=jellyfin_users,ou=groups,dc=franksdank,dc=cloud)`).
- `Non-Admin Users` = the plain-user marker group (precedents: qa-e2e, and
  every human account).
- **The human-user template** (ridley, kassie, alex, artem, kirby, ry, nic,
  debo, freya — kirby/ry/nic/debo/freya (pk 17–21) created 2026-10-09, kassie
  and alex created 2026-10-08, artem (pk 16) created 2026-10-07): `username` +
  `<user>@franksdank.cloud` + display name +
  `is_active=True` + the password set via `set_password`, membership in
  `Non-Admin Users` + `jellyfin_users` + `openwebui_users`. `alex` was
  recreated from scratch on 2026-10-08 into this shape (the old row had an
  empty email and no `openwebui_users`; audit first — the row carried no MFA
  device, token, entitlement, or proxy session, and events survive a user
  delete because `Event.user` is a denormalized JSON field, not a FK).
- EXCLUDE from a user-level account: `homepage-admins`, `jellyfin_admins`,
  `openwebui_admins`, `authentik Admins`, `authentik Read-only`.
- Other user-category vhosts (SearXNG, Gitea, Calibre-Web, RomM, Copyparty,
  Plex, Seerr, Marreta) need no per-user group: any authenticated user
  passes the outpost.

Verify login behaviorally — `authenticate()` is the exact PasswordStage code
path:

```python
from django.contrib.auth import authenticate
assert authenticate(request=None, username="ridley", password="...")   # -> User
assert authenticate(request=None, username="ridley", password="nope") is None
```

Look flows up by slug, not name:
`Flow.objects.get(slug="default-authentication-flow")` (its `.name` is
"Welcome to authentik!" — `.get(name=…)` fails). Stage order:
Identification(10) → Password(20) → AuthenticatorValidate(30; no-op with no
enrolled device) → UserLogin(100). With no MFA enrolled, nothing else blocks.

More field gotchas (each cost a failed exec): `ApplicationEntitlement.application`
can be null (guard with `getattr`); `LDAPProvider` has
`search_mode`/`bind_mode`/`base_dn`, lacks `search_base`/`of_groups`/
`organization`; `OAuth2Provider` has no `.scopes`; `Outpost.config` is an
`OutpostConfig` object, not a dict; `Policy` has no uniform `.policy_cls`
(use `type(p).__name__`); `User` has no `.status`.

Never rotate `AUTHENTIK_SECRET_KEY` to fix claim issues — it invalidates
every outpost session fleet-wide; purge the stale `proxysession` rows
instead (section above).

## Probing the outpost from inside the netns

Rootless podman gives each container its own netns; **host `ss -ltn` cannot
see listeners inside other containers' namespaces** — a missing listener on
the host is an observation artifact, not a fault. Distroless images have no
shell (`podman exec authentik-server sh` fails), so probe by joining the
namespace:

```
podman run --rm --network container:<container> --entrypoint /bin/busybox \
  docker.io/library/busybox:latest \
  wget -q -T 4 -t 1 -O /dev/null -S "<url>" 2>&1 | head -8
```

- Quote URLs over ssh: the remote shell is zsh and bare `?` in query strings
  triggers globbing (`no matches found`).
- Boot transience: immediately after a restart the same probe can return 503
  (configure task pending); poll until the answer changes before concluding.
- `podman inspect --format "{{.Config.NetMode}}"` errors on this podman
  version — the field is `.HostConfig.NetworkMode`.

## Failure modes

| Symptom | Cause | Fix |
|---|---|---|
| Every vhost 502s at the outpost | `authentik-server` down | `podman logs authentik-server`; check postgres health |
| Login loops (302 → login → 302) | `trusted_proxies` / `X-Forwarded-*` mismatch | Confirm Caddy sets `X-Forwarded-For`; the outpost trusts `private_ranges` |
| "Invalid signature" on the outpost | `AUTHENTIK_SECRET_KEY` differs between server and worker | Both services read the same `.env` var; re-deploy |
| Postgres "password authentication failed" | `AUTHENTIK_PG_PASS` changed after first init | Reset via `pg_ctl` or re-init the data dir |
| "Request has been denied: Invalid next URL" at sign-in | Flow entered with an absolute cross-host `next=` (old redirect design) | Do not hand-craft flow URLs; enter guarded hosts normally — the outpost carries state in its own cookie |
| `/outpost.goauthentik.io/*` 404s on every host | Any provider row with a null flow (`authentication`/`authorization`/`invalidation`) wedges the outpost config fetch — all guarded vhosts die at the next outpost reload | Backfill all three flows on every provider; watch `podman logs authentik-server` for `Failed to fetch outpost from API` |
| Login completes but lands on authentik's user UI (`/if/user/…`) instead of the app | Proxy provider has no attached Application (outpost cannot resolve the post-login redirect) | Create the Application with `provider_id` set — no error is logged, so test with a fresh browser context |
| Off-LAN clients get "hostname could not be found" | Guarded host missing from the public DNS zone | Every host in the lockstep set needs a public A record (Porkbun) as well as the Pi-hole LAN answer |
| Correct membership but one user still 403s at a header gate | Outpost claim cache: a `proxysession` row minted before the membership change serves old claims until expiry | Delete that user's `proxysession` rows (section above) — next visit re-authorizes silently |

## Cross-references

- [Caddy](Caddy.md) — the reverse proxy carrying the `:8016`/`:8017` auth-chain listeners
- [Pi-hole](Pi-hole.md) — the LAN DNS that resolves `auth.franksdank.cloud`
- [Jellyfin](../Dankserver/Jellyfin.md) — consumes the LDAP outpost via its LDAP-Auth plugin
- [Network Architecture](../Architecture/Network%20Architecture.md) — the flat naming scheme
- Research: `.docaudit/research/auth.md` — the full provider comparison