serverAppConfigs/serverAppConfigs-docs/Architecture/E2E Suite.md

type, description, tags, timestamp

type	description	tags	timestamp
reference	`tests/e2e/` drives the **live fleet** through a real browser: fresh contexts	architecture e2e homepage	2026-10-18

E2E Suite — homepage + guard chain (Playwright)

tests/e2e/ drives the live fleet through a real browser: fresh contexts hit https://www.franksdank.cloud, log in through the authentik two-stage flow, and crawl every service link through the Caddy → Anubis → outpost guard chain. It is the only test in this repo that exercises the chain as a user sees it; the bats seams assert config shape, not behavior.

Layout

Path	Role
playwright.config.ts	five projects (below), real-browser UA, trace on failure
specs/00-auth.setup.ts	setup project: two real authentik logins, saves storage states
specs/10-homepage.spec.ts	user panel set + open links derived from services.json (registry-derived — no hardcoded service lists)
specs/11-admin.spec.ts	admin project: admin grid + /admin/ RBAC (403 for standard user, members-only 200)
specs/12-crawl.spec.ts	every user open link reaches its real app through the guard chain
specs/19-frankbook.spec.ts	frankbook project: walks the six admin links that terminate on FrankBook-Pro (192.168.0.112) — conditional, see below
specs/13-sso.spec.ts	direct guarded-vhost visits resolve silently with ONLY the homepage session (single-login enforcement)
specs/14-guard.spec.ts	guest project, no storage state: unauthenticated traffic lands on the IdP login
specs/15-unauth-fetch.spec.ts	guest project: a wget-style client must never receive application bytes from a guarded service — it asserts on leakage (app-code markers), not on status code
specs/16-marreta.spec.ts	regression guard for the 2026-09-12 incident: the marreta tile resolving to the authentik login flow because the marreta-proxy ProxyProvider had no linked Application
specs/17-hermes-vhost.spec.ts	hermes.franksdank.cloud fronts a host-networked container, so the vhost upstream is the host LAN address; guards the 2026-09-16 loopback-bind 502 that the rest of the suite stayed green through
specs/18-openwebui-sso.spec.ts	ai.franksdank.cloud completes its OIDC round-trip silently with only the homepage session
lib/login.ts	akLogin helper (two-stage flow, shadow-DOM locators)
lib/env.ts	loads .env.local, exports REAL_UA
tests/e2e_scaffold.bats	bats seam: structure, placeholder-only example env, every spec matched by a playwright project (tests/e2e_scaffold.bats:51), and no credentials/auth state tracked. Never drives the fleet

Projects and auth

• setup — logs in as qa-e2e and qa-e2e-admin (fleet-managed IdP test users; the admin is a homepage-admins member), writes .auth/qa-e2e{,-admin}.json storage states.
• standard — 10/12/13/16/17/18 with the user state (tests/e2e/playwright.config.ts:33).
• admin — 11 with the admin state.
• guest — 14/15 with an empty state.
• frankbook — 19 with the admin state. Excluded from every default run: the *arr/qBittorrent links terminate on FrankBook-Pro, which sleeps when idle, and a sleeping Mac mini must never fail an unrelated deploy. The project joins only when E2E_INCLUDE_FRANKBOOK=1 (set by scripts/deploy.sh when a frankbook trigger path changed: tests/e2e/specs/19-frankbook.spec.ts, frankbook-pro/**, shared/homepage/services.json, or the dankclaw/Caddyfile serving the arr vhosts) or when an explicit --project argument names it. An 11-admin edit that touches the arr links must therefore be paired with a 19-frankbook run against an awake host before it is trusted live.

Credentials live in tests/e2e/.env.local (git-ignored; placeholders in .env.local.example). The scaffold bats seam fails the commit if creds or auth states are ever tracked.

Run

tests/e2e/run.sh                          # default set — frankbook project excluded
E2E_INCLUDE_FRANKBOOK=1 tests/e2e/run.sh  # default set + the frankbook walk
cd tests/e2e && npx playwright test       # ALL projects, frankbook included

Requires LAN reachability of www.franksdank.cloud (Pi-hole DNS or public). The suite is not part of ./tests/run.sh — it needs the live fleet and real IdP credentials. It is a deploy gate: scripts/deploy.sh runs it and aborts with DEPLOY INCOMPLETE: e2e gate FAILED when it does not pass (scripts/deploy.sh:278-285).

Gotchas (all paid for live)

1. Fresh context per login. Reusing a half-finished flow poisons authentik's pending_user stage and the second login hangs. The setup spec runs each login in its own context.
2. Lit stage re-render race. authentik's flow executor can re-render the password stage right after mount, wiping a programmatic fill; the submit then POSTs an empty form ("This field is required"). akLogin fills, verifies the value survived, submits via Enter, and retries.
3. Headless UA = bot. Anubis denies obvious headless UAs; the config overrides userAgent with a real Chrome UA (REAL_UA).
4. Cold Anubis PoW. The first challenge in a fresh context can outlive default assertion budgets; title asserts carry 60s budgets.
5. Solve-once cookie. Since Anubis v1.27.0 + COOKIE_DOMAIN (see Anubis), one PoW per fresh context covers every guarded vhost — crawl and SSO specs complete in ~0.5 s per vhost instead of a PoW each.
6. Apps navigate too — and may leave. 12-crawl returns to the homepage between links, and Plex Web (signed out in the qa-e2e profile) runs its own sign-in flow: it redirects the top frame, so Chromium cancelled our goto("/") with net::ERR_ABORTED (once as a 420 s timeout, once at 5 s on the same link), and the same redirect then lands the browser on app.plex.tv — Plex's hosted surface for a session-less client — which broke the hostname == target law. Two narrow answers: backHome() re-issues the navigation on that one error class, and the landing check takes an explicit hand-off list (HANDOFFS) instead of a single host. The guard contract is untouched: challenge cleared, no authentik uidField, status < 500, and every host NOT in the list still has to land on its own vhost.