Frank
d2587e1d29
Two faces of the same verification round. 12-crawl returned to the homepage between links with a bare page.goto, and Plex Web (signed out, so it renders its own sign-in iframe for its Plex-account flow) keeps redirecting the top frame: Chromium cancelled our goto with net::ERR_ABORTED, outside the try, so the whole crawl died — once as a 420 s timeout in the deploy gate, once at 5.4 s on the same link. backHome() tolerates that one error class and re-issues; any other error still propagates, and the per-app verdict (guard settled, hostname, no authentik uidField, status < 500) is unchanged. Plex was healthy throughout (container up 2 days, 32400 answering HTTPS 401 / plain-HTTP empty reply), so this was app behaviour, not a fault — recorded as gotcha 6 on the E2E page. The 200,704 boot printed 5.5 GiB reserved and 74.1 GiB in all (not the ~5.4/~74 my arithmetic predicted), so every prose figure is re-keyed to the ledger, and the runbook gains the second production-boot section: marker revision, pool reservation, shape line, prompt-cache entry size, and the refusal contract at the new ceiling (400 in 0.86 s naming 200704 and the 96,405-token overage, before any prefill). Cost stated (two maximum-length conversations never coexist; the 180k compaction target is unreachable inside a 155,648 window) and purchase stated (1.7 GiB of page cache, 9.1 GiB against the 2026-10-06 shape).
4.9 KiB
4.9 KiB
type, description, tags, timestamp
| type | description | tags | timestamp | |||
|---|---|---|---|---|---|---|
| reference | `tests/e2e/` drives the **live fleet** through a real browser: fresh contexts |
|
2026-09-01 |
E2E Suite — homepage + guard chain (Playwright)
tests/e2e/ drives the live fleet through a real browser: fresh contexts
hit https://www.franksdank.cloud, log in through the authentik two-stage
flow, and crawl every service link through the Caddy → Anubis → outpost
guard chain. It is the only test in this repo that exercises the chain as a
user sees it; the bats seams assert config shape, not behavior.
Layout
| Path | Role |
|---|---|
playwright.config.ts |
five projects (below), real-browser UA, trace on failure |
specs/00-auth.setup.ts |
setup project: two real authentik logins, saves storage states |
specs/10-homepage.spec.ts |
user panel set + open links derived from services.json (registry-derived — no hardcoded service lists) |
specs/11-admin.spec.ts |
admin project: admin grid + /admin/ RBAC (403 for standard user, members-only 200) |
specs/12-crawl.spec.ts |
every user open link reaches its real app through the guard chain |
specs/19-frankbook.spec.ts |
frankbook project: walks the six admin links that terminate on FrankBook-Pro (192.168.0.112) — conditional, see below |
specs/13-sso.spec.ts |
direct guarded-vhost visits resolve silently with ONLY the homepage session (single-login enforcement) |
specs/14-guard.spec.ts |
guest project, no storage state: unauthenticated traffic lands on the IdP login |
lib/login.ts |
akLogin helper (two-stage flow, shadow-DOM locators) |
lib/env.ts |
loads .env.local, exports REAL_UA |
tests/e2e_scaffold.bats |
bats seam: structure + no-credentials-tracked (never drives the fleet) |
Projects and auth
- setup — logs in as
qa-e2eandqa-e2e-admin(fleet-managed IdP test users; the admin is ahomepage-adminsmember), writes.auth/qa-e2e{,-admin}.jsonstorage states. - standard — 10/12/13 with the user state.
- admin — 11 with the admin state.
- guest — 14/15 with an empty state.
- frankbook — 19 with the admin state. Excluded from every default run: the
*arr/qBittorrent links terminate on FrankBook-Pro, which sleeps when idle, and a
sleeping Mac mini must never fail an unrelated deploy. The project joins only
when
E2E_INCLUDE_FRANKBOOK=1(set byscripts/deploy.shwhen a frankbook trigger path changed:tests/e2e/specs/19-frankbook.spec.ts,frankbook-pro/**,shared/homepage/services.json, or thedankclaw/Caddyfileserving the arr vhosts) or when an explicit--projectargument names it. An11-adminedit that touches the arr links must therefore be paired with a19-frankbookrun against an awake host before it is trusted live.
Credentials live in tests/e2e/.env.local (git-ignored; placeholders in
.env.local.example). The scaffold bats seam fails the commit if creds or
auth states are ever tracked.
Run
tests/e2e/run.sh # default set — frankbook project excluded
E2E_INCLUDE_FRANKBOOK=1 tests/e2e/run.sh # default set + the frankbook walk
cd tests/e2e && npx playwright test # ALL projects, frankbook included
Requires LAN reachability of www.franksdank.cloud (Pi-hole DNS or public).
The suite is not part of ./tests/run.sh — it needs the live fleet and
real IdP credentials.
Gotchas (all paid for live)
- Fresh context per login. Reusing a half-finished flow poisons
authentik's
pending_userstage and the second login hangs. The setup spec runs each login in its own context. - Lit stage re-render race. authentik's flow executor can re-render the
password stage right after mount, wiping a programmatic
fill; the submit then POSTs an empty form ("This field is required").akLoginfills, verifies the value survived, submits via Enter, and retries. - Headless UA = bot. Anubis denies obvious headless UAs; the config
overrides
userAgentwith a real Chrome UA (REAL_UA). - Cold Anubis PoW. The first challenge in a fresh context can outlive default assertion budgets; title asserts carry 60s budgets.
- Solve-once cookie. Since Anubis v1.27.0 +
COOKIE_DOMAIN(see Anubis), one PoW per fresh context covers every guarded vhost — crawl and SSO specs complete in ~0.5 s per vhost instead of a PoW each. - Apps navigate too.
12-crawlreturns to the homepage between links, and Plex Web (signed out, so it shows its own "would like to sign in to your Plex account" iframe) keeps redirecting the top frame — Chromium cancels ourgoto("/")withnet::ERR_ABORTED, which used to abort the whole crawl (once as a 420 s timeout, once at 5 s on the same link).backHome()re-issues the navigation; the app itself was already judged, so the abort is the app's behaviour, not a guard-chain failure.