0 Fork 0

Files

T

dank 183a6d5442 feat(fleet): give the media servers an I/O share that actually exists

A Plex section scan walked 50k files at ~850 r/s and nothing bounded it.
`IOWeight=` in a quadlet looks like the fix and does nothing: systemd delegates
`cpu memory pids` to user@.service and NOT io, and it drops a resource line it
cannot honour without a word — the container boots healthy and its cgroup has no
io.weight file at all. Same family as RequiresMountsFor= (inert in the user
manager) and ConditionPathIsMountPoint= (fail-dead).

Place the two system drop-ins that make the controller reachable (user.slice
IOAccounting=yes puts io in user.slice/cgroup.subtree_control, measured
immediately; user@1000.service DelegateControllers gains io and lands at the
next reboot, because restarting the user manager would kill every rootless
container on the host), then ask for the share: IOWeight=10 on jellyfin and plex.
10 is a convention, not a measured optimum — proportional share, only bites
under contention, only requirement is far below the default 100.

ensure_user_io_delegate places both from git and prints NOTE: io is not
delegated … inert until the next reboot, so the quadlet line never reads as live
config. Verified on the host: subtree_control flipped to `cpu io memory pids`,
the drop-in is attached and parsed (systemctl cat), and the podman 5.7 generator
emits IOWeight=10 for both units (bare key; UnitConfigItem= not needed).

Two limits recorded instead of hidden: a container's weight covers its whole
cgroup (scan and playback share it), and the comparison happens inside
user.slice, so it cannot push a rootless unit below a root unit such as
frank-archon — that pairing is user.slice's own weight, a separate call.

Docs: Quadlet Lifecycle 4b (the three-command read of the chain), Pi-hole (all
rootless containers share one 10.0.2.100, so one chatty container throttles DNS
for the host — 8 events today), E2E Suite gotcha 7 (the gate depends on the
runner's resolver: 21 reds on a lost wired link, 37/37 minutes later).

Seam: an in-repo IOWeight= may only exist when both drop-ins exist AND the
deployer places them. ./tests/run.sh: 789 ok, exit 0.

2026-10-31 18:42:27 -07:00

7.0 KiB

Raw Permalink Blame History

type, description, tags, timestamp

type

description

E2E Suite — homepage + guard chain (Playwright)

tests/e2e/ drives the live fleet through a real browser: fresh contexts hit https://www.franksdank.cloud, log in through the authentik two-stage flow, and crawl every service link through the Caddy → Anubis → outpost guard chain. It is the only test in this repo that exercises the chain as a user sees it; the bats seams assert config shape, not behavior.

Layout

Path	Role
`playwright.config.ts`	five projects (below), real-browser UA, trace on failure
`specs/00-auth.setup.ts`	setup project: two real authentik logins, saves storage states
`specs/10-homepage.spec.ts`	user panel set + open links derived from `services.json` (registry-derived — no hardcoded service lists)
`specs/11-admin.spec.ts`	`admin` project: admin grid + `/admin/` RBAC (403 for standard user, members-only 200)
`specs/12-crawl.spec.ts`	every user open link reaches its real app through the guard chain
`specs/19-frankbook.spec.ts`	`frankbook` project: walks the six admin links that terminate on FrankBook-Pro (192.168.0.112) — conditional, see below
`specs/13-sso.spec.ts`	direct guarded-vhost visits resolve silently with ONLY the homepage session (single-login enforcement)
`specs/14-guard.spec.ts`	`guest` project, no storage state: unauthenticated traffic lands on the IdP login
`specs/15-unauth-fetch.spec.ts`	`guest` project: a wget-style client must never receive application bytes from a guarded service — it asserts on leakage (app-code markers), not on status code
`specs/16-marreta.spec.ts`	regression guard for the 2026-09-12 incident: the marreta tile resolving to the authentik login flow because the `marreta-proxy` ProxyProvider had no linked Application
`specs/17-hermes-vhost.spec.ts`	`hermes.franksdank.cloud` fronts a host-networked container, so the vhost upstream is the host LAN address; guards the 2026-09-16 loopback-bind 502 that the rest of the suite stayed green through
`specs/18-openwebui-sso.spec.ts`	`ai.franksdank.cloud` completes its OIDC round-trip silently with only the homepage session
`lib/login.ts`	`akLogin` helper (two-stage flow, shadow-DOM locators)
`lib/env.ts`	loads `.env.local`, exports `REAL_UA`
`tests/e2e_scaffold.bats`	bats seam: structure, placeholder-only example env, every spec matched by a playwright project (`tests/e2e_scaffold.bats:51`), and no credentials/auth state tracked. Never drives the fleet

Projects and auth

setup — logs in as qa-e2e and qa-e2e-admin (fleet-managed IdP test users; the admin is a homepage-admins member), writes .auth/qa-e2e{,-admin}.json storage states.
standard — 10/12/13/16/17/18 with the user state (tests/e2e/playwright.config.ts:33).
admin — 11 with the admin state.
guest — 14/15 with an empty state.
frankbook — 19 with the admin state. Excluded from every default run: the *arr/qBittorrent links terminate on FrankBook-Pro, which sleeps when idle, and a sleeping Mac mini must never fail an unrelated deploy. The project joins only when E2E_INCLUDE_FRANKBOOK=1 (set by scripts/deploy.sh when a frankbook trigger path changed: tests/e2e/specs/19-frankbook.spec.ts, frankbook-pro/** shared/homepage/services.json, or the dankclaw/Caddyfile serving the arr vhosts) or when an explicit --project argument names it. An 11-admin edit that touches the arr links must therefore be paired with a 19-frankbook run against an awake host before it is trusted live.

Credentials live in tests/e2e/.env.local (git-ignored; placeholders in .env.local.example). The scaffold bats seam fails the commit if creds or auth states are ever tracked.

Run

tests/e2e/run.sh                          # default set — frankbook project excluded
E2E_INCLUDE_FRANKBOOK=1 tests/e2e/run.sh  # default set + the frankbook walk
cd tests/e2e && npx playwright test       # ALL projects, frankbook included

Requires LAN reachability of www.franksdank.cloud (Pi-hole DNS or public). The suite is not part of ./tests/run.sh — it needs the live fleet and real IdP credentials. It is a deploy gate: scripts/deploy.sh runs it and aborts with DEPLOY INCOMPLETE: e2e gate FAILED when it does not pass (scripts/deploy.sh:278-285).

Gotchas (all paid for live)

Fresh context per login. Reusing a half-finished flow poisons authentik's pending_user stage and the second login hangs. The setup spec runs each login in its own context.
Lit stage re-render race. authentik's flow executor can re-render the password stage right after mount, wiping a programmatic fill; the submit then POSTs an empty form ("This field is required"). akLogin fills, verifies the value survived, submits via Enter, and retries.
Headless UA = bot. Anubis denies obvious headless UAs; the config overrides userAgent with a real Chrome UA (REAL_UA).
Cold Anubis PoW. The first challenge in a fresh context can outlive default assertion budgets; title asserts carry 60s budgets.
Solve-once cookie. Since Anubis v1.27.0 + COOKIE_DOMAIN (see Anubis), one PoW per fresh context covers every guarded vhost — crawl and SSO specs complete in ~0.5 s per vhost instead of a PoW each.
Apps navigate too — and may leave. 12-crawl returns to the homepage between links, and Plex Web (signed out in the qa-e2e profile) runs its own sign-in flow: it redirects the top frame, so Chromium cancelled our goto("/") with net::ERR_ABORTED (once as a 420 s timeout, once at 5 s on the same link), and the same redirect then lands the browser on app.plex.tv — Plex's hosted surface for a session-less client — which broke the hostname == target law. Two narrow answers: backHome() re-issues the navigation on that one error class, and the landing check takes an explicit hand-off list (HANDOFFS) instead of a single host. The guard contract is untouched: challenge cleared, no authentik uidField, status < 500, and every host NOT in the list still has to land on its own vhost.
The runner's resolver is part of the suite. A 2026-10-31 deploy of one Dankserver quadlet failed the gate with 21 reds spread over SSO, homepage RBAC and every vhost crawl — precisely the specs that navigate to an absolute *.franksdank.cloud name, while every spec on the relative baseURL stayed green. The error was net::ERR_NAME_NOT_RESOLVED: the workstation had lost its wired link, and on the fallback its resolver was no longer pinned to Pi-hole. Nothing on the fleet was wrong — the same suite re-run minutes later passed 37/37. Before a mass red gets blamed on the guard chain, run resolvectl query grafana.franksdank.cloud on the box that runs the gate.

7.0 KiB Raw Permalink Blame History

E2E Suite — homepage + guard chain (Playwright)

Layout

Projects and auth

Run

Gotchas (all paid for live)

7.0 KiB

Raw Permalink Blame History