Frank
ab06e4c221
Audit of all 12 e2e specs for the 'green-but-vacuous' class that broke
Grafana's SSO. The suite is otherwise sound (incident-driven, leakage-based);
two real coverage gaps surfaced:
- notebook.franksdank.cloud (open-notebook, deployed 2026-10-19) was a guarded
vhost in NEITHER the 13-/14- GUARDED lists NOR any dedicated spec. A broken
upstream would stay invisible - the exact 17-hermes class. Added to the 14-
guard audit (unauth -> authentik gate, 'Open Notebook' leak marker) plus a
new 21- that proves authenticated reachability to open-notebook's own login
through the guard chain, with a positive app marker.
- prometheus.franksdank.cloud was only ever asserted '<500 and not a login
wall' (13-/11-), which a 404 also passes. New 22- adds the app-is-really-
there proof 20- gives Grafana: SSO handoff + its own /api/v1/status/buildinfo
answers {status:success}.
Both specs wired into the standard project (playwright testMatch), which the
e2e_scaffold bats gate enforces. Verified live against the fleet; full bats
suite green.
34 lines
1.8 KiB
TypeScript
34 lines
1.8 KiB
TypeScript
import { expect, test } from "@playwright/test";
|
|
|
|
// Prometheus (prometheus.franksdank.cloud) sits behind the same embedded outpost
|
|
// as Grafana. 13-sso and 11-admin only ever assert it is "not a login wall and
|
|
// under 500" - which a Prometheus serving a 404 or a blank page would ALSO pass.
|
|
// This is the app-is-really-there proof 20- provides for Grafana: with the
|
|
// standard session the outpost hands off silently to the Prometheus UI, and the
|
|
// app's own API answers success. A guard-route or upstream regression that lets
|
|
// any non-Prometheus answer through fails here.
|
|
|
|
test("SSO: Prometheus app answers its own API through authentik", async ({ page, request }) => {
|
|
test.setTimeout(180_000);
|
|
await page.goto("https://prometheus.franksdank.cloud/");
|
|
|
|
// Challenge clears, outpost hands off silently to the app (no credential prompt).
|
|
await expect
|
|
.poll(
|
|
async () => !(await page.locator("body").innerText()).includes("Making sure you're not a bot"),
|
|
{ timeout: 90_000 },
|
|
)
|
|
.toBe(true);
|
|
await expect
|
|
.poll(() => new URL(page.url()).hostname, { timeout: 60_000 })
|
|
.toBe("prometheus.franksdank.cloud");
|
|
expect(await page.locator('input[name="uidField"]').count(), "no authentik form").toBe(0);
|
|
await expect(page, "must land on the Prometheus UI").toHaveTitle(/prometheus/i, { timeout: 30_000 });
|
|
|
|
// Load-bearing assertion: the Prometheus API itself answers, authenticated,
|
|
// through the outpost. An IdP bounce, a 404, or a raw upstream error fails here.
|
|
const api = await request.get("https://prometheus.franksdank.cloud/api/v1/status/buildinfo");
|
|
expect(api.status(), "Prometheus API must answer 200 through the guard").toBe(200);
|
|
expect((await api.json()).status, "Prometheus API must report success").toBe("success");
|
|
});
|