tests: gate the hermes vhost, and stop specs from silently not running
Two gaps behind the same near-miss. 1. **No spec covered hermes.franksdank.cloud** — the one guarded vhost whose upstream is a host address (`reverse_proxy 192.168.0.143:8787`) rather than a container name, because its service is host-networked. That is how a loopback-only bind could take the vhost down with a 502 and leave every gate green. tests/e2e/specs/17-hermes-vhost.spec.ts asserts what a dead upstream violates: no 5xx through the guard chain, no Caddy error page, and no app bytes to an unauthenticated raw client. It does not assert SSO-silence, since this app has its own password login. 2. **Specs are matched by number prefix, not globbed.** A spec file whose prefix is in no project silently never runs — which is exactly what happened to the new one until it was registered. The prefix is registered now, the comment in playwright.config.ts says why, and tests/e2e_scaffold.bats fails if any spec file matches no project. The hermes-webui seam assertion is corrected too: it asserted the loopback bind I wrongly imposed, and now asserts the real contract — the unit's bind must serve the address the Caddyfile fronts it on.
This commit is contained in:
1 parent
9c8a62f9fe
commit
25f387dd76
5 files changed
+111
-18
No files matched your search
@@ -46,3 +46,24 @@ PY
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
@test "every e2e spec is matched by a playwright project" {
|
||||
# Playwright projects match specs by NUMBER PREFIX, not by glob, so a spec
|
||||
# whose prefix is not listed in any project silently never runs: the suite
|
||||
# reports green while the test does not exist. That happened on 2026-09-16
|
||||
# with 17-hermes-vhost.spec.ts — written to catch a real regression (a
|
||||
# loopback-only bind that killed hermes.franksdank.cloud's upstream while
|
||||
# every gate stayed green), and invisible until the prefix was registered.
|
||||
local cfg="$REPO_ROOT/tests/e2e/playwright.config.ts"
|
||||
python3 - "$cfg" <<'PYEOF'
|
||||
import pathlib, re, sys
|
||||
cfg = pathlib.Path(sys.argv[1]).read_text()
|
||||
patterns = [re.compile(p) for p in re.findall(r"testMatch:\s*/(.+?)/,", cfg)]
|
||||
assert patterns, "no testMatch patterns found in the playwright config"
|
||||
specs = sorted(pathlib.Path(sys.argv[1]).parent.joinpath("specs").glob("*.spec.ts"))
|
||||
assert specs, "no spec files found"
|
||||
unmatched = [s.name for s in specs if not any(p.search(s.name) for p in patterns)]
|
||||
assert not unmatched, f"specs matched by no project (they will never run): {unmatched}"
|
||||
print(f"{len(specs)} specs, all matched by {len(patterns)} projects")
|
||||
PYEOF
|
||||
}
|
||||
Reference in new issue
Block a user