deploy: make Playwright e2e a mandatory post-deploy gate
The 2026-09-12 homepage outage slipped through because the live-fleet Playwright suite was a manual 'MUST invoke' that deploys skipped. Wire it into deploy.sh as the [3/3] gate: after a behavior-affecting deploy, run tests/e2e/run.sh against the live fleet; a failure exits DEPLOY INCOMPLETE. Add --skip-e2e as an emergency escape hatch. Update the AGENTS.md deployment-completeness contract and flow diagram to match. Verified: bash -n clean, --skip-e2e parsed, unknown flags rejected, seam suite green (only the pre-existing comfyui test 73 fails).
This commit is contained in:
@@ -148,9 +148,12 @@ only when BOTH of the following pass:
|
||||
1. **Unit/config tests** — `./tests/run.sh` (all seams green), and
|
||||
2. **Playwright end-to-end tests** — `tests/e2e/run.sh` drives the LIVE fleet
|
||||
(homepage, Anubis/Authentik guard chain, guarded vhosts) in a real
|
||||
browser. The E2E suite is deliberately excluded from `./tests/run.sh`
|
||||
because it needs live hosts and credentials, so it MUST be invoked
|
||||
explicitly after every behavior-affecting deploy.
|
||||
browser. As of 2026-09-12 this runs **automatically** as the `[3/3]` gate
|
||||
at the end of `deploy.sh`: a behavior-affecting deploy that fails the e2e
|
||||
suite exits `DEPLOY INCOMPLETE` and must not be treated as good. The suite
|
||||
stays out of `./tests/run.sh` (it needs live hosts plus the `qa-e2e`
|
||||
credentials in `tests/e2e/.env.local`). Bypass only with
|
||||
`deploy.sh --skip-e2e` in an emergency.
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
@@ -161,6 +164,10 @@ graph LR
|
||||
E --> F[Target pulls with ff-only]
|
||||
F --> G[deploy-remote.sh dispatches selected service]
|
||||
G --> H[Per-service health check]
|
||||
H --> I[Playwright e2e gate on live fleet]
|
||||
I --> J{e2e green?}
|
||||
J -->|yes| K[Deploy complete]
|
||||
J -->|no| L[DEPLOY INCOMPLETE]
|
||||
```
|
||||
|
||||
## Boot Persistence & Self-Healing (MANDATORY)
|
||||
|
||||
@@ -29,6 +29,7 @@ Usage: deploy.sh <dankclaw|dankserver|frankbook-pro> [options]
|
||||
Options:
|
||||
--service NAME deploy only NAME (repeatable; bypasses automatic planning)
|
||||
--dry-run show the plan without pushing or connecting to a target
|
||||
--skip-e2e skip the mandatory Playwright live-fleet gate (emergency only)
|
||||
-h, --help show this help
|
||||
|
||||
Examples:
|
||||
@@ -74,6 +75,7 @@ case "$TARGET" in
|
||||
esac
|
||||
|
||||
DRY_RUN=0
|
||||
SKIP_E2E=0
|
||||
EXPLICIT_SERVICES=()
|
||||
while [[ "$#" -gt 0 ]]; do
|
||||
case "$1" in
|
||||
@@ -81,6 +83,10 @@ while [[ "$#" -gt 0 ]]; do
|
||||
DRY_RUN=1
|
||||
shift
|
||||
;;
|
||||
--skip-e2e)
|
||||
SKIP_E2E=1
|
||||
shift
|
||||
;;
|
||||
--service)
|
||||
[[ "$#" -ge 2 ]] || { echo "--service requires a name" >&2; exit 2; }
|
||||
EXPLICIT_SERVICES+=("$2")
|
||||
@@ -233,4 +239,24 @@ exec "$REMOTE_SHELL" ./scripts/deploy-remote.sh "$TARGET" "$NEW_REVISION" "$@" \
|
||||
--from-revision "$OLD_REVISION"
|
||||
REMOTE_EOF
|
||||
|
||||
# --- Playwright e2e gate (MANDATORY after behavior-affecting deploys) ---
|
||||
# The unit seams in tests/run.sh cannot see the live guard chain, LAN DNS, or
|
||||
# the rendered homepage — exactly the layers that broke on 2026-09-12. This gate
|
||||
# drives the LIVE fleet (Anubis -> Authentik outpost -> apps) and fails the
|
||||
# deploy on any regression. It needs tests/e2e/.env.local with the qa-e2e IdP
|
||||
# credentials (see .env.local.example). Use --skip-e2e for an emergency only.
|
||||
if [[ "$SKIP_E2E" -eq 1 ]]; then
|
||||
echo "[3/3] e2e gate SKIPPED (--skip-e2e): live fleet NOT verified." >&2
|
||||
else
|
||||
echo "[3/3] Running Playwright e2e gate against the live fleet..."
|
||||
if bash "$REPO_ROOT/tests/e2e/run.sh"; then
|
||||
echo "e2e gate PASSED."
|
||||
else
|
||||
echo "=== DEPLOY INCOMPLETE: e2e gate FAILED on $TARGET ===" >&2
|
||||
echo "The live-fleet Playwright suite did not pass. Do NOT treat this" >&2
|
||||
echo "deploy as good. Re-run manually: tests/e2e/run.sh" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "=== Deploy complete: $TARGET ==="
|
||||
|
||||
Reference in New Issue
Block a user