ci: run Playwright e2e suite on every PR by GirlBossRush · Pull Request #21996 · goauthentik/authentik

GirlBossRush · 2026-04-30T22:02:25Z

Depends on

ci: Consistent NPM versions via Corepack #20400

Summary

Wires up the existing web/test/browser/ Playwright suite as its own job in ci-web.yml, closing #21994.

Boots the full authentik stack from source (postgres + Go server + Rust worker) inside the workflow, applies migrations and the test-admin blueprint, then runs corepack npm run --prefix web test:e2e against http://localhost:9000. On failure, uploads the HTML report, traces/videos, and server + worker logs as artifacts so reviewers can debug without rerunning locally.

Also enables the HTML reporter and screenshot/video capture on CI in playwright.config.js, and updates the full dev-environment docs to point at the same npm scripts CI uses so local and CI runs stay in lockstep.

Stacking

This PR targets npm-corepack (#20400) intentionally — the new job uses corepack-prefixed npm calls and the composite setup-node action that #20400 introduces. Will rebase onto main once #20400 lands.

Notes for reviewer

Rust cold build is the long pole (~5-10 min cold, ~1 min warm via the existing rust toolchain cache keyed on Cargo.lock). 60-min job timeout chosen with that in mind.
Not added to ci-web-mark.needs — runs in parallel with lint/build to avoid cascading the heavy job into vitest. Branch protection should add e2e (playwright) directly as a required status check after the first green run.
No path filter — consistent with the rest of ci-web.yml.
Existing Django/Selenium suite under tests/e2e/ is untouched; both suites coexist as the issue requests.

Test plan

First run on this PR completes within the 60-min timeout (validates cold rust build assumption)
HTML report + traces + authentik logs are retrievable from a deliberately-failed run's artifacts
After merge, branch protection updated to require e2e (playwright) status check

Closes #21994

🤖 Generated with Claude Code

netlify · 2026-04-30T22:03:17Z

✅ Deploy Preview for authentik-integrations ready!

Name	Link
🔨 Latest commit	`e4ef898`
🔍 Latest deploy log	https://app.netlify.com/projects/authentik-integrations/deploys/69fa1da1b4181d000845baf1
😎 Deploy Preview	https://deploy-preview-21996--authentik-integrations.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

netlify · 2026-04-30T22:03:57Z

✅ Deploy Preview for authentik-docs ready!

Name	Link
🔨 Latest commit	`e4ef898`
🔍 Latest deploy log	https://app.netlify.com/projects/authentik-docs/deploys/69fa1da1207aa70008dd189a
😎 Deploy Preview	https://deploy-preview-21996--authentik-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

codecov · 2026-04-30T22:10:42Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.13%. Comparing base (6be7b2f) to head (e4ef898).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #21996      +/-   ##
==========================================
- Coverage   93.16%   93.13%   -0.04%     
==========================================
  Files        1024     1024              
  Lines       59287    59287              
  Branches      400      400              
==========================================
- Hits        55236    55216      -20     
- Misses       4051     4071      +20

Flag	Coverage Δ
conformance	`36.87% <ø> (+0.01%)`	⬆️
e2e	`41.31% <ø> (-0.79%)`	⬇️
integration	`33.30% <ø> (+<0.01%)`	⬆️
rust	`0.00% <ø> (ø)`
unit	`92.09% <ø> (+<0.01%)`	⬆️
unit-migrate	`92.11% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

netlify · 2026-04-30T22:12:43Z

✅ Deploy Preview for authentik-storybook ready!

Name	Link
🔨 Latest commit	`e4ef898`
🔍 Latest deploy log	https://app.netlify.com/projects/authentik-storybook/deploys/69fa1da1b4b9be00080f9be6
😎 Deploy Preview	https://deploy-preview-21996--authentik-storybook.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

github-actions · 2026-04-30T23:02:01Z

authentik PR Installation instructions

Instructions for docker-compose

Add the following block to your .env file:

AUTHENTIK_IMAGE=ghcr.io/goauthentik/dev-server
AUTHENTIK_TAG=gh-13d167e8d5c2317d071016b16415a9b7427c33d1
AUTHENTIK_OUTPOSTS__CONTAINER_IMAGE_BASE=ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s

Afterwards, run the upgrade commands from the latest release notes.

Instructions for Kubernetes

Add the following block to your values.yml file:

authentik:
    outposts:
        container_image_base: ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s
global:
    image:
        repository: ghcr.io/goauthentik/dev-server
        tag: gh-13d167e8d5c2317d071016b16415a9b7427c33d1

Afterwards, run the upgrade commands from the latest release notes.

github-actions · 2026-05-02T13:06:40Z

Playwright e2e — ✅ Passed

Result	Count
✅ Passed	22
❌ Failed	0
⚠️ Flaky	0
⏭️ Skipped	0

Workflow run · HTML report hosting is gated off until the authentik-playwright-artifacts S3 bucket is provisioned (vars.PLAYWRIGHT_S3_ENABLED). Until then, download the playwright-report artifact from the run page.

dominic-r · 2026-05-02T16:51:49Z

+        if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && vars.PLAYWRIGHT_S3_ENABLED == 'true' }}
+        uses: aws-actions/configure-aws-credentials@ec61189d14ec14c8efccab744f656cffd0e33f37 # v6.1.0
+        with:
+          role-to-assume: "arn:aws:iam::016170277896:role/github_goauthentik_authentik"


this might open us to more exposure/risk, I'd keep this as an env var

Seems to be safe :)

https://github.com/goauthentik/authentik/blame/0b6d3a2850b5678bb189cf845e23cb1e4d5abfe5/.github/workflows/release-publish.yml#L196

fda6054#diff-c6d47dacf31662f6791d0cf218b1d34ee0ab48348c037819431cdd40e3fc23e4R187

In that case, fair :)

Boots the full authentik stack (postgres + Go server + Rust worker) inside the existing ci-web workflow, applies migrations and the test-admin user blueprint, then runs `corepack npm run --prefix web test:e2e` against http://localhost:9000. Uploads the HTML report, traces/videos, and authentik logs as artifacts on failure so reviewers can debug without rerunning locally. Also enables the HTML reporter and screenshot/video capture on CI in playwright.config.js, and updates the full dev-environment docs to point at the same npm scripts CI uses so local and CI runs stay in lockstep. Closes #21994 Co-Authored-By: Agent (authentik-i21994-better-mobile-tangelo) <279763771+playpen-agent@users.noreply.github.com> ci/web: make test-admin blueprint self-contained The previous blueprint used !Find to look up the authentik Admins group, which raced against system/bootstrap.yaml and resolved to None when the explicit apply_blueprint step ran before the worker had applied bootstrap. The serializer rejected groups: [None] with Invalid pk "None". Define the group in the same blueprint with state: present and reference it via !KeyOf, so the test admin setup does not depend on any pre-existing data. If bootstrap has already created the group, state: present is a no-op on the identifiers; otherwise the group is created here. Co-Authored-By: Agent (authentik-i21994-better-mobile-tangelo) <279763771+playpen-agent@users.noreply.github.com> ci/web: format test-admin-user.yaml with prettier Pick up the 4-space indent that web/'s prettier config enforces. The file was added under issue #21994 with 2-space indent and tripped the ci-web format check on push. Co-Authored-By: Agent (authentik-i21994-better-mobile-tangelo) <279763771+playpen-agent@users.noreply.github.com> Use parallelism. Remove guard. Reorder tests. Ignore playwright-traces. Update expected path. Always parallel. Flesh out types. ci/web: post Playwright result comment + gated S3 upload + !cancelled() guards Three reviewer-facing improvements to the e2e job: 1. Idempotent PR comment summarising Playwright pass/fail/flaky/skipped counts. Marker `` lets re-runs edit the same comment instead of piling up. Skipped on fork PRs where the default GITHUB_TOKEN is read-only. 2. Optional S3 publish of the HTML report to `s3://authentik-playwright-artifacts/pr-<n>/run-<id>/attempt-<n>/`, gated behind `vars.PLAYWRIGHT_S3_ENABLED == 'true'`. The bucket is pending infra provisioning; the public URL pattern is already wired into the comment so flipping the variable on later requires no workflow changes. Borrows the OIDC + IAM role plumbing from `.github/workflows/release-publish.yml`. 3. Switch the failure-guarded reporting/upload steps to `!cancelled()` so a superseded (cancelled) run no longer emits failure-shaped noise, and so successful runs still produce the artifact bundle reviewers expect. Adds the Playwright JSON reporter so the parse step can pull pass/fail counts from `playwright-report/results.json` for the comment body. Co-Authored-By: Agent (authentik-i21996-internal-achievable-raisin) <279763771+playpen-agent@users.noreply.github.com> web/e2e: fix three regressions blocking the parallel suite Locally and in CI the new `e2e (playwright)` job appeared to "hang" under `fullyParallel: true` + `workers: "50%"`. The hang was actually five tests sharing two unrelated bugs that all manifest as 30s test timeouts; the cluster only *looks* like a parallelism issue because multiple workers stall on the same wall-clock window. With these three fixes the full suite is green in 1m48s on `--workers=2` (was: 5 failed / 17 passed in 5m30s). 1. `web/test/browser/600-providers.test.ts` PR #21647 dropped the `to:` argument on the `session.login()` call in this file's `beforeEach`. Without it, `SessionFixture.login()` waits for the auth-flow URL pattern to re-appear — which it does immediately, since we just navigated there — so the helper returns *before* the post-login redirect lands. The wizard buttons probed afterward live on `/if/admin/#/core/providers`, which the user never actually reaches; every test in the file then hits the 30s `beforeEach` timeout. Pin the destination explicitly, matching the shape of every other test file. 2. `web/src/admin/roles/ak-role-list.ts` The role-list row anchor had no aria-label, so its accessible name was the (random, generated) role name. `500-roles.test.ts` searches for that anchor with `getByRole("link", { name: "view details" })` — the same selector `400-groups.test.ts` uses against the group list, where `GroupListPage.row()` *does* set `aria-label="View details of group ..."`. Bring the role row to parity with groups; the test wasn't wrong, the UI was missing the accessibility hook. 3. `web/test/browser/500-roles.test.ts` ("Edit role from view page") The post-edit verification used `page.getByText(updatedName)`, but on the role view page the new name renders in two places (the "Role <name>" page-navbar heading and the description-list value), so the bare text match resolves to two elements and trips strict-mode. Add `{ exact: true }` so we assert the canonical value the edit wrote rather than the heading template. Co-Authored-By: Agent (authentik-i21996-internal-achievable-raisin) <279763771+playpen-agent@users.noreply.github.com> Use headless.

dominic-r · 2026-05-05T18:40:17Z

+      - name: Publish package
+        working-directory: gen-ts-api/
+        run: |
+          npm i


would there be a reason why we would need corepack'd npm in this file?

i'm thinking publish wise if anything changes and really making sure all our stuff uses the same version, no matter what it is

dominic-r · 2026-05-05T18:42:13Z

+      - name: Build authentik worker (Rust)
+        run: | # shell
+          cargo build --release --bin authentik
+          sudo install -m 0755 ./target/release/authentik /usr/local/bin/authentik


Suggested change

sudo install -m 0755 ./target/release/authentik /usr/local/bin/authentik

sudo install -m 0755 ./target/release/authentik /usr/local/bin/authentik-worker

dominic-r · 2026-05-05T18:44:22Z

+        working-directory: web
+        run: | # shell
+          if [ "${{ steps.playwright-cache.outputs.cache-hit }}" = "true" ]; then
+            corepack npm exec -- playwright install-deps chromium


does this use the project's playwright version or the one of npm? Also, I don't see an install step, but it might be handled by the setup action? If that's the case, it would use the project version, right?

dominic-r · 2026-05-05T18:45:13Z

+          if [ "${{ steps.playwright-cache.outputs.cache-hit }}" = "true" ]; then
+            corepack npm exec -- playwright install-deps chromium
+          else
+            corepack npm exec -- playwright install --with-deps chromium


do we want to dare to test firefox based browsers as well? as we have encountered a few strange/unique bugs on there as well

dominic-r · 2026-05-05T18:47:48Z

+            done'
+          timeout 300 bash -c '
+            until curl -fsS http://localhost:9000/if/flow/default-authentication-flow/ \
+              | grep -q "ak-flow-executor"; do


this feels a bit hacky? also, usually, would this be present even if no js is loaded client side via the bundles? mabye just check for the inline presence of a brand config since that's a pretty permanent item?

dominic-r · 2026-05-05T18:49:35Z

+          aws s3 cp \
+            --recursive \
+            --acl=public-read \
+            --cache-control "public, max-age=600" \


how likely are the reports to change once imported? i'd go for a longer cache control

dominic-r · 2026-05-05T18:50:42Z

+        if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && vars.PLAYWRIGHT_S3_ENABLED == 'true' }}
+        env:
+          S3_BUCKET: authentik-playwright-artifacts
+          S3_KEY_PREFIX: pr-${{ github.event.pull_request.number }}/run-${{ github.run_id }}/attempt-${{ github.run_attempt }}


ci-web also runs on main and version branches, wouldn't there be an issue with the key prefix to contain the pr number

GirlBossRush requested review from a team as code owners April 30, 2026 22:02

GirlBossRush changed the base branch from npm-corepack to main April 30, 2026 22:05

GirlBossRush requested a review from a team as a code owner April 30, 2026 22:05

GirlBossRush force-pushed the issue-21994-on-corepack branch from 68d394b to e5f0ff8 Compare April 30, 2026 22:09

GirlBossRush force-pushed the issue-21994-on-corepack branch 3 times, most recently from 2bb7e35 to 6fee629 Compare May 1, 2026 02:05

GirlBossRush force-pushed the issue-21994-on-corepack branch from 90f27f9 to a7337a3 Compare May 2, 2026 13:08

dominic-r reviewed May 2, 2026

View reviewed changes

rissson changed the title ~~ci/web: run Playwright e2e suite on every PR~~ ci: run Playwright e2e suite on every PR May 4, 2026

GirlBossRush force-pushed the issue-21994-on-corepack branch from 38f04f6 to 13d167e Compare May 4, 2026 23:18

GirlBossRush added 11 commits May 5, 2026 18:34

Fix mounted references.

67fa7bc

Flesh out node scripts.

da57283

Bump engines.

2a0d867

Prep containers.

5f614e1

Update makefile.

d537aee

Flesh out github actions.

8bf756e

Clean up docs container.

3e524a6

lint.

479d478

Bump.

afd1b7f

Lint.

d2e0d62

Bump NPM version.

f0a5444

GirlBossRush force-pushed the issue-21994-on-corepack branch from 13d167e to e4ef898 Compare May 5, 2026 16:41

dominic-r reviewed May 5, 2026

View reviewed changes

	sudo install -m 0755 ./target/release/authentik /usr/local/bin/authentik
	sudo install -m 0755 ./target/release/authentik /usr/local/bin/authentik-worker

Uh oh!

Conversation

GirlBossRush commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Depends on

Summary

Stacking

Notes for reviewer

Test plan

Uh oh!

netlify Bot commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for authentik-integrations ready!

Uh oh!

netlify Bot commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for authentik-docs ready!

Uh oh!

codecov Bot commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

netlify Bot commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for authentik-storybook ready!

Uh oh!

github-actions Bot commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Playwright e2e — ✅ Passed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

GirlBossRush commented Apr 30, 2026 •

edited

Loading

netlify Bot commented Apr 30, 2026 •

edited

Loading

netlify Bot commented Apr 30, 2026 •

edited

Loading

codecov Bot commented Apr 30, 2026 •

edited

Loading

netlify Bot commented Apr 30, 2026 •

edited

Loading

github-actions Bot commented Apr 30, 2026 •

edited

Loading

github-actions Bot commented May 2, 2026 •

edited

Loading