feat: e2e tests using docker compose #1165
Conversation
✅ Deploy Preview for endearing-brigadeiros-63f9d0 canceled.
|
coopernetes
force-pushed
the
e2e-testing
branch
2 times, most recently
from
d351d4b to
b5ed561
Compare
coopernetes
force-pushed
the
e2e-testing
branch
2 times, most recently
from
7765858 to
f3f02db
Compare
Codecov Report❌ Patch coverage is ❌ Your patch check has failed because the patch coverage (76.27%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #1165 +/- ##
==========================================
- Coverage 82.69% 82.51% -0.18%
==========================================
Files 70 69 -1
Lines 3005 3043 +38
Branches 499 509 +10
==========================================
+ Hits 2485 2511 +26
- Misses 417 428 +11
- Partials 103 104 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
dcoric
left a comment
dcoric
left a comment
There was a problem hiding this comment.
Excellent work! This is a comprehensive and well-architected E2E testing implementation.
I really appreciate the complete testing infrastructure with Docker and CI/CD - this addresses a common pain point I've experienced where tests would fail due to incorrect Node.js versions on different machines. Having everything containerized with consistent environments is a huge improvement for reliability.
The only area I'd suggest improving is the async configuration loading in src/ui/services/*.js, which currently has race conditions. This could potentially create behavioral issues if API calls are made before the runtime configuration is fully loaded, leading to requests being sent to incorrect endpoints.
Overall, this is solid work that will significantly improve our testing capabilities! 🎉
coopernetes
force-pushed
the
e2e-testing
branch
from
ea924dc to
03c1460
Compare
- Remove verbose logging from Dockerfile - pin dependent actions in new e2e workflow to their respective commits - refactor the tests to work slightly more robustly (handle creds, etc)
- Removed the obsolete 'version: 3.7' field from docker-compose.yml - This fixes the warning about the version field being obsolete in newer Docker Compose versions - The e2e tests now run without warnings
- Fix OIDC configuration syntax errors - Update Cypress baseUrl to use correct port (8080) - Fix CI workflow to use correct port and remove & from start command - Ensure frontend is built before running tests - CSRF protection already properly disabled in test environment Cypress tests now pass: - autoApproved.cy.js: 1/1 passing - login.cy.js: 8/8 passing - repo.cy.js: failing due to rate limiting (separate issue) This resolves the main issues mentioned in the failing job: - CSRF Token Missing Errors: Fixed by proper test environment config - Shell Script Syntax Error: Fixed by removing & from start command - Unknown Authentication Strategy: Fixed OIDC syntax errors - Route Not Hit: Fixed by building frontend and using correct port
- Fix CSRF protection by setting NODE_ENV=test in CI - Fix OIDC strategy by checking enabled flag before configuration - Fix port configuration by using correct server port (8080) - Add start:ci script to run server only (not dev client) - Set CYPRESS_BASE_URL environment variable for consistency This should resolve: - CSRF token missing errors in CI - Unknown authentication strategy errors - Port mismatch issues (3000 vs 8080) - Shell script syntax errors with & character
coopernetes
force-pushed
the
e2e-testing
branch
from
03c1460 to
4e82a52
Compare
|
@dcoric Can I get your 👀 on the API base URL handling? This should fix a number of unrelated issues regarding production deployments and it's a bit more robust now. Had to add a bit more logic for handling CORS properly as well. Thanks! cc @jescalada @kriswest (sorry for the delays on this one) Informing the rest of the maintainer team. There are "production deployment" related changes in this PR around CORS and base URL handling on the frontend required to make this testing setup work so hopefully we can merge that smoothly and add it to 2.x. |
coopernetes
left a comment
coopernetes
left a comment
There was a problem hiding this comment.
Some changes of note that I would appreciate a set of 👀 on. Thanks!
| push: | ||
| branches: [main] | ||
| issue_comment: | ||
| types: [created] |
There was a problem hiding this comment.
@kriswest we can do this in a follow-up PR but it may be worth adding workflow_dispatch and allowing this test suite to be executed as part of a pre-release step in the future.
| push: | |
| branches: [main] | |
| issue_comment: | |
| types: [created] | |
| push: | |
| branches: [main] | |
| issue_comment: | |
| types: [created] | |
| workflow_dispatch: |
| runs-on: ubuntu-latest | ||
| # Run on push/PR or when a maintainer comments "/test e2e" or "/run e2e" | ||
| if: | | ||
| github.event_name != 'issue_comment' || ( | ||
| github.event.issue.pull_request && | ||
| (contains(github.event.comment.body, '/test e2e') || contains(github.event.comment.body, '/run e2e')) && | ||
| (github.event.comment.author_association == 'OWNER' || | ||
| github.event.comment.author_association == 'MEMBER' || | ||
| github.event.comment.author_association == 'COLLABORATOR') | ||
| ) | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4 | ||
| with: | ||
| # When triggered by comment, checkout the PR branch | ||
| ref: ${{ github.event_name == 'issue_comment' && format('refs/pull/{0}/head', github.event.issue.number) || github.ref }} |
There was a problem hiding this comment.
This step makes me a bit nervous. It's not a great security practice to run Actions on a forked PR but it would be valuable to run these tests on specific PRs that may be adding breaking changes to validate full end-to-end app functionality. I'm not sure if there's a better way than this to run this workflow and would like the team's thoughts.
| "rateLimit": { | ||
| "windowMs": 60000, | ||
| "limit": 150 | ||
| "limit": 1000 |
There was a problem hiding this comment.
This was added to address flakiness in the Cypress tests. It might have been just a problem with my local machine so happy to revert it if necessary.
| credentials: true, | ||
| origin: true, | ||
| /** | ||
| * CORS Configuration |
There was a problem hiding this comment.
This probably belongs in the proper documentation site. Also a TODO - add a section for production deployments which take these new changes into account and properly document it for deployers.
| // legacy git proxy paths took the form: https://<git proxy domain>:<port>/<repoPath> | ||
| // by assuming the host was github.com | ||
| url = 'https://github.com' + (pathBreakdown?.repoPath ?? 'NOT-FOUND'); | ||
| // First, try to find a matching repository by checking both http:// and https:// protocols |
There was a problem hiding this comment.
Just calling this out - are we OK with supporting non-TLS git repositories? This is required for testing for now but I can fanagle a setup that uses a self-signed TLS certificate for the local git server if necessary and preferred.
| #!/usr/bin/env python3 | ||
| """ | ||
| CGI wrapper for git-http-backend that captures raw HTTP request/response data. | ||
| This wrapper intercepts git operations and saves the binary data to files for testing. |
3 participants
This is in draft until the workflow is fully tested - currently underway in a private forkcompletedThis PR adds a new end-to-end test setup using docker compose. It includes a few basic tests such as git clone (fetch) and a single push test with an expected failure. By moving end-to-end testing to a container-based setup, we can start to remove the state coupling that exists between various test suites at the moment. For example, attempting to run a single test can result in unexpected failures because it relies on setups from another test that starts a real GitProxy server, inserts some data into the database, etc. By refactoring those tests into these new e2e style tests, we can remove those dependencies and start to break apart the existing tests to focus more on the system-under-test.
In addition to this setup, a few other changes were made to support it as well as to cleanup some miscellaneous issues. If preferred, these can be moved out into their own PRs..
See below compose logs: