ci(test-suite): add e2e test orchestrator for merge queue #1076
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| echo "🔧 Parsing component versions..." | ||
| # Parse coprocessor array (7 elements) | ||
| coprocessor_input='${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.coprocessor || '[]' }}' |
Check warning
Code scanning / zizmor
code injection via template expansion Warning test
| echo "🔧 Parsing component versions..." | ||
| # Parse coprocessor array (7 elements) | ||
| coprocessor_input='${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.coprocessor || '[]' }}' |
Check failure
Code scanning / zizmor
code injection via template expansion Error test
| fi | ||
| # Parse gateway string | ||
| gateway_input='${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.gateway || '' }}' |
Check warning
Code scanning / zizmor
code injection via template expansion Warning test
| fi | ||
| # Parse gateway string | ||
| gateway_input='${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.gateway || '' }}' |
Check failure
Code scanning / zizmor
code injection via template expansion Error test
| [[ -n "${gateway_input}" ]] && echo "GATEWAY_VERSION_OVERRIDE=${gateway_input}" >> "$GITHUB_ENV" | ||
| # Parse host string | ||
| host_input='${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.host || '' }}' |
Check warning
Code scanning / zizmor
code injection via template expansion Warning test
| fi | ||
| # Parse test-suite string | ||
| test_suite_input='${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.test-suite || '' }}' |
Check failure
Code scanning / zizmor
code injection via template expansion Error test
| [[ -n "${test_suite_input}" ]] && echo "TEST_SUITE_VERSION_OVERRIDE=${test_suite_input}" >> "$GITHUB_ENV" | ||
| # Parse relayer string | ||
| relayer_input='${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.relayer || '' }}' |
Check warning
Code scanning / zizmor
code injection via template expansion Warning test
| [[ -n "${test_suite_input}" ]] && echo "TEST_SUITE_VERSION_OVERRIDE=${test_suite_input}" >> "$GITHUB_ENV" | ||
| # Parse relayer string | ||
| relayer_input='${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.relayer || '' }}' |
Check failure
Code scanning / zizmor
code injection via template expansion Error test
| [[ -n "${relayer_input}" ]] && echo "RELAYER_VERSION_OVERRIDE=${relayer_input}" >> "$GITHUB_ENV" | ||
| # Parse kms-core string | ||
| kms_core_input='${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.kms-core || '' }}' |
Check warning
Code scanning / zizmor
code injection via template expansion Warning test
| [[ -n "${relayer_input}" ]] && echo "RELAYER_VERSION_OVERRIDE=${relayer_input}" >> "$GITHUB_ENV" | ||
| # Parse kms-core string | ||
| kms_core_input='${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.kms-core || '' }}' |
Check failure
Code scanning / zizmor
code injection via template expansion Error test
| - name: Check if image was built | ||
| id: check-build | ||
| run: | | ||
| # Check if docker-e2e-image job ran and succeeded |
Check warning
Code scanning / zizmor
code injection via template expansion Warning test
| - name: Check if image was built | ||
| id: check-build | ||
| run: | | ||
| # Check if docker-e2e-image job ran and succeeded |
Check notice
Code scanning / zizmor
code injection via template expansion Note test
| run: | | ||
| # Check if docker-e2e-image job ran and succeeded | ||
| if [[ "${{ needs.docker-e2e-image.result }}" == "success" ]]; then | ||
| echo "image-built=true" >> "$GITHUB_OUTPUT" |
Check warning
Code scanning / zizmor
code injection via template expansion Warning test
| echo "✅ Image was built successfully" | ||
| else | ||
| echo "image-built=false" >> "$GITHUB_OUTPUT" | ||
| echo "image-tag=" >> "$GITHUB_OUTPUT" |
Check warning
Code scanning / zizmor
code injection via template expansion Warning test
| echo "✅ Image was built successfully" | ||
| else | ||
| echo "image-built=false" >> "$GITHUB_OUTPUT" | ||
| echo "image-tag=" >> "$GITHUB_OUTPUT" |
Check notice
Code scanning / zizmor
code injection via template expansion Note test
tawadaa
force-pushed
the
mq/ci/orchestrate-e2e
branch
from
e18de8f to
8c0a30e
Compare
There was a problem hiding this comment.
zizmor found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
tawadaa
force-pushed
the
mq/ci/orchestrate-e2e
branch
from
8c0a30e to
4340f77
Compare
| fhevm-e2e-test: | ||
| name: test-suite-e2e-tests-mq/fhevm-e2e-test (bpr) | ||
| needs: check-changes | ||
| if: ${{ needs.check-changes.outputs.changes-fhevm == 'true' || github.event_name == 'release' || github.event_name == 'workflow_dispatch' || startsWith(github.head_ref, 'mergify/merge-queue/') }} |
There was a problem hiding this comment.
Are we sure we want to run this workflow on github.event_name == 'release'?
Should we keep both test-suite-e2e-tests.yml and test-suite-e2e-tests-mq.yml?
There was a problem hiding this comment.
Also, I think we should delete test-suite-e2e-tests-with-build.yml right?
There was a problem hiding this comment.
If we keep both test-suite-e2e-tests.yml and test-suite-e2e-tests-mq.yml, what is the purpose of test-suite-orchestrate-e2e-tests.yml?
Couldn't we just use the trigger-builds job directly as the first job of the test-suite-e2e-tests-mq.yml?
There was a problem hiding this comment.
we don't need github.event_name == 'release', it's an old test, let me clean it
Yes, for now, we keep both of them, in order not to disturb the current e2e tests pipeline, until validating the merge queue pipeline
There was a problem hiding this comment.
If we keep both test-suite-e2e-tests.yml and test-suite-e2e-tests-mq.yml, what is the purpose of test-suite-orchestrate-e2e-tests.yml? Couldn't we just use the trigger-builds job directly as the first job of the test-suite-e2e-tests-mq.yml?
There are mainly 2 approches, either separate simple workflows or 1 complex workflow, I prefer the first approcah as it's easier to maintain.
In our case, it might appear that test-suite-e2e-tests-mq.yml and test-suite-orchestrate-e2e-tests.yml can be easily merged, but I prefer to keep them separate for few reasons:
- They have different roles, one is for orchestration and the other is for execution
- We might need to launch manually test-suite-e2e-tests-mq.yml for some reason e.g hotfix
There was a problem hiding this comment.
Fine for me, thx for the details 🙂
| output-build-status: | ||
| name: coprocessor-gw-listener-docker-build/output-build-status | ||
| needs: [check-changes, build] | ||
| if: always() | ||
| permissions: | ||
| contents: 'read' | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| image-built: ${{ steps.check-build.outputs.image-built }} | ||
| image-tag: ${{ steps.check-build.outputs.image-tag }} | ||
| steps: | ||
| - name: Check if image was built | ||
| id: check-build | ||
| run: | | ||
| # Check if docker build job ran and succeeded | ||
| if [[ "${{ needs.build.result }}" == "success" ]]; then |
There was a problem hiding this comment.
Are we sure all these output-build-status are userful?
AFAIK, the test-suite-orchestrate have access to the ${{ github.sha }} and thus the image tag already. Plus, I think it can check by itself whether the build.result step was equal to success or not right?
No description provided.