NONEVM-3123: Add staging contract compatibility testing #403
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| name: 'CCIP E2E Setup' | ||
| description: 'Common setup for CCIP E2E tests - caches, dependencies, and environment' | ||
|
|
||
| inputs: | ||
| contract_version: | ||
| description: 'Contract version to test against (local or commit SHA)' | ||
| required: true | ||
| default: 'local' | ||
| build_contracts: | ||
| description: 'Whether to build contracts locally' | ||
| required: false | ||
| default: 'true' | ||
|
|
||
| runs: | ||
| using: 'composite' | ||
| steps: | ||
| - name: Free Disk Space | ||
| uses: smartcontractkit/.github/actions/free-disk-space@free-disk-space/v1 | ||
|
|
||
| - name: Relocate Nix store to /mnt | ||
| shell: bash | ||
| run: | | ||
| sudo mkdir -p /mnt/nix | ||
| sudo mkdir -p /nix | ||
| sudo mount --bind /mnt/nix /nix | ||
| df -h /nix /mnt | ||
|
|
||
| - name: Restore cached docker images | ||
| uses: actions/cache@v4 | ||
| id: docker-cache-restore | ||
| with: | ||
| path: ${{ github.workspace }}/.cache/ccip-e2e-docker-images.tar | ||
| key: ccip-e2e-images-v2 | ||
|
|
||
| - name: Load docker images | ||
| if: steps.docker-cache-restore.outputs.cache-hit == 'true' | ||
| shell: bash | ||
| run: | | ||
| echo "Cache hit. Loading images from tarball..." | ||
| docker load -i ${{ github.workspace }}/.cache/ccip-e2e-docker-images.tar | ||
|
|
||
| - name: Install Nix | ||
| uses: cachix/install-nix-action@02a151ada4993995686f9ed4f1be7cfbb229e56f # v31 | ||
| with: | ||
| nix_path: nixpkgs=channel:nixos-unstable | ||
|
|
||
| - name: Cache Go modules | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: | | ||
| /home/runner/.cache/go-build | ||
| /home/runner/go/pkg/mod | ||
| key: go-${{ hashFiles('**/go.sum', 'chainlink/**/go.sum') }} | ||
|
|
||
| - name: Read Chainlink Core Ref | ||
| id: read_core_ref | ||
| shell: bash | ||
| run: | | ||
| CORE_REF=$(cat ./scripts/.core_version | tr -d '[:space:]') | ||
| # Validate it's a 40-char hex SHA (not a branch name) | ||
| if [[ ! "$CORE_REF" =~ ^[0-9a-f]{40}$ ]]; then | ||
| echo "ERROR: .core_version must be a full commit SHA, got: $CORE_REF" | ||
| exit 1 | ||
| fi | ||
| echo "CORE_REF=$CORE_REF" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Cache Chainlink Core repo | ||
| uses: actions/cache@v4 | ||
| id: core-cache | ||
| with: | ||
| path: chainlink | ||
| key: core-${{ steps.read_core_ref.outputs.CORE_REF }} | ||
|
|
||
| - name: Checkout Chainlink Core repo | ||
| if: steps.core-cache.outputs.cache-hit != 'true' | ||
| uses: actions/checkout@v5 | ||
| with: | ||
| repository: smartcontractkit/chainlink | ||
| ref: ${{ steps.read_core_ref.outputs.CORE_REF }} | ||
| path: chainlink | ||
| persist-credentials: false | ||
|
|
||
| - name: Build contracts | ||
| if: inputs.build_contracts == 'true' | ||
| shell: bash | ||
| run: | | ||
| cd contracts | ||
| nix develop .#contracts -c yarn && yarn build | ||
|
|
||
| - name: Set contract version environment | ||
| shell: bash | ||
| run: echo "CCIP_CONTRACTS_TON_VERSION=${{ inputs.contract_version }}" >> $GITHUB_ENV |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| # CCIP TON E2E Test Definitions | ||
| # This file is the single source of truth for CCIP TON integration tests. | ||
| # Both ccip-integration-test.yml and ccip-staging-compat.yml workflows read from this file. | ||
| # | ||
| # To add a new test: | ||
| # 1. Add an entry to the 'tests' list below | ||
| # 2. The test will automatically run in both local and staging compatibility checks | ||
|
|
||
| tests: | ||
| - id: ccip_messaging_ton2evm | ||
| name: "TON2EVM Messaging Test" | ||
| cmd: "cd integration-tests && go test ./smoke/ccip -run Test_CCIPMessaging_TON2EVM -timeout 20m -test.parallel=1 -count=1 -json" | ||
|
|
||
| - id: ccip_messaging_evm2ton | ||
| name: "EVM2TON Messaging Test" | ||
| cmd: "cd integration-tests && go test ./smoke/ccip -run Test_CCIPMessaging_EVM2TON -timeout 20m -test.parallel=1 -count=1 -json" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| name: "TON - CCIP Staging Compatibility" | ||
|
|
||
| # This workflow tests plugin compatibility with staging-deployed contracts. | ||
| # It serves as an early warning system - failures indicate that merging this PR | ||
| # will break staging until contracts are also updated. | ||
| # | ||
| # This workflow is INFORMATIONAL and should NOT be required for merge. | ||
| # Configure branch protection to only require "TON - CCIP Integration Tests". | ||
|
|
||
| on: | ||
| pull_request: | ||
| push: | ||
| branches: | ||
| - "main" | ||
|
|
||
| permissions: | ||
| contents: read | ||
| actions: read | ||
|
|
||
| env: | ||
| DOCKER_CACHE_KEY: ccip-e2e-images-v2 | ||
| DOCKER_CACHE_DIR: ${{ github.workspace }}/.cache | ||
| DOCKER_CACHE_TAR_NAME: ccip-e2e-docker-images.tar | ||
|
|
||
| jobs: | ||
| load-tests: | ||
| name: Load Test Matrix | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| matrix: ${{ steps.set-matrix.outputs.matrix }} | ||
| staging_version: ${{ steps.read-staging.outputs.version }} | ||
| steps: | ||
| - name: Checkout Repository | ||
| uses: actions/checkout@v5 | ||
|
|
||
| - name: Load test definitions | ||
| id: set-matrix | ||
| run: | | ||
| MATRIX=$(yq -o=json -I=0 '.tests' .github/ccip-ton-tests.yml) | ||
|
Collaborator
There was a problem hiding this comment. Think it would be great if immediately extend this to compat test matrix per env - it's gonna useful to run this against prod-testnet, and we can imagine we having more than one staging (sandbox?) envs. We might have prod-testnet live, and need to patch off-chain, this will be useful to run.
Collaborator
Author
There was a problem hiding this comment.
yeah that would be great, I'll see how we can extend it targeting different environment(importing each state file seems promising) |
||
| echo "matrix=$MATRIX" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Read staging contract version | ||
| id: read-staging | ||
| run: | | ||
| VERSION=$(cat ./scripts/.staging_contract_version | tr -d '[:space:]') | ||
| echo "version=$VERSION" >> $GITHUB_OUTPUT | ||
| echo "Staging contract version: $VERSION" | ||
|
|
||
| staging-compat-test: | ||
| name: "${{ matrix.test.name }} (Staging)" | ||
| needs: [load-tests] | ||
| runs-on: ubuntu-latest-4cores-16GB | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| test: ${{ fromJson(needs.load-tests.outputs.matrix) }} | ||
| steps: | ||
| - name: Checkout Repository | ||
| uses: actions/checkout@v5 | ||
|
|
||
| - name: Setup E2E Environment | ||
| uses: ./.github/actions/ccip-e2e-setup | ||
| with: | ||
| contract_version: ${{ needs.load-tests.outputs.staging_version }} | ||
jadepark-dev marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| build_contracts: 'false' | ||
|
|
||
| - name: Run Tests | ||
| run: | | ||
| echo "Testing staging compatibility with contract version: $CCIP_CONTRACTS_TON_VERSION" | ||
| nix develop .#ccip-e2e -c scripts/e2e/setup-env.sh --core-dir "${GITHUB_WORKSPACE}/chainlink" | ||
| nix develop .#ccip-e2e -c scripts/e2e/run-test.sh --core-dir "${GITHUB_WORKSPACE}/chainlink" --test-command "${{ matrix.test.cmd }}" --clean-logs | ||
|
|
||
| - name: Upload test logs on success | ||
| if: success() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: ccip-staging-compat-logs-${{ matrix.test.id }} | ||
| path: chainlink/integration-tests/smoke/ccip/logs/ | ||
| retention-days: 3 | ||
|
|
||
| - name: Upload test logs on failure | ||
| if: failure() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: ccip-staging-compat-logs-${{ matrix.test.id }} | ||
| path: chainlink/integration-tests/smoke/ccip/logs/ | ||
| retention-days: 7 | ||
|
|
||
| staging-compat-status: | ||
| name: Staging Compatibility Status | ||
| if: always() | ||
| runs-on: ubuntu-latest | ||
| needs: [staging-compat-test] | ||
| steps: | ||
| - name: Report staging compatibility | ||
| run: | | ||
| if [ "${{ needs.staging-compat-test.result }}" == "failure" ]; then | ||
| echo "::warning::Staging compatibility tests failed. After merging, staging contracts must be updated." | ||
| exit 1 | ||
| fi | ||
| echo "Staging compatibility tests passed." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| 43d7a93089fe | ||
jadepark-dev marked this conversation as resolved.
Show resolved
Hide resolved
Collaborator
There was a problem hiding this comment. Let's load this from the CLD state file which should be the source of truth here, right? vs. hardcoding a magic sha
Collaborator
Author
There was a problem hiding this comment. 100% agree, was looking for a shortcut 😅 Exploring state update here |
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is the single source truth for our tests/apps in .github CI context?
I would prefer this source of truth to be in the core project, some task manager (nix, makefile, just) to define and run common operations, so everybody can easily trigger, and then the CI uses that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm I thought this list is mostly for CI. On local development, devs usually run tests they're interested in(mostly single test). But I can see the point where CI would only pick up predefined test organization, not define by itself