-
Notifications
You must be signed in to change notification settings - Fork 24
feat: Add just task runner to ensure reproducibility of CI #1345
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
2e6a32a
Ensure golang input is up-to-date for nix flake
maru-ava 7951296
fixup: Update to use the non-deprecated flake update syntax
maru-ava 0dc23b7
feat: Add just task runner to ensure reproducibility of CI
maru-ava f544432
fixup: Exclude justfile and run-just.sh for licence header check
maru-ava 0ce83e7
fixup: Add output to differentiate test recipes
maru-ava e3493bc
fixup: Use flake path as intended
maru-ava 262b823
fixup: Update to use current flake update syntax
maru-ava efc90b6
fixup: Add separate command for updating the nix flake
maru-ava 734c3c7
fixup: Move run script and fix update flake command name
maru-ava 86ab4ed
fixup: Add check for golang version consistency
maru-ava b93f0da
fixup: Cleanup justfile comments
maru-ava 846c5cd
Merge branch 'main' into maru/just-ffi
rkuris 59146e8
Merge branch 'main' into maru/just-ffi
maru-ava 2384349
Merge branch 'main' into maru/just-ffi
maru-ava File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -53,6 +53,8 @@ | |
| "cliff.toml", | ||
| "clippy.toml", | ||
| "**/tests/compile_*/**", | ||
| "justfile", | ||
| "scripts/run-just.sh", | ||
| ], | ||
| } | ||
| ] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| # List available recipes | ||
| default: | ||
| ./scripts/run-just.sh --list | ||
|
|
||
| # Build ffi with nix | ||
| build-ffi-nix: check-nix | ||
| cd ffi && nix build | ||
|
|
||
| # Check if the git branch is clean | ||
| check-clean-branch: | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| git add --all | ||
| git update-index --really-refresh >> /dev/null | ||
|
|
||
| # Show the status of the working tree. | ||
| git status --short | ||
|
|
||
| # Exits if any uncommitted changes are found. | ||
| git diff-index --quiet HEAD | ||
|
|
||
| # Check if the FFI flake is up-to-date (requires clean git tree) | ||
| check-ffi-flake: check-nix | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
| ./scripts/run-just.sh update-ffi-flake | ||
| ./scripts/run-just.sh check-clean-branch | ||
|
|
||
| # Check if the golang version is set consistently (requires clean git tree) | ||
| check-golang-version: check-nix | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| # Exit only at the end if any of the checks set FAILED=1 | ||
| FAILED= | ||
|
|
||
| cd ffi | ||
|
|
||
| TOOLCHAIN_VERSION=$(nix develop --command bash -c "go mod edit -json | jq -r '.Toolchain'") | ||
| echo "toolchain version in ffi/go.mod is ${TOOLCHAIN_VERSION}" | ||
|
|
||
| ETH_TESTS_VERSION=$(nix develop --command bash -c "cd tests/eth && go mod edit -json | jq -r '.Toolchain'") | ||
| echo "toolchain version in ffi/tests/eth/go.mod is ${ETH_TESTS_VERSION}" | ||
|
|
||
| if [[ "${TOOLCHAIN_VERSION}" != "${ETH_TESTS_VERSION}" ]]; then | ||
| echo "❌ toolchain version in ffi/tests/eth/go.mod should be ${TOOLCHAIN_VERSION}" | ||
| FAILED=1 | ||
| fi | ||
|
|
||
| FIREWOOD_TESTS_VERSION=$(nix develop --command bash -c "cd tests/firewood && go mod edit -json | jq -r '.Toolchain'") | ||
| echo "toolchain version in ffi/tests/firewood/go.mod is ${FIREWOOD_TESTS_VERSION}" | ||
|
|
||
| if [[ "${TOOLCHAIN_VERSION}" != "${FIREWOOD_TESTS_VERSION}" ]]; then | ||
| echo "❌ toolchain version in ffi/tests/firewood/go.mod should be ${TOOLCHAIN_VERSION}" | ||
| FAILED=1 | ||
| fi | ||
|
|
||
| NIX_VERSION=$(nix run .#go -- version | awk '{print $3}') | ||
| echo "golang provided by ffi/flake.nix is ${NIX_VERSION}" | ||
|
|
||
| if [[ "${TOOLCHAIN_VERSION}" != "${NIX_VERSION}" ]]; then | ||
| echo "❌ golang provided by ffi/flake/nix should be ${TOOLCHAIN_VERSION}" | ||
| echo "It will be necessary to update the golang.url in ffi/flake.nix to point to a SHA of"\ | ||
| "AvalancheGo whose nix/go/flake.nix provides ${TOOLCHAIN_VERSION}." | ||
| fi | ||
|
|
||
| if [[ -n "${FAILED}" ]]; then | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Check if nix is installed | ||
| check-nix: | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
| if ! command -v nix &> /dev/null; then | ||
| echo "Error: 'nix' is not installed." >&2 | ||
| echo "" >&2 | ||
| echo "To install nix:" >&2 | ||
| echo " - Visit: https://github.com/DeterminateSystems/nix-installer" >&2 | ||
| echo " - Or run: curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Run all checks of ffi built with nix | ||
| test-ffi-nix: test-ffi-nix-build-equivalency test-ffi-nix-go-bindings | ||
|
|
||
| # Test ffi build equivalency between nix and cargo | ||
| test-ffi-nix-build-equivalency: check-nix | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| echo "Testing ffi build equivalency between nix and cargo" | ||
|
|
||
| bash -x ./ffi/test-build-equivalency.sh | ||
|
|
||
| # Test golang ffi bindings using the nix-built artifacts | ||
| test-ffi-nix-go-bindings: build-ffi-nix | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| echo "running ffi tests against bindings built by nix..." | ||
|
|
||
| cd ffi | ||
|
|
||
| # Need to capture the flake path before changing directories to | ||
| # result/ffi because `result` is a nix store symlink so ../../ | ||
| # won't resolve to the ffi path containing the flake. | ||
| FLAKE_PATH="$PWD" | ||
|
|
||
| # This runs golang outside a nix shell to validate viability | ||
| # without the env setup performed by a nix shell | ||
| GO="nix run $FLAKE_PATH#go" | ||
|
|
||
| cd result/ffi | ||
|
|
||
| # - cgocheck2 is expensive but provides complete pointer checks | ||
| # - use hash mode ethhash since the flake builds with `--features ethhash,logger` | ||
| GOEXPERIMENT=cgocheck2 TEST_FIREWOOD_HASH_MODE=ethhash ${GO} test ./... | ||
|
|
||
| # Ensure the FFI flake is up-to-date | ||
| update-ffi-flake: check-nix | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
| cd ffi | ||
|
|
||
| echo "ensuring flake lock file is current for golang" | ||
| nix flake update golang | ||
|
|
||
| echo "checking for a consistent golang verion" | ||
| ../scripts/run-just.sh check-golang-version |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| if command -v just &> /dev/null; then | ||
| exec just "$@" | ||
| elif command -v nix &> /dev/null; then | ||
| exec nix run nixpkgs#just -- "$@" | ||
| else | ||
| echo "Error: Neither 'just' nor 'nix' is installed." >&2 | ||
| echo "" >&2 | ||
| echo "Please install one of the following:" >&2 | ||
| echo "" >&2 | ||
| echo "Option 1 - Install just:" >&2 | ||
| echo " - Visit: https://github.com/casey/just#installation" >&2 | ||
| echo " - Or use cargo: cargo install just" >&2 | ||
| echo "" >&2 | ||
| echo "Option 2 - Install nix:" >&2 | ||
| echo " - Visit: https://github.com/DeterminateSystems/nix-installer" >&2 | ||
| echo " - Or run: curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install" >&2 | ||
| exit 1 | ||
| fi |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Used by the
check-golang-versionrecipe.