CLOUDP-315271: Onboard Kundukto to CI #3862
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,6 +5,7 @@ | |
| *.so | ||
| *.dylib | ||
| bin/** | ||
| compliance/** | ||
| dist/** | ||
| # mac notarization service | ||
| linux_amd64/** | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -117,7 +117,7 @@ addcopy: ## Add missing license to files | |
| @scripts/add-copy.sh | ||
|
|
||
| .PHONY: generate | ||
| generate: gen-docs gen-mocks gen-api-commands gen-purls ## Generate docs, mocks, code, api commands, all auto generated assets | ||
|
|
||
| .PHONY: apply-overlay | ||
| apply-overlay: ## Apply overlay on openapi spec | ||
|
|
@@ -149,6 +149,14 @@ gen-docs: gen-docs-metadata ## Generate docs for atlascli commands | |
| @echo "==> Generating docs" | ||
| go run -ldflags "$(LINKER_FLAGS)" ./tools/cmd/docs | ||
|
|
||
| .PHONY: gen-purls | ||
| gen-purls: # Generate purls on linux os | ||
| @echo "==> Generating purls" | ||
| GOOS=linux GOARCH=amd64 go build -trimpath -mod=readonly -o bin/atlas-linux ./cmd/atlas | ||
|
There was a problem hiding this comment. there are some dependencies which are used only by some builds. For the moment, we will only generate purls for linux (in the interest of reducing this PRs scope) and will include generation for all builds, compiling them into one list which will be used for sbom generation in CLOUDP-316920 |
||
| go version -m ./bin/atlas-linux | \ | ||
| awk '$$1 == "dep" || $$1 == "=>" { print "pkg:golang/" $$2 "@" $$3 }' | \ | ||
| LC_ALL=C sort > build/package/purls.txt | ||
|
|
||
| .PHONY: build | ||
| build: ## Generate an atlas binary in ./bin | ||
| @echo "==> Building $(ATLAS_BINARY_NAME) binary" | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| # Copyright 2025 MongoDB Inc | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| set -Eeou pipefail | ||
|
|
||
| if ! git diff --quiet --exit-code build/package/purls.txt; then | ||
| echo "build/package/purls.txt is out of date. Please run 'make gen-purls' and commit the result." | ||
| git --no-pager diff build/package/purls.txt | ||
| exit 1 | ||
| fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -533,6 +533,21 @@ functions: | |
| binary: make | ||
| args: | ||
| - otel | ||
| "check purls": | ||
|
There was a problem hiding this comment. Example of check failing if purls.txt is not up-to-date: |
||
| - command: subprocess.exec | ||
| type: test | ||
| params: | ||
| <<: *go_options | ||
| binary: make | ||
| args: | ||
| - gen-purls | ||
| - command: subprocess.exec | ||
| params: | ||
| <<: *go_options | ||
| include_expansions_in_env: | ||
| - workdir | ||
| binary: build/ci/check-purls.sh | ||
|
|
||
| tasks: | ||
| - name: compile | ||
| tags: ["code_health"] | ||
|
|
@@ -1726,6 +1741,10 @@ tasks: | |
| vars: | ||
| span: "coverage" | ||
| attr: "total=${percentage},count=${count}" | ||
| - name: check_purls | ||
| tags: ["code_health"] | ||
| commands: | ||
| - func: "check purls" | ||
| - name: snyk_monitor | ||
| tags: | ||
| - snyk | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
There was a problem hiding this comment. general design comment, can we commit the purl file and make it a CI check (GH or EVG no preference) that is kept up to date, this is similar to how other repos manage this file There was a problem hiding this comment. check the library owners check which is kind of similar There was a problem hiding this comment. I would't go for this approach we would jump from 2 files storing dependency information (library_owners.json and go.mod) to 3 files (library_owners.json, go.mod and purls.txt). I would prefer for the tooling to translate dependencies into purls and later json on the fly as needed. I'm inclined to even remove library_owners.json, given only our team has code in atlasCLI since the kubernetes plugin extraction. There was a problem hiding this comment. I'm ok to remove lib owners (same reason) but I see value on purls being committed given shipped dependencies are not the same as the ones in go.mod and this raises awareness when adding new libs and the implications, this also comes with the comment that I'd like if purls are auto generated by the precommit hook, similar to mms There was a problem hiding this comment. I have no strong opinion here. Having the purl file committed could definitely help with making dependencies more visible, but I see @fmenezes's point about not wanting to manage yet another file. I can create a make command to generate purls from the binary and set up a GH action to check that it's up-to-date similar to how we do the docs check. @fmenezes, any strong feelings about this idea? There was a problem hiding this comment. sure that is fine, we can commit it but bear in mind the extra checks |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| # Copyright 2025 MongoDB Inc | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| set -Eeou pipefail | ||
|
|
||
| export WORKDIR=${workdir:?} | ||
|
|
||
| # Authenticate Docker to AWS ECR | ||
| aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 901841024863.dkr.ecr.us-east-1.amazonaws.com | ||
|
|
||
| echo "Generating SBOMs..." | ||
| docker run --rm \ | ||
| -v "$WORKDIR/src/github.com/mongodb/mongodb-atlas-cli:/pwd" \ | ||
| 901841024863.dkr.ecr.us-east-1.amazonaws.com/release-infrastructure/silkbomb:2.0 \ | ||
| update \ | ||
| --purls /pwd/build/package/purls.txt \ | ||
| --sbom-out /pwd/sbom.json | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| pkg:golang/cloud.google.com/go/auth/[email protected] | ||
| pkg:golang/cloud.google.com/go/[email protected] | ||
| pkg:golang/cloud.google.com/go/compute/[email protected] | ||
| pkg:golang/cloud.google.com/go/[email protected] | ||
| pkg:golang/cloud.google.com/go/[email protected] | ||
| pkg:golang/cloud.google.com/go/[email protected] | ||
| pkg:golang/github.com/AlecAivazis/survey/[email protected] | ||
| pkg:golang/github.com/Azure/azure-sdk-for-go/sdk/[email protected] | ||
| pkg:golang/github.com/Azure/azure-sdk-for-go/sdk/[email protected] | ||
| pkg:golang/github.com/Azure/azure-sdk-for-go/sdk/[email protected] | ||
| pkg:golang/github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/[email protected] | ||
| pkg:golang/github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/[email protected] | ||
| pkg:golang/github.com/AzureAD/[email protected] | ||
| pkg:golang/github.com/Masterminds/semver/[email protected] | ||
| pkg:golang/github.com/PaesslerAG/[email protected] | ||
| pkg:golang/github.com/PaesslerAG/[email protected] | ||
| pkg:golang/github.com/ProtonMail/[email protected] | ||
| pkg:golang/github.com/STARRY-S/[email protected] | ||
| pkg:golang/github.com/andybalholm/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/feature/ec2/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/internal/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/internal/endpoints/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/internal/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/service/internal/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/service/internal/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/service/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/service/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/service/[email protected] | ||
| pkg:golang/github.com/aws/aws-sdk-go-v2/service/[email protected] | ||
| pkg:golang/github.com/aws/[email protected] | ||
| pkg:golang/github.com/aws/[email protected] | ||
| pkg:golang/github.com/bodgit/[email protected] | ||
| pkg:golang/github.com/bodgit/[email protected] | ||
| pkg:golang/github.com/bodgit/[email protected] | ||
| pkg:golang/github.com/briandowns/[email protected] | ||
| pkg:golang/github.com/cloudflare/[email protected] | ||
| pkg:golang/github.com/denisbrodbeck/[email protected] | ||
| pkg:golang/github.com/dsnet/[email protected] | ||
| pkg:golang/github.com/fatih/[email protected] | ||
| pkg:golang/github.com/felixge/[email protected] | ||
| pkg:golang/github.com/fsnotify/[email protected] | ||
| pkg:golang/github.com/go-logr/[email protected] | ||
| pkg:golang/github.com/go-logr/[email protected] | ||
| pkg:golang/github.com/go-viper/mapstructure/[email protected] | ||
| pkg:golang/github.com/golang-jwt/jwt/[email protected] | ||
| pkg:golang/github.com/golang/[email protected] | ||
| pkg:golang/github.com/golang/[email protected] | ||
| pkg:golang/github.com/google/go-github/[email protected] | ||
| pkg:golang/github.com/google/[email protected] | ||
| pkg:golang/github.com/google/[email protected] | ||
| pkg:golang/github.com/google/[email protected] | ||
| pkg:golang/github.com/googleapis/[email protected] | ||
| pkg:golang/github.com/googleapis/gax-go/[email protected] | ||
| pkg:golang/github.com/hashicorp/[email protected] | ||
| pkg:golang/github.com/hashicorp/[email protected] | ||
| pkg:golang/github.com/hashicorp/golang-lru/[email protected] | ||
| pkg:golang/github.com/iancoleman/[email protected] | ||
| pkg:golang/github.com/kballard/[email protected] | ||
| pkg:golang/github.com/klauspost/[email protected] | ||
| pkg:golang/github.com/klauspost/[email protected] | ||
| pkg:golang/github.com/kylelemons/[email protected] | ||
| pkg:golang/github.com/mattn/[email protected] | ||
| pkg:golang/github.com/mattn/[email protected] | ||
| pkg:golang/github.com/mgutz/[email protected] | ||
| pkg:golang/github.com/mholt/[email protected] | ||
| pkg:golang/github.com/minio/[email protected] | ||
| pkg:golang/github.com/mongodb-forks/[email protected] | ||
| pkg:golang/github.com/montanaflynn/[email protected] | ||
| pkg:golang/github.com/nwaples/rardecode/[email protected] | ||
| pkg:golang/github.com/pelletier/go-toml/[email protected] | ||
| pkg:golang/github.com/pelletier/[email protected] | ||
| pkg:golang/github.com/pierrec/lz4/[email protected] | ||
| pkg:golang/github.com/pkg/[email protected] | ||
| pkg:golang/github.com/sagikazarmark/[email protected] | ||
| pkg:golang/github.com/shirou/gopsutil/[email protected] | ||
| pkg:golang/github.com/sorairolake/[email protected] | ||
| pkg:golang/github.com/sourcegraph/[email protected] | ||
| pkg:golang/github.com/spf13/[email protected] | ||
| pkg:golang/github.com/spf13/[email protected] | ||
| pkg:golang/github.com/spf13/[email protected] | ||
| pkg:golang/github.com/spf13/[email protected] | ||
| pkg:golang/github.com/spf13/[email protected] | ||
| pkg:golang/github.com/subosito/[email protected] | ||
| pkg:golang/github.com/tangzero/[email protected] | ||
| pkg:golang/github.com/therootcompany/[email protected] | ||
| pkg:golang/github.com/tklauser/[email protected] | ||
| pkg:golang/github.com/tklauser/[email protected] | ||
| pkg:golang/github.com/ulikunitz/[email protected] | ||
| pkg:golang/github.com/xdg-go/[email protected] | ||
| pkg:golang/github.com/xdg-go/[email protected] | ||
| pkg:golang/github.com/xdg-go/[email protected] | ||
| pkg:golang/github.com/youmark/[email protected] | ||
| pkg:golang/go.mongodb.org/atlas-sdk/[email protected] | ||
| pkg:golang/go.mongodb.org/atlas-sdk/[email protected] | ||
| pkg:golang/go.mongodb.org/[email protected] | ||
| pkg:golang/go.mongodb.org/[email protected] | ||
| pkg:golang/go.opentelemetry.io/auto/[email protected] | ||
| pkg:golang/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/[email protected] | ||
| pkg:golang/go.opentelemetry.io/contrib/instrumentation/net/http/[email protected] | ||
| pkg:golang/go.opentelemetry.io/otel/[email protected] | ||
| pkg:golang/go.opentelemetry.io/otel/[email protected] | ||
| pkg:golang/go.opentelemetry.io/[email protected] | ||
| pkg:golang/[email protected] | ||
| pkg:golang/golang.org/x/[email protected] | ||
| pkg:golang/golang.org/x/[email protected] | ||
| pkg:golang/golang.org/x/[email protected] | ||
| pkg:golang/golang.org/x/[email protected] | ||
| pkg:golang/golang.org/x/[email protected] | ||
| pkg:golang/golang.org/x/[email protected] | ||
| pkg:golang/golang.org/x/[email protected] | ||
| pkg:golang/golang.org/x/[email protected] | ||
| pkg:golang/golang.org/x/[email protected] | ||
| pkg:golang/golang.org/x/[email protected] | ||
| pkg:golang/google.golang.org/[email protected] | ||
| pkg:golang/google.golang.org/genproto/googleapis/[email protected] | ||
| pkg:golang/google.golang.org/genproto/googleapis/[email protected] | ||
| pkg:golang/google.golang.org/[email protected] | ||
| pkg:golang/google.golang.org/[email protected] | ||
| pkg:golang/google.golang.org/[email protected] | ||
| pkg:golang/gopkg.in/[email protected] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will hooking this to the pre commit be a separate task?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't intended to since, as a part of CI, we do a check to see if purls.txt is up-to-date. Devs would have to manually run this command. My intent here was to give devs visual on changes to sbom because of their work.
I'm happy to include this command in pre commit. To verify @gssbzn, this would make the CI check is redundant, correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No cause having the pre commit installed is opt-in, and you can also do
git commit --no-verifyso you still need to check the files is up to date on the CIAlso just inc ase, ideally this should only run if the go.mod is modified