Skip to content

Conversation

daniel-cit
Copy link
Contributor

This PR adds a retry when a build in cloud build fails with an error that can be tried again.
It uses the same configuration for the retry used for deploying with terraform locally

@daniel-cit daniel-cit marked this pull request as ready for review September 26, 2025 20:10
@daniel-cit
Copy link
Contributor Author

@apeabody Could you PTAL?

@daniel-cit
Copy link
Contributor Author

retry in action

waiting for build facd1734-6410-427b-9244-ba8afee7f2b5 execution.
build status is QUEUED
build status is QUEUED
build status is QUEUED
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
final build status is FAILURE
error 'Error setting IAM policy' is worth of a retry
build failed with retryable error. a new build will be triggered.
triggered new build with ID: f22d917b-ab0c-4722-878b-9606391c645a (attempt 1/2)
waiting for build f22d917b-ab0c-4722-878b-9606391c645a execution.
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
build status is WORKING
final build status is SUCCESS
# completing step 'gcp-org.shared' execution

Copy link
Contributor

@apeabody apeabody left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @daniel-cit!

@apeabody
Copy link
Contributor

/gemini review

1 similar comment
@apeabody
Copy link
Contributor

/gemini review

Copy link

Important

Installation incomplete: to start using Gemini Code Assist, please ask the organization owner(s) to visit the Gemini Code Assist Admin Console and sign the Terms of Services.

@apeabody
Copy link
Contributor

/gemini review

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a retry mechanism for Cloud Build builds that fail with transient errors. The changes are mostly in helpers/foundation-deployer/gcp/gcp.go, with corresponding updates to tests and call sites. The overall approach is sound, but I've identified a critical issue that could cause a panic, along with a few other suggestions to improve robustness and code quality. Specifically, there's a potential panic when fetching the last build status if no builds are found. I've also suggested improvements to error handling in the init function, handling of cases where no build is found in the retry loop, and a recommendation to avoid using a package-level context for API calls.

@daniel-cit
Copy link
Contributor Author

/gemini review

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This PR introduces a retry mechanism for failed Cloud Build builds, which is a great addition for resiliency. The implementation is solid, with good use of the Go SDK and comprehensive tests. I've added a few suggestions to improve code style consistency, error message clarity, and code organization. These are minor changes to further improve the maintainability of the code.

@apeabody
Copy link
Contributor

Thanks @daniel-cit!

@apeabody apeabody merged commit 2fcd799 into terraform-google-modules:main Sep 29, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants