Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: command flag to override parallel pool size #5422

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

itays333
Copy link

@itays333 itays333 commented Mar 17, 2025

what

add -P / --parallel-pool-size flag to allow overriding pool size during plan/apply commands

why

Can be helpful in many scenarios IMO, but my main reason is that in some cases running multiple projects in parallel, which access the same resource (e.g via google_project_iam_member) can cause apply failures or sometimes state inconsistency. It should probably be addressed in the terraform provider, but i saw many cases of this in multiple resources. It would be good to have the option to apply serially with -P 1 from time to time

tests

  • Added flag parsing test cases
  • Tested manually e2e

@itays333 itays333 requested review from a team as code owners March 17, 2025 21:47
@itays333 itays333 requested review from GenPage, lukemassa and X-Guardian and removed request for a team March 17, 2025 21:47
@github-actions github-actions bot added the go Pull requests that update Go code label Mar 17, 2025
@dosubot dosubot bot added the feature New functionality/enhancement label Mar 17, 2025
@itays333 itays333 force-pushed the itay_command_parallel_exec branch from a2b82ec to 916a834 Compare March 17, 2025 21:56
@itays333 itays333 force-pushed the itay_command_parallel_exec branch from 1fe0aeb to 781f38e Compare March 18, 2025 11:18
@lukemassa
Copy link
Contributor

It seems like this is more about toggling between running in parallel vs running in serial, does --parallel-plan work for this use case?

@itays333
Copy link
Author

It seems like this is more about toggling between running in parallel vs running in serial, does --parallel-plan work for this use case?

--parallel-plan is a server-level configuration that requires redeploying Atlantis to modify. This PR provides more flexibility by:

  • Enabling dynamic adjustment of the pool size for specific PR runs when serial execution is needed, without server redeployment
  • Providing the ability to temporarily increase pool size beyond the default configuration. for example - useful when when spinning up resources that takes long time to provision over many projects

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New functionality/enhancement go Pull requests that update Go code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants