Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ProvisioningModelMix to InstanceFlexibilityPolicy for Dataproc cluster #12327

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ahzaz
Copy link

@ahzaz ahzaz commented Nov 14, 2024

This PR adds ProvisioningModelMix to enable support for spot and on-demand instance mixing for Datparoc secondary workers.

Release Note Template for Downstream PRs (will be copied)

See Write release notes for guidance.

dataproc: added `cluster_config.preemptible_worker_config.instance_flexibility_policy.provisioning_model_mix` field to `resource_dataproc_cluster` resource

Copy link

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@zli82016, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 129 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 2 files changed, 129 insertions(+), 2 deletions(-))

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 127 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 2 files changed, 127 insertions(+), 2 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 55
Skipped tests: 6
Affected tests: 2

Click here to see the affected service packages
  • dataproc

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataprocCluster_spotOnDemandMixing
  • TestAccDataprocCluster_spotWithInstanceFlexibilityPolicy

Get to know how VCR tests work

1 similar comment
@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 55
Skipped tests: 6
Affected tests: 2

Click here to see the affected service packages
  • dataproc

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataprocCluster_spotOnDemandMixing
  • TestAccDataprocCluster_spotWithInstanceFlexibilityPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccDataprocCluster_spotOnDemandMixing [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🔴 Tests failed during RECORDING mode:
TestAccDataprocCluster_spotWithInstanceFlexibilityPolicy [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccDataprocCluster_spotOnDemandMixing [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🔴 Tests failed during RECORDING mode:
TestAccDataprocCluster_spotWithInstanceFlexibilityPolicy [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

…ng of spot and on-demands instances for Dataproc cluster
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 126 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 2 files changed, 126 insertions(+), 2 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 55
Skipped tests: 6
Affected tests: 2

Click here to see the affected service packages
  • dataproc

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataprocCluster_spotOnDemandMixing
  • TestAccDataprocCluster_spotWithInstanceFlexibilityPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataprocCluster_spotOnDemandMixing [Error message] [Debug log]
TestAccDataprocCluster_spotWithInstanceFlexibilityPolicy [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 2 files changed, 127 insertions(+), 2 deletions(-))
google-beta provider: Diff ( 2 files changed, 127 insertions(+), 2 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 57
Skipped tests: 6
Affected tests: 0

Click here to see the affected service packages
  • dataproc

🟢 All tests passed!

View the build log

"provisioning_model_mix": {
Type: schema.TypeList,
Optional: true,
Computed: true,
Copy link
Member

@zli82016 zli82016 Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think Computed is needed here. It is used for the case that a default value is returned from API side when no value for this field is provided in the request. Is there a default value for this field returned in the response in API side?

Comment on lines +2013 to +2017
instance_flexibility_policy {
provisioning_model_mix {
standard_capacity_base = 1
standard_capacity_percent_above_base = 50
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

weird indentations

@zli82016
Copy link
Member

Also, is the new filed updatable? I don't see the new field is added to the updateMask in the code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants