Skip to content

Adding support for batched provisioning#20

Open
sahan-ctds wants to merge 1 commit into
datastax:masterfrom
sahan-ctds:add-support-for-batched-provisioning
Open

Adding support for batched provisioning#20
sahan-ctds wants to merge 1 commit into
datastax:masterfrom
sahan-ctds:add-support-for-batched-provisioning

Conversation

@sahan-ctds

Copy link
Copy Markdown

Description

This PR adds optional batch provisioning support to Fallout, allowing NodeGroups to be provisioned in sequential batches instead of all in parallel. This feature helps prevent resource exhaustion (e.g., memory, CPU) when provisioning many NodeGroups simultaneously.

This PR introduces a new optional system property fallout.system.provisioning.batch_size that allows users to control how many NodeGroups are provisioned simultaneously. When configured, NodeGroups are divided into batches and provisioned sequentially:

  1. NodeGroups within a batch provision in parallel
  2. Each batch waits for completion before the next batch starts
  3. If any batch fails, subsequent batches are skipped

Format:

ensemble:
  fallout.system.provisioning.batch_size: 5
  servers:
    ...
  clients:
    ...
  observer:
    ...

Changes

  1. FalloutPropertySpecs.java: Added provisioningBatchSizePropertySpec property definition
  2. Ensemble.java:
    • Added provisioningBatchSize field and getter
    • Added transitionStateWithFunction() method to support batch processing with per-NodeGroup state determination
    • Modified transitionState() to implement batch logic for uniform state transitions
  3. EnsembleBuilder.java: Added batch size field and builder method
  4. ActiveTestRunBuilder.java: Added YAML parsing for batch size property
  5. ActiveTestRun.java: Modified transitionEnsembleStateUpwards() to use ensemble-level batch-aware transitions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant