Skip to content

fix: Avoid redundant autoscaling start request #4683

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jwongmongodb merged 3 commits into from
Aug 26, 2026
+32 −18
Merged

fix: Avoid redundant autoscaling start request #4683

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a912ff7
fix: Avoid redundant autoscaling start request
jwongmongodb Aug 26, 2026
45629a6
chore: Add changelog entry for #4683
jwongmongodb Aug 26, 2026
e85b20d
test: Cover direct stream processor start
jwongmongodb Aug 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 0 additions & 16 deletions internal/service/streamprocessor/resource.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,14 +111,6 @@ func (r *streamProcessorRS) Create(ctx context.Context, req resource.CreateReque
if plan.Tier.ValueString() != "" {
startWithOptions.SetTier(plan.Tier.ValueString())
}
// On the :startWith endpoint, `autoscaling` is TOP-LEVEL (no options wrapper).
autoscaling, diags := autoscalingFromOptions(ctx, plan.Options)
if diags.HasError() {
resp.Diagnostics.Append(diags...)
return
}
startWithOptions.Autoscaling = autoscaling
Comment on lines -114 to -120

@AgustinBettati AgustinBettati Aug 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to understand, was this additional property simply redundant, or actually causing a failures? And what would be the change planned on the API side?

@jwongmongodb jwongmongodb Aug 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is redundant, but there appears to be a race condition with the service that the api calls when the autoscaling payload was included. I believe that this 404 returned in the endpoint call is a bug that we will fix. But it's good to also remove the redundant config and fix this error path in the meantime. My understanding that any payload sent the startsWithOptions in this flow will trigger this issue: here is the ticket filed


_, err := connV2.StreamsAPI.StartStreamProcessorWith(ctx, projectID, workspaceOrInstanceName, processorName, startWithOptions).Execute()
if err != nil {
resp.Diagnostics.AddError(errorCreateStart, err.Error())
Expand Down Expand Up @@ -247,14 +239,6 @@ func (r *streamProcessorRS) Update(ctx context.Context, req resource.UpdateReque
if plan.Tier.ValueString() != "" {
startWithOptions.SetTier(plan.Tier.ValueString())
}
// On the :startWith endpoint, `autoscaling` is TOP-LEVEL (see create path).
autoscaling, diags := autoscalingFromOptions(ctx, plan.Options)
if diags.HasError() {
resp.Diagnostics.Append(diags...)
return
}
startWithOptions.Autoscaling = autoscaling

_, err := r.Client.AtlasV2.StreamsAPI.StartStreamProcessorWith(ctx, projectID, workspaceOrInstanceName, processorName, startWithOptions).Execute()
if err != nil {
resp.Diagnostics.AddError("Error starting stream processor", err.Error())
Expand Down
34 changes: 32 additions & 2 deletions internal/service/streamprocessor/resource_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,6 @@ func TestAccStreamProcessor_withOptionsDLQAutoscaling(t *testing.T) {
),
},
{
// Starts a configured autoscaling processor through the top-level
// autoscaling field of the :startWith request.
Config: configWithOptionsDLQAutoscaling(t, projectID, workspaceName, clusterName, processorName, streamProcessorOptionsConfig{
includeDLQ: true,
autoscalingMinTier: "SP10",
Expand Down Expand Up @@ -198,6 +196,38 @@ func TestAccStreamProcessor_withOptionsDLQAutoscaling(t *testing.T) {
}})
}

func TestAccStreamProcessor_withOptionsDLQAutoscalingCreateStarted(t *testing.T) {
var (
projectID, workspaceName = acc.ProjectIDExecutionWithStreamInstance(t)
_, clusterName = acc.ClusterNameExecution(t, false)
randomSuffix = acctest.RandString(5)
Comment on lines +199 to +203
processorName = "new-processor-autoscaling-started-" + randomSuffix
)

resource.Test(t, resource.TestCase{

@jwongmongodb jwongmongodb Aug 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this tests the STARTED flow from the CREATE call which is not covered in the TestAccStreamProcessor_withOptionsDLQAutoscaling which calls the STARTED flow in the UPDATE call, which is why this was missed.

This edge case bug only affected the from scratch STARTED state generation, this will be fixed separately internally, but this change is still worth pushing to work in prod and also to clean up the redundant autoscaling config being set.

PreCheck: func() { acc.PreCheckBasic(t) },
ProtoV6ProviderFactories: acc.TestAccProviderV6Factories,
CheckDestroy: checkDestroyStreamProcessor,
Steps: []resource.TestStep{
{
Config: configWithOptionsDLQAutoscaling(t, projectID, workspaceName, clusterName, processorName, streamProcessorOptionsConfig{
includeDLQ: true,
autoscalingMinTier: "SP10",
autoscalingMaxTier: "SP50",
state: streamprocessor.StartedState,
}),
Check: resource.ComposeAggregateTestCheckFunc(
resource.TestCheckResourceAttrSet(resourceName, "id"),
resource.TestCheckResourceAttr(resourceName, "state", streamprocessor.StartedState),
resource.TestCheckResourceAttr(resourceName, "options.autoscaling.min_tier", "SP10"),
resource.TestCheckResourceAttr(resourceName, "options.autoscaling.max_tier", "SP50"),
resource.TestCheckResourceAttrSet(resourceName, "effective_tier"),
),
},
importStep(),
}})
}

func basicTestCase(t *testing.T) *resource.TestCase {
t.Helper()
var (
Expand Down
Loading