KAFKA-20169: Support static membership for Kafka Streams with the streams rebalance protocol at Client Side.

[chickenchickenlove]

Description

This PR adds client-side support for static membership in the Streams group protocol, aligned with KIP-1071.

What changed

• Allowed group.instance.id with group.protocol=streams in Streams config validation.
• Wired group.instance.id into StreamsMembershipManager creation path.
• Updated Streams close flow to honor CloseOptions.GroupMembershipOperation and support static member leave epoch handling (-2 for remain-in-group path).
• Added UNRELEASED_INSTANCE_ID handling in StreamsGroupHeartbeatRequestManager as a fatal error path.
• Added/updated unit tests:
  • StreamsMembershipManagerTest for static member close/leave epoch behavior.
  • StreamsConfigTest for group.protocol=streams + group.instance.id acceptance.

Scope

• Client-side changes only.
• Client↔Server integration tests are out of scope for this PR (server-side is in a separate PR).
• There is a mismatch in the Javadocs for the close() method here. This will be aligned in KIP-1284.

Related PR

• Server Side - KAFKA-20169: Support static membership for Kafka Streams with the streams rebalance protocol at Server Side. #21565

Reviewers: Lucas Brutschy lbrutschy@confluent.io

[chickenchickenlove]

@m1a2st , Hi!
I’m currently working on implementing KAFKA-20169, which adds Static Membership support for StreamsGroup.

During the client-side work, I realized I had to implement part of what’s being discussed in KIP-1284. Specifically, for static members, the client should be able to leave with epoch -2, but at the moment it always leaves with -1. Because of that, I had to adjust a portion of the code related to close(CloseOptions).

To complete this PR, I think there are a couple of possible approaches:

• I can remove the CloseOptions-related changes from this PR and keep it focused only on the group.instance.id / static membership parts. If leaving with epoch -2 is needed for integration tests, I could rebase and add it later after KIP-1284 is implemented.
• Alternatively, I can include a partial implementation of the CloseOptions changes in this PR, and you could follow up with the remaining parts based on the direction you prefer for KIP-1284.

I want to be respectful of the PR that will come out of KIP-1284.
If you could let me know which direction you’d prefer, I’m happy to align with that.
And if there’s a better approach than the options above, I’m very open to suggestions!

Thanks in advance for your guidance. 🙇‍♂️

Small world — looks like we’re at the same company. Nice to connect here. 😄

[m1a2st]

Hello @chickenchickenlove,

I think we can wait until KIP-1284 is approved. In the meantime, I’ll complete the implementation without static member support first, and then you can follow up with the static member–related logic afterward. That might make the development process smoother overall.

It’s great to connect with you as well 😊

[chickenchickenlove]

@m1a2st
Thanks for your comments and sorry for late!
Sounds great. Please let me know when the work is done.
Also, if you’d like my review as well, feel free to mention me anytime. I’d be happy to take a look 🙇‍♂️

[github-actions]

This PR is being marked as stale since it has not had any activity in 90 days. If you
would like to keep this PR alive, please leave a comment asking for a review. If the PR has
merge conflicts, update it with the latest from the base branch.

If you are having difficulty finding a reviewer, please reach out on the [mailing list](https://kafka.apache.org/contact).

If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.

[lucasbru]

Seems likle #22559 replaced this