Skip to content

KAFKA-19543: fix(consumer): keep metrics available after close#22598

Open
wilmerdooley wants to merge 1 commit into
apache:trunkfrom
wilmerdooley:oss/kafka-19543
Open

KAFKA-19543: fix(consumer): keep metrics available after close#22598
wilmerdooley wants to merge 1 commit into
apache:trunkfrom
wilmerdooley:oss/kafka-19543

Conversation

@wilmerdooley

Copy link
Copy Markdown

What changed

Previously, calling Consumer#metrics() returned a live view of the underlying Metrics instance. Once close() was invoked, the metric registry was torn down, so any reference obtained before close would lose metrics that were unregistered as part of shutdown. This change ensures that a Map<MetricName, ? extends Metric> returned before close() continues to expose the same set of metrics after the consumer has been closed.

Implementation

  • A new internal class ConsumerMetricsView (in clients/.../internals) wraps Metrics and lazily freezes a snapshot of the current metric set when freeze() is called. Reads from the view return the live map while unfrozen, and the frozen snapshot afterwards.
  • AsyncKafkaConsumer, ClassicKafkaConsumer, and ShareConsumerImpl now track weak references to every ConsumerMetricsView they hand out from metrics(). On close(), each still-reachable view is frozen so it keeps serving its captured set of metrics even after the underlying Metrics registry has been cleared. If the consumer has already been closed, metrics() simply returns the current metrics.metrics() map as before.
  • New tests in KafkaConsumerTest and KafkaShareConsumerMetricsTest capture the metrics map before close and assert that the same metric names remain visible afterwards.

Testing strategy

The behaviour change is covered by unit tests:

  • KafkaConsumerTest#testReturnedMetricsRemainAvailableAfterClose exercises both classic (GroupProtocol.CLASSIC) and consumer group (GroupProtocol.CONSUMER) paths and verifies the pre-close metrics map is unchanged after close().
  • KafkaShareConsumerMetricsTest#testReturnedMetricsRemainAvailableAfterClose does the same for the share consumer using the existing mock client infrastructure.

Existing metrics tests continue to apply and were not modified.


Delete this text and replace it with a detailed description of your change. The
PR title and body will become the squashed commit message.

If you would like to tag individuals, add some commentary, upload images, or
include other supplemental information that should not be part of the eventual
commit message, please use a separate comment.

If applicable, please include a summary of the testing strategy (including
rationale) for the proposed change. Unit and/or integration tests are expected
for any behavior change and system tests should be considered for larger
changes.

  • The pull request description includes the appropriate summary of the change.
  • The commit message(s) follow the project's conventions.
  • New and existing unit tests pass locally and on the build server.
  • The change is sufficiently documented (API docs, Javadoc, comments).
  • New public classes, interfaces, methods, fields, etc. are linked in the documentation.
  • Larger changes have been discussed with the project maintainers.
  • I have considered possible security implications and do not see any.

JIRA: https://issues.apache.org/jira/browse/KAFKA-19543

…ges after Consumer.close()

Signed-off-by: wilmerdooley <wilmerdooley1@gmail.com>
@github-actions github-actions Bot added triage PRs from the community consumer clients labels Jun 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clients consumer triage PRs from the community

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant