feat: add ServiceLoader support for plugin.discovery=service_load #1568
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Explained in issue #1567
Root Cause Analysis
The issue appears to be that
kafka-connect-jdbcis missing the required ServiceLoader configuration files.For ServiceLoader to discover connector implementations, the JAR needs to contain:
META-INF/services/org.apache.kafka.connect.source.SourceConnectorMETA-INF/services/org.apache.kafka.connect.sink.SinkConnectorThese files should list the fully qualified class names of the connector implementations.
Proposed Solution
Add ServiceLoader configuration files to the
kafka-connect-jdbcJAR:1. Create Source Connector Service File
File:
META-INF/services/org.apache.kafka.connect.source.SourceConnector2. Create Sink Connector Service File
File:
META-INF/services/org.apache.kafka.connect.sink.SinkConnectorDoes this solution apply anywhere else?
: yes
If yes, where?
here: https://github.com/apache/kafka/tree/trunk/connect/file/src/main/resources/META-INF/services
Test Strategy
Manual Testing Performed
I manually tested this change by spinning up Kafka brokers and Connect servers locally.
Test Environment
plugin.discovery=service_loadTest Steps
plugin.discovery=service_loadRelease Plan
Proposed Approach
masterbranch for next releaseCompatibility
Suggested Target