Fix issue 434: Add connector level proxy configurations by wangshu3000 · Pull Request #436 · splunk/kafka-connect-splunk

wangshu3000 · 2024-06-24T00:46:19Z

This is a PR to fix issue: #434

In this PR, i added 2 parameters to the connector configurations.
splunk.hec.http.proxy.host - The proxy host
splunk.hec.http.proxy.port - The proxy port

This proxy is setup at the connector level, the splunk connector can use it's own proxy.
If these 2 parameters were configured, it'll use this proxy when connecting to HEC endpoint.
Otherwise, it'll use the JVM level proxy setting in JVM_OPTS.

This provides us the capability to use a different proxy for this connector. No matter how the global JVM_OPTS proxy was configured.

wangshu3000 · 2024-07-08T07:31:19Z

Hi @wojtekzyla @VihasMakwana
Could you please hep to review this PR if you have time?
This is a PR to add proxy related configuration settings, so that we can configure connector level proxy.

Thanks.

wojtekzyla · 2024-07-16T09:31:21Z

Hi @wangshu3000 sure I'll review this

wangshu3000 · 2024-08-27T01:52:45Z

@wojtekzyla Not sure if you got chance to review my change, i have updated the description based on your suggestion. Could you please have a review? Thanks.

shijiadong2022 · 2024-10-16T03:26:44Z

@wojtekzyla I wonder if you could help review this one when you have a chance?

wojtekzyla · 2024-10-22T09:30:24Z

Could you add some empty commit to restart the workflow?

Signed-off-by: Wang, Shu <shu.wang@fmr.com>

wangshu3000 · 2024-10-29T00:58:57Z

Thanks @wojtekzyla for your approval, i have retriggered the CI Build Test, could you please approve the flow? Thanks.

github-actions · 2024-10-29T10:07:50Z

Unit Test Results

175 tests 175 ✅ 38s ⏱️
24 suites 0 💤
24 files 0 ❌

Results for commit b45a55d.

wangshu3000 · 2024-10-29T10:52:45Z

Thanks @wojtekzyla
Looks like the CI Build was failed at fossa-scan step, looks like some dependencies are not compliant, not sure if i should bump the version in my PR or it can be bypassed?

wojtekzyla · 2024-11-14T10:00:54Z

Thanks @wojtekzyla Looks like the CI Build was failed at fossa-scan step, looks like some dependencies are not compliant, not sure if i should bump the version in my PR or it can be bypassed?

Yes, if you could do that it would be helpful. You could also add it to PR description

Copilot

Pull request overview

This PR addresses issue #434 by adding connector-level HTTP proxy configuration for the Splunk HEC client, allowing a specific connector to use a dedicated outbound proxy instead of relying solely on JVM-wide proxy settings.

Changes:

Added new connector configs splunk.hec.http.proxy.host / splunk.hec.http.proxy.port and plumbed them through to HecConfig.
Updated the HEC HTTP client construction to apply a per-connector proxy via Apache HttpClient RequestConfig.
Added/updated unit tests and documentation, and refreshed the JaCoCo coverage report artifacts.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
target/site/jacoco/jacoco.csv	Updates committed coverage metrics after code/test changes.
src/test/java/com/splunk/kafka/connect/UnitUtil.java	Adds proxy settings into generated test task configs.
src/test/java/com/splunk/kafka/connect/SplunkSinkConnectorConfigTest.java	Adds a unit test asserting proxy config is propagated into `HecConfig`.
src/test/java/com/splunk/kafka/connect/ConfigProfile.java	Extends test config profiles with proxy fields and string representation updates.
src/test/java/com/splunk/hecclient/HttpClientBuilderTest.java	Adds a smoke test ensuring `HttpClientBuilder` can be built with proxy inputs.
src/test/java/com/splunk/hecclient/HecConfigTest.java	Extends getter/setter coverage for new proxy fields on `HecConfig`.
src/main/java/com/splunk/kafka/connect/SplunkSinkConnectorConfig.java	Introduces new connector config keys/docs and passes proxy settings into `HecConfig`.
src/main/java/com/splunk/hecclient/HttpClientBuilder.java	Adds proxy fields/setters and applies proxy in `RequestConfig` when configured.
src/main/java/com/splunk/hecclient/HecConfig.java	Adds new proxy host/port fields with getters/setters.
src/main/java/com/splunk/hecclient/Hec.java	Plumbs `HecConfig` proxy settings into HTTP client creation paths.
README.md	Documents the new connector properties and defaults.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

-                .build();
+        RequestConfig.Builder requestConfigBuilder = RequestConfig.custom()
+            .setCookieSpec(CookieSpecs.STANDARD);
+        if (StringUtils.isNotEmpty(this.httpProxyHost) && this.httpProxyPort != 0) {


        sourcetypes = getString(SOURCETYPE_CONF);
        sources = getString(SOURCE_CONF);
        httpKeepAlive = getBoolean(HTTP_KEEPALIVE_CONF);
+        httpProxyHost = getString(HTTP_PROXY_HOST_CONF);
+        httpProxyPort = getInt(HTTP_PROXY_PORT_CONF);


            return new HttpClientBuilder().setDisableSSLCertVerification(config.getDisableSSLCertVerification())
                    .setMaxConnectionPoolSizePerDestination(poolSizePerDest)
                    .setMaxConnectionPoolSize(poolSizePerDest * config.getUris().size())
+                    .setHttpProxyHost(config.getHttpProxyHost())
+                    .setHttpProxyPort(config.getHttpProxyPort())
                    .build();


+| `splunk.hec.http.proxy.host` | This setting is the http proxy server hostname. Configure it to use connector level proxy when connecting to HEC endpoint, otherwise, it'll use JVM level proxy setting in JVM_OPTS. | `""` |
+| `splunk.hec.http.proxy.port` | This setting is the http proxy server port. Configure it to use connector level proxy when connecting to HEC endpoint, otherwise, it'll use JVM level proxy setting in JVM_OPTS. | `0` |



    @Override public String toString() {
-        return "ConfigProfile{" + "topics='" + topics + '\'' + ", topics.regex='" + topicsRegex +  '\'' + ", token='" + token + '\'' + ", uri='" + uri + '\'' + ", raw=" + raw + ", ack=" + ack + ", indexes='" + indexes + '\'' + ", sourcetypes='" + sourcetypes + '\'' + ", sources='" + sources + '\'' + ", httpKeepAlive=" + httpKeepAlive + ", validateCertificates=" + validateCertificates + ", hasTrustStorePath=" + hasTrustStorePath + ", trustStorePath='" + trustStorePath + '\'' + ", trustStoreType='" + trustStoreType + '\'' + ", trustStorePassword='" + trustStorePassword + '\'' + ", eventBatchTimeout=" + eventBatchTimeout + ", ackPollInterval=" + ackPollInterval + ", ackPollThreads=" + ackPollThreads + ", maxHttpConnPerChannel=" + maxHttpConnPerChannel + ", totalHecChannels=" + totalHecChannels + ", socketTimeout=" + socketTimeout + ", enrichements='" + enrichements + '\'' + ", enrichementMap=" + enrichementMap + ", trackData=" + trackData + ", maxBatchSize=" + maxBatchSize + ", numOfThreads=" + numOfThreads + '}';
+        return "ConfigProfile{" + "topics='" + topics + '\'' + ", topics.regex='" + topicsRegex +  '\'' + ", token='" + token + '\'' + ", uri='" + uri + '\'' + ", raw=" + raw + ", ack=" + ack + ", indexes='" + indexes + '\'' + ", sourcetypes='" + sourcetypes + '\'' + ", sources='" + sources + '\'' + ", httpKeepAlive=" + httpKeepAlive + ", httpProxyHost=" + httpProxyHost + ", httpProxyPort=" + httpProxyPort + ", validateCertificates=" + validateCertificates + ", hasTrustStorePath=" + hasTrustStorePath + ", trustStorePath='" + trustStorePath + '\'' + ", " + "trustStoreType='" + trustStoreType + '\'' + ", trustStorePassword='" + trustStorePassword + '\'' + ", eventBatchTimeout=" + eventBatchTimeout + ", ackPollInterval=" + ackPollInterval + ", ackPollThreads=" + ackPollThreads + ", maxHttpConnPerChannel=" + maxHttpConnPerChannel + ", totalHecChannels=" + totalHecChannels + ", socketTimeout=" + socketTimeout + ", enrichements='" + enrichements + '\'' + ", enrichementMap=" + enrichementMap + ", trackData=" + trackData + ", maxBatchSize=" + maxBatchSize + ", numOfThreads=" + numOfThreads + '}';


wangshu3000 had a problem deploying to workflow-approval June 24, 2024 00:46 — with GitHub Actions Failure

wangshu3000 changed the title ~~Fix issue 434~~ Fix issue 434: Add connector level proxy configurations Jun 24, 2024

wojtekzyla suggested changes Jul 23, 2024

View reviewed changes

Comment thread README.md Outdated

brianwarner force-pushed the fix-issue-434 branch from 2498574 to c6b8471 Compare July 24, 2024 21:14

brianwarner had a problem deploying to workflow-approval July 24, 2024 21:14 — with GitHub Actions Failure

wojtekzyla self-requested a review October 22, 2024 09:14

wojtekzyla approved these changes Oct 22, 2024

View reviewed changes

fix-issue-434 To add proxy related settings to splunk connector

b45a55d

Signed-off-by: Wang, Shu <shu.wang@fmr.com>

kayhern force-pushed the fix-issue-434 branch from c6b8471 to b45a55d Compare October 28, 2024 17:31

kayhern temporarily deployed to workflow-approval October 28, 2024 17:31 — with GitHub Actions Inactive

pszkamruk-splunk requested a review from Copilot May 6, 2026 11:26

Copilot started reviewing on behalf of pszkamruk-splunk May 6, 2026 11:27 View session

Copilot AI reviewed May 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix issue 434: Add connector level proxy configurations#436

Fix issue 434: Add connector level proxy configurations#436
wangshu3000 wants to merge 1 commit into
splunk:developfrom
fidelity-contributions:fix-issue-434

wangshu3000 commented Jun 24, 2024

Uh oh!

wangshu3000 commented Jul 8, 2024

Uh oh!

wojtekzyla commented Jul 16, 2024

Uh oh!

Uh oh!

wangshu3000 commented Aug 27, 2024

Uh oh!

shijiadong2022 commented Oct 16, 2024

Uh oh!

wojtekzyla commented Oct 22, 2024

Uh oh!

wangshu3000 commented Oct 29, 2024

Uh oh!

github-actions Bot commented Oct 29, 2024

Uh oh!

wangshu3000 commented Oct 29, 2024

Uh oh!

wojtekzyla commented Nov 14, 2024

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

		\| `splunk.hec.http.proxy.host` \| This setting is the http proxy server hostname. Configure it to use connector level proxy when connecting to HEC endpoint, otherwise, it'll use JVM level proxy setting in JVM_OPTS. \| `""` \|
		\| `splunk.hec.http.proxy.port` \| This setting is the http proxy server port. Configure it to use connector level proxy when connecting to HEC endpoint, otherwise, it'll use JVM level proxy setting in JVM_OPTS. \| `0` \|

Uh oh!

Conversation

wangshu3000 commented Jun 24, 2024

Uh oh!

wangshu3000 commented Jul 8, 2024

Uh oh!

wojtekzyla commented Jul 16, 2024

Uh oh!

Uh oh!

wangshu3000 commented Aug 27, 2024

Uh oh!

shijiadong2022 commented Oct 16, 2024

Uh oh!

wojtekzyla commented Oct 22, 2024

Uh oh!

wangshu3000 commented Oct 29, 2024

Uh oh!

github-actions Bot commented Oct 29, 2024

Unit Test Results

Uh oh!

wangshu3000 commented Oct 29, 2024

Uh oh!

wojtekzyla commented Nov 14, 2024

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants