Enable TLS for Netty4GrpcServerTransport

[finnegancarroll]

Description

Introduces SecureNetty4GrpcServerTransport, a TLS enabled alternative to Netty4GrpcServerTransport.

Mirrors many setting options of the secure http transport, except under the plugins.security.ssl.aux. setting prefix.
For information on specific settings supported please see the security plugin companion PR:
opensearch-project/security#5152

Building and running the secure transport is a little awkward as security plugin must link with OpenSearch core to build a distribution, which then must be installed into an OpenSearch distribution.

Use the following steps to compile and test this feature branch (darwin-tar used in this example):

export OS_REPO=<os-feature-branch>
export SEC_REPO=<security-plugin-feature-branch>
export OS_INSTALL=${OS_REPO}/distribution/archives/darwin-tar/build/install/opensearch-3.0.0-SNAPSHOT
export SEC_TAR=${SEC_REPO}/build/distributions/opensearch-security-3.0.0.0-alpha1-SNAPSHOT.zip
export GRPC_TAR=${OS_REPO}/plugins/transport-grpc/build/distributions/transport-grpc-3.0.0-SNAPSHOT.zip

# Publish OS to maven local and build security plugin
cd ${OS_REPO} && ./gradlew :distribution:archives:darwin-tar:assemble
cd ${OS_REPO} && ./gradlew publishToMavenLocal
cd ${OS_REPO} && ./gradlew plugins:transport-grpc:assemble
cd ${SEC_REPO} && ./gradlew :assemble

# Install security plugin into OS distribution
${OS_INSTALL}/bin/opensearch-plugin install file://${GRPC_TAR}
${OS_INSTALL}/bin/opensearch-plugin install file://${SEC_TAR}

# If needed, generate demo certs: https://github.com/opensearch-project/security/blob/main/DEVELOPER_GUIDE.md#refreshing-demo-certificates
chmod +x ${OS_INSTALL}/plugins/opensearch-security/tools/install_demo_configuration.sh
# yes yes no - keep enable cluster mode off
export OPENSEARCH_INITIAL_ADMIN_PASSWORD=SomeTestPass246!
${OS_INSTALL}/plugins/opensearch-security/tools/install_demo_configuration.sh 

# Enable TLS for aux transport
echo "aux.transport.types: experimental-secure-transport-grpc" >> ${OS_INSTALL}/config/opensearch.yml
echo "aux.transport.experimental-secure-transport-grpc.port: '9400-9500'" >> ${OS_INSTALL}/config/opensearch.yml
echo "plugins.security.ssl.aux.enabled: true" >> ${OS_INSTALL}/config/opensearch.yml

# Configure TLS w/ demo pem key and cert
echo "plugins.security.ssl.aux.pemcert_filepath: esnode.pem" >> ${OS_INSTALL}/config/opensearch.yml
echo "plugins.security.ssl.aux.pemkey_filepath: esnode-key.pem" >> ${OS_INSTALL}/config/opensearch.yml

# Configure client auth mode
echo "plugins.security.ssl.aux.client_authentication: NONE" >> ${OS_INSTALL}/config/opensearch.yml

# Launch OpenSearch
${OS_INSTALL}/bin/opensearch

Related Issues

Partially resolves #16905

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for fddab56: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for a70fd93: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 0ac61ba: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 709cbbb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for 1135fd6: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Attention: Patch coverage is 71.42857% with 20 lines in your changes missing coverage. Please review.

Project coverage is 72.37%. Comparing base (6d53f9d) to head (34c86e7).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...ansport/grpc/ssl/SecureAuxTransportSslContext.java	65.71%	10 Missing and 2 partials ⚠️
...a/org/opensearch/common/network/NetworkModule.java	61.53%	3 Missing and 2 partials ⚠️
...java/org/opensearch/transport/grpc/GrpcPlugin.java	0.00%	2 Missing ⚠️
...ch/plugins/SecureAuxTransportSettingsProvider.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #17406      +/-   ##
============================================
- Coverage     72.40%   72.37%   -0.03%     
+ Complexity    65828    65786      -42     
============================================
  Files          5316     5319       +3     
  Lines        305294   305402     +108     
  Branches      44289    44299      +10     
============================================
- Hits         221033   221029       -4     
- Misses        66187    66301     +114     
+ Partials      18074    18072       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

❕ Gradle check result for f88e461: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions]

❌ Gradle check result for 675c9a6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 681f228: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for bae5abe:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 13ec303: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for caa6605: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 2309786:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for ebe3557: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[finnegancarroll]

@finnegancarroll Are you now able to run core w/ the grpc plugin installed and configure TLS w/ security plugin settings? Would I be able to make a localDistro of core install the security plugin and start making requests using this transport after this change?

Updated PR description with steps for installing with security plugin companion PR here: opensearch-project/security#5152. I've additionally moved most of the testing for client/cert authentication into this plugin.