Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
0a25c2a
Introduce Interceptor to Grpc Plugin - Signed-off-by: Aparajita Pande…
aparajita31pandey Aug 24, 2025
cd161e7
This is a combination of 7 commits.
aparajita31pandey Sep 17, 2025
79c0b10
javadoc
aparajita31pandey Sep 19, 2025
6923d44
cleanuo
aparajita31pandey Sep 19, 2025
a9b473d
move GrpcInterceptor Interface to spi
aparajita31pandey Oct 2, 2025
b5c3c6f
move GrpcInterceptor Interface to spi
aparajita31pandey Oct 2, 2025
365647c
build.gradle fixes
aparajita31pandey Oct 3, 2025
2bd2b21
GrpcInterceptorChain refactoring
aparajita31pandey Oct 3, 2025
3715209
GrpcInterceptorChain refactoring
aparajita31pandey Oct 3, 2025
f3cd4f9
rebase
aparajita31pandey Oct 8, 2025
84ac010
coverage
aparajita31pandey Oct 8, 2025
aa4cf6f
spotless
aparajita31pandey Oct 8, 2025
06b1fa7
Add logic to throw exception on workload group deletion with associat…
kaushalmahi12 Oct 8, 2025
3290574
add documentation about debug-server-jvm in testing.md (#19567)
vigyasharma Oct 8, 2025
f9f2ba2
Adds a mapper for context aware segments grouping criteria (#19233)
shatejas Oct 8, 2025
01853c5
Refactor README.md and GrpcInterceptorChain Constructor
aparajita31pandey Oct 9, 2025
18d5c2f
UpdateREADME & switch to implmentation
aparajita31pandey Oct 9, 2025
77455a7
refactoring
aparajita31pandey Oct 9, 2025
bbab74b
refactoring
aparajita31pandey Oct 9, 2025
1a86f7e
Address PR review comments
aparajita31pandey Oct 9, 2025
d1935e4
empty commit
aparajita31pandey Oct 9, 2025
e8c7e4d
Merge branch 'main' into introduceGprcInterceptor
aparajita31pandey Oct 9, 2025
9784239
resolve conflict
aparajita31pandey Oct 9, 2025
020a0aa
Merge branch 'main' into introduceGprcInterceptor
aparajita31pandey Oct 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Use Lucene `pack` method for `half_float` and `usigned_long` when using `ApproximatePointRangeQuery`.
- Add a mapper for context aware segments grouping criteria ([#19233](https://github.com/opensearch-project/OpenSearch/pull/19233))
- Return full error for GRPC error response ([#19568](https://github.com/opensearch-project/OpenSearch/pull/19568))
- Add pluggable gRPC interceptors with explicit ordering([#19005](https://github.com/opensearch-project/OpenSearch/pull/19005))

### Changed
- Faster `terms` query creation for `keyword` field with index and docValues enabled ([#19350](https://github.com/opensearch-project/OpenSearch/pull/19350))
Expand Down
148 changes: 128 additions & 20 deletions modules/transport-grpc/spi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ The `transport-grpc-spi` module enables plugin developers to:
- Implement custom query converters for gRPC transport
- Extend gRPC protocol buffer handling
- Register custom query types that can be processed via gRPC
- Register gRPC interceptors with explicit ordering

## Key Components

Expand All @@ -26,6 +27,26 @@ public interface QueryBuilderProtoConverter {

Interface for accessing the query converter registry. This provides a clean abstraction for plugins that need to convert nested queries without exposing internal implementation details.

### GrpcInterceptorProvider

Interface for providing gRPC interceptors that can intercept and process all incoming gRPC requests. This enables plugins to implement cross-cutting concerns like authentication, authorization, logging, and metrics collection.

```java
public interface GrpcInterceptorProvider {
List<OrderedGrpcInterceptor> getOrderedGrpcInterceptors();

/**
* Nested interface for ordered gRPC interceptors.
* Interceptors are executed in order based on their order() value,
* with lower values executing first (e.g., 10 before 20).
*/
interface OrderedGrpcInterceptor {
int order(); // Lower values execute first
ServerInterceptor getInterceptor();
}
}
```

## Usage for Plugin Developers

### 1. Add Dependency
Expand All @@ -36,10 +57,47 @@ Add the SPI dependency to your plugin's `build.gradle`:
dependencies {
compileOnly 'org.opensearch.plugin:transport-grpc-spi:${opensearch.version}'
compileOnly 'org.opensearch:protobufs:${protobufs.version}'
compileOnly 'io.grpc:grpc-api:${versions.grpc}'
}
```

### 2. Implement Custom Query Converter
### 2. Declare Extension in build.gradle

In your `build.gradle`, declare that your plugin extends `transport-grpc`. This automatically adds the `extended.plugins=transport-grpc` entry to the auto-generated `plugin-descriptor.properties` file: :

```groovy
opensearchplugin {
name 'your-plugin-name'
description 'Your plugin description'
classname 'org.opensearch.yourplugin.YourPlugin'
extendedPlugins = ['transport-grpc'] // Declare extension here
}
```
**Real-world examples:**
- [OpenSearch Reporting Plugin](https://github.com/opensearch-project/reporting/blob/main/build.gradle#L92)
- [OpenSearch k-NN Plugin](https://github.com/opensearch-project/k-NN/blob/main/build.gradle#L319)

### 3. Create SPI Registration File(s)

Create a service file denoting your plugin's implementation of a service interface.

For QueryBuilderProtoConverter implementations:
`src/main/resources/META-INF/services/org.opensearch.transport.grpc.spi.QueryBuilderProtoConverter`:

```
org.opensearch.mypackage.MyCustomQueryConverter
```

For `GrpcInterceptorProvider` implementations: `src/main/resources/META-INF/services/org.opensearch.transport.grpc.spi.GrpcInterceptorProvider`:

```
org.opensearch.mypackage.SampleInterceptorProvider
```



## QueryBuilderProtoConverter
### 1. Implement Custom Query Converter

```java
public class MyCustomQueryConverter implements QueryBuilderProtoConverter {
Expand All @@ -58,7 +116,7 @@ public class MyCustomQueryConverter implements QueryBuilderProtoConverter {
}
```

### 3. Register Your Converter
### 2. Register Your Converter

In your plugin's main class, return the converter from createComponents:

Expand All @@ -80,23 +138,7 @@ public class MyPlugin extends Plugin {
}
```

**Step 3b: Create SPI Registration File**

Create a file at `src/main/resources/META-INF/services/org.opensearch.transport.grpc.spi.QueryBuilderProtoConverter`:

```
org.opensearch.mypackage.MyCustomQueryConverter
```

**Step 3c: Declare Extension in Plugin Descriptor**

In your `plugin-descriptor.properties`, declare that your plugin extends transport-grpc:

```properties
extended.plugins=transport-grpc
```

### 4. Accessing the Registry (For Complex Queries)
### 3. Accessing the Registry (For Complex Queries)

If your converter needs to handle nested queries (like k-NN's filter clause), you'll need access to the registry to convert other query types. The transport-grpc plugin will inject the registry into your converter.

Expand Down Expand Up @@ -188,7 +230,7 @@ public class KNNQueryBuilderProtoConverter implements QueryBuilderProtoConverter
./gradlew :modules:transport-grpc:spi:test
```

### Testing Your Custom Converter
### 4. Testing Your Custom Converter

```java
@Test
Expand Down Expand Up @@ -275,3 +317,69 @@ org.opensearch.knn.grpc.proto.request.search.query.KNNQueryBuilderProtoConverter

**Why k-NN needs the registry:**
The k-NN query's `filter` field is a `QueryContainer` protobuf type that can contain any query type (MatchAll, Term, Terms, etc.). The k-NN converter needs access to the registry to convert these nested queries to their corresponding QueryBuilder objects.


## gRPC Interceptors

### Overview

Intercept incoming gRPC requests for authentication, authorization, logging, metrics, rate limiting,etc

### Basic Usage

**1. Implement Provider:**
```java
public class SampleInterceptorProvider implements GrpcInterceptorProvider {
@Override
public List<GrpcInterceptorProvider.OrderedGrpcInterceptor> getOrderedGrpcInterceptors() {
return Arrays.asList(
// First interceptor (order = 5, runs first)
new GrpcInterceptorProvider.OrderedGrpcInterceptor() {
@Override
public int order() { return 5; } // Lower = higher priority

@Override
public ServerInterceptor getInterceptor() {
return (call, headers, next) -> {
String methodName = call.getMethodDescriptor().getFullMethodName();
System.out.println("First interceptor - Method: " + methodName);
return next.startCall(call, headers);
};
}
},

// Second interceptor (order = 10, runs after first)
new GrpcInterceptorProvider.OrderedGrpcInterceptor() {
@Override
public int order() { return 10; }

@Override
public ServerInterceptor getInterceptor() {
return (call, headers, next) -> {
System.out.println("Second interceptor - Processing request");
return next.startCall(call, headers);
};
}
}
);
}
}
```

### Understanding Interceptor Ordering

#### How Order Values Work

Interceptors are executed based on their `order()` value. Lower order values execute first. The interceptor chain processes requests from lowest to highest order value.

```
Request → [order=5] → [order=10] → [order=100] → Service Handler
```

#### Duplicate Order Values

Each interceptor must have a unique order value. If duplicate order values are detected, OpenSearch will fail to start with an `IllegalArgumentException`.
```
IllegalArgumentException: Multiple gRPC interceptors have the same order value: 10.
Each interceptor must have a unique order value.
```
24 changes: 22 additions & 2 deletions modules/transport-grpc/spi/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,28 @@ base {
}

dependencies {
api project(":server")
api "org.opensearch:protobufs:${versions.opensearchprotobufs}"
implementation project(":server")
implementation "org.opensearch:protobufs:${versions.opensearchprotobufs}"
implementation "io.grpc:grpc-api:${versions.grpc}"

testImplementation project(":test:framework")
}

thirdPartyAudit {
ignoreMissingClasses(
'com.google.common.base.Joiner',
'com.google.common.base.MoreObjects',
'com.google.common.base.MoreObjects$ToStringHelper',
'com.google.common.base.Objects',
'com.google.common.base.Preconditions',
'com.google.common.base.Strings',
'com.google.common.base.Throwables',
'com.google.common.collect.ImmutableList',
'com.google.common.collect.ImmutableMap',
'com.google.common.collect.Maps',
'com.google.common.collect.Sets',
'com.google.common.io.BaseEncoding',
'com.google.common.io.ByteStreams',
'com.google.common.util.concurrent.ListenableFuture'
)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
18ddd409fb9bc0209d216854ca584d027e68210b
Loading
Loading