[Sandbox] Add local index authorization check to analytics engine before planning phase

[finnegancarroll]

Description

The analytics engine's query execution path (PPL, SQL) bypasses the security plugin's index-level privilege evaluation. The FragmentExecutionRequest on the data node is executed with dispatchFragmentStreaming and never evaluated for index permissions by the security plugin SecurityFilter (wraps the ActionFilter).

SQL / PPL queries with analytics engine

In a typical SQL / PPL query authorization is handled at the node-to-node transport layer on the coordinator as the request is being dispatched to shards. The security plugin provides an ActionFilter which every TransportAction child holds a reference to, and executes on new requests.

The overall flow for a regular DSL search query is then:

NodeClient.execute(SearchAction.INSTANCE, searchRequest)
    → NodeClient.executeLocally(action, request, listener)
      → transportAction(action).execute(request, listener)
        → TransportAction.execute(task, request, listener)
          → RequestFilterChain.proceed()

The path for an analytics engine SQL / PPL query avoids the RequestFilterChain entirely. The ShardTaskRunner is responsible for dispatching a FragmentExecutionRequest on the transport layer.

    @Override
    public void run(ShardStageTask task, ActionListener<Void> listener) {
        ShardExecutionTarget target = (ShardExecutionTarget) task.target();
        FragmentExecutionRequest request = requestBuilder.apply(target);
        PendingExecutions pending = pendingFor(target);
        transport.dispatchFragmentStreaming(request, target.node(), stage.responseListenerFor(listener), config.parentTask(), pending);
    }

The FragmentExecutionRequest fetches a new connection from the transport service, and directly executes the request itself. Skipping the TransportAction framework which typically holds and applies the ActionFilter.

        pending.tryRun(() -> {
            try {
                Transport.Connection connection = getConnection(null, targetNode.getId());
                transportService.sendChildRequest(connection, FragmentExecutionAction.NAME, request, parentTask, options, handler);
            } catch (Exception e) {
                try {
                    listener.onFailure(e);
                } finally {
                    pending.finishAndRunNext();
                }
            }
        });

DSL queries with analytics engine

DSL permissions are actually evaluated correctly already. This comes from how DSL is routed into the analytics plugin. DSL queries construct a regular SearchRequest which is handled by the TransportSearchAction and are executed on the transport layer with the local node client as usual.

Handling of DSL queries is done by a new SearchActionFilter which is also injected into the RequestFilterChain and redirects these SearchRequests to the DslExecuteAction.INSTANCE handler instead of executing them as normal search requests. Since this SearchActionFilter executes AFTER the security plugin ActionFilter (which has highest priority), privileges are evaluated correctly.

Path	Unauthorized Index	Status
DSL (POST /{index}/_search)	✅ Denied	Secure
PPL (POST /_plugins/_ppl)	❌ Allowed	Gap
SQL (POST /_plugins/_sql)	❌ Allowed	Gap

Proposed fix

These changes somewhat mimic the functionality of DSL queries for SQL and PPL by adding a pre-planning authorization check which passes a no-op AnalyticsAuthRequest through the ActionFilter to determine if permissions are valid for the equivalent set of FragmentExecutionRequests.

No actual request is executed on the transport layer, the AnalyticsAuthRequest is passed through the full ActionFilter chain to invoke security plugin privilege evaluation, and proceeds with the analytics engine planning + query only if this probing AnalyticsAuthRequest is authorized for the same indices.

Testing

Testing this feature requires an analytics engine enabled OpenSearch node, with SQL plugin installed, and security plugin installed. Currently SQL plugin supports this environment with distribution level security enabled tests. This seems like the most ideal place to house ITs validating index level security over the analytics plugin.

Test implementation is pending. Currently changes are validated with local single node testing.

Related Issues

N/A

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit ae21cbe.

Path	Line	Severity	Description
sandbox/plugins/analytics-engine/src/main/java/org/opensearch/analytics/exec/action/TransportAnalyticsAuthAction.java	33	medium	doExecute is a permanent no-op that always returns success. The authorization model relies entirely on the security plugin intercepting the action before doExecute is called. If the security plugin is absent, misconfigured, or does not match this action name, every analytics query is unconditionally authorized with no access control whatsoever. There is no defensive check (e.g., verifying a security plugin is active) to prevent silent fallthrough to full access.
sandbox/plugins/analytics-engine/src/main/java/org/opensearch/analytics/exec/DefaultPlanExecutor.java	155	medium	collectIndices only walks LogicalTableScan nodes. Calcite plans can access data through other RelNode subtypes (e.g., LogicalJoin with subquery inputs, LogicalValues, or custom nodes introduced by adapters). Any plan node type other than TableScan is silently skipped, potentially allowing queries that access real index data to proceed without the corresponding index names being included in the authorization probe request.
sandbox/plugins/analytics-engine/src/main/java/org/opensearch/analytics/exec/DefaultPlanExecutor.java	122	low	When extractIndices returns an empty array the code skips the authorization probe entirely and proceeds directly to doExecuteOnSearchPool. A crafted query plan that produces no TableScan nodes (e.g., a pure VALUES clause, a correlated subquery, or an unsupported plan shape) would bypass index-level authorization completely.

The table above displays the top 10 most important findings.

Total: 3 | Critical: 0 | High: 0 | Medium: 2 | Low: 1

---

Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.

---

⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

[Bukhtawar]

I was wondering if we add a transport interceptor that can also validate if the request was indeed authorised.

public class AnalyticsAuthInterceptor implements TransportInterceptor {
      @Override
      public <T extends TransportRequest> TransportRequestHandler<T> interceptHandler(
          String action, ..., TransportRequestHandler<T> actualHandler, ...) {

          if (action.equals(FragmentExecutionAction.NAME)) {
              return (request, channel, task) -> {
                  String token = threadContext.getHeader("auth_token");
                  if (token == null || !verify(token)) {
                      throw new SecurityException("Analytics auth not performed");
                  }
                  actualHandler.messageReceived(request, channel, task);
              };
          }
          return actualHandler;
      }
  }

[finnegancarroll]

I was wondering if we add a transport interceptor that can also validate if the request was indeed authorised.

Thanks for taking a look @Bukhtawar, I spent a small bit of time trying to implement this and it seems like we would need to work this authorization token down to the thread context from the QueryPlanExecutor with some additional plumbing.

I can look at implementing this but i'm wondering if we are guarding against anything additional here since the QueryPlanExecutor is the single entry point for analytics engine.

[cwperks]

@finnegancarroll another problem to think about with regards to security is how to extract indices from a request pertinent to indices.

There currently are 2 ways:

1. Inside the IndexResolverReplacer of the security plugin
2. The modern way is to implement TransportIndicesResolvingAction so that security puts that onus on the transport action to be able to resolve the indices.

[cwperks]

The FragmentExecutionRequest on the data node is executed with dispatchFragmentStreaming and never evaluated for index permissions by the security plugin SecurityFilter (wraps the ActionFilter).

Agreed on putting the call to dispatchFragmentStreaming within a transport action. @finnegancarroll should we get the security plugin in a state where we can test with the sandbox from core?

[cwperks]

Quickly came up with cwperks/security#97 which adds test that would be expected to fail until a fix like this lands.

[cwperks]

@finnegancarroll I'm also seeing some action names that I think we should rename: cwperks#362

^ I haven't been tracking sandbox too closely so lmk what you think and if there are other actions similar.