ESQL: Makes ENRICH scope local to subqueries#153296
Conversation
6980591 to
96fd217
Compare
b63eb62 to
d260e86
Compare
| import static org.hamcrest.Matchers.greaterThanOrEqualTo; | ||
|
|
||
| public abstract class AbstractEnrichBasedCrossClusterTestCase extends AbstractMultiClustersTestCase { | ||
| public abstract class AbstractEnrichBasedCrossClusterTestCase extends AbstractMultiClustersTestCase implements EnrichClusterPluginSupport { |
There was a problem hiding this comment.
This file has been refactored moving relevant code for having ENRICH in a cluster setup into EnrichClusterPluginSupport so the CrossClusterInSubqueryIT and CrossClsuterSubqueryIT files don't have to repeat the same setup
There was a problem hiding this comment.
Pull request overview
This PR updates ES|QL cross-cluster ENRICH policy resolution so each ENRICH occurrence is validated only against the cluster(s) that actually feed rows into that ENRICH (including subquery branches and WHERE ... IN (subquery) shapes), similar to existing LOOKUP JOIN scoping.
Changes:
- Add per-ENRICH “feeding cluster scope” computation in
EsqlSessionand pass that scope intoEnrichPolicyResolver. - Key
EnrichResolutionby the originatingEnrichnode’sSource(instead of(policyName, mode)), enabling independent resolution of same-named policies across differently-scoped subqueries/views. - Extend unit and internal cluster tests to cover ENRICH scoping across FROM-subqueries and WHERE-IN subqueries, and add shared ENRICH test-cluster setup helpers.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/telemetry/PlanExecutorMetricsTests.java | Updates resolver mocking to match new resolvePolicies signature. |
| x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/session/EsqlSessionTests.java | Adds thorough unit tests for computeEnrichScope(s) across unions, ROW sources, FROM-subqueries, and WHERE-IN subqueries. |
| x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/AbstractLocalPhysicalPlanOptimizerTests.java | Adapts test enrich resolution setup to Source-keyed EnrichResolution. |
| x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/enrich/EnrichPolicyResolverTests.java | Updates tests to supply per-occurrence Source+scope and validate Source-keyed resolution/errors. |
| x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/session/EsqlSession.java | Introduces ENRICH feeding-scope computation and wires scopes into enrich resolution. |
| x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/enrich/EnrichPolicyResolver.java | Accepts per-ENRICH scopes, narrows target clusters per policy/mode, and stores results keyed by Source. |
| x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/EnrichResolution.java | Switches resolution/error maps to be keyed by Source. |
| x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/Analyzer.java | Consumes EnrichResolution by plan.source() instead of policy-name/mode. |
| x-pack/plugin/esql/src/internalClusterTest/java/org/elasticsearch/xpack/esql/action/EnrichClusterPluginSupport.java | Adds reusable ENRICH plugin + policy setup helpers for CCS ITs. |
| x-pack/plugin/esql/src/internalClusterTest/java/org/elasticsearch/xpack/esql/action/CrossClusterSubqueryIT.java | Enables ENRICH in CCS subquery ITs and adds coverage for ENRICH modes across FROM-subquery unions. |
| x-pack/plugin/esql/src/internalClusterTest/java/org/elasticsearch/xpack/esql/action/CrossClusterInSubqueryIT.java | Enables ENRICH in CCS WHERE-IN ITs and adds extensive mode/scope coverage, including view/source collisions. |
| x-pack/plugin/esql/src/internalClusterTest/java/org/elasticsearch/xpack/esql/action/AbstractEnrichBasedCrossClusterTestCase.java | Refactors plugin/settings wiring to share ENRICH cluster setup logic. |
| x-pack/plugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/TestAnalyzer.java | Adapts test fixtures to Source-keyed enrich resolution via deferred (pending) registrations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Closes https://github.com/elastic/esql-planning/issues/997
Similar to what we did for LOOKUP JOIN, we scope ENRICH policy resolution to the part of the query that actually feeds that specific ENRICH (its own FROM-subquery branch or WHERE IN subquery), instead of resolving it against every cluster touched anywhere in the whole query.
For example, in this query:
we only need
values_enrich_ato exist oncluster-aandvalues_enrich_bto exist onremote-b— not on every cluster the query touches.Implementation:
EsqlSessionresolves all the relevant clusters for each ENRICH policy in the query, reusing the same tree walk we did forLOOKUP JOINEnrichPolicyResolvernarrows each scope having into account the ENRICH mode.