Fixing indexing regression and bug fixes for grouping criteria #20145
Conversation
RS146BIJAY
requested review from
a team,
Bukhtawar,
CEHENKLE,
Rishikesh1159,
anasalkouz,
andrross,
ashking94,
cwperks,
dbwiddis,
gbbafna,
jed326,
kotwanikunal,
mch2,
msfroh,
owaiskazi19,
reta,
sachinpkale,
saratvemulapalli,
shwetathareja and
sohami
as code owners
WalkthroughAdjusts lookup-map lock retry behavior by lowering the default and increasing the maximum retry setting, propagating a max-retry parameter into CompositeIndexWriter lookup paths using bounded tryAcquire loops; removes exception-driven retry from bulk action; narrows a mapper return set; updates tests and CHANGELOG entries. (≤50 words) Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Suggested labels
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (2)
215-215: Reduced encapsulation ofmapReadLock.The visibility of
mapReadLockhas been changed fromprivate finalto package-private, allowing direct access from other classes in the same package. This field controls critical concurrency behavior, and exposing it directly increases the risk of misuse.Consider:
- Keeping the field
privateand exposing only necessary operations through methods (e.g.,tryAcquireLock()).- If package-private access is required for the retry logic, add clear documentation about proper usage patterns and thread-safety requirements.
- Restrict access using a package-private accessor method rather than exposing the field directly.
498-498: Simplify boolean comparisons.The condition uses explicit
== falseand== truecomparisons which are redundant in Java.Apply this diff:
-if (success == false && current != null && current.mapReadLock.isHeldByCurrentThread() == true) { +if (!success && current != null && current.mapReadLock.isHeldByCurrentThread()) { current.mapReadLock.close(); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
CHANGELOG.md(1 hunks)server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java(1 hunks)server/src/main/java/org/opensearch/index/IndexSettings.java(1 hunks)server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java(10 hunks)server/src/main/java/org/opensearch/index/mapper/MapperService.java(2 hunks)server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java(0 hunks)server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java(4 hunks)server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java(1 hunks)
💤 Files with no reviewable changes (1)
- server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: gradle-check
🔇 Additional comments (13)
server/src/main/java/org/opensearch/index/mapper/MapperService.java (2)
87-87: LGTM!Import correctly added to support the stream operations in
getCompositeFieldTypes().
693-697: Verify the behavior change scope and call frequency.The filtering to return only
StarTreeMapper.StarTreeFieldTypeinstances represents a narrowed scope from returning all composite field types. Confirm this change is intentional and whether any callers expect otherCompositeMappedFieldTypeimplementations. Additionally, verify the call frequency of this method; if invoked on hot paths, consider caching the filtered result to avoid repeated stream collection operations.server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java (1)
106-106: LGTM!The test constant is appropriately set to a lower value (20) than the production default (100) for faster test execution while still being within the valid range (5-500).
server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java (5)
44-46: LGTM!Mockito imports correctly added to support the new verification test.
71-77: LGTM!Method call correctly updated to include
MAX_NUMBER_OF_RETRIESparameter, aligning with the new bounded retry API.
141-146: LGTM!Method call correctly updated with retry parameter.
197-202: LGTM!Method call correctly updated with retry parameter.
208-227: Test validates bounded retry semantics correctly.The test properly verifies:
LookupMapLockAcquisitionExceptionis thrown after exhausting retriestryAcquire()is called exactlyMAX_NUMBER_OF_RETRIEStimesOne consideration: the mock setup directly assigns to
map.currentandmap.current.mapReadLockwhich accesses package-private fields. This works for testing but creates tight coupling to internal implementation details.server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java (1)
724-753: Retry logic moved to lower layer - verify exception handling.The
LookupMapLockAcquisitionExceptionretry logic has been removed from bulk action handling and moved toCompositeIndexWriterwith bounded retries. This architectural approach places retry logic closer to where the exception originates.Ensure that when
LookupMapLockAcquisitionExceptionpropagates up after max retries are exhausted, it's properly handled and doesn't cause unexpected bulk operation failures.server/src/main/java/org/opensearch/index/IndexSettings.java (1)
499-506: Significant default value change - verify upgrade impact.The default retry count increased to 100 with a maximum of 500. Since this is a dynamic setting, existing indices will apply the new default upon upgrade. Consider whether this change should be documented in release notes for operators who have tuned their clusters based on previous defaults.
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (3)
691-693: LGTM: Metrics gathering refactoring.The refactoring from stream-based iteration to explicit for-loops improves code clarity and performance for these simple aggregation operations. The logic is correct in all cases, with proper handling of both current and old maps where necessary, and appropriate locking in
ramBytesUsed().Also applies to: 702-704, 731-742, 758-770, 796-806
210-210: Verify removal offinalmodifier is intentional.The
finalmodifier has been removed fromCriteriaBasedIndexWriterLookup,CriteriaBasedWriterLock, andLiveIndexWriterDeletesMap. This allows subclassing of these internal implementation classes. Confirm whether:
- Subclassing is required for test mocking/stubbing.
- If so, consider restricting visibility to test scope or use sealed classes.
- If intentional for production extensibility, document extension points and invariants.
Also applies to: 301-301, 406-406
678-679: Verify retry configuration defaults and bounds.The
maxRetryOnLookupMapAcquisitionExceptionsetting controls retry behavior for lookup map acquisition. Without access to the codebase, I cannot confirm the specific default values, upper/lower bounds, or whether they align with industry best practices for lock acquisition retry mechanisms. Ensure the following:
- Default retry count is reasonable (typically 3–10 attempts for in-thread locks per best practices)
- Upper bounds prevent excessive retries that could cause performance degradation
- Configuration is documented and tunable per workload expectations
- Retry strategy includes backoff/jitter to avoid retry storms (not just blind spinning)
CHANGELOG.md
Outdated
| - Fix node bootstrap error when enable stream transport and remote cluster state ([#19948](https://github.com/opensearch-project/OpenSearch/pull/19948)) | ||
| - Fix deletion failure/error of unused index template; case when an index template matches a data stream but has a lower priority. ([#20102](https://github.com/opensearch-project/OpenSearch/pull/20102)) | ||
| - Fix toBuilder method in EngineConfig to include mergedSegmentTransferTracker([20105](https://github.com/opensearch-project/OpenSearch/pull/20105)) | ||
| - Fix indexing regression and bug fixes for grouping criteria([20145](https://github.com/opensearch-project/OpenSearch/pull/20145)) |
There was a problem hiding this comment.
Minor formatting inconsistency.
The PR link format is inconsistent with other entries in the changelog. Missing # before the PR number.
Apply this diff to fix the formatting:
-- Fix indexing regression and bug fixes for grouping criteria([20145](https://github.com/opensearch-project/OpenSearch/pull/20145))
+- Fix indexing regression and bug fixes for grouping criteria ([#20145](https://github.com/opensearch-project/OpenSearch/pull/20145))📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - Fix indexing regression and bug fixes for grouping criteria([20145](https://github.com/opensearch-project/OpenSearch/pull/20145)) | |
| - Fix indexing regression and bug fixes for grouping criteria ([#20145](https://github.com/opensearch-project/OpenSearch/pull/20145)) |
🤖 Prompt for AI Agents
CHANGELOG.md around line 94: the PR link is missing the '#' before the PR number
which makes it inconsistent with other entries; update the link by inserting '#'
before the PR number so it reads .../pull/#20145 (i.e. change
(https://github.com/opensearch-project/OpenSearch/pull/20145) to include the
hash) and ensure the rest of the entry formatting matches surrounding lines.
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
475-489: Add exponential backoff to retry loop.The retry loop performs a busy-wait by calling
tryAcquire()repeatedly without any delay between attempts. This pattern wastes CPU cycles under contention when the lock is frequently unavailable.As suggested in the previous review, add exponential backoff between retry attempts with
Thread.sleep(Math.min(1L << counter, 100L))and handleInterruptedExceptionby restoring the interrupt flag and propagatingLookupMapLockAcquisitionException.
🧹 Nitpick comments (1)
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
483-483: Simplify boolean comparisons.The expressions
current.isClosed() == true(line 483) andcurrent.mapReadLock.isHeldByCurrentThread() == true(line 498) contain redundant boolean comparisons.Apply this diff to simplify:
- if (current != null && current.isClosed() == true) { + if (current != null && current.isClosed()) {- if (success == false && current != null && current.mapReadLock.isHeldByCurrentThread() == true) { + if (!success && current != null && current.mapReadLock.isHeldByCurrentThread()) {Also applies to: 498-498
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
CHANGELOG.md(1 hunks)server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java(1 hunks)server/src/main/java/org/opensearch/index/IndexSettings.java(1 hunks)server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java(10 hunks)server/src/main/java/org/opensearch/index/mapper/MapperService.java(2 hunks)server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java(0 hunks)server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java(4 hunks)server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java(1 hunks)
💤 Files with no reviewable changes (1)
- server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java
🚧 Files skipped from review as they are similar to previous changes (5)
- server/src/main/java/org/opensearch/index/mapper/MapperService.java
- CHANGELOG.md
- server/src/main/java/org/opensearch/index/IndexSettings.java
- server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java
- server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java
🔇 Additional comments (6)
server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java (1)
717-731: Behavior change: retries now only on version conflicts inonCompleteThe retry path in
onCompletenow exclusively handlesVersionConflictEngineException(viaisConflictException) bounded byretryOnConflict, with the previousLookupMapLockAcquisitionException-specific retry branch removed. That seems consistent with moving lock-acquisition retry logic down intoCompositeIndexWriterand keeps this layer from depending on engine-internal exception types.Please double-check that:
LookupMapLockAcquisitionExceptionis fully retried/consumed withinCompositeIndexWriter(or below), and- it does not bubble back up to this point expecting a retry here anymore,
so we don’t inadvertently change failure behavior for callers that previously relied on the bulk action to transparently retry on transient lookup-map lock contention.
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (5)
210-210: Visibility changes support the new retry mechanism.The removal of
finalmodifiers from inner classes and the change ofmapReadLockto package-private access are necessary to support the refactored retry logic. The package-private access onmapReadLock(line 215) enablesLiveIndexWriterDeletesMap.computeIndexWriterIfAbsentForCriteriato calltryAcquire()directly at line 482.Also applies to: 215-215, 301-301, 406-406, 408-408
466-471: Proper parameter propagation for configurable retry limit.The addition of
maxRetryOnLookupMapAcquisitionExceptionparameter enables configurable retry behavior, and the value is correctly sourced fromengineConfig.getIndexSettings().Also applies to: 675-680
691-693: Simplified iteration improves readability.The refactoring from stream-based iteration to direct iteration over
criteriaBasedIndexWriterMap.values()is cleaner and avoids unnecessary intermediate operations.Also applies to: 702-704
731-750: Proper tragic exception checking across all writers.The iteration through both current and old writer maps to detect tragic exceptions is thorough and correctly checks if writers are closed before accessing their tragic exception state.
753-774: Correct synchronization and state checks.The
ramBytesUsed()method properly acquires write locks before iterating, and therollback()method correctly checks if writers are open before attempting rollback operations.Also applies to: 794-811
|
❌ Gradle check result for 4b247a0: null Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java (1)
44-46: Retry wiring in tests looks correct; consider simplifying the mocking for readability.
- Updating all
computeIndexWriterIfAbsentForCriteriainvocations to passMAX_NUMBER_OF_RETRIESkeeps the tests consistent with the new API and the retry semantics; this looks correct.testMaxRetryCountWhenWriteLockDuringIndexingcorrectly verifies thattryAcquire()is invoked exactlyMAX_NUMBER_OF_RETRIEStimes when the lock is never obtained, and theLookupMapLockAcquisitionExceptionis thrown as expected.As a minor test ergonomics tweak, you could stub and verify directly on
writerLockinstead of going throughmap.current.mapReadLockin thewhen(...)andverify(...)calls. That would make the test a bit less coupled to the internal layout ofLiveIndexWriterDeletesMapandCriteriaBasedIndexWriterLookupwhile preserving the behavior being asserted.Also applies to: 72-77, 141-146, 197-202, 208-227
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
210-215: Visibility and mutability changes for nested types are acceptable but could use an explicit “for testing” annotation.Making
CriteriaBasedIndexWriterLookupandCriteriaBasedWriterLockmore visible, and relaxingmapReadLockandLiveIndexWriterDeletesMap.currentfromfinal, is understandable to support the new tests that need to mock and override these internals.To keep the public surface area tidy and signal intent, consider adding an explicit
@opensearch.internal(or similar) Javadoc tag or comment on these nested types/fields indicating that they are exposed primarily for testing. That helps discourage external production code from depending on them and makes future refactors easier.Also applies to: 301-301, 406-412
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
CHANGELOG.md(1 hunks)server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java(1 hunks)server/src/main/java/org/opensearch/index/IndexSettings.java(1 hunks)server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java(10 hunks)server/src/main/java/org/opensearch/index/mapper/MapperService.java(2 hunks)server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java(0 hunks)server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java(4 hunks)server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java(1 hunks)
💤 Files with no reviewable changes (1)
- server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java
🚧 Files skipped from review as they are similar to previous changes (4)
- server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java
- server/src/main/java/org/opensearch/index/IndexSettings.java
- CHANGELOG.md
- server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (20)
- GitHub Check: gradle-check
- GitHub Check: precommit (21, windows-2025, true)
- GitHub Check: precommit (25, ubuntu-24.04-arm)
- GitHub Check: precommit (21, ubuntu-latest)
- GitHub Check: precommit (25, macos-15)
- GitHub Check: precommit (25, macos-15-intel)
- GitHub Check: precommit (25, windows-latest)
- GitHub Check: precommit (21, ubuntu-24.04-arm)
- GitHub Check: precommit (21, macos-15)
- GitHub Check: precommit (25, ubuntu-latest)
- GitHub Check: precommit (21, windows-latest)
- GitHub Check: precommit (21, macos-15-intel)
- GitHub Check: assemble (25, windows-latest)
- GitHub Check: Analyze (java)
- GitHub Check: assemble (21, windows-latest)
- GitHub Check: assemble (25, ubuntu-24.04-arm)
- GitHub Check: assemble (21, ubuntu-24.04-arm)
- GitHub Check: assemble (21, ubuntu-latest)
- GitHub Check: assemble (25, ubuntu-latest)
- GitHub Check: detect-breaking-change
🔇 Additional comments (3)
server/src/main/java/org/opensearch/index/mapper/MapperService.java (2)
87-87: LGTM!The import is necessary for the
Collectors.toSet()operation used in thegetCompositeFieldTypes()method below.
694-696: Verify initialization safety, caller expectations, and method naming alignment.The method now filters to return only
StarTreeMapper.StarTreeFieldTypeinstances fromcompositeMappedFieldTypes:
Potential NPE risk: Verify that
getCompositeFieldTypes()is never called beforeinternalMerge()initializescompositeMappedFieldTypesat line 552. If called during early initialization phases,.stream()could fail on a null reference.Semantic narrowing: Confirm whether the method name
getCompositeFieldTypes()still accurately reflects its behavior. If other composite field type implementations exist or may be added, consider renaming togetStarTreeFieldTypes()or updating documentation to clarify the filtering behavior.Performance: If
getCompositeFieldTypes()is called frequently in hot paths, consider caching the filtered result to avoid recreating the set on each invocation.server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
691-705: Iteration refactors over current/old writer maps look correct and improve clarity.
getFlushingBytes()andgetPendingNumDocs()now iterate directly overliveIndexWriterDeletesMap.current.criteriaBasedIndexWriterMap.values(), summing per-child metrics before adding the accumulating writer’s values. This preserves behavior and is straightforward.getTragicException()now checks bothcurrentandoldchild writers for a tragic exception before falling back to the accumulating writer, which ensures group-specific failures are surfaced.ramBytesUsed()androllback()explicitly iterate over bothcurrentandoldwriters, and the use ofmapWriteLock.acquire()around theramBytesUsed()traversals is appropriate for a consistent snapshot.Overall, these loops are clear and consistent with the data structures being used; no issues from a correctness or concurrency standpoint.
Also applies to: 731-742, 757-772, 796-805
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java
Show resolved
Hide resolved
|
❌ Gradle check result for dbcae67: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
server/src/main/java/org/opensearch/index/mapper/MapperService.java (2)
694-696: Consider renaming method to reflect StarTree-specific filtering.The method name
getCompositeFieldTypes()suggests it returns all composite field types, but it now filters to return onlyStarTreeMapper.StarTreeFieldTypeinstances. This could mislead callers who expect all composite types.Consider renaming to
getStarTreeFieldTypes()or adding a clarifying comment:+ /** + * Returns only StarTree composite field types. + * For all composite field types, use getCompositeFieldTypesFromMapper(). + */ public Set<CompositeMappedFieldType> getCompositeFieldTypes() { return compositeMappedFieldTypes.stream() .filter(compositeMappedFieldType -> compositeMappedFieldType instanceof StarTreeMapper.StarTreeFieldType) .collect(Collectors.toSet()); }
694-696: Consider caching the filtered result to avoid repeated stream operations.The method creates a new stream, filters, and collects to a Set on every invocation. Since
compositeMappedFieldTypesonly changes during merge operations (line 552), the filtered result could be cached in a separate volatile field and updated alongsidecompositeMappedFieldTypes.Example optimization:
private volatile Set<CompositeMappedFieldType> compositeMappedFieldTypes; +private volatile Set<CompositeMappedFieldType> starTreeFieldTypes; // In internalMerge() after line 552: this.compositeMappedFieldTypes = getCompositeFieldTypesFromMapper(); +this.starTreeFieldTypes = compositeMappedFieldTypes.stream() + .filter(type -> type instanceof StarTreeMapper.StarTreeFieldType) + .collect(Collectors.toSet()); public Set<CompositeMappedFieldType> getCompositeFieldTypes() { - return compositeMappedFieldTypes.stream() - .filter(compositeMappedFieldType -> compositeMappedFieldType instanceof StarTreeMapper.StarTreeFieldType) - .collect(Collectors.toSet()); + return starTreeFieldTypes; }server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
475-489: Consider adding exponential backoff between retry attempts.The retry loop repeatedly calls
tryAcquire()without delay, creating a busy-wait pattern. With a default max retry of 100 (configurable up to 500), this can waste significant CPU cycles under contention.A past review comment suggested adding exponential backoff, but the current code doesn't implement it. Consider adding a small delay (e.g.,
Thread.sleep(Math.min(1L << counter, 100L))) between attempts, with appropriateInterruptedExceptionhandling.int counter = 0; while ((current == null || current.isClosed()) && counter < maxRetryOnLookupMapAcquisitionException) { // This function acquires a first read lock on a map which does not have any write lock present. Current keeps // on getting rotated during refresh, so there will be one current on which read lock can be obtained. // Validate that no write lock is applied on the map and the map is not closed. Idea here is write lock was // never applied on this map as write lock gets only during closing time. We are doing this instead of acquire, // because acquire can also apply a read lock in case refresh completed and map is closed. current = this.current.mapReadLock.tryAcquire(); if (current != null && current.isClosed() == true) { current.mapReadLock.close(); current = null; } + if (current == null && counter < maxRetryOnLookupMapAcquisitionException - 1) { + try { + Thread.sleep(Math.min(1L << counter, 100L)); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new LookupMapLockAcquisitionException(shardId, "Interrupted during retry", e); + } + } ++counter; }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
CHANGELOG.md(1 hunks)server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java(1 hunks)server/src/main/java/org/opensearch/index/IndexSettings.java(1 hunks)server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java(10 hunks)server/src/main/java/org/opensearch/index/mapper/MapperService.java(2 hunks)server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java(0 hunks)server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java(4 hunks)server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java(1 hunks)
💤 Files with no reviewable changes (1)
- server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java
🚧 Files skipped from review as they are similar to previous changes (2)
- CHANGELOG.md
- server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (20)
- GitHub Check: gradle-check
- GitHub Check: assemble (21, ubuntu-24.04-arm)
- GitHub Check: assemble (25, windows-latest)
- GitHub Check: assemble (25, ubuntu-latest)
- GitHub Check: assemble (25, ubuntu-24.04-arm)
- GitHub Check: assemble (21, ubuntu-latest)
- GitHub Check: assemble (21, windows-latest)
- GitHub Check: Analyze (java)
- GitHub Check: precommit (25, windows-latest)
- GitHub Check: precommit (25, ubuntu-24.04-arm)
- GitHub Check: precommit (21, windows-2025, true)
- GitHub Check: precommit (25, macos-15)
- GitHub Check: precommit (21, ubuntu-latest)
- GitHub Check: precommit (21, ubuntu-24.04-arm)
- GitHub Check: precommit (21, macos-15-intel)
- GitHub Check: precommit (25, macos-15-intel)
- GitHub Check: precommit (21, macos-15)
- GitHub Check: precommit (25, ubuntu-latest)
- GitHub Check: precommit (21, windows-latest)
- GitHub Check: detect-breaking-change
🔇 Additional comments (9)
server/src/main/java/org/opensearch/index/IndexSettings.java (1)
499-506: LGTM! Increased retry limits for lock acquisition.The changes to
INDEX_MAX_RETRY_ON_LOOKUP_MAP_LOCK_ACQUISITION_EXCEPTIONappropriately raise the default from 15 to 100 and the maximum from 100 to 500, providing better tolerance for lock contention scenarios during index writes with context-aware grouping criteria.server/src/main/java/org/opensearch/index/mapper/MapperService.java (2)
87-87: LGTM!The
Collectorsimport is necessary for the stream operations added ingetCompositeFieldTypes().
694-696: Verify that filtering to StarTree types only doesn't break existing functionality.This change narrows the return value to only
StarTreeMapper.StarTreeFieldTypeinstances. Ensure that:
- Other composite field type implementations (if any) are correctly handled elsewhere
- The lookup sets (
fieldsPartOfCompositeMappings,nestedFieldsPartOfCompositeMappings) built from allcompositeMappedFieldTypesremain sufficient for other composite types- External callers of
getCompositeFieldTypes()andisCompositeIndexPresent()are not expecting all composite field typesTo verify: Search the codebase for other
CompositeMappedFieldTypeimplementations, all call sites ofgetCompositeFieldTypes()andisCompositeIndexPresent(), and confirm how the filtered results are used downstream.server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java (1)
730-730: LGTM: Retry logic correctly moved to lower layer.The removal of the exception-driven retry loop simplifies the bulk action layer. Retry handling for lookup map lock acquisition is now managed within
CompositeIndexWriter.computeIndexWriterIfAbsentForCriteria(with boundedtryAcquireloops and a configurable max retry parameter), which provides better encapsulation.server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (3)
497-502: LGTM: Finally block correctly handles lock cleanup.The finally block now uses
assertfor theisHeldByCurrentThread()check, which addresses the previous concern aboutUnsupportedOperationExceptionwhen assertions are disabled. In production, the lock is unconditionally closed on error; in tests/debug builds, ownership is validated.
692-693: LGTM: Improved encapsulation in iteration patterns.The refactoring to iterate directly over
DisposableIndexWritervalues (instead of extracting and operating onIndexWriterinstances) improves encapsulation and reduces coupling. The consistent pattern acrossgetFlushingBytes(),getPendingNumDocs(),getTragicException(),ramBytesUsed(), androllback()enhances maintainability.Also applies to: 703-704, 732-743, 759-772, 797-806
210-210: Verify that visibility relaxations don't expose internal state inappropriately.The removal of
finalmodifiers fromCriteriaBasedIndexWriterLookup,CriteriaBasedWriterLock, andLiveIndexWriterDeletesMap, along with wideningmapReadLockto package-private and makingcurrentnon-final, enables mocking in tests.While these changes support testing, ensure they don't inadvertently expose internal state or allow unintended subclassing or mutation by other classes in the package. Specifically, verify that:
mapReadLock(now package-private) is not accessed outside of test and core composite writer code- The inner classes cannot be extended by unintended classes in the same package
current(now non-final) is not reassigned outside controlled contextsserver/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java (2)
72-77: LGTM: Test calls updated consistently with new signature.The calls to
computeIndexWriterIfAbsentForCriteriacorrectly pass the newMAX_NUMBER_OF_RETRIESparameter, maintaining existing test coverage while adapting to the updated signature.Also applies to: 141-146, 197-202
44-46: LGTM: New test provides good coverage of retry exhaustion.The new
testMaxRetryCountWhenWriteLockDuringIndexingtest effectively validates the bounded retry behavior by:
- Mocking lock acquisition to always fail (return
null)- Asserting that
LookupMapLockAcquisitionExceptionis thrown after exhausting retries- Verifying via Mockito that
tryAcquire()is called exactlyMAX_NUMBER_OF_RETRIEStimesThis complements the existing integration test at line 179 and provides focused unit-level coverage of the retry mechanism.
Also applies to: 208-227
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
server/src/main/java/org/opensearch/index/mapper/MapperService.java (2)
694-696: Consider renaming method to reflect StarTree-specific filtering.The method name
getCompositeFieldTypes()suggests it returns all composite field types, but it now filters to return onlyStarTreeMapper.StarTreeFieldTypeinstances. This could mislead callers who expect all composite types.Consider renaming to
getStarTreeFieldTypes()or adding a clarifying comment:+ /** + * Returns only StarTree composite field types. + * For all composite field types, use getCompositeFieldTypesFromMapper(). + */ public Set<CompositeMappedFieldType> getCompositeFieldTypes() { return compositeMappedFieldTypes.stream() .filter(compositeMappedFieldType -> compositeMappedFieldType instanceof StarTreeMapper.StarTreeFieldType) .collect(Collectors.toSet()); }
694-696: Consider caching the filtered result to avoid repeated stream operations.The method creates a new stream, filters, and collects to a Set on every invocation. Since
compositeMappedFieldTypesonly changes during merge operations (line 552), the filtered result could be cached in a separate volatile field and updated alongsidecompositeMappedFieldTypes.Example optimization:
private volatile Set<CompositeMappedFieldType> compositeMappedFieldTypes; +private volatile Set<CompositeMappedFieldType> starTreeFieldTypes; // In internalMerge() after line 552: this.compositeMappedFieldTypes = getCompositeFieldTypesFromMapper(); +this.starTreeFieldTypes = compositeMappedFieldTypes.stream() + .filter(type -> type instanceof StarTreeMapper.StarTreeFieldType) + .collect(Collectors.toSet()); public Set<CompositeMappedFieldType> getCompositeFieldTypes() { - return compositeMappedFieldTypes.stream() - .filter(compositeMappedFieldType -> compositeMappedFieldType instanceof StarTreeMapper.StarTreeFieldType) - .collect(Collectors.toSet()); + return starTreeFieldTypes; }server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
475-489: Consider adding exponential backoff between retry attempts.The retry loop repeatedly calls
tryAcquire()without delay, creating a busy-wait pattern. With a default max retry of 100 (configurable up to 500), this can waste significant CPU cycles under contention.A past review comment suggested adding exponential backoff, but the current code doesn't implement it. Consider adding a small delay (e.g.,
Thread.sleep(Math.min(1L << counter, 100L))) between attempts, with appropriateInterruptedExceptionhandling.int counter = 0; while ((current == null || current.isClosed()) && counter < maxRetryOnLookupMapAcquisitionException) { // This function acquires a first read lock on a map which does not have any write lock present. Current keeps // on getting rotated during refresh, so there will be one current on which read lock can be obtained. // Validate that no write lock is applied on the map and the map is not closed. Idea here is write lock was // never applied on this map as write lock gets only during closing time. We are doing this instead of acquire, // because acquire can also apply a read lock in case refresh completed and map is closed. current = this.current.mapReadLock.tryAcquire(); if (current != null && current.isClosed() == true) { current.mapReadLock.close(); current = null; } + if (current == null && counter < maxRetryOnLookupMapAcquisitionException - 1) { + try { + Thread.sleep(Math.min(1L << counter, 100L)); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new LookupMapLockAcquisitionException(shardId, "Interrupted during retry", e); + } + } ++counter; }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
CHANGELOG.md(1 hunks)server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java(1 hunks)server/src/main/java/org/opensearch/index/IndexSettings.java(1 hunks)server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java(10 hunks)server/src/main/java/org/opensearch/index/mapper/MapperService.java(2 hunks)server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java(0 hunks)server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java(4 hunks)server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java(1 hunks)
💤 Files with no reviewable changes (1)
- server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java
🚧 Files skipped from review as they are similar to previous changes (2)
- CHANGELOG.md
- server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (20)
- GitHub Check: gradle-check
- GitHub Check: assemble (21, ubuntu-24.04-arm)
- GitHub Check: assemble (25, windows-latest)
- GitHub Check: assemble (25, ubuntu-latest)
- GitHub Check: assemble (25, ubuntu-24.04-arm)
- GitHub Check: assemble (21, ubuntu-latest)
- GitHub Check: assemble (21, windows-latest)
- GitHub Check: Analyze (java)
- GitHub Check: precommit (25, windows-latest)
- GitHub Check: precommit (25, ubuntu-24.04-arm)
- GitHub Check: precommit (21, windows-2025, true)
- GitHub Check: precommit (25, macos-15)
- GitHub Check: precommit (21, ubuntu-latest)
- GitHub Check: precommit (21, ubuntu-24.04-arm)
- GitHub Check: precommit (21, macos-15-intel)
- GitHub Check: precommit (25, macos-15-intel)
- GitHub Check: precommit (21, macos-15)
- GitHub Check: precommit (25, ubuntu-latest)
- GitHub Check: precommit (21, windows-latest)
- GitHub Check: detect-breaking-change
🔇 Additional comments (9)
server/src/main/java/org/opensearch/index/IndexSettings.java (1)
499-506: LGTM! Increased retry limits for lock acquisition.The changes to
INDEX_MAX_RETRY_ON_LOOKUP_MAP_LOCK_ACQUISITION_EXCEPTIONappropriately raise the default from 15 to 100 and the maximum from 100 to 500, providing better tolerance for lock contention scenarios during index writes with context-aware grouping criteria.server/src/main/java/org/opensearch/index/mapper/MapperService.java (2)
87-87: LGTM!The
Collectorsimport is necessary for the stream operations added ingetCompositeFieldTypes().
694-696: Verify that filtering to StarTree types only doesn't break existing functionality.This change narrows the return value to only
StarTreeMapper.StarTreeFieldTypeinstances. Ensure that:
- Other composite field type implementations (if any) are correctly handled elsewhere
- The lookup sets (
fieldsPartOfCompositeMappings,nestedFieldsPartOfCompositeMappings) built from allcompositeMappedFieldTypesremain sufficient for other composite types- External callers of
getCompositeFieldTypes()andisCompositeIndexPresent()are not expecting all composite field typesTo verify: Search the codebase for other
CompositeMappedFieldTypeimplementations, all call sites ofgetCompositeFieldTypes()andisCompositeIndexPresent(), and confirm how the filtered results are used downstream.server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java (1)
730-730: LGTM: Retry logic correctly moved to lower layer.The removal of the exception-driven retry loop simplifies the bulk action layer. Retry handling for lookup map lock acquisition is now managed within
CompositeIndexWriter.computeIndexWriterIfAbsentForCriteria(with boundedtryAcquireloops and a configurable max retry parameter), which provides better encapsulation.server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (3)
497-502: LGTM: Finally block correctly handles lock cleanup.The finally block now uses
assertfor theisHeldByCurrentThread()check, which addresses the previous concern aboutUnsupportedOperationExceptionwhen assertions are disabled. In production, the lock is unconditionally closed on error; in tests/debug builds, ownership is validated.
692-693: LGTM: Improved encapsulation in iteration patterns.The refactoring to iterate directly over
DisposableIndexWritervalues (instead of extracting and operating onIndexWriterinstances) improves encapsulation and reduces coupling. The consistent pattern acrossgetFlushingBytes(),getPendingNumDocs(),getTragicException(),ramBytesUsed(), androllback()enhances maintainability.Also applies to: 703-704, 732-743, 759-772, 797-806
210-210: Verify that visibility relaxations don't expose internal state inappropriately.The removal of
finalmodifiers fromCriteriaBasedIndexWriterLookup,CriteriaBasedWriterLock, andLiveIndexWriterDeletesMap, along with wideningmapReadLockto package-private and makingcurrentnon-final, enables mocking in tests.While these changes support testing, ensure they don't inadvertently expose internal state or allow unintended subclassing or mutation by other classes in the package. Specifically, verify that:
mapReadLock(now package-private) is not accessed outside of test and core composite writer code- The inner classes cannot be extended by unintended classes in the same package
current(now non-final) is not reassigned outside controlled contextsserver/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java (2)
72-77: LGTM: Test calls updated consistently with new signature.The calls to
computeIndexWriterIfAbsentForCriteriacorrectly pass the newMAX_NUMBER_OF_RETRIESparameter, maintaining existing test coverage while adapting to the updated signature.Also applies to: 141-146, 197-202
44-46: LGTM: New test provides good coverage of retry exhaustion.The new
testMaxRetryCountWhenWriteLockDuringIndexingtest effectively validates the bounded retry behavior by:
- Mocking lock acquisition to always fail (return
null)- Asserting that
LookupMapLockAcquisitionExceptionis thrown after exhausting retries- Verifying via Mockito that
tryAcquire()is called exactlyMAX_NUMBER_OF_RETRIEStimesThis complements the existing integration test at line 179 and provides focused unit-level coverage of the retry mechanism.
Also applies to: 208-227
|
❌ Gradle check result for 7ee65b1: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #20145 +/- ##
============================================
- Coverage 73.30% 73.20% -0.10%
+ Complexity 71732 71706 -26
============================================
Files 5793 5793
Lines 328056 328047 -9
Branches 47245 47243 -2
============================================
- Hits 240476 240162 -314
- Misses 68264 68596 +332
+ Partials 19316 19289 -27 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| if (current != null && current.isClosed() == true) { | ||
| current.mapReadLock.close(); | ||
| current = null; | ||
| } |
There was a problem hiding this comment.
Should we move this logic centrally in close(). Also didn't quite understand why the close operation done as a part of write lock acquisition doesn't handle this logic?
There was a problem hiding this comment.
This handles the scenario where try acquire succeded in obtaining the lock on the current writer but the map itself rotated and the writer got closed. In that case, we close the old writer as we retry to obtain lock again on the current. As this ensures the lock is correctly released on the old writer before we try acquiring lock on new writer.
| for (DisposableIndexWriter disposableIndexWriter : liveIndexWriterDeletesMap.old.criteriaBasedIndexWriterMap.values()) { | ||
| if (disposableIndexWriter.getIndexWriter().isOpen() == true) { | ||
| disposableIndexWriter.getIndexWriter().rollback(); |
There was a problem hiding this comment.
This might not be thread-safe for instance the index writer might be closed while we are doing a rollback?
|
❌ Gradle check result for 4f48472: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
462-465: Potential criteria lookup bug across current/old maps when resolving writers by UIDIn
LiveIndexWriterDeletesMap:String getCriteriaForDoc(BytesRef key) { return current.getCriteriaForDoc(key); }and in
CompositeIndexWriter.getIndexWriterForIdFromLookup:String criteria = getCriteriaForDoc(uid); ... DisposableIndexWriter disposableIndexWriter = indexWriterLookup.getIndexWriterForCriteria(criteria);
getIndexWriterForIdFromLookupis used for both:
getIndexWriterForIdFromCurrent(passingliveIndexWriterDeletesMap.current), andgetIndexWriterForIdFromOld(passingliveIndexWriterDeletesMap.old).But
getCriteriaForDoc(uid)always queries the current lookup, never theindexWriterLookuppassed in. Combined withputCriteriaForDoconly updating thecurrentlookup, this means:
- After a refresh, documents whose criteria were recorded in the map that is now
oldwill still have theircriteriastored inold.criteria, not in the newcurrentmap.getIndexWriterForIdFromOldacquires the old map’s read lock but looks up the criteria only incurrent, so it will often fail to find the criteria for pre‑refresh documents and returnnull, skipping the intended partial delete on the old child writer.A minimal local fix is to have the map consult both
currentandoldwhen resolving criteria:- String getCriteriaForDoc(BytesRef key) { - return current.getCriteriaForDoc(key); - } + String getCriteriaForDoc(BytesRef key) { + // Prefer the current map, but fall back to old for documents that were + // written before the last refresh and still live in the old map. + String criteria = current.getCriteriaForDoc(key); + if (criteria == null) { + criteria = old.getCriteriaForDoc(key); + } + return criteria; + }This keeps the
getIndexWriterForIdFromLookupAPI untouched while ensuring deletes/update clean‑up can still locate writers in both generations of the lookup map.Also applies to: 630-647, 655-657
🧹 Nitpick comments (4)
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (4)
210-215: Visibility & mutability changes for lookup/lock classes – consider tightening API surface
CriteriaBasedIndexWriterLookupis nowpublic,LiveIndexWriterDeletesMapis non‑final, andmapReadLockis package‑visible and no longerfinal. This all widens the surface for external or test code to interact with internal locking structures.If the only consumer is tests, consider:
- Making
CriteriaBasedIndexWriterLookuppackage‑private instead ofpublic.- Keeping
mapReadLockprivate finaland exposing it via a package‑private accessor (possibly annotated@VisibleForTesting).- Keeping
LiveIndexWriterDeletesMapfinalunless subclassing is required.This preserves invariants while still allowing test access.
Also applies to: 301-305, 406-409
466-471: Bounded lock‑acquisition retry logic looks sound; document lock‑lifecycle expectations for callersThe new
computeIndexWriterIfAbsentForCriteria(..., ShardId shardId, int maxRetryOnLookupMapAcquisitionException)plus thewhileloop usingmapReadLock.tryAcquire()correctly:
- Bounds retries by
maxRetryOnLookupMapAcquisitionException.- Avoids using a
CriteriaBasedIndexWriterLookupthat has been marked closed (and closes its read lock before retrying).- Ensures that on failure (
success == false) any acquired read lock is released, while on success the read lock remains held for the caller to release later.Two follow‑ups to keep this robust:
- Call‑site discipline: every successful call must be paired with a later
current.mapReadLock.close()(as done by the try‑with‑resources aroundgetLookupMap().getMapReadLock()in the indexing paths). It’s worth adding a short comment to this method indicating that the read lock remains held on the success path and must be closed by the caller.- Config validation: ensure
IndexSettings.getMaxRetryOnLookupMapAcquisitionException()never returns a negative value; a value of0is handled (immediateLookupMapLockAcquisitionException), but negative values would skip the loop and behave the same as0, which may surprise operators.Overall, the retry/error path and assertion‑only use of
isHeldByCurrentThread()avoid the earlierUnsupportedOperationExceptionrisk.Also applies to: 475-489, 498-503, 671-681
692-694: Flushing/pending stats only account for current child writers; consider including old for completeness
getFlushingBytes()andgetPendingNumDocs()now iterate only over:liveIndexWriterDeletesMap.current.criteriaBasedIndexWriterMap.values()and ignore
liveIndexWriterDeletesMap.old. This is consistent with the existing TODO ingetPendingNumDocs, but note that:
ramBytesUsed()andgetTragicException()do consider both current and old maps.- During refresh, old writers can still hold in‑flight bytes/docs that won’t be reflected in these metrics.
If you want these stats to reflect all outstanding work (not just the newest generation), consider also iterating over
liveIndexWriterDeletesMap.old.criteriaBasedIndexWriterMap.values()here, or at least documenting that these methods intentionally report only the current generation.Also applies to: 703-705
754-775:ramBytesUsedlocking strategy is correct but could share code for current/old loops
ramBytesUsed()now:
- Acquires
mapWriteLockoncurrentandoldin two separatetry (ReleasableLock ...)blocks.- Iterates
criteriaBasedIndexWriterMap.values()and sumsramBytesUsed()only for open writers.This is a solid pattern: taking the write lock avoids concurrent rotation/mutation while measuring, and filtering by
isOpen()avoids touching already‑closed writers.To reduce duplication and risk of future drift between the two blocks, consider extracting a small helper that sums RAM usage for a single
CriteriaBasedIndexWriterLookup, and call it for bothcurrentandold. Behaviour would remain unchanged.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
CHANGELOG.md(1 hunks)server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java(1 hunks)server/src/main/java/org/opensearch/index/IndexSettings.java(1 hunks)server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java(10 hunks)server/src/main/java/org/opensearch/index/mapper/ContextAwareGroupingFieldMapper.java(2 hunks)server/src/main/java/org/opensearch/index/mapper/MapperService.java(2 hunks)server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java(0 hunks)server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java(4 hunks)server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java(1 hunks)
💤 Files with no reviewable changes (1)
- server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java
🚧 Files skipped from review as they are similar to previous changes (2)
- CHANGELOG.md
- server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java
🧰 Additional context used
🧬 Code graph analysis (1)
server/src/main/java/org/opensearch/index/mapper/MapperService.java (1)
server/src/main/java/org/opensearch/search/aggregations/metrics/TopHitsAggregator.java (1)
Collectors(85-95)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (20)
- GitHub Check: gradle-check
- GitHub Check: precommit (21, windows-2025, true)
- GitHub Check: precommit (25, macos-15-intel)
- GitHub Check: precommit (25, ubuntu-24.04-arm)
- GitHub Check: precommit (21, ubuntu-24.04-arm)
- GitHub Check: precommit (21, macos-15)
- GitHub Check: precommit (25, macos-15)
- GitHub Check: precommit (25, windows-latest)
- GitHub Check: precommit (21, macos-15-intel)
- GitHub Check: precommit (25, ubuntu-latest)
- GitHub Check: precommit (21, windows-latest)
- GitHub Check: detect-breaking-change
- GitHub Check: precommit (21, ubuntu-latest)
- GitHub Check: assemble (25, ubuntu-24.04-arm)
- GitHub Check: assemble (21, windows-latest)
- GitHub Check: assemble (25, windows-latest)
- GitHub Check: assemble (25, ubuntu-latest)
- GitHub Check: assemble (21, ubuntu-latest)
- GitHub Check: assemble (21, ubuntu-24.04-arm)
- GitHub Check: Analyze (java)
🔇 Additional comments (9)
server/src/main/java/org/opensearch/index/mapper/ContextAwareGroupingFieldMapper.java (1)
186-199: LGTM - appropriate no-op overrides for context-aware field mapper.The empty implementations for
canDeriveSource()andderiveSource()correctly exempt this synthetic field from source derivation since it's not part of the ingested document. The Javadoc clearly explains the rationale.server/src/main/java/org/opensearch/index/IndexSettings.java (1)
499-506: Significant increase in retry defaults - ensure monitoring is in place.The default retry count increased from 15 to 100, and the maximum from 100 to 500. This addresses the indexing regression by allowing more retries during lock acquisition contention. However, under heavy contention, this could increase indexing latency.
Consider adding metrics/logging around retry counts so operators can monitor lock contention and tune this setting appropriately.
server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java (1)
106-106: LGTM - appropriate test constant for retry behavior verification.Using a smaller value (20) than the production default (100) is a reasonable choice to keep test execution times manageable while still validating retry behavior.
server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java (3)
44-46: LGTM - necessary imports for retry verification tests.
71-77: LGTM - updated to pass retry parameter.The call sites are correctly updated to pass
MAX_NUMBER_OF_RETRIESto match the new method signature.
208-227: Good test for validating bounded retry behavior.The test correctly verifies that:
LookupMapLockAcquisitionExceptionis thrown after retries are exhaustedtryAcquire()is called exactlyMAX_NUMBER_OF_RETRIEStimesNote: The direct field assignment pattern (
map.current = mock(...),map.current.mapReadLock = writerLock) relies on these fields having package-private visibility, which aligns with the visibility changes made inCompositeIndexWriter.server/src/main/java/org/opensearch/index/mapper/MapperService.java (1)
693-697: Verify the behavioral change of excluding non-StarTreeFieldType composite mappings.This change narrows
getCompositeFieldTypes()to return onlyStarTreeFieldTypeinstances, excluding any otherCompositeMappedFieldTypeimplementations (e.g.,ContextAwareGroupingFieldMapper). Ensure this is the intended behavior and that all callers of this method do not rely on receiving other composite field types.Additionally, consider caching the filtered result instead of streaming on each call if this method is invoked frequently.
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (2)
732-736: Including both current and old child writers ingetTragicExceptionis a good improvementThe updated
getTragicException()now scans bothcurrentandoldcriteriaBasedIndexWriterMap.values()before falling back to the accumulating writer. This avoids dropping tragic exceptions that occur in writers that have just rotated to the old map but haven’t yet been merged/closed.The additional
isOpen() == falseguard also prevents spurious reads while writers are still healthy. This change looks correct and improves observability of failures.Also applies to: 739-743
795-805: Rollback over both current and old child writers looks consistent with live‑map semanticsThe
rollback()implementation now explicitly iterates over:liveIndexWriterDeletesMap.current.criteriaBasedIndexWriterMap.values() liveIndexWriterDeletesMap.old.criteriaBasedIndexWriterMap.values()and calls
rollback()on open child writers before rolling back the accumulating writer and marking the composite as closed. This matches the two‑generationLiveIndexWriterDeletesMapmodel and ensures no child writer is left unrolled back.There is still inherent race potential if some other code closes a child writer concurrently, but given the error‑path nature of
rollback()and theisOpen()guard, this is acceptable and not a regression from prior behaviour.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
CHANGELOG.md (1)
99-104: Deduplicate entry and normalize PR link formatting.There is now a duplicate
EngineConfig.toBuilderfix (20105) and both new PR links (20105, 20145) lack the leading#, and one is missing a space before(. To keep the changelog clean and consistent with surrounding entries, drop the duplicate line and fix link formatting:- - Fix toBuilder method in EngineConfig to include mergedSegmentTransferTracker([#20105](https://github.com/opensearch-project/OpenSearch/pull/20105)) - - Fix toBuilder method in EngineConfig to include mergedSegmentTransferTracker([20105](https://github.com/opensearch-project/OpenSearch/pull/20105)) - - Fix indexing regression and bug fixes for grouping criteria. ([20145](https://github.com/opensearch-project/OpenSearch/pull/20145)) + - Fix toBuilder method in EngineConfig to include mergedSegmentTransferTracker ([#20105](https://github.com/opensearch-project/OpenSearch/pull/20105)) + - Fix indexing regression and bug fixes for grouping criteria ([#20145](https://github.com/opensearch-project/OpenSearch/pull/20145))
🧹 Nitpick comments (2)
server/src/main/java/org/opensearch/index/mapper/MapperService.java (1)
87-88: Clarify thatgetCompositeFieldTypesnow returns only StarTree-based composites.The method now filters
compositeMappedFieldTypesdown toStarTreeMapper.StarTreeFieldTypeinstances before returning. That’s fine if only StarTree-based composites are ever expected here, but the method name reads like it returns all composite field types.Consider either:
- Updating the Javadoc to state it only returns StarTree composite field types, or
- Introducing a more specific helper (e.g.,
getStarTreeFieldTypes) and delegating from existing callers,to avoid surprises if other composite index types are added later.
Also applies to: 693-697
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
210-215: Limit exposure of internal lookup/lock types if only used for testing.
CriteriaBasedIndexWriterLookupandCriteriaBasedWriterLockare nowpublic/non-final, andmapReadLockandLiveIndexWriterDeletesMap.currentare mutable, primarily to support tests that override these internals.If these types and fields are not meant for general external use, consider:
- Keeping them package-private instead of public, and/or
- Adding a brief comment (e.g., “visible for testing”) near the loosened visibility/mutability,
to signal that they’re internal implementation details and avoid accidental production coupling.
Also applies to: 301-381, 406-409
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
CHANGELOG.md(1 hunks)server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java(1 hunks)server/src/main/java/org/opensearch/index/IndexSettings.java(1 hunks)server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java(10 hunks)server/src/main/java/org/opensearch/index/mapper/ContextAwareGroupingFieldMapper.java(1 hunks)server/src/main/java/org/opensearch/index/mapper/MapperService.java(2 hunks)server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java(0 hunks)server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java(4 hunks)server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java(1 hunks)
💤 Files with no reviewable changes (1)
- server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java
🚧 Files skipped from review as they are similar to previous changes (2)
- server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java
- server/src/main/java/org/opensearch/index/mapper/ContextAwareGroupingFieldMapper.java
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (20)
- GitHub Check: gradle-check
- GitHub Check: precommit (25, ubuntu-24.04-arm)
- GitHub Check: precommit (21, windows-2025, true)
- GitHub Check: precommit (25, macos-15-intel)
- GitHub Check: precommit (25, macos-15)
- GitHub Check: precommit (21, ubuntu-latest)
- GitHub Check: precommit (21, macos-15-intel)
- GitHub Check: precommit (21, macos-15)
- GitHub Check: precommit (25, windows-latest)
- GitHub Check: precommit (21, windows-latest)
- GitHub Check: precommit (21, ubuntu-24.04-arm)
- GitHub Check: precommit (25, ubuntu-latest)
- GitHub Check: assemble (25, windows-latest)
- GitHub Check: assemble (25, ubuntu-24.04-arm)
- GitHub Check: assemble (25, ubuntu-latest)
- GitHub Check: assemble (21, ubuntu-24.04-arm)
- GitHub Check: Analyze (java)
- GitHub Check: detect-breaking-change
- GitHub Check: assemble (21, ubuntu-latest)
- GitHub Check: assemble (21, windows-latest)
🔇 Additional comments (4)
server/src/main/java/org/opensearch/index/IndexSettings.java (1)
499-506: Retry-setting bound changes look consistent with new lock-acquisition retry logic.Raising the default/max for
index.context_aware.max_retry_on_lookup_map_acquisition_exception(100 / 500) is consistent with delegating lock-acquisition retries intoCompositeIndexWriter. No structural issues seen with the updated bounds or flags.server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java (1)
717-733: Update retry behavior now only handles version conflicts.The
onCompletepath for updates now retries only onVersionConflictEngineException; special retry handling for lookup-map lock acquisition failures has been removed, which matches delegating that retry logic down into the index writer layer.No issues from a bulk action perspective.
server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java (1)
44-47: Tests correctly cover new lookup-map retry semantics.The updates to pass
MAX_NUMBER_OF_RETRIESintocomputeIndexWriterIfAbsentForCriteria, and the newtestMaxRetryCountWhenWriteLockDuringIndexingthat mocksmap.current/writerLockand assertstryAcquire()is called exactlyMAX_NUMBER_OF_RETRIES, provide good coverage of the bounded-retry and failure behavior.The direct field manipulation of
LiveIndexWriterDeletesMap.currentandmapReadLockdoes couple the test to internals, but it’s reasonable here to validate the retry loop.Also applies to: 72-78, 141-147, 195-203, 208-227
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
466-503: Bounded lock-acquisition retry and settings wiring look correct.The new
computeIndexWriterIfAbsentForCriteriainLiveIndexWriterDeletesMap:
- Uses a bounded loop over
tryAcquire()oncurrent.mapReadLock, checkingcurrent == null || current.isClosed()and retrying up tomaxRetryOnLookupMapAcquisitionException.- Throws
LookupMapLockAcquisitionExceptionwith the shard id once the retry budget is exhausted.- Ensures any acquired read lock is released on failure in the
finallyblock.
CompositeIndexWriter.computeIndexWriterIfAbsentForCriteriacorrectly threads through:
- The current
LiveIndexWriterDeletesMapinstance, andengineConfig.getIndexSettings().getMaxRetryOnLookupMapAcquisitionException(),so behavior is fully governed by the new index setting.
No correctness issues stand out in the new retry logic; just be aware that increasing the max to higher values will proportionally extend worst-case wait time before surfacing
LookupMapLockAcquisitionException.Also applies to: 671-681
|
❌ Gradle check result for 1c15758: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: RS146BIJAY <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
210-241: Lock leak in retry loop can block refresh and cause deadlocksThe new
LiveIndexWriterDeletesMap.computeIndexWriterIfAbsentForCriteriaimplementation acquires a read lock viathis.current.mapReadLock.tryAcquire()but never releases it on the success path:
- On each successful
tryAcquire(), the underlyingReentrantReadWriteLockread lock is taken.successis set totrueafter creating/looking up the writer, so thefinallyblock does not closecurrent.mapReadLock.- As a result, at least one read lock is left held on the old map’s
mapLock.During
beforeRefresh()you then do:liveIndexWriterDeletesMap = liveIndexWriterDeletesMap.buildTransitionMap(); try (Releasable ignore = liveIndexWriterDeletesMap.old.mapWriteLock.acquire(), ...) { ... }
old.mapWriteLock.acquire()needs all read locks on the old map to be released. Since the compute method never closes the read lock on success, this write-lock acquisition can block indefinitely, stalling refresh and rotation (and any logic that depends on it).The new tests exercise the failure path (write lock held /
tryAcquire()returning null) but do not cover the successful acquisition path or verify that the read lock is released. This bug is therefore not caught by the current tests.You can fix this by tracking the successfully acquired
CriteriaBasedWriterLockseparately and always closing it infinally, regardless of success/failure, while still preserving the bounded retry semantics. For example:@@ - DisposableIndexWriter computeIndexWriterIfAbsentForCriteria( - String criteria, - CheckedBiFunction<String, CriteriaBasedIndexWriterLookup, DisposableIndexWriter, IOException> indexWriterSupplier, - ShardId shardId, - int maxRetryOnLookupMapAcquisitionException - ) { - boolean success = false; - CriteriaBasedIndexWriterLookup current = null; - try { - int counter = 0; - while ((current == null || current.isClosed()) && counter < maxRetryOnLookupMapAcquisitionException) { - // This function acquires a first read lock on a map which does not have any write lock present. Current keeps - // on getting rotated during refresh, so there will be one current on which read lock can be obtained. - // Validate that no write lock is applied on the map and the map is not closed. Idea here is write lock was - // never applied on this map as write lock gets only during closing time. We are doing this instead of acquire, - // because acquire can also apply a read lock in case refresh completed and map is closed. - current = this.current.mapReadLock.tryAcquire(); - if (current != null && current.isClosed() == true) { - current.mapReadLock.close(); - current = null; - } - - ++counter; - } - - if (current == null || current.isClosed()) { - throw new LookupMapLockAcquisitionException(shardId, "Unable to obtain lock on the current Lookup map", null); - } - DisposableIndexWriter writer = current.computeIndexWriterIfAbsentForCriteria(criteria, indexWriterSupplier); - success = true; - return writer; - } finally { - if (success == false && current != null) { - assert current.mapReadLock.isHeldByCurrentThread() == true; - current.mapReadLock.close(); - } - } - } + DisposableIndexWriter computeIndexWriterIfAbsentForCriteria( + String criteria, + CheckedBiFunction<String, CriteriaBasedIndexWriterLookup, DisposableIndexWriter, IOException> indexWriterSupplier, + ShardId shardId, + int maxRetryOnLookupMapAcquisitionException + ) { + CriteriaBasedIndexWriterLookup current = null; + CriteriaBasedWriterLock acquiredLock = null; + try { + int counter = 0; + while ((current == null || current.isClosed()) && counter < maxRetryOnLookupMapAcquisitionException) { + // This function acquires a read lock on the current map. Current keeps getting rotated during refresh, + // so there will eventually be a map on which a read lock can be obtained. + final CriteriaBasedWriterLock candidateLock = this.current.mapReadLock; + final CriteriaBasedIndexWriterLookup lookup = candidateLock.tryAcquire(); + if (lookup == null) { + // Could not obtain the read lock due to a concurrent writer; retry. + ++counter; + continue; + } + if (lookup.isClosed()) { + // Map was already closed; release the lock and retry. + candidateLock.close(); + ++counter; + continue; + } + current = lookup; + acquiredLock = candidateLock; + break; + } + + if (current == null || current.isClosed()) { + throw new LookupMapLockAcquisitionException(shardId, "Unable to obtain lock on the current Lookup map", null); + } + + return current.computeIndexWriterIfAbsentForCriteria(criteria, indexWriterSupplier); + } finally { + if (acquiredLock != null) { + acquiredLock.close(); + } + } + }This preserves the bounded retry behavior and ensures every successful
tryAcquire()is matched with a correspondingclose(), sobeforeRefresh()can still obtain the write lock on the old map.You may also want to add a test that covers the successful acquisition path and asserts that
mapReadLock.close()is called exactly once per successfultryAcquire()to guard against regressions.Also applies to: 301-381, 406-417, 466-503
🧹 Nitpick comments (2)
server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java (1)
208-227: New max-retry test is precise and robust
testMaxRetryCountWhenWriteLockDuringIndexingcleanly validates that:
tryAcquire()is attempted exactlyMAX_NUMBER_OF_RETRIEStimes, and- a
LookupMapLockAcquisitionExceptionis thrown when the read lock cannot be acquired.Using a mocked
CriteriaBasedIndexWriterLookupandCriteriaBasedWriterLockto controltryAcquire()is a good way to isolate this behavior.If you want even stronger coverage, you could add a complementary test where
tryAcquire()succeeds after N failed attempts and assert thattryAcquire()is not called more than necessary.server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
692-705: Iteration refactors over writer maps look correctThe refactors to iterate directly over
criteriaBasedIndexWriterMap.values()for:
getFlushingBytesgetPendingNumDocsgetTragicException(both current and old maps)ramBytesUsed(with write locks held)rollback(for both current and old maps)are straightforward and maintain the original intent:
- Metrics and state are still aggregated across all relevant child writers.
ramBytesUsedandrollbackcorrectly guard map traversal with the write lock.- The checks on
isOpen()before callingramBytesUsed(),getTragicException(), orrollback()remain intact.No functional regressions spotted here.
If desired, you could factor the duplicated “iterate over writer map and apply X” pattern into a small helper to reduce repetition, but it’s not necessary for correctness.
Also applies to: 732-736, 739-743, 759-762, 767-770, 797-806
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
CHANGELOG.md(1 hunks)server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java(1 hunks)server/src/main/java/org/opensearch/index/IndexSettings.java(1 hunks)server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java(10 hunks)server/src/main/java/org/opensearch/index/mapper/ContextAwareGroupingFieldMapper.java(2 hunks)server/src/main/java/org/opensearch/index/mapper/MapperService.java(2 hunks)server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java(0 hunks)server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java(4 hunks)server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java(1 hunks)
💤 Files with no reviewable changes (1)
- server/src/test/java/org/opensearch/action/bulk/TransportShardBulkActionTests.java
🚧 Files skipped from review as they are similar to previous changes (3)
- server/src/main/java/org/opensearch/action/bulk/TransportShardBulkAction.java
- server/src/main/java/org/opensearch/index/mapper/MapperService.java
- CHANGELOG.md
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (20)
- GitHub Check: gradle-check
- GitHub Check: Analyze (java)
- GitHub Check: assemble (25, ubuntu-24.04-arm)
- GitHub Check: assemble (21, ubuntu-latest)
- GitHub Check: assemble (21, ubuntu-24.04-arm)
- GitHub Check: assemble (25, windows-latest)
- GitHub Check: detect-breaking-change
- GitHub Check: precommit (25, ubuntu-24.04-arm)
- GitHub Check: assemble (25, ubuntu-latest)
- GitHub Check: precommit (21, windows-2025, true)
- GitHub Check: precommit (25, windows-latest)
- GitHub Check: precommit (25, ubuntu-latest)
- GitHub Check: assemble (21, windows-latest)
- GitHub Check: precommit (25, macos-15)
- GitHub Check: precommit (21, ubuntu-24.04-arm)
- GitHub Check: precommit (25, macos-15-intel)
- GitHub Check: precommit (21, ubuntu-latest)
- GitHub Check: precommit (21, macos-15)
- GitHub Check: precommit (21, windows-latest)
- GitHub Check: precommit (21, macos-15-intel)
🔇 Additional comments (6)
server/src/main/java/org/opensearch/index/mapper/ContextAwareGroupingFieldMapper.java (2)
11-12: New imports are correctly scoped to source-derivation override
LeafReaderandXContentBuilderare only used in the newderiveSourceoverride, so adding these imports is appropriate and keeps dependencies minimal.
186-199: No-op overrides appropriately exclude CAS mapper from source derivationGiven
parseCreateFieldalready prevents this field from being ingested, overridingcanDeriveSourceandderiveSourceas no-ops cleanly keeps the Context Aware Segment mapper out of generic source validation/generation, matching the Javadoc intent.server/src/test/java/org/opensearch/index/engine/CriteriaBasedCompositeIndexWriterBaseTests.java (1)
106-106: Shared retry constant for tests looks goodExposing
MAX_NUMBER_OF_RETRIESas a test base constant is reasonable and keeps retry-related tests consistent; no issues from a correctness standpoint.server/src/main/java/org/opensearch/index/IndexSettings.java (1)
499-506: Retry setting bounds change is reasonable, but behavior depends on callerThe new
INDEX_MAX_RETRY_ON_LOOKUP_MAP_LOCK_ACQUISITION_EXCEPTIONrange (min 5, max 500, default 100) is fine and keeps retries bounded. The practical impact now fully depends on howgetMaxRetryOnLookupMapAcquisitionException()is honored inCompositeIndexWriter’s retry loop, which is reviewed separately.server/src/test/java/org/opensearch/index/engine/CompositeIndexWriterForAppendTests.java (1)
44-47: Test updates correctly track new retry-aware APIThe added Mockito imports and the updated calls to
computeIndexWriterIfAbsentForCriteria(..., new ShardId(...), MAX_NUMBER_OF_RETRIES)keep the tests in sync with the new method signature and with the shared retry constant from the base class. The wiring looks consistent across all affected tests.Also applies to: 72-77, 141-146, 197-203
server/src/main/java/org/opensearch/index/engine/CompositeIndexWriter.java (1)
671-681: Index settings are correctly wired into retry behaviorThe wrapper
computeIndexWriterIfAbsentForCriterianow forwards both the shard ID andengineConfig.getIndexSettings().getMaxRetryOnLookupMapAcquisitionException()into theLiveIndexWriterDeletesMapretry loop. This cleanly centralizes retry configuration inIndexSettingsand avoids hard‑coding retry counts in production code.
2 participants
Description
Fixing indexing regression and bug fixes for grouping criteria. For testing grouping criteria changes, enabled the grouping criteria on local and tested with setting criteria. Wil raise the changes for integ test enablement for CAS in a separate PR as that require decent changes in integ test as well.
Related Issues
#19919
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.
Summary by CodeRabbit
Bug Fixes
Tests
Documentation
✏️ Tip: You can customize this high-level summary in your review settings.