Updated nomenclature to differentiate between hot and warm tiering implementation. #17490

mayanksharma27 · 2025-02-28T14:01:51Z

Description

We currently only have tiering implementation for warm indices.
The current nomenclature for feature flags and settings are too generic and cant differentiate between hot and warm indices.
Since we want to implement tiering for hot indices as well in near future; we would want to update the nomenclature to be extensible and easy to understand, this PR present the nomenclature changes.

Related Issues

Resolves #[17489]

Meta Issue #13149

Check List

[NA] Functionality includes testing.
[NA ] API changes companion pull request created, if applicable.
[NA] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-02-28T15:17:10Z

❌ Gradle check result for e920df4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/cluster/routing/RoutingPool.java

Signed-off-by: Mayank Sharma <[email protected]>

mayanksharma27

Addressed comments from @gbbafna

Signed-off-by: Mayank Sharma <[email protected]>

…omments opensearch-project#17490 Signed-off-by: Mayank Sharma <[email protected]>

github-actions · 2025-03-06T10:19:02Z

❌ Gradle check result for d18bd6e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-06T12:08:55Z

✅ Gradle check result for 5552edd: SUCCESS

codecov · 2025-03-06T12:09:33Z

Codecov Report

Attention: Patch coverage is 69.56522% with 7 lines in your changes missing coverage. Please review.

Project coverage is 72.42%. Comparing base (2ee8660) to head (a32221e).
Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
.../opensearch/index/engine/NRTReplicationEngine.java	0.00%	0 Missing and 2 partials ⚠️
...in/java/org/opensearch/index/shard/IndexShard.java	0.00%	1 Missing and 1 partial ⚠️
...ing/allocation/allocator/RemoteShardsBalancer.java	0.00%	1 Missing ⚠️
...c/main/java/org/opensearch/index/IndexService.java	50.00%	0 Missing and 1 partial ⚠️
.../indices/replication/SegmentReplicationTarget.java	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #17490      +/-   ##
============================================
- Coverage     72.43%   72.42%   -0.02%     
- Complexity    65694    65728      +34     
============================================
  Files          5311     5311              
  Lines        304937   304933       -4     
  Branches      44226    44225       -1     
============================================
- Hits         220872   220836      -36     
- Misses        65912    65995      +83     
+ Partials      18153    18102      -51

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…ntation for warm and upcoming hot implementation commit 5552edd Author: Mayank Sharma <[email protected]> Date: Thu Mar 6 16:38:38 2025 +0530 Empty commit to rerun flaky test Signed-off-by: Mayank Sharma <[email protected]> commit d18bd6e Author: Mayank Sharma <[email protected]> Date: Thu Mar 6 14:25:33 2025 +0530 Ran spotlessapply Signed-off-by: Mayank Sharma <[email protected]> commit d5ac854 Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 19:59:05 2025 +0530 Empty commit to rerun flaky test Signed-off-by: Mayank Sharma <[email protected]> commit 44000ee Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 19:17:00 2025 +0530 Fixed failing UTs for ShardsTieringAllocationTests Signed-off-by: Mayank Sharma <[email protected]> commit 03a9305 Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 17:54:33 2025 +0530 Empty commit to rerun flaky test Signed-off-by: Mayank Sharma <[email protected]> commit 795936a Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 12:10:18 2025 +0530 Updated settings from "index.is.warm" to "index.warm" as part of PR comments opensearch-project#17490 Signed-off-by: Mayank Sharma <[email protected]> commit 724b934 Author: Mayank Sharma <[email protected]> Date: Tue Mar 4 17:09:19 2025 +0530 Addressed comments for PR opensearch-project#17490 Signed-off-by: Mayank Sharma <[email protected]> commit f1a249f Author: Mayank Sharma <[email protected]> Date: Fri Feb 28 19:02:24 2025 +0530 Updated nomenclature to differentiate between current tiering implementation for warm and upcoming hot implementation Signed-off-by: Mayank Sharma <[email protected]> Signed-off-by: Mayank Sharma <[email protected]>

…current tiering implementation for warm and upcoming hot implementation commit 9cdec49 Author: Mayank Sharma <[email protected]> Date: Thu Mar 6 18:04:16 2025 +0530 Updated nomenclature to differentiate between current tiering implementation for warm and upcoming hot implementation commit 5552edd Author: Mayank Sharma <[email protected]> Date: Thu Mar 6 16:38:38 2025 +0530 Empty commit to rerun flaky test Signed-off-by: Mayank Sharma <[email protected]> commit d18bd6e Author: Mayank Sharma <[email protected]> Date: Thu Mar 6 14:25:33 2025 +0530 Ran spotlessapply Signed-off-by: Mayank Sharma <[email protected]> commit d5ac854 Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 19:59:05 2025 +0530 Empty commit to rerun flaky test Signed-off-by: Mayank Sharma <[email protected]> commit 44000ee Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 19:17:00 2025 +0530 Fixed failing UTs for ShardsTieringAllocationTests Signed-off-by: Mayank Sharma <[email protected]> commit 03a9305 Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 17:54:33 2025 +0530 Empty commit to rerun flaky test Signed-off-by: Mayank Sharma <[email protected]> commit 795936a Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 12:10:18 2025 +0530 Updated settings from "index.is.warm" to "index.warm" as part of PR comments opensearch-project#17490 Signed-off-by: Mayank Sharma <[email protected]> commit 724b934 Author: Mayank Sharma <[email protected]> Date: Tue Mar 4 17:09:19 2025 +0530 Addressed comments for PR opensearch-project#17490 Signed-off-by: Mayank Sharma <[email protected]> commit f1a249f Author: Mayank Sharma <[email protected]> Date: Fri Feb 28 19:02:24 2025 +0530 Updated nomenclature to differentiate between current tiering implementation for warm and upcoming hot implementation Signed-off-by: Mayank Sharma <[email protected]> Signed-off-by: Mayank Sharma <[email protected]> Signed-off-by: Mayank Sharma <[email protected]>

github-actions · 2025-03-06T14:20:54Z

❌ Gradle check result for 3c323e2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

mayanksharma27 · 2025-03-06T20:09:05Z

Flaky integ tests failing - #11037

github-actions · 2025-03-10T05:10:41Z

❌ Gradle check result for 33f94d1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-12T08:56:17Z

❌ Gradle check result for e6e82ca: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-12T09:20:57Z

❌ Gradle check result for 33f94d1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…current tiering implementation for warm and upcoming hot implementation commit 9cdec49 Author: Mayank Sharma <[email protected]> Date: Thu Mar 6 18:04:16 2025 +0530 Updated nomenclature to differentiate between current tiering implementation for warm and upcoming hot implementation commit 5552edd Author: Mayank Sharma <[email protected]> Date: Thu Mar 6 16:38:38 2025 +0530 Empty commit to rerun flaky test Signed-off-by: Mayank Sharma <[email protected]> commit d18bd6e Author: Mayank Sharma <[email protected]> Date: Thu Mar 6 14:25:33 2025 +0530 Ran spotlessapply Signed-off-by: Mayank Sharma <[email protected]> commit d5ac854 Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 19:59:05 2025 +0530 Empty commit to rerun flaky test Signed-off-by: Mayank Sharma <[email protected]> commit 44000ee Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 19:17:00 2025 +0530 Fixed failing UTs for ShardsTieringAllocationTests Signed-off-by: Mayank Sharma <[email protected]> commit 03a9305 Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 17:54:33 2025 +0530 Empty commit to rerun flaky test Signed-off-by: Mayank Sharma <[email protected]> commit 795936a Author: Mayank Sharma <[email protected]> Date: Wed Mar 5 12:10:18 2025 +0530 Updated settings from "index.is.warm" to "index.warm" as part of PR comments opensearch-project#17490 Signed-off-by: Mayank Sharma <[email protected]> commit 724b934 Author: Mayank Sharma <[email protected]> Date: Tue Mar 4 17:09:19 2025 +0530 Addressed comments for PR opensearch-project#17490 Signed-off-by: Mayank Sharma <[email protected]> commit f1a249f Author: Mayank Sharma <[email protected]> Date: Fri Feb 28 19:02:24 2025 +0530 Updated nomenclature to differentiate between current tiering implementation for warm and upcoming hot implementation Signed-off-by: Mayank Sharma <[email protected]> Signed-off-by: Mayank Sharma <[email protected]> Signed-off-by: Mayank Sharma <[email protected]>

github-actions · 2025-03-12T21:55:10Z

❌ Gradle check result for a32221e: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-13T07:08:15Z

❌ Gradle check result for a32221e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-13T09:23:07Z

❌ Gradle check result for a32221e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Bukhtawar

Is there a difference between searchable snapshot index vs tiered warm index?

github-actions · 2025-03-17T07:20:38Z

✅ Gradle check result for a32221e: SUCCESS

gbbafna · 2025-03-17T08:07:17Z

Is there a difference between searchable snapshot index vs tiered warm index?

Yes, index settings are different for both of them . For tiered warm index , store.data_locality is set to partial , whereas for searchable snapshots index.store.type: remote_snapshot is used.

… current tiering implementation for warm and upcoming hot implementation (opensearch-project#17490) Signed-off-by: Mayank Sharma <[email protected]> Signed-off-by: Vinay Krishna Pudyodu <[email protected]>

opensearch-ci-bot mentioned this pull request Feb 28, 2025

[AUTOCUT] Gradle Check Flaky Test Report for DedicatedClusterSnapshotRestoreIT #15806

Open

gbbafna reviewed Mar 1, 2025

View reviewed changes

server/src/main/java/org/opensearch/cluster/routing/RoutingPool.java Outdated Show resolved Hide resolved

mayanksharma27 added a commit to mayanksharma27/OpenSearch that referenced this pull request Mar 4, 2025

Addressed comments for PR opensearch-project#17490

38cf2ad

Signed-off-by: Mayank Sharma <[email protected]>

mayanksharma27 commented Mar 4, 2025

View reviewed changes

mayanksharma27 added a commit to mayanksharma27/OpenSearch that referenced this pull request Mar 6, 2025

Addressed comments for PR opensearch-project#17490

724b934

Signed-off-by: Mayank Sharma <[email protected]>

mayanksharma27 added a commit to mayanksharma27/OpenSearch that referenced this pull request Mar 6, 2025

Updated settings from "index.is.warm" to "index.warm" as part of PR c…

795936a

…omments opensearch-project#17490 Signed-off-by: Mayank Sharma <[email protected]>

mayanksharma27 force-pushed the smynk-tiering-1 branch from be698df to d18bd6e Compare March 6, 2025 09:22

mayanksharma27 force-pushed the smynk-tiering-1 branch from 5552edd to 9cdec49 Compare March 6, 2025 13:22

mayanksharma27 force-pushed the smynk-tiering-1 branch from 9cdec49 to 3c323e2 Compare March 6, 2025 13:56

opensearch-ci-bot mentioned this pull request Mar 7, 2025

[AUTOCUT] Gradle Check Flaky Test Report for S3BlobContainerRetriesTests #17551

Open

mayanksharma27 force-pushed the smynk-tiering-1 branch from e6e82ca to 33f94d1 Compare March 12, 2025 09:13

mayanksharma27 force-pushed the smynk-tiering-1 branch from 33f94d1 to a32221e Compare March 12, 2025 19:51

opensearch-ci-bot mentioned this pull request Mar 12, 2025

[AUTOCUT] Gradle Check Flaky Test Report for RemotePrimaryRelocationIT #17364

Closed

Bukhtawar reviewed Mar 15, 2025

View reviewed changes

gbbafna approved these changes Mar 17, 2025

View reviewed changes

gbbafna merged commit e27cf58 into opensearch-project:main Mar 17, 2025
31 of 32 checks passed

opensearch-ci-bot mentioned this pull request Mar 17, 2025

[AUTOCUT] Gradle Check Flaky Test Report for S3BlobStoreRepositoryTests #14299

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated nomenclature to differentiate between hot and warm tiering implementation. #17490

Updated nomenclature to differentiate between hot and warm tiering implementation. #17490

mayanksharma27 commented Feb 28, 2025

github-actions bot commented Feb 28, 2025

mayanksharma27 left a comment

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

codecov bot commented Mar 6, 2025 •

edited

Loading

github-actions bot commented Mar 6, 2025

mayanksharma27 commented Mar 6, 2025

github-actions bot commented Mar 10, 2025

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 13, 2025

github-actions bot commented Mar 13, 2025

Bukhtawar left a comment

github-actions bot commented Mar 17, 2025

gbbafna commented Mar 17, 2025

Updated nomenclature to differentiate between hot and warm tiering implementation. #17490

Updated nomenclature to differentiate between hot and warm tiering implementation. #17490

Conversation

mayanksharma27 commented Feb 28, 2025

Description

Related Issues

Check List

github-actions bot commented Feb 28, 2025

mayanksharma27 left a comment

Choose a reason for hiding this comment

github-actions bot commented Mar 6, 2025

github-actions bot commented Mar 6, 2025

codecov bot commented Mar 6, 2025 • edited Loading

Codecov Report

github-actions bot commented Mar 6, 2025

mayanksharma27 commented Mar 6, 2025

github-actions bot commented Mar 10, 2025

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 13, 2025

github-actions bot commented Mar 13, 2025

Bukhtawar left a comment

Choose a reason for hiding this comment

github-actions bot commented Mar 17, 2025

gbbafna commented Mar 17, 2025

codecov bot commented Mar 6, 2025 •

edited

Loading