Skip to content

Add Metrics For The Merged Segment Warmer Flow #18929

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 30 commits into from
Oct 14, 2025
Merged

Add Metrics For The Merged Segment Warmer Flow #18929

merged 30 commits into from
Oct 14, 2025

Conversation

kh3ra
Copy link
Contributor

@kh3ra kh3ra commented Aug 5, 2025 •
edited
Loading

Description

This PR introduces comprehensive metrics for monitoring merged segment warming operations, building upon the work from PR#18683, Issue#17528. These metrics provide insights into warming operations, data transfer, and performance characteristics.

Metrics Added

Cumulative Metrics

Added the following metrics under merges.warmer:

  • total_invocations_count: Tracks total number of warm operations
  • total_time_millis: Total time spent in warming operations
  • total_failure_count: Number of failed warming attempts
  • total_bytes_sent: Total data volume uploaded
  • total_bytes_received: Total data volume downloaded
  • total_send_time: Cumulative time spent in uploads
  • total_receive_time: Cumulative time spent in downloads

Point-in-Time Metrics

  • ongoing_count: Current number of active warming operations

API Changes

/_nodes/stats/indices/merge

{
  "_nodes": {
    "total": 4,
    "successful": 4,
    "failed": 0
  },
  "cluster_name": "opensearch-cluster",
  "nodes": {
    "n3zTQMDcQYOca2jeSt9ALA": {
      "timestamp": 1757059591666,
      "name": "opensearch-node2",
      "transport_address": "172.18.0.3:9300",
      "host": "172.18.0.3",
      "ip": "172.18.0.3:9300",
      "roles": [
        "data",
        "ingest",
        "remote_cluster_client"
      ],
      "attributes": {
        "remote_store.repository.state.settings.location": "/usr/share/opensearch/remote/state",
        "remote_store.repository.translog.type": "fs",
        "remote_store.repository.state.type": "fs",
        "remote_store.state.repository": "state",
        "remote_store.repository.segment.type": "fs",
        "shard_indexing_pressure_enabled": "true",
        "remote_store.repository.segment.settings.location": "/usr/share/opensearch/remote/segment",
        "remote_store.segment.repository": "segment",
        "remote_store.translog.repository": "translog",
        "remote_store.repository.translog.settings.location": "/usr/share/opensearch/remote/translog"
      },
      "indices": {
        "merges": {
          "current": 0,
          "current_docs": 0,
          "current_size_in_bytes": 0,
          "total": 1,
          "total_time_in_millis": 325,
          "total_docs": 37,
          "total_size_in_bytes": 316031,
          "total_stopped_time_in_millis": 0,
          "total_throttled_time_in_millis": 0,
          "total_auto_throttle_in_bytes": 20971520,
          "unreferenced_file_cleanups_performed": 0,
          "warmer": {
            "total_invocations_count": 1,
            "total_time_millis": 184,
            "total_failure_count": 0,
            "total_bytes_sent": 158956,
            "total_bytes_received": 0,
            "total_send_time_millis": 74,
            "total_receive_time_millis": 0,
            "ongoing_count": 0
          }
        }
      }
    },
    "2-by2PpBRRq7Z39w8Lvstw": {
      "timestamp": 1757059591669,
      "name": "opensearch-node3",
      "transport_address": "172.18.0.5:9300",
      "host": "172.18.0.5",
      "ip": "172.18.0.5:9300",
      "roles": [
        "data",
        "ingest",
        "remote_cluster_client"
      ],
      "attributes": {
        "remote_store.repository.state.settings.location": "/usr/share/opensearch/remote/state",
        "remote_store.repository.translog.type": "fs",
        "remote_store.repository.state.type": "fs",
        "remote_store.state.repository": "state",
        "remote_store.repository.segment.type": "fs",
        "shard_indexing_pressure_enabled": "true",
        "remote_store.repository.segment.settings.location": "/usr/share/opensearch/remote/segment",
        "remote_store.segment.repository": "segment",
        "remote_store.translog.repository": "translog",
        "remote_store.repository.translog.settings.location": "/usr/share/opensearch/remote/translog"
      },
      "indices": {
        "merges": {
          "current": 0,
          "current_docs": 0,
          "current_size_in_bytes": 0,
          "total": 0,
          "total_time_in_millis": 0,
          "total_docs": 0,
          "total_size_in_bytes": 0,
          "total_stopped_time_in_millis": 0,
          "total_throttled_time_in_millis": 0,
          "total_auto_throttle_in_bytes": 0,
          "unreferenced_file_cleanups_performed": 0,
          "warmer": {
            "total_invocations_count": 0,
            "total_time_millis": 0,
            "total_failure_count": 0,
            "total_bytes_sent": 0,
            "total_bytes_received": 158956,
            "total_send_time_millis": 0,
            "total_receive_time_millis": 88,
            "ongoing_count": 0
          }
        }
      }
    },
    "SvhMX0XxSJak03cuzK3XJA": {
      "timestamp": 1757059591667,
      "name": "opensearch-node1",
      "transport_address": "172.18.0.4:9300",
      "host": "172.18.0.4",
      "ip": "172.18.0.4:9300",
      "roles": [
        "data",
        "ingest",
        "remote_cluster_client"
      ],
      "attributes": {
        "remote_store.repository.state.settings.location": "/usr/share/opensearch/remote/state",
        "remote_store.repository.translog.type": "fs",
        "remote_store.repository.state.type": "fs",
        "remote_store.state.repository": "state",
        "remote_store.repository.segment.type": "fs",
        "shard_indexing_pressure_enabled": "true",
        "remote_store.repository.segment.settings.location": "/usr/share/opensearch/remote/segment",
        "remote_store.segment.repository": "segment",
        "remote_store.translog.repository": "translog",
        "remote_store.repository.translog.settings.location": "/usr/share/opensearch/remote/translog"
      },
      "indices": {
        "merges": {
          "current": 0,
          "current_docs": 0,
          "current_size_in_bytes": 0,
          "total": 0,
          "total_time_in_millis": 0,
          "total_docs": 0,
          "total_size_in_bytes": 0,
          "total_stopped_time_in_millis": 0,
          "total_throttled_time_in_millis": 0,
          "total_auto_throttle_in_bytes": 0,
          "unreferenced_file_cleanups_performed": 0,
          "warmer": {
            "total_invocations_count": 0,
            "total_time_millis": 0,
            "total_failure_count": 0,
            "total_bytes_sent": 0,
            "total_bytes_received": 0,
            "total_send_time_millis": 0,
            "total_receive_time_millis": 0,
            "ongoing_count": 0
          }
        }
      }
    }
}

/_stats/merge

{
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "_all": {
    "primaries": {
      "merges": {
        "current": 0,
        "current_docs": 0,
        "current_size_in_bytes": 0,
        "total": 19,
        "total_time_in_millis": 5299,
        "total_docs": 2524,
        "total_size_in_bytes": 12307726,
        "total_stopped_time_in_millis": 0,
        "total_throttled_time_in_millis": 0,
        "total_auto_throttle_in_bytes": 20971520,
        "unreferenced_file_cleanups_performed": 0,
        "warmer": {
          "total_invocations_count": 19,
          "total_time_millis": 3589,
          "total_failure_count": 0,
          "total_bytes_sent": 9071479,
          "total_bytes_received": 0,
          "total_send_time_millis": 1271,
          "total_receive_time_millis": 0,
          "ongoing_count": 0
        }
      }
    },
    "total": {
      "merges": {
        "current": 0,
        "current_docs": 0,
        "current_size_in_bytes": 0,
        "total": 19,
        "total_time_in_millis": 5299,
        "total_docs": 2524,
        "total_size_in_bytes": 12307726,
        "total_stopped_time_in_millis": 0,
        "total_throttled_time_in_millis": 0,
        "total_auto_throttle_in_bytes": 20971520,
        "unreferenced_file_cleanups_performed": 0,
        "warmer": {
          "total_invocations_count": 19,
          "total_time_millis": 3589,
          "total_failure_count": 0,
          "total_bytes_sent": 9071479,
          "total_bytes_received": 17725826,
          "total_send_time_millis": 1271,
          "total_receive_time_millis": 4018,
          "ongoing_count": 0
        }
      }
    }
  },
  "indices": {
    "test-index": {
      "uuid": "VxNpGxAESxWv4d6bzUm86w",
      "primaries": {
        "merges": {
          "current": 0,
          "current_docs": 0,
          "current_size_in_bytes": 0,
          "total": 19,
          "total_time_in_millis": 5299,
          "total_docs": 2524,
          "total_size_in_bytes": 12307726,
          "total_stopped_time_in_millis": 0,
          "total_throttled_time_in_millis": 0,
          "total_auto_throttle_in_bytes": 20971520,
          "unreferenced_file_cleanups_performed": 0,
          "warmer": {
            "total_invocations_count": 19,
            "total_time_millis": 3589,
            "total_failure_count": 0,
            "total_bytes_sent": 9071479,
            "total_bytes_received": 0,
            "total_send_time_millis": 1271,
            "total_receive_time_millis": 0,
            "ongoing_count": 0
          }
        }
      },
      "total": {
        "merges": {
          "current": 0,
          "current_docs": 0,
          "current_size_in_bytes": 0,
          "total": 19,
          "total_time_in_millis": 5299,
          "total_docs": 2524,
          "total_size_in_bytes": 12307726,
          "total_stopped_time_in_millis": 0,
          "total_throttled_time_in_millis": 0,
          "total_auto_throttle_in_bytes": 20971520,
          "unreferenced_file_cleanups_performed": 0,
          "warmer": {
            "total_invocations_count": 19,
            "total_time_millis": 3589,
            "total_failure_count": 0,
            "total_bytes_sent": 9071479,
            "total_bytes_received": 17725826,
            "total_send_time_millis": 1271,
            "total_receive_time_millis": 4018,
            "ongoing_count": 0
          }
        }
      }
    }
  }
}

/_stats/merge?pretty&level=shards

{
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "_all": {
    "primaries": {
      "merges": {
        "current": 0,
        "current_docs": 0,
        "current_size_in_bytes": 0,
        "total": 23,
        "total_time_in_millis": 6323,
        "total_docs": 3428,
        "total_size_in_bytes": 16071365,
        "total_stopped_time_in_millis": 0,
        "total_throttled_time_in_millis": 0,
        "total_auto_throttle_in_bytes": 20971520,
        "unreferenced_file_cleanups_performed": 0,
        "warmer": {
          "total_invocations_count": 23,
          "total_time_millis": 4217,
          "total_failure_count": 0,
          "total_bytes_sent": 12107729,
          "total_bytes_received": 0,
          "total_send_time_millis": 1459,
          "total_receive_time_millis": 0,
          "ongoing_count": 0
        }
      }
    },
    "total": {
      "merges": {
        "current": 0,
        "current_docs": 0,
        "current_size_in_bytes": 0,
        "total": 23,
        "total_time_in_millis": 6323,
        "total_docs": 3428,
        "total_size_in_bytes": 16071365,
        "total_stopped_time_in_millis": 0,
        "total_throttled_time_in_millis": 0,
        "total_auto_throttle_in_bytes": 20971520,
        "unreferenced_file_cleanups_performed": 0,
        "warmer": {
          "total_invocations_count": 23,
          "total_time_millis": 4217,
          "total_failure_count": 0,
          "total_bytes_sent": 12107729,
          "total_bytes_received": 29833555,
          "total_send_time_millis": 1459,
          "total_receive_time_millis": 6444,
          "ongoing_count": 0
        }
      }
    }
  },
  "indices": {
    "test-index": {
      "uuid": "VxNpGxAESxWv4d6bzUm86w",
      "primaries": {
        "merges": {
          "current": 0,
          "current_docs": 0,
          "current_size_in_bytes": 0,
          "total": 23,
          "total_time_in_millis": 6323,
          "total_docs": 3428,
          "total_size_in_bytes": 16071365,
          "total_stopped_time_in_millis": 0,
          "total_throttled_time_in_millis": 0,
          "total_auto_throttle_in_bytes": 20971520,
          "unreferenced_file_cleanups_performed": 0,
          "warmer": {
            "total_invocations_count": 23,
            "total_time_millis": 4217,
            "total_failure_count": 0,
            "total_bytes_sent": 12107729,
            "total_bytes_received": 0,
            "total_send_time_millis": 1459,
            "total_receive_time_millis": 0,
            "ongoing_count": 0
          }
        }
      },
      "total": {
        "merges": {
          "current": 0,
          "current_docs": 0,
          "current_size_in_bytes": 0,
          "total": 23,
          "total_time_in_millis": 6323,
          "total_docs": 3428,
          "total_size_in_bytes": 16071365,
          "total_stopped_time_in_millis": 0,
          "total_throttled_time_in_millis": 0,
          "total_auto_throttle_in_bytes": 20971520,
          "unreferenced_file_cleanups_performed": 0,
          "warmer": {
            "total_invocations_count": 23,
            "total_time_millis": 4217,
            "total_failure_count": 0,
            "total_bytes_sent": 12107729,
            "total_bytes_received": 29833555,
            "total_send_time_millis": 1459,
            "total_receive_time_millis": 6444,
            "ongoing_count": 0
          }
        }
      },
      "shards": {
        "0": [
          {
            "routing": {
              "state": "STARTED",
              "primary": true,
              "node": "n3zTQMDcQYOca2jeSt9ALA",
              "relocating_node": null
            },
            "merges": {
              "current": 0,
              "current_docs": 0,
              "current_size_in_bytes": 0,
              "total": 23,
              "total_time_in_millis": 6323,
              "total_docs": 3428,
              "total_size_in_bytes": 16071365,
              "total_stopped_time_in_millis": 0,
              "total_throttled_time_in_millis": 0,
              "total_auto_throttle_in_bytes": 20971520,
              "unreferenced_file_cleanups_performed": 0,
              "warmer": {
                "total_invocations_count": 23,
                "total_time_millis": 4217,
                "total_failure_count": 0,
                "total_bytes_sent": 12107729,
                "total_bytes_received": 0,
                "total_send_time_millis": 1459,
                "total_receive_time_millis": 0,
                "ongoing_count": 0
              }
            },
            "commit": {
              "id": "8rgsCwSDGwTMVTE7UXlz2w==",
              "generation": 3,
              "user_data": {
                "min_retained_seq_no": "0",
                "translog_uuid": "mM3XtVkzRBmZMbK7YktNpg",
                "local_checkpoint": "-1",
                "history_uuid": "hNsSVAB0RLWewv9HSWiI9A",
                "max_seq_no": "-1",
                "max_unsafe_auto_id_timestamp": "-1"
              },
              "num_docs": 0
            },
            "seq_no": {
              "max_seq_no": 799,
              "local_checkpoint": 799,
              "global_checkpoint": 799
            },
            "retention_leases": {
              "primary_term": 1,
              "version": 6,
              "leases": [
                {
                  "id": "peer_recovery/n3zTQMDcQYOca2jeSt9ALA",
                  "retaining_seq_no": 797,
                  "timestamp": 1757059730737,
                  "source": "peer recovery"
                }
              ]
            },
            "shard_path": {
              "state_path": "/usr/share/opensearch/data/nodes/0",
              "data_path": "/usr/share/opensearch/data/nodes/0",
              "is_custom_data_path": false
            }
          },
          {
            "routing": {
              "state": "STARTED",
              "primary": false,
              "node": "2-by2PpBRRq7Z39w8Lvstw",
              "relocating_node": null
            },
            "merges": {
              "current": 0,
              "current_docs": 0,
              "current_size_in_bytes": 0,
              "total": 0,
              "total_time_in_millis": 0,
              "total_docs": 0,
              "total_size_in_bytes": 0,
              "total_stopped_time_in_millis": 0,
              "total_throttled_time_in_millis": 0,
              "total_auto_throttle_in_bytes": 0,
              "unreferenced_file_cleanups_performed": 0,
              "warmer": {
                "total_invocations_count": 0,
                "total_time_millis": 0,
                "total_failure_count": 0,
                "total_bytes_sent": 0,
                "total_bytes_received": 29833555,
                "total_send_time_millis": 0,
                "total_receive_time_millis": 6444,
                "ongoing_count": 0
              }
            },
            "commit": {
              "id": "Ouxkq5fRF2ZOcdcXMMF4DA==",
              "generation": 6,
              "user_data": {
                "min_retained_seq_no": "0",
                "translog_uuid": "mM3XtVkzRBmZMbK7YktNpg",
                "local_checkpoint": "-1",
                "history_uuid": "hNsSVAB0RLWewv9HSWiI9A",
                "max_seq_no": "-1",
                "max_unsafe_auto_id_timestamp": "-1"
              },
              "num_docs": 0
            },
            "seq_no": {
              "max_seq_no": 799,
              "local_checkpoint": -1,
              "global_checkpoint": -1
            },
            "retention_leases": {
              "primary_term": 1,
              "version": 0,
              "leases": []
            },
            "shard_path": {
              "state_path": "/usr/share/opensearch/data/nodes/0",
              "data_path": "/usr/share/opensearch/data/nodes/0",
              "is_custom_data_path": false
            }
          }
        ]
      }
    }
  }
}

/_cat/shards?v&h=index,shard,prirep,merges.*&format=json

[
  {
    "index": "test-index",
    "shard": "0",
    "prirep": "p",
    "merges.current": "0",
    "merges.current_docs": "0",
    "merges.current_size": "0b",
    "merges.total": "23",
    "merges.total_docs": "3428",
    "merges.total_size": "15.3mb",
    "merges.total_time": "6.3s",
    "merges.warmer.total_invocations": "23",
    "merges.warmer.total_time": "4.2s",
    "merges.warmer.ongoing_count": "0",
    "merges.warmer.total_bytes_received": "0b",
    "merges.warmer.total_bytes_sent": "11.5mb",
    "merges.warmer.total_receive_time": "0s",
    "merges.warmer.total_failure_count": "0",
    "merges.warmer.total_send_time": "1.4s"
  },
  {
    "index": "test-index",
    "shard": "0",
    "prirep": "r",
    "merges.current": "0",
    "merges.current_docs": "0",
    "merges.current_size": "0b",
    "merges.total": "0",
    "merges.total_docs": "0",
    "merges.total_size": "0b",
    "merges.total_time": "0s",
    "merges.warmer.total_invocations": "0",
    "merges.warmer.total_time": "0s",
    "merges.warmer.ongoing_count": "0",
    "merges.warmer.total_bytes_received": "63mb",
    "merges.warmer.total_bytes_sent": "0b",
    "merges.warmer.total_receive_time": "13.7s",
    "merges.warmer.total_failure_count": "0",
    "merges.warmer.total_send_time": "0s"
  }
]

/_cat/nodes?v&h=id,merges.*&format=json

[
  {
    "id": "n3zT",
    "merges.current": "0",
    "merges.current_docs": "0",
    "merges.current_size": "0b",
    "merges.total": "23",
    "merges.total_docs": "3428",
    "merges.total_size": "15.3mb",
    "merges.total_time": "6.3s",
    "merges.warmer.total_invocations": "23",
    "merges.warmer.total_time": "4.2s",
    "merges.warmer.ongoing_count": "0",
    "merges.warmer.total_bytes_received": "0b",
    "merges.warmer.total_bytes_sent": "11.5mb",
    "merges.warmer.total_receive_time": "0s",
    "merges.warmer.total_failure_count": "0",
    "merges.warmer.total_send_time": "1.4s"
  },
  {
    "id": "2-by",
    "merges.current": "0",
    "merges.current_docs": "0",
    "merges.current_size": "0b",
    "merges.total": "0",
    "merges.total_docs": "0",
    "merges.total_size": "0b",
    "merges.total_time": "0s",
    "merges.warmer.total_invocations": "0",
    "merges.warmer.total_time": "0s",
    "merges.warmer.ongoing_count": "0",
    "merges.warmer.total_bytes_received": "86.1mb",
    "merges.warmer.total_bytes_sent": "0b",
    "merges.warmer.total_receive_time": "18.5s",
    "merges.warmer.total_failure_count": "0",
    "merges.warmer.total_send_time": "0s"
  },
  {
    "id": "SvhM",
    "merges.current": "0",
    "merges.current_docs": "0",
    "merges.current_size": "0b",
    "merges.total": "0",
    "merges.total_docs": "0",
    "merges.total_size": "0b",
    "merges.total_time": "0s",
    "merges.warmer.total_invocations": "0",
    "merges.warmer.total_time": "0s",
    "merges.warmer.ongoing_count": "0",
    "merges.warmer.total_bytes_received": "0b",
    "merges.warmer.total_bytes_sent": "0b",
    "merges.warmer.total_receive_time": "0s",
    "merges.warmer.total_failure_count": "0",
    "merges.warmer.total_send_time": "0s"
  },
  {
    "id": "SBeY",
    "merges.current": "0",
    "merges.current_docs": "0",
    "merges.current_size": "0b",
    "merges.total": "0",
    "merges.total_docs": "0",
    "merges.total_size": "0b",
    "merges.total_time": "0s",
    "merges.warmer.total_invocations": "0",
    "merges.warmer.total_time": "0s",
    "merges.warmer.ongoing_count": "0",
    "merges.warmer.total_bytes_received": "0b",
    "merges.warmer.total_bytes_sent": "0b",
    "merges.warmer.total_receive_time": "0s",
    "merges.warmer.total_failure_count": "0",
    "merges.warmer.total_send_time": "0s"
  }
]

/_cat/indices?v&h=id,merges.warmer.*&format=json

[
  {
    "id": "brrP7gIbR2ivDZ7PGKIBWA",
    "merges.warmer.total_invocations": "0",
    "merges.warmer.total_time": "0s",
    "merges.warmer.ongoing_count": "0",
    "merges.warmer.total_bytes_received": "0b",
    "merges.warmer.total_bytes_sent": "0b",
    "merges.warmer.total_receive_time": "0s",
    "merges.warmer.total_failure_count": "0",
    "merges.warmer.total_send_time": "0s"
  }
]

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@kh3ra kh3ra marked this pull request as draft August 5, 2025 18:24
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 5, 2025

❌ Gradle check result for 3dfd64f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@kh3ra kh3ra force-pushed the dev/remote-merged-segment-warmer-stats branch from 3dfd64f to 166ec29 Compare August 5, 2025 18:43
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 5, 2025

❌ Gradle check result for 166ec29: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 12721aa: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for b5bc126: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@kh3ra kh3ra force-pushed the dev/remote-merged-segment-warmer-stats branch from 1c95c87 to 66fa8f1 Compare August 12, 2025 05:53
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 66fa8f1: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 2a071c8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Aditya Khera added 13 commits October 14, 2025 12:36
spotlessApply
2bafefd 
Signed-off-by: Aditya Khera <[email protected]>
Empty commit
e9cd37f 
Signed-off-by: Aditya Khera <[email protected]>
Empty commit to trigger build
d2f2dc2 
Signed-off-by: Aditya Khera <[email protected]>
Test changes
d37c9d5 
Signed-off-by: Aditya Khera <[email protected]>
Empty commit
9e5924b 
Signed-off-by: Aditya Khera <[email protected]>
Empty commit to trigger build
3db6936 
Signed-off-by: Aditya Khera <[email protected]>
Fixing tests
055fd6b 
Signed-off-by: Aditya Khera <[email protected]>
fixes to merged segment warmer + tests
a11a127 
Signed-off-by: Aditya Khera <[email protected]>
spotlessApply
28a61cc 
Signed-off-by: Aditya Khera <[email protected]>
Fixing ITs after rebase
c13f652 
Signed-off-by: Aditya Khera <[email protected]>
Empty commit
4d8d2b7 
Signed-off-by: Aditya Khera <[email protected]>
Addressing review comments
bf68923 
Signed-off-by: Aditya Khera <[email protected]>
rebased with main
a142adb 
Signed-off-by: Aditya Khera <[email protected]>
@kh3ra kh3ra force-pushed the dev/remote-merged-segment-warmer-stats branch from 44437d4 to a142adb Compare October 14, 2025 07:15
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for a142adb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Fixing tests
a68d57b 
Signed-off-by: Aditya Khera <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for a68d57b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

spotlessApply
d6be469 
Signed-off-by: Aditya Khera <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for d6be469: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

YAML test fix
a6c9a97 
Signed-off-by: Aditya Khera <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for a6c9a97: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

yaml test fix
ba2cec5 
Signed-off-by: Aditya Khera <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for ba2cec5: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

test fixes
1739f47 
Signed-off-by: Aditya Khera <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

✅ Gradle check result for 1739f47: SUCCESS

@ashking94 ashking94 merged commit f15cbbd into opensearch-project:main Oct 14, 2025
32 checks passed
@codecov Codecov
Copy link

codecov bot commented Oct 14, 2025

Codecov Report

❌ Patch coverage is 80.07117% with 56 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.12%. Comparing base (252cff8) to head (1739f47).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
.../opensearch/rest/action/cat/RestIndicesAction.java 52.63% 0 Missing and 18 partials ⚠️
...rg/opensearch/rest/action/cat/RestNodesAction.java 47.05% 0 Missing and 9 partials ⚠️
...g/opensearch/rest/action/cat/RestShardsAction.java 75.00% 0 Missing and 8 partials ⚠️
...in/java/org/opensearch/index/merge/MergeStats.java 75.00% 2 Missing and 3 partials ⚠️
...g/opensearch/index/engine/MergedSegmentWarmer.java 77.77% 1 Missing and 3 partials ⚠️
.../opensearch/index/engine/NRTReplicationEngine.java 0.00% 3 Missing ⚠️
...es/replication/MergedSegmentReplicationTarget.java 40.00% 3 Missing ⚠️
...on/checkpoint/AbstractPublishCheckpointAction.java 57.14% 3 Missing ⚠️
...ckpoint/RemoteStorePublishMergedSegmentAction.java 80.00% 2 Missing ⚠️
...ensearch/index/merge/MergedSegmentWarmerStats.java 98.36% 1 Missing ⚠️
Additional details and impacted files 
@@             Coverage Diff              @@
##               main   #18929      +/-   ##
============================================
+ Coverage     73.11%   73.12%   +0.01%     
- Complexity    70661    70764     +103     
============================================
  Files          5724     5726       +2     
  Lines        323498   323750     +252     
  Branches      46852    46882      +30     
============================================
+ Hits         236518   236748     +230     
- Misses        67846    67941      +95     
+ Partials      19134    19061      -73

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@BrewTestBot BrewTestBot mentioned this pull request Oct 15, 2025
Merged
peteralfonsi pushed a commit to peteralfonsi/OpenSearch that referenced this pull request Oct 15, 2025
@kh3ra
Add Metrics For The Merged Segment Warmer Flow (opensearch-project#18929
551f321 
)

* Adding stats for merged segment warmer

Signed-off-by: kh3ra <[email protected]>
Signed-off-by: Aditya Khera <[email protected]>

* Adding unit tests

Signed-off-by: kh3ra <[email protected]>
Signed-off-by: Aditya Khera <[email protected]>

* SpotlessApply

Signed-off-by: kh3ra <[email protected]>
Signed-off-by: Aditya Khera <[email protected]>

* Minor fix

Signed-off-by: kh3ra <[email protected]>
Signed-off-by: Aditya Khera <[email protected]>

* minor fix

Signed-off-by: kh3ra <[email protected]>
Signed-off-by: Aditya Khera <[email protected]>

* addressing review comments - tests to follow

Signed-off-by: Aditya Khera <[email protected]>

* Addressing review comments + added ITs

Signed-off-by: Aditya Khera <[email protected]>

* addressing review comments - tests to follow

Signed-off-by: Aditya Khera <[email protected]>

* Changelog + minor bug fix

Signed-off-by: Aditya Khera <[email protected]>

* Addressing review comments + added UTs

Signed-off-by: Aditya Khera <[email protected]>

* Fixing tests

Signed-off-by: Aditya Khera <[email protected]>

* Restored breaking publicAPIs

Signed-off-by: Aditya Khera <[email protected]>

* spotlessApply

Signed-off-by: Aditya Khera <[email protected]>

* Empty commit

Signed-off-by: Aditya Khera <[email protected]>

* Empty commit to trigger build

Signed-off-by: Aditya Khera <[email protected]>

* Test changes

Signed-off-by: Aditya Khera <[email protected]>

* Empty commit

Signed-off-by: Aditya Khera <[email protected]>

* Empty commit to trigger build

Signed-off-by: Aditya Khera <[email protected]>

* Fixing tests

Signed-off-by: Aditya Khera <[email protected]>

* fixes to merged segment warmer + tests

Signed-off-by: Aditya Khera <[email protected]>

* spotlessApply

Signed-off-by: Aditya Khera <[email protected]>

* Fixing ITs after rebase

Signed-off-by: Aditya Khera <[email protected]>

* Empty commit

Signed-off-by: Aditya Khera <[email protected]>

* Addressing review comments

Signed-off-by: Aditya Khera <[email protected]>

* rebased with main

Signed-off-by: Aditya Khera <[email protected]>

* Fixing tests

Signed-off-by: Aditya Khera <[email protected]>

* spotlessApply

Signed-off-by: Aditya Khera <[email protected]>

* YAML test fix

Signed-off-by: Aditya Khera <[email protected]>

* yaml test fix

Signed-off-by: Aditya Khera <[email protected]>

* test fixes

Signed-off-by: Aditya Khera <[email protected]>

---------

Signed-off-by: kh3ra <[email protected]>
Signed-off-by: Aditya Khera <[email protected]>
Co-authored-by: Aditya Khera <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ashking94 ashking94 ashking94 approved these changes

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@andrross andrross Awaiting requested review from andrross andrross is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@cwperks cwperks Awaiting requested review from cwperks cwperks is a code owner

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@jed326 jed326 Awaiting requested review from jed326 jed326 is a code owner

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@reta reta Awaiting requested review from reta reta is a code owner

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@VachaShah VachaShah Awaiting requested review from VachaShah

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kh3ra @ashking94