[FEATURE] Create REST APIs to list all registered jobs and get jobs actively running in the cluster

[cwperks]

Is your feature request related to a problem?

Currently, job-scheduler has REST APIs, but they are only applicable in the context of extensions and not for OpenSearch plugins.

As an OpenSearch operator, I would like APIs to interact with the job-scheduler to see all registered jobs in my cluster, what the job type is, whether it is enabled/disabled, the job's schedule and when it was last run.

Additionally, I would like another API to fetch information about jobs actively running in the cluster, where they are running and total time elapsed.

What solution would you like?

Create 2 new REST APIs to:

1. Get all registered jobs
2. Get information about currently running jobs

I could imagine that in the future that job-scheduler could better keep track of job history with stats and then also get more administrative APIs that limit where jobs can run in a cluster to give cluster admins more control.

[Jeremydupras]

Here's the readout for the new API which gets all jobs in the cluster

It incudes: job type, job id, last run time, next run time, the node where the job is located and some other information about the job. As of now the jobs are grouped by node, I'll be adding a url param to give the option of getting all jobs on the cluster or grouping by node.

{
  "nodes": [
    {
      "node_id": "z58k9f9fS4Smb3I8bvBRCA",
      "node_name": "opensearch-node1",
      "scheduled_job_info": {
        "total_jobs": 1,
        "jobs": [
          {
            "job_type": "reports-scheduler",
            "job_id": "VjpYiZcBCJelw0UblVf0",
            "index_name": ".opendistro-reports-definitions",
            "name": "FAILED_LOGIN",
            "descheduled": false,
            "enabled": true,
            "enabled_time": "2025-06-19T17:59:33.618Z",
            "last_update_time": "2025-06-19T17:59:33.618Z",
            "next_expected_execution_time": "2025-06-20T17:59:32.014588127Z",
            "next_time_to_execute": "2025-06-20T17:59:32.014631842Z",
            "schedule": {
              "start_time": "2025-06-19T17:59:32.014Z",
              "unit": "Days",
              "interval": 1,
              "type": "interval"
            },
            "delay": "none",
            "jitter": "none",
            "lock_duration": "no_lock"
          }
        ]
      }
    },
    {
      "node_id": "ZUFW8GfLQOG_kE_02lhtMA",
      "node_name": "opensearch-node2",
      "scheduled_job_info": {
        "total_jobs": 1,
        "jobs": [
          {
            "job_type": "reports-scheduler",
            "job_id": "dzJYiZcBZQ2faWqS04Cr",
            "index_name": ".opendistro-reports-definitions",
            "name": "FAILED_LOGIN2",
            "descheduled": false,
            "enabled": true,
            "enabled_time": "2025-06-19T17:59:49.419Z",
            "last_update_time": "2025-06-19T17:59:49.419Z",
            "next_expected_execution_time": "2025-06-23T02:15:00Z",
            "next_time_to_execute": "2025-06-23T02:14:59.999992708Z",
            "schedule": {
              "type": "cron",
              "timezone": "Africa/Abidjan",
              "expression": "15 2 1,15 * 1"
            },
            "delay": "none",
            "jitter": "none",
            "lock_duration": "no_lock"
          }
        ]
      }
    }
  ],
  "failures": []
}