[WIP] Metrics framework integration with ml-commons #3661
Conversation
| } | ||
|
|
||
| private String getRemoteModelType() { | ||
| return "remote_sub_tye"; |
| @@ -95,6 +105,40 @@ public void onClusterManager() { | |||
| TimeValue.timeValueSeconds(jobInterval), | |||
| GENERAL_THREAD_POOL | |||
| ); | |||
| // startStatsCollectorJob(); | |||
There was a problem hiding this comment.
Is it commented intentionally ?
There was a problem hiding this comment.
this is a draft PR, it is not complete, still in development - raised a PR just to show the structure of adding metrics to ml-commons
pyek-bot
changed the title
| public static final String TASK_POLLING_JOB_INDEX = ".ml_commons_task_polling_job"; | ||
| // index created in 3.0 to track all ml jobs created via job scheduler | ||
| public static final String ML_JOBS_INDEX = ".plugins-ml-jobs"; |
There was a problem hiding this comment.
We already have multiple system index. To avoid adding too many system index, can we reuse .plugins-ml-task index ?
There was a problem hiding this comment.
I believe the job scheduler monitors a particular index for its documents. If a new document is added, it starts a new job using certain parameters in document defined in MLJobParameter. If the index has different documents with different format, I'm not sure how this will react. At the same time, how will the existing tasks work if job scheduler documents are present in it? I can test this out and get back to you
Description
[Describe what this change achieves]
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.