Skip to content

Commit 6717299

Browse files
committed
Squashed commit of the following:
commit 303ceb5b320f148dcd015ab6a11f870ae3e23125 Author: Joshua Palis <[email protected]> Date: Mon Mar 18 16:49:59 2024 -0700 Adding 2.13 release notes (#1170) Signed-off-by: Joshua Palis <[email protected]> commit 1507dd4b4045bd8e8cdf505d07041ce304788ca0 Author: Tyler Ohlsen <[email protected]> Date: Wed Feb 21 15:59:34 2024 -0800 Inject NamedWriteableRegistry in AD node client (#1164) Signed-off-by: Tyler Ohlsen <[email protected]> commit 5b85720d442d1b43001dde0c20248fd4a20a64b0 Author: Owais Kazi <[email protected]> Date: Tue Feb 13 10:20:20 2024 -0800 Fixed lucene url (#1158) Signed-off-by: owaiskazi19 <[email protected]> commit 290bdcd832a0cae6d039787d9f0af73ea6be2ec2 Author: Kaituo Li <[email protected]> Date: Tue Feb 6 16:14:49 2024 -0800 add 2.12 release notes (#1152) Signed-off-by: Kaituo Li <[email protected]> commit 6a6eec347379c997f0f7acc88b027bc1ffdd8f54 Author: Tyler Ohlsen <[email protected]> Date: Tue Jan 30 18:20:36 2024 -0800 Add ser/deser to get AD transport action request (#1150) Signed-off-by: Tyler Ohlsen <[email protected]> commit 475f0c36e5b6267aaeb101627b9c3ae1fcca3a88 Author: Owais Kazi <[email protected]> Date: Thu Jan 25 14:37:12 2024 -0800 Updated lucene snapshot url (#1146) Signed-off-by: owaiskazi19 <[email protected]> commit 7192d848d2923566a409901909f342a63695e1c2 Author: Jackie Han <[email protected]> Date: Thu Jan 25 09:48:37 2024 -0800 Remove default admin credentials (#1134) * Remove default admin credentials Signed-off-by: Jackie Han <[email protected]> * Replace default password with a placeholder in README files Signed-off-by: Jackie Han <[email protected]> --------- Signed-off-by: Jackie Han <[email protected]> commit 97482c19b0eded34a7f435e589303a834c258aa3 Author: Jackie Han <[email protected]> Date: Wed Jan 24 09:39:55 2024 -0800 Require JDK version for java spotless check (#1129) Signed-off-by: Jackie Han <[email protected]> commit 708e0462189677a217b35905c2feba5486c77cdc Author: Andriy Redko <[email protected]> Date: Tue Jan 23 15:11:39 2024 -0500 Update to Jackson 2.16.1 (#1135) Signed-off-by: Andriy Redko <[email protected]> commit 7c0ce4cb554a05230ecf5365b903219f357997bf Author: Andriy Redko <[email protected]> Date: Tue Jan 16 17:51:37 2024 -0500 Update to Gradle 8.5 (#1131) Signed-off-by: Andriy Redko <[email protected]> commit e5cbe93c0e9cadebe2a30cba83f99ae8a3f8fe57 Author: zane-neo <[email protected]> Date: Thu Jan 4 16:50:47 2024 +0800 add publishToMavenLocal task to fix zip plugin not pulished to maven local issue when running this script (#1121) Signed-off-by: zane-neo <[email protected]> commit 106dc25b3ef36beae329dc7882bcdf9c23347fb5 Author: Tyler Ohlsen <[email protected]> Date: Wed Dec 27 16:33:25 2023 -0800 Refactor client's getDetectorProfile to use GetAnomalyDetectorTransportAction (#1124) Signed-off-by: Tyler Ohlsen <[email protected]> commit 59b4ebecfa11cac977f49c999ce85637fb6fac58 Author: Tyler Ohlsen <[email protected]> Date: Wed Dec 27 11:56:00 2023 -0800 Add profile transport action to AD client (#1119) Signed-off-by: Tyler Ohlsen <[email protected]> commit 29711f96dbb2c9abc1181b933a8b334eed63ca66 Author: Tyler Ohlsen <[email protected]> Date: Tue Dec 12 08:32:53 2023 -0800 Add an AD transport client (#1110) Signed-off-by: Tyler Ohlsen <[email protected]> commit 353dcaeb05f8b75fa74d92e2ae8415ab018bc68e Author: Daniel Widdis <[email protected]> Date: Wed Nov 22 13:59:53 2023 -0800 Fix build, update CVE-affected versions (#1102) * Fix build, update CVE-affected versions Signed-off-by: Daniel Widdis <[email protected]> * Spotless depends on CVE-impacted eclipse dependency, now needs JDK17+ Signed-off-by: Daniel Widdis <[email protected]> --------- Signed-off-by: Daniel Widdis <[email protected]> commit 4c6ba48bf9fb9234ad8bc0b9193f3c68409acfb9 Author: Peter Zhu <[email protected]> Date: Mon Oct 30 19:35:01 2023 -0400 Fix the bwc test version retrieval (#1093) * Onboarding jenkins prod docker images to github actions Signed-off-by: Peter Zhu <[email protected]> * Add temurin Signed-off-by: Peter Zhu <[email protected]> * Add temurin Signed-off-by: Peter Zhu <[email protected]> * Add more Signed-off-by: Peter Zhu <[email protected]> * Add more Signed-off-by: Peter Zhu <[email protected]> * Add more Signed-off-by: Peter Zhu <[email protected]> --------- Signed-off-by: Peter Zhu <[email protected]> commit a1da3aa0eefc6671446d2eb3bc670bad6449d2f0 Author: Peter Zhu <[email protected]> Date: Mon Oct 30 19:14:31 2023 -0400 Onboarding jenkins prod docker images to github actions (#1092) * Onboarding jenkins prod docker images to github actions Signed-off-by: Peter Zhu <[email protected]> * Add temurin Signed-off-by: Peter Zhu <[email protected]> * Add temurin Signed-off-by: Peter Zhu <[email protected]> * Add more Signed-off-by: Peter Zhu <[email protected]> --------- Signed-off-by: Peter Zhu <[email protected]> commit 040cd7e33f8e696ffe7748140c9b0ecc7475d513 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Oct 10 12:43:02 2023 -0700 dependabot: bump net.bytebuddy:byte-buddy from 1.14.6 to 1.14.9 (#1081) Bumps [net.bytebuddy:byte-buddy](https://github.com/raphw/byte-buddy) from 1.14.6 to 1.14.9. - [Release notes](https://github.com/raphw/byte-buddy/releases) - [Changelog](https://github.com/raphw/byte-buddy/blob/master/release-notes.md) - [Commits](https://github.com/raphw/byte-buddy/compare/byte-buddy-1.14.6...byte-buddy-1.14.9) --- updated-dependencies: - dependency-name: net.bytebuddy:byte-buddy dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit dd9c1521bc1946027245d65e39a7844cdf04a8b7 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Oct 10 12:42:15 2023 -0700 dependabot: bump aws-actions/configure-aws-credentials (#1076) Bumps [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials) from 1.7.0 to 4.0.1. - [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases) - [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws-actions/configure-aws-credentials/compare/v1.7.0...v4.0.1) --- updated-dependencies: - dependency-name: aws-actions/configure-aws-credentials dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit d6e3fd7a69b1daea45d7a455335e55b6de118033 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Oct 10 12:05:00 2023 -0700 dependabot: bump com.netflix.nebula.ospackage from 11.0.0 to 11.5.0 (#1066) Bumps com.netflix.nebula.ospackage from 11.0.0 to 11.5.0. --- updated-dependencies: - dependency-name: com.netflix.nebula.ospackage dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 57eacf611ce44559e14aa37f2e51dfbb82d53cc2 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Oct 10 12:04:19 2023 -0700 dependabot: bump org.apiguardian:apiguardian-api from 1.1.0 to 1.1.2 (#1058) Bumps [org.apiguardian:apiguardian-api](https://github.com/apiguardian-team/apiguardian) from 1.1.0 to 1.1.2. - [Release notes](https://github.com/apiguardian-team/apiguardian/releases) - [Commits](https://github.com/apiguardian-team/apiguardian/compare/r1.1.0...r1.1.2) --- updated-dependencies: - dependency-name: org.apiguardian:apiguardian-api dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 6b9faf2b71e5481be366b12bd6f16440c6c8ef89 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Oct 10 12:04:01 2023 -0700 dependabot: bump org.junit.platform:junit-platform-launcher (#1056) Bumps [org.junit.platform:junit-platform-launcher](https://github.com/junit-team/junit5) from 1.9.2 to 1.10.0. - [Release notes](https://github.com/junit-team/junit5/releases) - [Commits](https://github.com/junit-team/junit5/commits) --- updated-dependencies: - dependency-name: org.junit.platform:junit-platform-launcher dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 2a957f2086688b5ee76703406105797d31207cc5 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Oct 10 12:03:11 2023 -0700 Bump urllib3 from 1.26.9 to 1.26.17 in /dataGeneration (#1068) Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.9 to 1.26.17. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](https://github.com/urllib3/urllib3/compare/1.26.9...1.26.17) --- updated-dependencies: - dependency-name: urllib3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit a12e33214f9c138946975801cefa2bd39c7978a9 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Oct 10 11:29:37 2023 -0700 dependabot: bump org.objenesis:objenesis from 3.0.1 to 3.3 (#1040) Bumps [org.objenesis:objenesis](https://github.com/easymock/objenesis) from 3.0.1 to 3.3. - [Release notes](https://github.com/easymock/objenesis/releases) - [Commits](https://github.com/easymock/objenesis/compare/3.0.1...3.3) --- updated-dependencies: - dependency-name: org.objenesis:objenesis dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 5a4ba14a431fbc53f757416542cda25ef2fae58c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Oct 10 11:28:46 2023 -0700 dependabot: bump tibdex/github-app-token from 2.0.0 to 2.1.0 (#1065) Bumps [tibdex/github-app-token](https://github.com/tibdex/github-app-token) from 2.0.0 to 2.1.0. - [Release notes](https://github.com/tibdex/github-app-token/releases) - [Commits](https://github.com/tibdex/github-app-token/compare/v2.0.0...v2.1.0) --- updated-dependencies: - dependency-name: tibdex/github-app-token dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 713d556d980ccb144eb6ae952cebc7b3963ed28c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Sep 13 15:05:44 2023 -0700 dependabot: bump actions/setup-java from 1 to 3 (#1035) Bumps [actions/setup-java](https://github.com/actions/setup-java) from 1 to 3. - [Release notes](https://github.com/actions/setup-java/releases) - [Commits](https://github.com/actions/setup-java/compare/v1...v3) --- updated-dependencies: - dependency-name: actions/setup-java dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 574ac521653ede84e7b821bc3c06049f1054a6ad Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Sep 13 10:53:02 2023 -0700 dependabot: bump tibdex/github-app-token from 1.5.0 to 2.0.0 (#1042) Bumps [tibdex/github-app-token](https://github.com/tibdex/github-app-token) from 1.5.0 to 2.0.0. - [Release notes](https://github.com/tibdex/github-app-token/releases) - [Commits](https://github.com/tibdex/github-app-token/compare/v1.5.0...v2.0.0) --- updated-dependencies: - dependency-name: tibdex/github-app-token dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 65e207aef49a6872408ae02e4f49cdd9060c1ef0 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Sep 13 10:52:40 2023 -0700 dependabot: bump org.apache.commons:commons-pool2 from 2.10.0 to 2.11.1 (#1045) Bumps org.apache.commons:commons-pool2 from 2.10.0 to 2.11.1. --- updated-dependencies: - dependency-name: org.apache.commons:commons-pool2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit a4ff71926af39726d9b4584102706e3ea7f4fa05 Author: Craig Perkins <[email protected]> Date: Mon Sep 11 14:16:31 2023 -0400 React to Tracer changes in TransportService (#1034) Signed-off-by: Craig Perkins <[email protected]> commit 05d0a3b905cae9d25f6d5d91f30f1b842b9a792f Author: Craig Perkins <[email protected]> Date: Mon Sep 11 14:14:20 2023 -0400 Ensure integ tests run with security after plugin rename (#1023) * Ensure integ tests run with security after plugin rename Signed-off-by: Craig Perkins <[email protected]> * Rename to time-series-analytics Signed-off-by: Craig Perkins <[email protected]> * Switch folder back Signed-off-by: Craig Perkins <[email protected]> * Run integTest with -i Signed-off-by: Craig Perkins <[email protected]> * Remove opensearch-anomaly-detection if installed Signed-off-by: Craig Perkins <[email protected]> * Update password rules and change expected error msg Signed-off-by: Craig Perkins <[email protected]> * Update password generation Signed-off-by: Craig Perkins <[email protected]> * Fix indexOf condition Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> commit 72210f014ab54bf4a5b567ebe9dabe13b66c6cbb Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Sep 11 11:13:39 2023 -0700 dependabot: bump codecov/codecov-action from 1 to 3 (#1038) Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1 to 3. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/codecov/codecov-action/compare/v1...v3) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit b77c3a7406e94424550ec1f22f084bc20533994f Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Sep 11 11:13:14 2023 -0700 dependabot: bump com.google.guava:guava from 32.0.1-jre to 32.1.2-jre (#1037) Bumps [com.google.guava:guava](https://github.com/google/guava) from 32.0.1-jre to 32.1.2-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 18defcc16e81abe590e8f109d6df3c4a060f74ce Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Sep 11 11:13:03 2023 -0700 dependabot: bump VachaShah/backport from 1.1.4 to 2.2.0 (#1039) Bumps [VachaShah/backport](https://github.com/vachashah/backport) from 1.1.4 to 2.2.0. - [Release notes](https://github.com/vachashah/backport/releases) - [Changelog](https://github.com/VachaShah/backport/blob/main/CHANGELOG.md) - [Commits](https://github.com/vachashah/backport/compare/v1.1.4...v2.2.0) --- updated-dependencies: - dependency-name: VachaShah/backport dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 726759f104117e1a28648a1524cdface49168b79 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Sep 11 11:12:53 2023 -0700 dependabot: bump actions/checkout from 2 to 4 (#1041) Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit f6ac01886a662264323d1e11a2199fb190148c80 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Sep 11 11:12:38 2023 -0700 dependabot: bump org.jacoco:org.jacoco.ant from 0.8.5 to 0.8.10 (#1044) Bumps [org.jacoco:org.jacoco.ant](https://github.com/jacoco/jacoco) from 0.8.5 to 0.8.10. - [Release notes](https://github.com/jacoco/jacoco/releases) - [Commits](https://github.com/jacoco/jacoco/compare/v0.8.5...v0.8.10) --- updated-dependencies: - dependency-name: org.jacoco:org.jacoco.ant dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit d30bcc394e6f281b6de504f102a0085ae56e78e4 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Sep 11 11:12:22 2023 -0700 dependabot: bump org.junit.jupiter:junit-jupiter-api (#1043) Bumps [org.junit.jupiter:junit-jupiter-api](https://github.com/junit-team/junit5) from 5.9.2 to 5.10.0. - [Release notes](https://github.com/junit-team/junit5/releases) - [Commits](https://github.com/junit-team/junit5/compare/r5.9.2...r5.10.0) --- updated-dependencies: - dependency-name: org.junit.jupiter:junit-jupiter-api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 92256db92f9ed78cb1de9925297c9ff5cce6da74 Author: Craig Perkins <[email protected]> Date: Mon Sep 11 14:04:23 2023 -0400 Add dependabot.yml (#1026) Signed-off-by: Craig Perkins <[email protected]> commit 23611f6d8ccdc9a4c41469ec0a4746290f976b4f Author: Jackie Han <[email protected]> Date: Fri Sep 8 11:19:31 2023 -0700 Add 2.10 release notes (#1031) Signed-off-by: Jackie Han <[email protected]> commit ee1db57058e08c36e4820637973f18016cf45015 Author: Owais Kazi <[email protected]> Date: Wed Sep 6 16:55:00 2023 -0700 Updates demo certs for integ tests (#1018) Signed-off-by: Owais Kazi <[email protected]> commit 338d72eb692c81802a70fe42556cde70c01cad6a Author: Kaituo Li <[email protected]> Date: Wed Sep 6 16:08:01 2023 -0700 refactor job response (#1017) Signed-off-by: Kaituo Li <[email protected]> commit db456c2543a6f1ff6b77db10bd19952768d16182 Author: Jackie Han <[email protected]> Date: Tue Sep 5 15:47:50 2023 -0700 upgrading commons-lang3 version to fix conflict issue (#1012) * force commons-lang3 version to fix conflict issue Signed-off-by: Jackie Han <[email protected]> * getting commons-lang3 version from upstream version properties file Signed-off-by: Jackie Han <[email protected]> * getting commons-lang3 version from core Signed-off-by: Jackie Han <[email protected]> * getting commons-lang3 version from core Signed-off-by: Jackie Han <[email protected]> * upgrading commons-lang3 version to align with core Signed-off-by: Jackie Han <[email protected]> --------- Signed-off-by: Jackie Han <[email protected]> commit db788f4714233121eba4238869b8db89ae0037d4 Author: Owais Kazi <[email protected]> Date: Tue Sep 5 13:58:31 2023 -0700 Adds auto release workflow (#1003) Signed-off-by: owaiskazi19 <[email protected]> commit 327c141e9b70465b1760f9ebf8c001a2e439b01f Author: Jackie Han <[email protected]> Date: Fri Sep 1 13:00:23 2023 -0700 Revert "Enforce DOCUMENT Replication for AD Indices and Adjust Primary Shards (#948)" (#999) This reverts commit bc1649922ef39619424fc16c64af5836e48a4a12. commit 4a09b741e1c127a92775cb55f4c56aa5c4f3f6a3 Author: Kaituo Li <[email protected]> Date: Mon Aug 21 11:21:36 2023 -0700 Update Gradle Wrapper to 8.2.1 for Enhanced JDK20 Compatibility (#985) - **Reason for Upgrade**: The migration to JDK20 ([reference](https://github.com/opensearch-project/opensearch-build/blob/aa65a8ecd69f77c3d3104043dd1c48dff708bffa/manifests/3.0.0/opensearch-3.0.0.yml#L9)) rendered the current Gradle version (7.6.1) incompatible. - **Actions Taken**: - **Gradle Wrapper Update**: Upgraded the Gradle wrapper to version 8.2.1 to maintain compatibility with JDK20. The gradle wrapper files are generated using the `./gradlew wrapper` command. - Applied `spotless` due to new formatting requirements in Gradle 8. - Resolved test "jar hell" issues. Gradle 8 introduced internal JARs to the test classpath that conflicted with dependencies from `org.junit.vintage:junit-vintage-engine`. As a remedy, these conflicting JARs have been excluded. - **Relevant Pull Requests**: - [Alerting#893](https://github.com/opensearch-project/alerting/pull/893/files) - [ML-Commons#892](https://github.com/opensearch-project/ml-commons/pull/892) - [Security PR](https://github.com/opensearch-project/security/pull/2978) - **Verification**: Successfully verified the changes using `gradle build`. Signed-off-by: Kaituo Li <[email protected]> commit 5ac6390e7de643d8177f0a5833995032f3a279ad Author: Kaituo Li <[email protected]> Date: Thu Aug 17 11:08:36 2023 -0700 Refactoring task cache manager for forecasting (#982) * Refactoring Shared Functionality for Broader Task Compatibility This PR extracts shared components from `ADTaskCacheManager` into `TaskCacheManager` to support the requirements of the forecasting feature. **Renamings**: - **Method-level in `ADTaskCacheManager`**: - `addDeletedDetector` to `addDeletedConfig` - `addDeletedDetectorTask` to `addDeletedTask` - `hasDeletedDetectorTask` to `hasDeletedTask` - `pollDeletedDetector` to `pollDeletedConfig` - `pollDeletedDetectorTask` to `pollDeletedTask` - **Variable-level in `AnomalyDetectorSettings`**: - `CHECKPOINT_MAINTAIN_QUEUE_MAX_HEAP_PERCENT` to `AD_CHECKPOINT_MAINTAIN_QUEUE_MAX_HEAP_PERCENT` - `CHECKPOINT_READ_QUEUE_MAX_HEAP_PERCENT` to `AD_CHECKPOINT_READ_QUEUE_MAX_HEAP_PERCENT` - `CHECKPOINT_SAVING_FREQ` to `AD_CHECKPOINT_SAVING_FREQ` - `CHECKPOINT_TTL` to `AD_CHECKPOINT_TTL` - `CHECKPOINT_WRITE_QUEUE_MAX_HEAP_PERCENT` to `AD_CHECKPOINT_WRITE_QUEUE_MAX_HEAP_PERCENT` - `COLD_ENTITY_QUEUE_MAX_HEAP_PERCENT` to `AD_COLD_ENTITY_QUEUE_MAX_HEAP_PERCENT` - `DEDICATED_CACHE_SIZE` to `AD_DEDICATED_CACHE_SIZE` - `ENTITY_COLD_START_QUEUE_CONCURRENCY` to `AD_ENTITY_COLD_START_QUEUE_CONCURRENCY` - `ENTITY_COLD_START_QUEUE_MAX_HEAP_PERCENT` to `AD_ENTITY_COLD_START_QUEUE_MAX_HEAP_PERCENT` - `EXPECTED_COLD_ENTITY_EXECUTION_TIME_IN_MILLISECS` to `AD_EXPECTED_COLD_ENTITY_EXECUTION_TIME_IN_MILLISECS` - `FILTER_BY_BACKEND_ROLES` to `AD_FILTER_BY_BACKEND_ROLES` - `MAX_ENTITIES_PER_QUERY` to `AD_MAX_ENTITIES_PER_QUERY` - `MAX_MODEL_SIZE_PER_NODE` to `AD_MAX_MODEL_SIZE_PER_NODE` - `MAX_MULTI_ENTITY_ANOMALY_DETECTORS` to `AD_MAX_HC_ANOMALY_DETECTORS` - `MAX_RETRY_FOR_END_RUN_EXCEPTION` to `AD_MAX_RETRY_FOR_END_RUN_EXCEPTION` - `MAX_SINGLE_ENTITY_ANOMALY_DETECTORS` to `AD_MAX_SINGLE_ENTITY_ANOMALY_DETECTORS` - `MODEL_MAX_SIZE_PERCENTAGE` to `AD_MODEL_MAX_SIZE_PERCENTAGE` - `PAGE_SIZE` to `AD_PAGE_SIZE` - `RESULT_WRITE_QUEUE_MAX_HEAP_PERCENT` to `AD_RESULT_WRITE_QUEUE_MAX_HEAP_PERCENT` - **Class-level**: - `ADRealtimeTaskCache` renamed to `RealtimeTaskCache` - **Package-level**: - Shifted from `org.opensearch.ad.breaker` to `org.opensearch.timeseries.breaker` **Migrations**: - Variables transferred from `AnomalyDetectorSettings` to `TimeSeriesSettings`: - `BATCH_BOUNDING_BOX_CACHE_RATIO` - `CHECKPOINT_MAINTAIN_REQUEST_SIZE_IN_BYTES` - `CHECKPOINT_WRITE_QUEUE_SIZE_IN_BYTES` - `DEFAULT_AD_JOB_LOC_DURATION_SECONDS` (renamed to `DEFAULT_JOB_LOC_DURATION_SECONDS`) - `ENTITY_REQUEST_SIZE_IN_BYTES` - `HOURLY_MAINTENANCE` - `INTERVAL_RATIO_FOR_REQUESTS` - `LOW_SEGMENT_PRUNE_RATIO` - `MAINTENANCE_FREQ_CONSTANT` - `MAX_COLD_START_ROUNDS` - `MAX_QUEUED_TASKS_RATIO` - `MAX_TOTAL_RCF_SERIALIZATION_BUFFERS` - `MAX_CHECKPOINT_BYTES` - `MEDIUM_SEGMENT_PRUNE_RATIO` - `NUM_MIN_SAMPLES` - `NUM_SAMPLES_PER_TREE` - `NUM_TREES` - `QUEUE_MAINTENANCE` - `RESULT_WRITE_QUEUE_SIZE_IN_BYTES` - `SERIALIZATION_BUFFER_BYTES` - `THRESHOLD_MIN_PVALUE` - `TIME_DECAY` **Deletions**: - Obsolete settings and methods: - `DESIRED_MODEL_SIZE_PERCENTAGE` in `AnomalyDetectorSettings` - `getDesiredModelSize` in `MemoryTracker` **Modifications**: - `MemoryTracker` enums renamed for clear differentiation between real-time and historical memory usage, adding `REAL_TIME_FORECASTER` for a harmonized single-stream and HC analysis approach. **Tests**: - Changes validated with a successful Gradle build. Signed-off-by: Kaituo Li <[email protected]> * improve comments Signed-off-by: Kaituo Li <[email protected]> * improve comments Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit 0c5b4b94cb5b192160463fa8a5f58eac587fe95d Author: Kaituo Li <[email protected]> Date: Tue Aug 15 13:09:17 2023 -0700 Update RCF to v3.8 and Enable Auto AD with 'Alert Once' Option (#979) * Update RCF to v3.8 and Enable Auto AD with 'Alert Once' Option This PR added support for automatic Anomaly Detection (AD) and the 'Alert Once' option introduced in RCF 3.8. Testing done: 1. Deserialization Test: * Verified model deserialization from 3.0-rc3. * Ensured consistent scoring using the rc3 checkpoint and rc3 dependency on identical data. 2. Backward Compatibility Test: * Executed a mixed cluster with versions 2.10 and 3.0. * Validated that older detectors still produce results without throwing any exceptions in a blue-green simulation scenario. Signed-off-by: Kaituo Li <[email protected]> * reduce recall since alertOnce reduced recall Signed-off-by: Kaituo Li <[email protected]> * remove commented out code Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit cadc9bbd949daf4f2669335bfb8629e2137c108d Author: Joshua Palis <[email protected]> Date: Fri Aug 11 16:02:38 2023 -0700 Fixing imports, adding TermsAggregator BucketCountThresholds for StringTerms, setting AnomalyResult grade to 0 if negative value (#976) Signed-off-by: Joshua Palis <[email protected]> commit 3985cd2f82f2cfa57604ecf5a512bb042e632cae Author: Romain Tartière <[email protected]> Date: Wed Aug 9 08:59:30 2023 -1000 Fix build with latest OpenSearch (for real this time) (#972) This is a follow-up to #971, where commit 2839bc2c6f470639b68901086c80cb2152d96558 only fixed _some_ but not _all_ of the build errors (it allowed `./gradlew run` to pass but `./gradlew build` was failing). Fix the build errors when we run `./gradlew build`. Signed-off-by: Romain Tartière <[email protected]> commit 2839bc2c6f470639b68901086c80cb2152d96558 Author: Romain Tartière <[email protected]> Date: Tue Aug 8 13:15:04 2023 -1000 Fix build with latest OpenSearch (#971) OpenSearch recently merged some code that cause buid failure for this module, as spotted in #585. This change the import to match the new one after https://github.com/opensearch-project/OpenSearch/pull/9103 Signed-off-by: Romain Tartière <[email protected]> commit d4946f04e7ad44f0bc79215962c6678786c592ff Author: Kaituo Li <[email protected]> Date: Mon Aug 7 14:21:56 2023 -0700 Refactor ADTask and Related Components (#969) * Refactor ADTask and Related Components This PR includes several key refactoring changes: - Extracts common code from ADTask into TimeSeriesTask, creating ForecastTask for forecasting-specific logic. - Consolidates common code from ADTaskType into TaskType and introduces ForecastTaskType for forecasting-related purposes. - Renames ADTaskState to TaskState for consistent code reuse. - Renames the method getId in ADTask to getConfigId to differentiate it from other IDs like task id. Testing done: 1. Added unit tests for the new code to ensure functionality. 2. Executed a successful Gradle build. Signed-off-by: Kaituo Li <[email protected]> * add comments and address compiler errors Signed-off-by: Kaituo Li <[email protected]> * address Amit's comments and address compiler failure Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit 1130a1ba6fc3c08f375dd99f98e93e4ee36331fb Author: Kaituo Li <[email protected]> Date: Thu Jul 27 10:39:39 2023 -0700 Refactoring NodeStateManager etc. to support forecasting functionality (#965) * Refactoring NodeStateManager etc. to support forecasting functionality This commit extends the codebase to support both Anomaly Detection (AD) and forecasting. It contains a mixture of refactoring, renaming, removal of unused code, and package moving tasks. Here are the details: Refactoring: - `NodeStateManager.getAnomalyDetector` is now `getConfig`, with added functionality to fetch a Forecaster. The method comments are updated for clarity. - Existing methods (`getFeatureSamplesForPeriods`, `getColdStartSamplesForPeriods`, `createPreviewSearchRequest`, `getMinDataTime`) have been added in `SearchFeatureDao` to handle forecasting logic. - Adjusted `SecurityClientUtil` and `ParseUtils` to handle forecasting logic. - Cleaned up `NodeState` to differentiate state for AD and forecasting. Renaming: - `AnomalyDetectorJob` is renamed to `Job` to facilitate reuse for forecasting. - `NodeStateManager.getAnomalyDetectorJob` is renamed to `getJob`. - Certain settings in `AnomalyDetectorSettings` are renamed to reflect they are meant for the AD setting. They have been marked as deprecated and new settings are used in `TimeSeriesSettings` instead. - `IndexAnomalyDetectorJobActionHandler.getAnomalyDetectorJobForWrite` is renamed to `getJobForWrite`. - `ADSafeSecurityInjector` is renamed to `TimeSeriesSafeSecurityInjector`. Removing unused code: - Synchronous code in `ClientUtil`, `IndexUtils`, and `CheckpointDao` is removed. - The unused class `Throttler` is deleted. - Mapping file names are changed, and the code referencing these files is adjusted. Package moving: - Several classes (`ClientUtil`, `MultiResponsesDelegateActionListener`, `SafeSecurityInjector`, `SecurityUtil`, `ExceptionUtil`, `SearchFeatureDao`, `CleanState`, `ExpiringState`, `MaintenanceState`, `NodeState`, `SingleStreamModelIdMapper`, `BackPressureRouting`) are moved to the respective `org.opensearch.timeseries` packages. Miscellaneous: - Fixed compiler failures caused by changes in https://github.com/opensearch-project/OpenSearch/pull/8730 by replacing `DoubleArrayList` with `java.util.ArrayList`. - Updates the Backwards Compatibility (bwc) version to align with the core's incremented bwc version as per [OpenSearch PR #8670](https://github.com/opensearch-project/OpenSearch/pull/8670). This change prevents the issue described in [OpenSearch Issue #5076](https://github.com/opensearch-project/OpenSearch/issues/5076). Testing: - Executed a `gradle build`. - Added new tests for `ClientUtil` and `NodeStateManager`. Signed-off-by: Kaituo Li <[email protected]> * improve comment Signed-off-by: Kaituo Li <[email protected]> * fix compiler error and comments Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit f0ed43b1aa3c5d80dc6f23a5dd4465bb03ef1739 Author: Kaituo Li <[email protected]> Date: Mon Jul 17 15:18:03 2023 -0700 Rename AD to time series analytics & Resolve Compiler Errors (#951) This commit focuses on renaming the AnomalyDetectorPlugin to TimeSeriesAnalyticsPlugin as well as addressing compiler errors due to package import changes. The primary transformations are: * Renamed AnomalyDetectorPlugin to TimeSeriesAnalyticsPlugin. * Updated all references from AnomalyDetectorPlugin to TimeSeriesAnalyticsPlugin. * Amended plugin information in build.gradle and src/main/resources/es-plugin.properties. * Switched job type in TimeSeriesAnalyticsPlugin from opendistro_anomaly_detector to opensearch_time_series_analytics. These changes led to the modification of our plugin binary from opensearch-anomaly-detection-3.0.0.0-SNAPSHOT.jar to opensearch-time-series-analytics-3.0.0.0-SNAPSHOT.jar. Moreover, following changes from OpenSearch Pull Request #8157, compiler errors were encountered, which were resolved by updating the import packages * org.opensearch.common.io.stream.NamedWriteableRegistry => org.opensearch.core.common.io.stream.NamedWriteableRegistry * org.opensearch.common.xcontent.XContentParserUtils => * org.opensearch.core.xcontent.XContentParserUtils * org.opensearch.common.bytes.BytesArray => org.opensearch.core.common.bytes.BytesArray * org.opensearch.common.io.stream.StreamInput => * org.opensearch.core.common.io.stream.StreamInput ** org.opensearch.common.io.stream.StreamOutput => org.opensearch.core.common.io.stream.StreamOutput * org.opensearch.common.io.stream.Writeable => org.opensearch.core.common.io.stream.Writeable * org.opensearch.common.io.stream.NotSerializableExceptionWrapper => org.opensearch.core.common.io.stream.NotSerializableExceptionWrapper * org.opensearch.rest.RestStatus => org.opensearch.core.rest.RestStatus * org.opensearch.common.ParsingException => org.opensearch.core.common.ParsingException * org.opensearch.common.bytes.BytesReference => org.opensearch.core.common.bytes.BytesReference * org.opensearch.common.io.stream.InputStreamStreamInput => org.opensearch.core.common.io.stream.InputStreamStreamInput * org.opensearch.common.io.stream.OutputStreamStreamOutput => org.opensearch.core.common.io.stream.OutputStreamStreamOutput * org.opensearch.index.shard.ShardId => org.opensearch.core.index.shard.ShardId* org.opensearch.index.Index => org.opensearch.core.index.Index * org.opensearch.core.index.IndexNotFoundException => org.opensearch.index.IndexNotFoundException Validation steps were executed to ensure these changes did not break functionality: * gradle build was successful. A local build was required due to outdated OpenSearch and job scheduler nightly builds. * Backward compatibility tests were performed on a two-node 2.9 cluster running some detectors. These detectors continued to produce results on a 3.0 cluster with these updates. * I can start new detectors on the 3.0 node. * After shutting down a 2.9 node, the job transfer to the 3.0 node was successful and the job continued to perform as expected. Signed-off-by: Kaituo Li <[email protected]> commit c7c6a46e88e15f615fc9dae0499cf0d674662057 Author: Tyler Ohlsen <[email protected]> Date: Thu Jul 13 15:42:46 2023 -0700 Add 2.9 release notes (#952) Signed-off-by: Tyler Ohlsen <[email protected]> commit bc1649922ef39619424fc16c64af5836e48a4a12 Author: Kaituo Li <[email protected]> Date: Tue Jul 11 13:34:09 2023 -0700 Enforce DOCUMENT Replication for AD Indices and Adjust Primary Shards (#948) * Enforce DOCUMENT Replication for AD Indices and Adjust Primary Shards In this PR, we temporarily enforce DOCUMENT replication for AD indices. This change is necessary due to the current limitation of SegRep, which doesn't support Get/MultiGet by ID. This measure will be in place until SegRep adds support for these operations. This adjustment aligns with the modification made in the referenced PR: job-scheduler PR #417 Additionally, this PR increases the number of primary shards for forecasting indices from 10 to 20, recognizing the higher write intensity of these indices. For instance, when the horizon is 24, we are expected to save 25 documents: 24 for each forecast and 1 for the actual value. In contrast, for AD, we save one document per actual value. Testing done: 1. gradle build passed Signed-off-by: Kaituo Li <[email protected]> * Fix comments Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit a1203827617ee42b35d41cb8b010719f334f73b7 Author: Kaituo Li <[email protected]> Date: Mon Jul 10 10:53:17 2023 -0700 Add ForecastResult and Refactor Shared Code (#941) This commit adds the functionality of ForecastResult, while also performing several necessary refactorings. Firstly, we introduce ForecastResult, which closely mirrors the functionality of AnomalyResult. Any shared code between these two classes has been abstracted out into a new parent class, IndexableResult. This refactoring was undertaken to facilitate code reuse and improve maintainability, as both classes deal with storing fields that need to be saved in result indices. Secondly, we've also extracted common code from ThresholdingResult into another new class, IntermediateResult. This work is in preparation for a forthcoming PR which will introduce RCFCasterResult. In this change, we moved the 'toAnomalyResult' method to IndexableResult, and renamed it to 'toIndexableResults'. This renaming allows for reuse in the upcoming RCFCasterResult. The updated method now returns a list of results instead of a single result. The shift caters for the RCFCasterResult use case where one RCFCasterResult is stored across multiple ForecastResult documents. Furthermore, this commit modifies the 'entity' field in several methods to be of the Optional type, indicating that it can be empty. This change provides a clear signal to other developers about the optional nature of these fields. In addition, this commit updates the mapping for the forecast result index. The prior mapping was a placeholder and this change brings it up to date. We've also moved to using httpcore5 and httpclient5 versions from OpenSearch core, as necessitated by recent changes in [OpenSearch PR #8434](https://github.com/opensearch-project/OpenSearch/pull/8434/files). This shift resolves potential jar hell issues with httpcore5 and httpclient5. Lastly, we've made name and package changes: 1. Renamed 'initDetectionStateIndex' in ADIndexManagement to 'initStateIndex' to align with ForecastIndexManagement naming. 2. Moved the 'DataByFeatureId' class to the 'timeseries' package for better organization. This commit also completes the 'createDummyIndexRequest' method in ForecastIndexManagement by invoking 'ForecastResult.getDummyResult()', which is now possible due to the introduction of the ForecastResult implementation. Testing done: 1. added tests for new code 2. gralde build passes 3. e2e tests Signed-off-by: Kaituo Li <[email protected]> commit f24d9e32ff739abf29cac3da15fd00f74219e213 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu Jul 6 17:28:09 2023 -0700 Bump scipy from 1.8.0 to 1.10.0 in /dataGeneration (#943) Bumps [scipy](https://github.com/scipy/scipy) from 1.8.0 to 1.10.0. - [Release notes](https://github.com/scipy/scipy/releases) - [Commits](https://github.com/scipy/scipy/compare/v1.8.0...v1.10.0) --- updated-dependencies: - dependency-name: scipy dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 9b9fd2ba3f3e678d717c0a45af8069bb67828808 Author: Amit Galitzky <[email protected]> Date: Tue Jun 27 16:08:55 2023 -0700 bump guava version to 32.0.1 (#933) Signed-off-by: Amit Galitzky <[email protected]> commit 99749dfa6bfc8a2f2c4660ca01d75d1ad57124a5 Author: Kaituo Li <[email protected]> Date: Mon Jun 26 16:46:24 2023 -0700 Refactoring Index Creation for Improved Code Reuse and Consistency (#932) This Pull Request significantly refactors the index creation module to promote greater code reusability and consistency. The primary changes are outlined as follows: 1. Code Migration: Common code segments have been moved from AnomalyDetectionIndices to IndexManagement. In addition, component-specific code has been established in ADIndexManagement and ForecastIndexManagement. The bulk of changes are concentrated in these three classes, with the remainder of the modifications relating to reference/test changes. 2. Function Renaming: Several functions have been renamed for broader applicability and consistency: * AnomalyDetectorFunction has been renamed to ExecutorFunction for potential use in Forecasting. * AnomalyDetectorSettings.MAX_PRIMARY_SHARDS has been renamed to AnomalyDetectorSettings.AD_MAX_PRIMARY_SHARDS in light of the new ForecastSettings.FORECAST_MAX_PRIMARY_SHARDS. * doesAnomalyDetectorJobIndexExist() has been renamed to doesJobIndexExist() to allow for job index reusability across Anomaly Detection (AD) and forecasting. Analogous changes have been made to other job index-related functions. * doesAnomalyDetectorIndexExist() has been renamed to doesConfigIndexExist() to allow for config index reusability across AD and forecasting. Analogous changes have been made to other config index-related functions. * detectionIndices.doesDetectorStateIndexExist() has been renamed to detectionIndices.doesStateIndexExist(), as the former name was unnecessarily redundant. Similar modifications have been made to the result and checkpoint index. 3. Class Migration: The classes ThrowingSupplierWrapper, ThrowingConsumer, and ThrowingSupplier have been moved from org.opensearch.ad.util to org.opensearch.timeseries.function to promote code reuse. 4. Initial Forecast Indices' Mapping: An initial version of forecast indices' mapping has been added, which can be adjusted as required in the future. 5. In terms of version updates, 2.x has been bumped to 2.9, prompting an increment of the Backward Compatibility (BWC) test version to 2.9. 6. Update dependency com.google.guava:guava to v32 for cve fix. Testing done: Testing has been performed with new tests added for forecast index creation, and the grade build passes successfully. Signed-off-by: Kaituo Li <[email protected]> commit aedb781a6b1df984636d72d50b65d41138539918 Author: Craig Perkins <[email protected]> Date: Tue Jun 20 16:55:19 2023 -0400 Fix main build - update import of Releasable and remove reference to BaseExceptionsHelper (#930) * Fix main build - update import of Releasable and remove reference to BaseExceptionsHelper Signed-off-by: Craig Perkins <[email protected]> * Update imports in test classes Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> commit fce78c29c0375ae06f56e16c85fb98933401e28f Author: Kaituo Li <[email protected]> Date: Wed Jun 14 10:36:40 2023 -0700 Refactor: Reorganize code for Forecasting and AnomalyDetector packages (#925) * Refactor: Reorganize code for Forecasting and AnomalyDetector packages This PR includes several refactorings to increase code reuse and improve readability: 1. Entity, FeatureData, and RestHandlerUtils have been moved from the 'ad' package to the 'timeseries' package, enabling their use within the Forecasting code. 2. A new class, DataByFeatureId, has been created to hold shared code previously located in FeatureData. FeatureData now inherits from DataByFeatureId. The standalone usage of DataByFeatureId will be detailed in subsequent PRs. 3. Renamed checkAnomalyDetectorFeaturesSyntax to checkFeaturesSyntax in RestHandlerUtils to make it more generic and usable for Forecasting. 4. Constants AGG_NAME_MAX_TIME, AGG_NAME_MIN_TIME, DATE_HISTOGRAM, and FEATURE_AGGS have been moved from ADCommonName to CommonName to make them accessible for Forecasting. 5. A new method, parseConfig, has been added to the Config class. This method can parse configuration for either AnomalyDetector or Forecaster based on input. Testing: The changes have been validated with a successful gradle build. Signed-off-by: Kaituo Li <[email protected]> * address Amit and Owais's comments Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit 02c120f025f73f3d45a8477c7ef8b4873befebfa Author: Kaituo Li <[email protected]> Date: Mon Jun 12 11:54:39 2023 -0700 Updated Maintainers and CODE_OWNERS list (#926) Signed-off-by: Kaituo Li <[email protected]> commit 2ea4dcf430b40a710a0babb04881eeff9fb2d689 Author: Kaituo Li <[email protected]> Date: Thu Jun 8 15:11:44 2023 -0700 Add Forecaster class (#920) * Add Forecaster class This PR adds class Forecaster that serves as the configuration POJO for forecasting. Shared code between AnomalyDetector and Forecaster is extracted and moved to the Config class to reduce duplication and promote reusability. References to the common code in related classes have also been adjusted. Testing done: 1. gradle build. Signed-off-by: Kaituo Li <[email protected]> * fix compiler error due to a recent core change Signed-off-by: Kaituo Li <[email protected]> * address Sudipto and Amit's comments Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit ee04225349a5f0f003d56e77a3243386f4a9673d Author: Kaituo Li <[email protected]> Date: Fri Jun 2 15:42:19 2023 -0700 Refactor: Migrate files from 'ad' to 'timeseries' package (#918) * Refactor: Migrate files from 'ad' to 'timeseries' package This commit involves moving various files from the 'ad' package to the 'timeseries' package. All changes to dependent imports were handled automatically through the IDE, and no changes to file contents were made in the process. Testing done: - gradle build Signed-off-by: Kaituo Li <[email protected]> * move ParseUtils to util package Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit 472868d165c2234eece8827584f64086aaf7ae92 Author: Kaituo Li <[email protected]> Date: Thu Jun 1 11:17:16 2023 -0700 Refactor Shared Settings, Introduce Dynamic Forecast Settings, Rename (#912) * Refactor Shared Settings, Introduce Dynamic Forecast Settings, and Rename Checkpoint mapping File This commit has undertaken three changes: * Refactoring of Shared Settings: I've performed a refactoring to unify the settings that are shared between the AD and forecasting modules. * Introduction of Dynamic Forecast Settings: To increase the adaptability of our system, I've implemented dynamic forecast settings. These settings mirror the existing structure of the AD dynamic settings and will enable us to adjust forecast settings on-the-fly. * Renaming of Checkpoint File: To enhance the consistency across our AD mapping files, I've renamed checkpoint.json to anomaly-checkpoint.java. Testing done: 1. added tests for new settings. Signed-off-by: Kaituo Li <[email protected]> * name change and delete files Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit 04ab91e60e9d71765a7b9df10b86e0e53ba4d715 Author: Kaituo Li <[email protected]> Date: Wed May 31 12:18:54 2023 -0700 add 2.8 release notes (#915) Signed-off-by: Kaituo Li <[email protected]> commit 98da8dffe8d336793ae35658015650b0f83aeef1 Author: Kaituo Li <[email protected]> Date: Wed May 24 14:37:38 2023 -0700 Implementing Multiple Imputation Methods for Missing Values (#907) * Implementing Multiple Imputation Methods for Missing Values This Pull Request introduces multiple imputation methods to handle missing data in Anomaly Detection (AD) and forecasting. The ability to handle missing data is crucial to improve the robustness and accuracy of our models. The following imputation methods have been implemented: * Zero Imputation (ZERO): This method replaces all missing values with 0's. It's a simple approach, but it may introduce bias if the data is not centered around zero. * Fixed Values Imputation (FIXED_VALUES): This method replaces missing values with a predefined set of values. The values are the same for each input dimension, and they need to be specified by the user. * Previous Value Imputation (PREVIOUS): This method replaces missing values with the last known value in the respective input dimension. It's a commonly used method for time series data, where temporal continuity is expected. * Linear Interpolation (LINEAR): This method estimates missing values by interpolating linearly between known values in the respective input dimension. This method assumes that the data follows a linear trend. These methods are designed to provide a set of options for users to handle missing data based on their specific needs and the nature of their data. Testing Done: The code changes in this pull request have been validated through a gradle build to ensure that all new and existing tests pass successfully. Signed-off-by: Kaituo Li <[email protected]> * Various changes The commit makes all new files use the shortened header text and addresses compiler/test issues due to core refactoring Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit da608aaa3d3f955efe32b0cb33b3c8cfdd7a7718 Author: Kaituo Li <[email protected]> Date: Tue May 16 11:37:34 2023 -0700 Code Refactoring for CommonMessages (#902) * Code Refactoring for CommonMessages In this pull request, I have refactored the code related to shared log or error messages in both AD and forecasting modules to CommonMessages. Additionally, the previously used CommonErrorMessages has been renamed to ADCommonMessages. For the Forecasting module, I have introduced new names in ForecastCommonMessages. Testing done: * gradle build Signed-off-by: Kaituo Li <[email protected]> * Update string constants wording Signed-off-by: Kaituo Li <[email protected]> * improve wording and remove redundant variables Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit 041d6ce73d37f1697013b2d2f0683fbfe012247c Author: Kaituo Li <[email protected]> Date: Fri May 12 13:05:56 2023 -0700 Code Refactoring for CommonName (#901) In this pull request, I have refactored the code related to shared names in both AD and forecasting modules to CommonNames. Additionally, the previously used CommonName has been renamed to ADCommonName. For the Forecasting module, I have introduced new names in ForecastCommonNames. Testing done: * gradle build Signed-off-by: Kaituo Li <[email protected]> commit e2fee9d6ba59cc2a23eadbb50a80414fcc7dbf8f Author: Kaituo Li <[email protected]> Date: Wed May 10 17:26:31 2023 -0700 Update outdated MAINTAINERS and OWNERS list (#898) Signed-off-by: Kaituo Li <[email protected]> commit 98b566d1ec57ffd8970b5180c54b493deb1a24ac Author: Daniel (dB.) Doubrovkine <[email protected]> Date: Wed May 10 18:03:04 2023 -0400 Sync up CODEOWNERS with MAINTAINERS.md. (#897) Signed-off-by: dblock <[email protected]> commit 1f8415aa39fa4ec3ba016214255aa0e79b7c7d63 Author: Kaituo Li <[email protected]> Date: Tue May 9 12:55:35 2023 -0700 Various fixes (#886) * Various fixes This pull request addresses several issues related to the compiler and tests in the current main branch. The first major change involves replacing ImmutableOpenMap with java.util.Map in the core. This modification is implemented in the following pull requests: https://github.com/opensearch-project/OpenSearch/pull/7165 https://github.com/opensearch-project/OpenSearch/pull/7301 To accommodate this change, the codebase of the AD (Anomaly Detection) module has been refactored to utilize JDK maps. As a consequence of this change, passing null to the custom parameters of ClusterState is no longer permissible, as it leads to a NullPointerException. The error stack trace is as follows: java.lang.NullPointerException: Cannot invoke "Object.getClass()" because "m" is null at __randomizedtesting.SeedInfo.seed([60CDDB34427ACD0C:6E72DB4ED18E018D]:0) at java.base/java.util.Collections.unmodifiableMap(Collections.java:1476) at org.opensearch.cluster.ClusterState.<init>(ClusterState.java:219) at org.opensearch.ad.transport.DeleteAnomalyDetectorTests.createClusterState(DeleteAnomalyDetectorTests.java:216) at org.opensearch.ad.transport.DeleteAnomalyDetectorTests.testDeleteADTransportAction_LatestDetectorLevelTask(DeleteAnomalyDetectorTests.java:160) To address this issue, we have replaced the usage of null with new HashMap<>(). The second change in this pull request aligns with the modifications introduced in PR https://github.com/opensearch-project/anomaly-detection/pull/878. The third issue is related to the incompatibility between tests that utilize the @Parameters annotation and those that do not, as explained in https://tinyurl.com/2y265s2w. Specifically, the SearchFeatureDaoTests class runes tests with the @Parameters annotation, whereas SearchFeatureDao tests do not. Testing done: 1. gradle build. Signed-off-by: Kaituo Li <[email protected]> commit 0a48da9e1a0cd70c64cc2da228674baaf377c89e Author: Jackie Han <[email protected]> Date: Mon Apr 17 12:44:06 2023 -0700 Add 2.7 release notes (#871) Signed-off-by: Jackie Han <[email protected]> commit 1edc2ef1ccfe20938711984744b20d1f2ec96a3c Author: Jackie Han <[email protected]> Date: Tue Apr 11 17:53:57 2023 -0700 Update OpenSearchRejectedExecutionException import (#861) * Update OpenSearchRejectedExecutionException import Signed-off-by: Jackie Han <[email protected]> * Update OpenSearchRejectedExecutionException import Signed-off-by: Jackie Han <[email protected]> * Update OpenSearchRejectedExecutionException import Signed-off-by: Jackie Han <[email protected]> * Update NodeInfo constructor and javadoc for isAdmin method Signed-off-by: Jackie Han <[email protected]> --------- Signed-off-by: Jackie Han <[email protected]> commit b8df172c2364997c446096e960068d6fc5d2e6f6 Author: Amit Galitzky <[email protected]> Date: Tue Apr 11 10:55:28 2023 -0700 Giving admin priority over backendrole filtering (#850) * giving admin priority over backend role filtering Signed-off-by: Amit Galitzky <[email protected]> * fix security tests Signed-off-by: Amit Galitzky <[email protected]> * remove redundent line in test case Signed-off-by: Amit Galitzky <[email protected]> --------- Signed-off-by: Amit Galitzky <[email protected]> commit fd2fd5d5f63c9004b9f265e3c1e4475282447da8 Author: Daniel (dB.) Doubrovkine <[email protected]> Date: Thu Mar 16 14:30:32 2023 -0400 Created untriaged issue workflow. (#809) Signed-off-by: dblock <[email protected]> commit fd547014fdde5114bbc9c8e49fe7aaa37eb6e793 Author: Kaituo Li <[email protected]> Date: Wed Mar 1 16:41:05 2023 -0800 Publish snapshots to maven via GHA, fix spotless errors, and bump bwc version (#827) We are de-coupling the task of publishing the maven snapshots from centralized build workflow to individual repositories. This PR publishes maven snapshots using GitHub Actions. Testing done: 1. ran ./gradlew publishPluginZipPublicationToSnapshotsRepository according to https://github.com/opensearch-project/anomaly-detection/issues/813 Signed-off-by: Kaituo Li <[email protected]> commit b6be5960fb8cff41f0e9d13d5f5a971c289b80ed Author: Jackie Han <[email protected]> Date: Thu Feb 23 12:23:01 2023 -0800 Update xcontent imports (#822) Signed-off-by: Jackie Han <[email protected]> commit 1dcb78a6c4d1331133aefe5af7adf137daa8846f Author: Jackie Han <[email protected]> Date: Wed Feb 22 11:17:21 2023 -0800 Add 2.6 release note (#818) Signed-off-by: Jackie Han <[email protected]> commit 553688aae45590c37cc4d86102642d0b3a6ebb33 Author: Kaituo Li <[email protected]> Date: Thu Feb 9 13:38:14 2023 -0800 Revert changes to exception message (#803) * Revert changes to exception message We used to log "No RCF models are available either because RCF models are not ready..." and now we log "resource_not_found_exception: No checkpoints found for model id...". This change caused an integration test failure. This PR reverts it back. Testing done: 1. verified that the original log is back. Signed-off-by: Kaituo Li <[email protected]> * check NO_MODEL_ERR_MSG instead of NO_CHECKPOINT_ERR_MSG Signed-off-by: Kaituo Li <[email protected]> --------- Signed-off-by: Kaituo Li <[email protected]> commit c9bdc628a03c4ced3cd1ebeec44ebff792d7ff98 Author: Kaituo Li <[email protected]> Date: Fri Feb 3 11:15:37 2023 -0800 fixing dls/fls logic around numeric aggregations (#801) Signed-off-by: Kaituo Li <[email protected]> commit 9cebc5bc97a84dd78ac8892863ff7d674428bced Author: Amit Galitzky <[email protected]> Date: Tue Jan 10 02:17:04 2023 +0000 adding 2.5 release notes Signed-off-by: Amit Galitzky <[email protected]> commit 2ace84643fd199a5e5179e6d21ab76ae212cb7d8 Author: Ryan Bogan <[email protected]> Date: Tue Jan 10 13:31:49 2023 -0800 Add jackson back to build.gradle (#780) Signed-off-by: Ryan Bogan <[email protected]> commit a5d60eefab5ac30ef5243da8e7a88b7a2189cd5c Author: Kaituo Li <[email protected]> Date: Mon Jan 9 15:49:35 2023 -0800 bump bwc to 2.6.0 (#775) We need to bump the version because only the latest 2.x will be upgradable to 3.0 when that ship. Since OpenSearch core bumped to 2.6, we need to do that as well. Note after bumping version, bwc still fails to find 2.6.0 core zip. Signed-off-by: Kaituo Li <[email protected]> commit 27ed49863bfbef90551cdfacc6b51703bf622e25 Author: Kaituo Li <[email protected]> Date: Mon Jan 9 12:17:57 2023 -0800 Fix the discrepancy between Profile API and real time tasks API (#770) * Fix the discrepancy between Profile API and real time tasks API This PR fixes the discrepancy by querying the result index when the total updates is less than 32. We have done similar things in profile API so I refactored reusable code to ProfileUtil. We also cached whether we have queried the result index and won't repeatedly issue the extra query. Testing done: 1. repeated repro steps in https://github.com/opensearch-project/anomaly-detection/issues/502 and verified the issue has been resolved. Signed-off-by: Kaituo Li <[email protected]> commit b49a36b3e7962a205695d451e0497ee4f57fb2cb Author: Daniel (dB.) Doubrovkine <[email protected]> Date: Thu Jan 5 10:19:51 2023 -0800 Updated MAINTAINERS.md to match recommended opensearch-project format. (#771) Signed-off-by: dblock <[email protected]> commit 20d2c5f94811f62a7077142ba505bed272a83549 Author: Amit Galitzky <[email protected]> Date: Tue Dec 20 13:16:21 2022 -0800 update bwc to 2.5 (#765) Signed-off-by: Amit Galitzky <[email protected]> commit e08314f43a22d2692e8467efae3699fce0440611 Author: Amit Galitzky <[email protected]> Date: Tue Dec 20 11:00:45 2022 -0800 [Forward-Port to main] Fix _source bug (#749) (#764) * Fix _source bug (#749) Signed-off-by: Amit Galitzky <[email protected]> * fixed strings method Signed-off-by: Amit Galitzky <[email protected]> commit 134942ed61b984e57b1f9c6bc7d416232d8f59d2 Author: Kaituo Li <[email protected]> Date: Fri Dec 16 09:38:46 2022 -0800 Speed up cold start (#753) * Speed up cold start If historical data is enough, a single stream detector takes 1 interval for cold start to be triggered + 1 interval for the state document to be updated. Similar to single stream detectors, HCAD cold start needs 2 intervals and one more interval to make sure an entity appears more than once. So HCAD needs three intervals to complete cold starts. Long initialization is the single most complained problem of AD. This PR reduces both single stream and HCAD detectors' initialization time to 1 minute by * delaying real time cache update by one minute when we receive ResourceNotFoundException in single stream detectors or when the init progress of HCAD real time cache is 0. Thus, we can finish the cold start and writing checkpoint one minute later and update the state document accordingly. This optimization saves one interval to wait for the state document update. * disable the door keeper by default so that we won't have to wait an extra interval in HCAD. * trigger cold start when starting a real time detector. This optimization saves one interval to wait for the cold start to be triggered. Testing done: * verified the cold start time is reduced to 1 minute. * added tests for new code. Signed-off-by: Kaituo Li <[email protected]> commit 609abe42df8af450e48bfa122746361f7a8201c7 Author: Daniel (dB.) Doubrovkine <[email protected]> Date: Thu Dec 15 17:11:10 2022 -0500 Fix: typo in ohltyler. (#760) Signed-off-by: dblock <[email protected]> commit ce3d747317aec5ccef080c767d4ba8e7015c5a2a Author: Varun Jain <[email protected]> Date: Tue Dec 6 15:44:33 2022 -0800 Model Profile Test (#748) commit 09b4cc1ac3ab9fb4e589754acba619d265603580 Author: Amit Galitzky <[email protected]> Date: Mon Dec 5 12:06:16 2022 -0800 Use sonatype to fetch JS instead of ci.opensearch.org (#740) Signed-off-by: Amit Galitzky <[email protected]> commit a3e2b994ed347263a8dbaf5beed6bd5ddd6931a4 Author: Kaituo Li <[email protected]> Date: Wed Nov 30 16:49:23 2022 -0800 AD model performance benchmark (#728) * AD model performance benchmark This PR adds an AD model performance benchmark so that we can compare model performance across versions. For the single stream detector, we refactored tests in DetectionResultEvalutationIT and moved it to SingleStreamModelPerfIT. For the HCAD detector, we randomly generated synthetic data with known anomalies inserted throughout the signal. In particular, these are one/two/four dimensional data where each dimension is a noisy cosine wave. Anomalies are inserted into one dimension with 0.003 probability. Anomalies across each dimension can be independent or dependent. We have approximately 5000 observations per data set. The data set is generated using the same random seed so the result is comparable across versions. We also backported #600 so that we can capture the performance data in CI output. We also fixed #712 by revising the client setup code. Testing done: * added unit tests to run the benchmark. Signed-off-by: Kaituo Li <[email protected]> commit d8f0c355c026bd60b991a055f5d35dc0c8f654bd Author: Owais Kazi <[email protected]> Date: Wed Nov 30 09:53:05 2022 -0800 Added coalesceToEmpty method (#736) Signed-off-by: Owais Kazi <[email protected]> commit 7cefb143d61178223eec0dc1aa5aa2a16534316e Author: Kaituo Li <[email protected]> Date: Tue Nov 8 10:19:50 2022 -0800 remove bwc code related to 1.0 and earlier version (#714) This PR addresses AD compile failure because the core removed 1x Version constants. (check https://github.com/opensearch-project/OpenSearch/pull/5021) OpenSearch does not support N-2 version compatibility. This is inherited from Elasticsearch and Lucene. So version 1x is not compatible with 3.0. Thus removal of deprecated 1x code. This PR follows suite and removes backward compatibility code on OpenSearch 1.0 and older versions. So we won't support direct upgrade from 1.x domains to 3.x. Testing done: 1. gradle build. Note that CI workflow will fail due to missing job scheduler. We are using a job scheduler from distribution. But due to a circular dependency on -SNAPSHOT builds being published to maven that require the distribution build to be successful (check https://github.com/opensearch-project/opensearch-build/issues/1463), AD compilation failure caused the job scheduler to be taken out. Not including the latest job scheduler will cause AD build to fail. So we have a chicken and egg problem: this PR fixes AD build failure and the PR itself cannot build due to missing job scheduler. To run gradle build, I changed to use local job scheduler and verified gradle build succeeded. Once I merge this PR. job scheduler should be able to build and be published to maven. Future AD CI should be unblocked after that. Signed-off-by: Kaituo Li <[email protected]> commit bcfe952f0bfced908ce1c5b9547d23bd77cbb9f4 Author: Amit Galitzky <[email protected]> Date: Mon Nov 7 09:00:56 2022 -0800 Fixing Docker CI for security enabled tests (#710) Signed-off-by: Amit Galitzky <[email protected]> commit 06e5edb68686bb8ca9dfb63f7a9c92b0b44bc295 Author: Amit Galitzky <[email protected]> Date: Fri Oct 28 11:44:40 2022 -0700 windows CI for AD (#703) Signed-off-by: Amit Galitzky <[email protected]> commit c43ddbfce5ba5ee179b082fbeacd0c12422a71cb Author: Jackie Han <[email protected]> Date: Thu Oct 27 10:55:41 2022 -0700 Add 2.4 release notes (#699) Signed-off-by: Jackie Han <[email protected]> Signed-off-by: Jackie Han <[email protected]> commit 7a05826a9f26ebd1e7c9bc8c821e7034e453562d Author: Jackie Han <[email protected]> Date: Fri Oct 21 15:04:02 2022 -0700 Fixing issues caused by 3.0 version bumping (#696) * Update BaseNodeRequest dependency Signed-off-by: Jackie Han <[email protected]> * Migrate client transports to Apache HttpClient / Core 5.x Signed-off-by: Jackie Han <[email protected]> * Address spotlessCheck failures Signed-off-by: Jackie Han <[email protected]> * Upgrade bwc version because 1.1.0 cannot be upgraded directly to version 3.0.0 Signed-off-by: Jackie Han <[email protected]> * update bwc version to 2.4 Signed-off-by: Jackie Han <[email protected]> * Update bwc version Signed-off-by: Jackie Han <[email protected]> * update bwc download link Signed-off-by: Jackie Han <[email protected]> * use latest 2.4 build for bwc test Signed-off-by: Jackie Han <[email protected]> * move bwc constants to build script block Signed-off-by: Jackie Han <[email protected]> Signed-off-by: Jackie Han <[email protected]> commit 24232aea08cf9217600b4ec7c00742b078c99a47 Author: Tyler Ohlsen <[email protected]> Date: Fri Oct 21 13:32:02 2022 -0700 Bump jackson-databind to 2.13.4.2 (#697) Signed-off-by: Tyler Ohlsen <[email protected]> commit e5e15a6f9acfa9588abeefc8a6666bbb5f602488 Author: Jackie Han <[email protected]> Date: Mon Oct 17 15:46:55 2022 -0700 bump version to 3.0.0 (#693) Signed-off-by: Jackie Han <[email protected]> Signed-off-by: Jackie Han <[email protected]> commit 16cee09f557b4c65edfd8961b25f200e466a01de Author: Prudhvi Godithi <[email protected]> Date: Thu Oct 13 09:26:16 2022 -0700 add group = org.opensearch.plugin (#690) Signed-off-by: prudhvigodithi <[email protected]> commit 27942cb37a73cd5f48764f14a72670bdda1c30a3 Author: Amit Galitzky <[email protected]> Date: Mon Oct 3 15:16:38 2022 -0700 update jackson dependency version (#678) Signed-off-by: Amit Galitzky <[email protected]> commit fd718445fc65934c8e2ff074194ff1f000cae952 Author: Amit Galitzky <[email protected]> Date: Sun Sep 25 11:25:04 2022 -0700 Fix window delay test (#674) Signed-off-by: Amit Galitzky <[email protected]> commit 426f713b8043c93494189271138e4c5b1ce4593f Author: Leonidas Spyropoulos <[email protected]> Date: Thu Sep 22 19:48:35 2022 +0100 Add support for reproducible builds (#579) As per gradle [docs] add support to remove timestamps and package with same order which is required from [reproducible] builds [docs]: https://docs.gradle.org/current/userguide/working_with_files.html#sec:archives [reproducible]: https://reproducible-builds.org/ Signed-off-by: Leonidas Spyropoulos <[email protected]> commit 397ebcc052110f9d3db1d312fc28b38c8530377e Author: Owais Kazi <[email protected]> Date: Wed Sep 14 10:31:23 2022 -0700 Removed Github DCO action since DCO runs via Github App now (#664) Signed-off-by: Owais Kazi <[email protected]> commit 19a6c84ad1b8048a571b7a1a1345e646f587508d Author: Tyler Ohlsen <[email protected]> Date: Wed Sep 7 12:01:25 2022 -0700 Add 2.3 release notes (#660) * Add 2.3 release notes Signed-off-by: Tyler Ohlsen <[email protected]> * Add other documentation PR Signed-off-by: Tyler Ohlsen <[email protected]> Signed-off-by: Tyler Ohlsen <[email protected]> commit 06d53fb28c12366ea29d88d2186d55c7295544c2 Author: Tyler Ohlsen <[email protected]> Date: Wed Sep 7 12:01:12 2022 -0700 Bump to version 2.3 (#658) Signed-off-by: Tyler Ohlsen <[email protected]> Signed-off-by: Tyler Ohlsen <[email protected]> commit d9d956dc138d66981a94765ad8229a9eab6b8c35 Author: Chris Moore <[email protected]> Date: Wed Sep 7 11:41:03 2022 -0700 “fix#921-README-forum-link-AD” (#659) Signed-off-by: cwillum <[email protected]> Signed-off-by: cwillum <[email protected]> commit 884573fc4d7343ddea6c7b8d638234a2805b87c2 Author: Amit Galitzky <[email protected]> Date: Wed Sep 7 10:56:07 2022 +0300 Removed additional non-inclusive terms (#644) Signed-off-by: Amit Galitzky <[email protected]> commit bd063bd0778400161ed1e75e3314ec7bb67e31af Author: Rishikesh Pasham <[email protected]> Date: Fri Sep 2 20:12:25 2022 +0000 Adding external property customDistributionUrl to let developer override default distribution Download url (#380) * Adding uasage and external property customDistributionUrl to let developer override default distribution Download url Signed-off-by: Rishikesh1159 <[email protected]> * Adding doc and removing system property from build.gradle Signed-off-by: Rishikesh1159 <[email protected]> Signed-off-by: Rishikesh1159 <[email protected]> commit c92cdc82fb2aef58b3c0b95fafa90da81bfc24cf Author: Kaituo Li <[email protected]> Date: Tue Aug 9 09:36:08 2022 -0700 Fix PowerMock related tests due to version bump (#637) 2.2 added a new dependency of ClusterSettings like TaskResourceTrackingService. It is possible that the classloader that loadsTaskResourceTrackingService is different from the classloader that loads PowerMock and related dependencies. PowerMock reports java.lang.NoClassDefFoundError when initializing ClusterSettings. Since we are not actually using the value of ClusterSettings in the tests, we make it null to avoid initializing it. The change fixed failed tests. This PR also fixed spotless errors in FakeNode due to recent changes. Testing done: * gradle build Signed-off-by: Kaituo Li <[email protected]> commit 103f03492681ce999fb87d13ed0218461b084c96 Author: Tyler Ohlsen <[email protected]> Date: Mon Aug 8 17:14:30 2022 -0400 Fix taskmanager compilation error in FakeNode (#634) Signed-off-by: Tyler Ohlsen <[email protected]> commit 01bbfc525b93068145f09e3e468dec23d1bbea26 Author: Amit Galitzky <[email protected]> Date: Fri Aug 5 16:30:29 2022 -0700 2.2 release notes (#631) Signed-off-by: Amit Galitzky <[email protected]> commit 12f9f2e337ee81c4882bc02fb68f631ea9ab1430 Author: Tyler Ohlsen <[email protected]> Date: Thu Aug 4 20:25:45 2022 -0400 Bump version to 2.2 (#627) Signed-off-by: Tyler Ohlsen <[email protected]> commit 7f846016c92800383ed848936ed82251108c8947 Author: Tyler Ohlsen <[email protected]> Date: Thu Aug 4 19:46:25 2022 -0400 Update BWC zip links (#625) Signed-off-by: Tyler Ohlsen <[email protected]> commit 08fdbdde4c2f97c27f63d3a4ba31f8acd6dbf570 Author: Kaituo Li <[email protected]> Date: Wed Aug 3 16:40:17 2022 -0700 make 1M1min possible (#620) * make 1M1min possible This PR improves performance to make the 1M1min experiment possible. First, I changed coordinating node pagination from sync to async mode in AnomalyResultTransportAction so that the coordinating node does not have to wait for model nodes' responses before fetching the next page. Second, during the million-entity evaluation, CPU is mostly around 1% with hourly spikes up to 65%. An internal hourly maintenance job can account for the spike due to saving hundreds of thousands of model checkpoints, clearing unused models, and performing bookkeeping for internal states. This PR evens out the resource usage more fairly across a large maintenance window by introducing CheckpointMaintainWorker. Third, during a model corruption, I retrigger cold start for mitigation. Check ModelManager.score, EntityResultTransportAction, and CheckpointReadWorker. Testing done: 1. Added unit tests. 2. Manually confirmed 1M1min is possible after the above changes. Signed-off-by: Kaituo Li <[email protected]> commit eb7bd07774658c7e0216e5436cfc613b15547d3d Author: Prudhvi Godithi <[email protected]> Date: Mon Aug 1 15:35:09 2022 -0400 Staging for version increment automation (#608) Signed-off-by: pgodithi <[email protected]> commit 9f6a5abb9652fb010decde8778c56c292eac90e2 Author: Amit Galitzky <[email protected]> Date: Wed Jul 13 10:07:23 2022 -0700 fix zip fetching issue on version increment (#611) Signed-off-by: Amit Galitzky <[email protected]> commit f630c8f97070e35a50fca8a7725767286c548a47 Author: Kaituo Li <[email protected]> Date: Wed Jun 29 10:41:57 2022 -0700 Expose model accuracy metrics in tests (#600) * Expose model accuracy metrics in tests This PR adds an option flag to print logs during tests and turn on the flag in CI workflow. The flag is disabled by default. By doing this, we can record model accuracy metrics in git workflows and later retrieve it for analysis. Testing done: 1. We can turn on/off logs during tests. 2. The accuracy logs are recorded. Signed-off-by: Kaituo Li <[email protected]> commit d484f9baa9aedf0a48215a82043107cd23c50c0b Author: Tyler Ohlsen <[email protected]> Date: Tue Jun 28 16:00:56 2022 -0400 Add 2.1.0 release notes (#597) Signed-off-by: Tyler Ohlsen <[email protected]> commit 6e373a7452f2d38795c5b73d77282a5239382915 Author: Amit Galitzky <[email protected]> Date: Mon Jun 27 16:01:32 2022 -0700 adding custom plugin to upload ad zip to maven (#594) Signed-off-by: Amit Galitzky <[email protected]> commit 0bd6e0e6aa933e6cd6ada8dfe88762b6e0db441a Author: Amit Galitzky <[email protected]> Date: Mon Jun 27 13:59:36 2022 -0700 Update ingestion (#592) Signed-off-by: Amit Galitzky <[email protected]> commit 4d7a8a452524e47b10fdba24f60c83f3103571ae Author: Amit Galitzky <[email protected]> Date: Fri Jun 24 16:46:04 2022 -0700 Adding HCAD data ingestion script to AD (#585) Signed-off-by: Amit Galitzky <[email protected]> commit c6f9b20334a4d280defe7f4bd680a69437358ba1 Author: Amit Galitzky <[email protected]> Date: Fri Jun 24 13:06:29 2022 -0700 Cluster manager revert fix (#584) Signed-off-by: Amit Galitzky <[email protected]> commit 0e7079e31ffab5439ff386fbd1a52e734a5d6500 Author: Amit Galitzky <[email protected]> Date: Thu Jun 23 15:38:22 2022 -0700 2.1 version bump and Gradle bump (#582) Signed-off-by: Amit Galitzky <[email protected]> commit 03e04d70556a2785782bbfd1603f3c19fbd8c895 Author: Kaituo Li <[email protected]> Date: Wed Jun 22 11:55:00 2022 -0700 Disable interpolation in HCAD cold start (#575) * Disable interpolation in HCAD cold start Previously, we used interpolation in HCAD cold start for the purpose of efficiency. This caused problems for model accuracy. This PR removes interpolation in the cold start step. Testing done: 1. added unit tests to verify precision boosted. Signed-off-by: Kaituo Li <[email protected]> commit 7f3820a9bca636b76f6f85601ad8439e1b55a0d2 Author: Tyler Ohlsen <[email protected]> Date: Tue Jun 14 11:11:57 2022 -0700 Add 2.0.1 release notes (#572) Signed-off-by: Tyler Ohlsen <[email protected]> commit 5f4f9cd9957e72f7844ad8dd66dd6ee2ad8615e0 Author: Amit Galitzky <[email protected]> Date: Thu Jun 9 18:13:11 2022 +0000 bump rcf to 3.0-rc3 Signed-off-by: Amit Galitzky <[email protected]> commit 17bf3f83e37735f9f9562ae51530123cfb7bb252 Author: Kaituo Li <[email protected]> Date: Mon May 23 14:24:19 2022 -0700 Use current time as training data end time (#547) (#557) * Use current time as training data end time The bug happens because we use job enabled time as training data end time. But if the historical data before that time is deleted or does not exist at all, cold start might never finish. This PR uses current time as the training data end time so that cold start has a chance to succeed later. This PR also removes the code that combines cold start data and existing samples in EntityColdStartWorker because we don't add samples until cold start succeeds. Combining cold start data and existing samples is thus unnecessary. Testing done: 1. manually verified the bug is fixed. 2. fixed all related unit tests. Signed-off-by: Kaituo Li <[email protected]> commit 1ada5d73d0e96a58cc8c461cf8b4d757b762f348 Author: aksingh-es <[email protected]> Date: Mon May 16 16:13:31 2022 -0700 GA release notes (#550) Signed-off-by: aksingh-es <[email protected]> commit dbd311a9ed6022b6d42db8ea432cc478d29a0947 Author: Amit Galitzky <[email protected]> Date: Thu May 12 10:03:33 2022 -0700 remove rc1 qualifier and _type from test (#543) Signed-off-by: Amit Galitzky <[email protected]> commit 42b7f50890ad0e60e93ef6aa5c5389621edcf7ef Author: Xun Zhang <[email protected]> Date: Tue May 3 13:45:21 2022 -0700 Increase more coverage and reduce jacocoExclusions (#533) Signed-off-by: Xun Zhang <[email protected]> commit 8227e32b692dc137112f6cc42b475a465aeec347 Author: Amit Galitzky <[email protected]> Date: Tue May 3 10:31:16 2022 -0700 bump rcf to 3.0-rc2.1 (#519) Signed-off-by: Amit Galitzky <[email protected]> commit b17d483610826fa283639b2218050c05cef3d257 Author: Xun Zhang <[email protected]> Date: Wed Apr 27 14:57:21 2022 -0700 refactor SearchADResultTransportAction to be more testable (#517) Signed-off-by: Xun Zhang <[email protected]> commit be319d1a7712b9d6634b6bba06dbff399cd44c98 Author: AMIT KUMAR SINGH <[email protected]> Date: Thu Apr 21 10:57:45 2022 -0700 Fix OS version in notes (#512) Signed-off-by: aksingh-es <[email protected]> commit a44ffec2b3cd6de3f92ec2c1f7994b1cbbf31485 Author: AMIT KUMAR SINGH <[email protected]> Date: Wed Apr 20 17:28:15 2022 -0700 Re-enable Tests for CI workflows (#509) * Re enable tests for CI Signed-off-by: aksingh-es <[email protected]> * reenable CI tests Signed-off-by: aksingh-es <[email protected]> * fixing minor issue Signed-off-by: aksingh-es <[email protected]> * fixing minor issue for CI Signed-off-by: aksingh-es <[email protected]> commit d874960d7afe77bb48796ced2c5b214ca5303d8c Author: Tyler Ohlsen <[email protected]> Date: Wed Apr 20 12:39:57 2022 -0700 Update labeler to default backport to 2.x (#507) Signed-off-by: Tyler Ohlsen <[email protected]> Signed-off-by: Kaituo Li <[email protected]>
1 parent aecf6d7 commit 6717299

File tree

500 files changed

+16802
-11652
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

500 files changed

+16802
-11652
lines changed

.github/CODEOWNERS

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
# This should match the owning team set up in https://github.com/orgs/opensearch-project/teams
2-
* @opensearch-project/anomaly-detection
1+
* @jmazanec15 @jngz-es @kaituo @saratvemulapalli @ohltyler @vamshin @VijayanB @ylwu-amzn @amitgalitz @sudiptoguha @jackiehanyang @sean-zheng-amazon @dbwiddis @owaiskazi19 @joshpalis
2+

.github/labeler.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
backport 1.x:
1+
backport 2.x:
22
- "*"
33
- "*/*"
44
- "*/**/*"

.github/workflows/auto-release.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ jobs:
1313
steps:
1414
- name: GitHub App token
1515
id: github_app_token
16-
uses: tibdex/github-app-token@v2.0.0
16+
uses: tibdex/github-app-token@v2.1.0
1717
with:
1818
app_id: ${{ secrets.APP_ID }}
1919
private_key: ${{ secrets.APP_PRIVATE_KEY }}

.github/workflows/backport.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ jobs:
1515
steps:
1616
- name: GitHub App token
1717
id: github_app_token
18-
uses: tibdex/github-app-token@v2.0.0
18+
uses: tibdex/github-app-token@v2.1.0
1919
with:
2020
app_id: ${{ secrets.APP_ID }}
2121
private_key: ${{ secrets.APP_PRIVATE_KEY }}

.github/workflows/labeler.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ jobs:
1515
steps:
1616
- name: GitHub App token
1717
id: github_app_token
18-
uses: tibdex/github-app-token@v2.0.0
18+
uses: tibdex/github-app-token@v2.1.0
1919
with:
2020
app_id: ${{ secrets.APP_ID }}
2121
private_key: ${{ secrets.APP_PRIVATE_KEY }}

.github/workflows/test_build_multi_platform.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@ jobs:
127127
./gradlew assemble
128128
- name: Build and Run Tests
129129
run: |
130-
./gradlew build -Dtest.logs=true
130+
./gradlew build
131131
- name: Publish to Maven Local
132132
run: |
133133
./gradlew publishToMavenLocal

.github/workflows/test_security.yml

+7-8
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,6 @@ jobs:
1616

1717
name: Security test workflow for Anomaly Detection
1818
runs-on: ubuntu-latest
19-
env:
20-
JENKINS_URL: build.ci.opensearch.org
2119

2220
steps:
2321
- name: Setup Java ${{ matrix.java }}
@@ -26,22 +24,22 @@ jobs:
2624
distribution: 'temurin'
2725
java-version: ${{ matrix.java }}
2826

29-
# anomaly-detection
27+
# time-series-analytics
3028
- name: Checkout AD
3129
uses: actions/checkout@v4
3230

3331
- name: Build Anomaly Detection
3432
run: |
3533
./gradlew assemble
3634
# example of variables:
37-
# plugin = opensearch-anomaly-detection-2.4.0.0-SNAPSHOT.zip
38-
# version = 2.4.0, plugin_version = 2.4.0.0, qualifier = SNAPSHOT
35+
# plugin = opensearch-time-series-analytics-2.10.0.0-SNAPSHOT.zip
36+
# version = 2.10.0, plugin_version = 2.10.0.0, qualifier = SNAPSHOT
3937
- name: Pull and Run Docker
4038
run: |
4139
plugin=`basename $(ls build/distributions/*.zip)`
42-
version=`echo $plugin|awk -F- '{print $4}'| cut -d. -f 1-3`
43-
plugin_version=`echo $plugin|awk -F- '{print $4}'| cut -d. -f 1-4`
44-
qualifier=`echo $plugin|awk -F- '{print $5}'| cut -d. -f 1-1`
40+
version=`echo $plugin|awk -F- '{print $5}'| cut -d. -f 1-3`
41+
plugin_version=`echo $plugin|awk -F- '{print $5}'| cut -d. -f 1-4`
42+
qualifier=`echo $plugin|awk -F- '{print $6}'| cut -d. -f 1-1`
4543
4644
if $qualifier!=SNAPSHOT
4745
then
@@ -57,6 +55,7 @@ jobs:
5755
then
5856
echo "FROM opensearchstaging/opensearch:$docker_version" >> Dockerfile
5957
echo "RUN if [ -d /usr/share/opensearch/plugins/opensearch-anomaly-detection ]; then /usr/share/opensearch/bin/opensearch-plugin remove opensearch-anomaly-detection; fi" >> Dockerfile
58+
echo "RUN if [ -d /usr/share/opensearch/plugins/opensearch-time-series-analytics ]; then /usr/share/opensearch/bin/opensearch-plugin remove opensearch-time-series-analytics; fi" >> Dockerfile
6059
echo "ADD anomaly-detection/build/distributions/$plugin /tmp/" >> Dockerfile
6160
echo "RUN /usr/share/opensearch/bin/opensearch-plugin install --batch file:/tmp/$plugin" >> Dockerfile
6261
docker build -t opensearch-ad:test .

MAINTAINERS.md

+25-13
Original file line numberDiff line numberDiff line change
@@ -4,16 +4,28 @@ This document contains a list of maintainers in this repo. See [opensearch-proje
44

55
## Current Maintainers
66

7-
| Maintainer | GitHub ID | Affiliation |
8-
| ----------------------- | ------------------------------------------------------- | ----------- |
9-
| Hanguang Zhang | [zhanghg08](https://github.com/zhanghg08) | Amazon |
10-
| Jack Mazanec | [jmazanec15](https://github.com/jmazanec15) | Amazon |
11-
| Jing Zhang | [jngz-es](https://github.com/jngz-es) | Amazon |
12-
| Kaituo Li | [kaituo](https://github.com/kaituo) | Amazon |
13-
| Lai Jiang | [wnbts](https://github.com/wnbts) | Amazon |
14-
| Sarat Vemulapalli | [saratvemulapalli](https://github.com/saratvemulapalli) | Amazon |
15-
| Tyler Ohlsen | [ohltyler](https://github.com/ohltyler) | Amazon |
16-
| Vamshi Vijay Nakkirtha | [vamshin](https://github.com/vamshin) | Amazon |
17-
| Vijayan Balasubramanian | [VijayanB](https://github.com/VijayanB) | Amazon |
18-
| Yaliang Wu | [ylwu-amzn](https://github.com/ylwu-amzn) | Amazon |
19-
| Yizhe Liu | [yizheliu-amazon](https://github.com/yizheliu-amazon) | Amazon |
7+
| Maintainer | GitHub ID | Affiliation |
8+
| ----------------------- | ---------------------------------------------------------| ----------- |
9+
| Jack Mazanec | [jmazanec15](https://github.com/jmazanec15) | Amazon |
10+
| Jing Zhang | [jngz-es](https://github.com/jngz-es) | Amazon |
11+
| Kaituo Li | [kaituo](https://github.com/kaituo) | Amazon |
12+
| Sarat Vemulapalli | [saratvemulapalli](https://github.com/saratvemulapalli) | Amazon |
13+
| Tyler Ohlsen | [ohltyler](https://github.com/ohltyler) | Amazon |
14+
| Vamshi Vijay Nakkirtha | [vamshin](https://github.com/vamshin) | Amazon |
15+
| Vijayan Balasubramanian | [VijayanB](https://github.com/VijayanB) | Amazon |
16+
| Yaliang Wu | [ylwu-amzn](https://github.com/ylwu-amzn) | Amazon |
17+
| Amit Galitzky | [amitgalitz](https://github.com/amitgalitz) | Amazon |
18+
| Sudipto Guha | [sudiptoguha](https://github.com/sudiptoguha) | Amazon |
19+
| Jackie Han | [jackiehanyang](https://github.com/jackiehanyang) | Amazon |
20+
| Sean Zheng | [sean-zheng-amazon](https://github.com/sean-zheng-amazon)| Amazon |
21+
| Dan Widdis | [dbwiddis](https://github.com/dbwiddis) | Amazon |
22+
| Owais Kazi | [owaiskazi19](https://github.com/owaiskazi19) | Amazon |
23+
| Josh Palis | [joshpalis](https://github.com/joshpalis) | Amazon |
24+
25+
## Emeritus
26+
27+
| Maintainer | GitHub ID | Affiliation |
28+
| -------------- | ----------------------------------------------------- | ----------- |
29+
| Hanguang Zhang | [zhanghg08](https://github.com/zhanghg08) | Amazon |
30+
| Yizhe Liu | [yizheliu-amazon](https://github.com/yizheliu-amazon) | Amazon |
31+
| Lai Jiang | [wnbts](https://github.com/wnbts) | Amazon |

0 commit comments

Comments
 (0)