Skip to content

Commit fcbfc10

Browse files
cwperksXtansia
cwperks
and
Xtansia
authored
Update PluginStats with new field in response optional_extended_plugins and upgrade to 2.19.0 (#814)
* Update PluginStats with new field in response optional_extended_plugins Signed-off-by: Craig Perkins <[email protected]> * Fix lint error and add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Update test-spec Signed-off-by: Craig Perkins <[email protected]> * Add test for /_cluster/stats/{metric}/nodes/{node_id} Signed-off-by: Craig Perkins <[email protected]> * Add mechanism to assert that response contains given a list of target values Signed-off-by: Craig Perkins <[email protected]> * Add additional unit test Signed-off-by: Craig Perkins <[email protected]> * Revert "Add additional unit test" This reverts commit e555bc2. Signed-off-by: Craig Perkins <[email protected]> * Revert "Add mechanism to assert that response contains given a list of target values" This reverts commit e9c9eb1. Signed-off-by: Craig Perkins <[email protected]> * Remove test Signed-off-by: Craig Perkins <[email protected]> * Remove from required and update sha Signed-off-by: Craig Perkins <[email protected]> * Update FlowFrameworkDeleteResponse to use WriteResponseBase Signed-off-by: Thomas Farr <[email protected]> * Fix k-NN model training tests to work with small training sample set Signed-off-by: Thomas Farr <[email protected]> * Add `id` and `group_by` to `TopQuery` Signed-off-by: Thomas Farr <[email protected]> * Even smaller k-NN model training Signed-off-by: Thomas Farr <[email protected]> * Fix k-NN tests Signed-off-by: Thomas Farr <[email protected]> * Fix flow framework test Signed-off-by: Thomas Farr <[email protected]> * Split k-NN plugin tests out of default Signed-off-by: Thomas Farr <[email protected]> * Simplify flow framework delete response Signed-off-by: Thomas Farr <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Thomas Farr <[email protected]> Co-authored-by: Thomas Farr <[email protected]>
1 parent 595fb05 commit fcbfc10

File tree

15 files changed

+157
-82
lines changed

15 files changed

+157
-82
lines changed

‎.github/workflows/test-spec.yml

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -28,40 +28,42 @@ jobs:
2828
admin_password: admin
2929
- version: 2.0.0
3030
admin_password: admin
31-
- version: 2.18.0
32-
- version: 2.18.0
31+
- version: 2.19.0
32+
- version: 2.19.0
3333
tests: plugins/index_state_management
34-
- version: 2.18.0
34+
- version: 2.19.0
3535
tests: plugins/ml
36-
- version: 2.18.0
36+
- version: 2.19.0
3737
tests: routing
38-
- version: 2.18.0
38+
- version: 2.19.0
3939
tests: snapshot
40-
- version: 2.18.0
40+
- version: 2.19.0
4141
tests: remote_store
42-
- version: 2.18.0
42+
- version: 2.19.0
4343
tests: dangling
4444
url: http://localhost:9200
45-
- version: 2.18.0
45+
- version: 2.19.0
4646
tests: plugins/replication
4747
url: http://localhost:9200
48-
- version: 2.18.0
48+
- version: 2.19.0
4949
tests: plugins/streaming
50-
- version: 2.18.0
50+
- version: 2.19.0
5151
tests: plugins/notifications
52-
- version: 2.18.0
52+
- version: 2.19.0
5353
tests: plugins/query_insights
54-
- version: 2.18.0
54+
- version: 2.19.0
5555
tests: plugins/workload-management
56-
- version: 2.18.0
56+
- version: 2.19.0
5757
tests: plugins/analysis
58-
- version: 2.18.0
58+
- version: 2.19.0
59+
tests: plugins/knn
60+
- version: 2.19.0
5961
tests: plugins/security
6062
cert: tests/plugins/security/.kirk.pem
6163
key: tests/plugins/security/.kirk-key.pem
62-
- version: 2.19.0
64+
- version: 2.20.0
6365
hub: opensearchstaging
64-
ref: '@sha256:4da23e0137b2b67206d23b36fcf0914cc39b3bf19310c782f536e4934b86f6cc'
66+
ref: '@sha256:037a2eb5a4f48b9a9a29950be5e5fae4ebd82d41de1066e00bb8d8b0ce7871b1'
6567
- version: 3.0.0
6668
hub: opensearchstaging
6769
ref: '@sha256:727643acdfebed77bfdb26362dbcff536b7ea02a0cc4ae2da2521729171333de'

‎CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,8 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
6161
- Added `POST /_plugins/_ml/_execute/{algorithm_name}` ([#811](https://github.com/opensearch-project/opensearch-api-specification/pull/811))
6262
- Added search suggester types ([#817](https://github.com/opensearch-project/opensearch-api-specification/pull/817))
6363
- Added `SimplePatternTokenizer` and `SimplePatternSplitTokenizer` ([#820](https://github.com/opensearch-project/opensearch-api-specification/pull/820))
64+
- Added `optional_extended_plugins` field to `PluginStats` ([#814](https://github.com/opensearch-project/opensearch-api-specification/pull/814))
65+
- Added `id` & `group_by` fields to `TopQuery` ([#814](https://github.com/opensearch-project/opensearch-api-specification/pull/814))
6466

6567
### Removed
6668
- Removed unsupported `_common.mapping:SourceField`'s `mode` field and associated `_common.mapping:SourceFieldMode` enum ([#652](https://github.com/opensearch-project/opensearch-api-specification/pull/652))
@@ -113,6 +115,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
113115
- Fixed GeoDistanceSort schema ([#805](https://github.com/opensearch-project/opensearch-api-specification/pull/805))
114116
- Fixed Aggregation schemas ([#801](https://github.com/opensearch-project/opensearch-api-specification/pull/801))
115117
- Fixed FilterQueryRequestProcessor to use correct query type ([#821](https://github.com/opensearch-project/opensearch-api-specification/pull/821))
118+
- Fixed `knn.train_model`'s request body `method` field to accept an object rather than a string ([#814](https://github.com/opensearch-project/opensearch-api-specification/pull/814))
116119

117120
### Changed
118121
- Changed `tasks._common:TaskInfo` and `tasks._common:TaskGroup` to be composed of a `tasks._common:TaskInfoBase` ([#683](https://github.com/opensearch-project/opensearch-api-specification/pull/683))
@@ -121,6 +124,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
121124
- Changed naming of `snapshot._common`'s `Status`, `ShardsStats`, `ShardsStatsStage`, `ShardsStatsSummary` and `ShardsStatsSummaryItem` schemas to be prefixed with `Snapshot` ([#730](https://github.com/opensearch-project/opensearch-api-specification/pull/730))
122125
- Changed `ml.get_memory` and `ml.get_message` to split out `get_all` variants ([#796](https://github.com/opensearch-project/opensearch-api-specification/pull/796))
123126
- Changed `ml.get_tools` to have two different operation groups `ml.get_all_tools` and `ml.get_tool` ([#799](https://github.com/opensearch-project/opensearch-api-specification/pull/799))
127+
- Changed `FlowFrameworkDeleteResponse` to utilize `WriteResponseBase` ([#814](https://github.com/opensearch-project/opensearch-api-specification/pull/814))
124128

125129
## [0.1.0] - 2024-10-25
126130

‎spec/namespaces/flow_framework.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -444,7 +444,7 @@ components:
444444
content:
445445
application/json:
446446
schema:
447-
$ref: '../schemas/flow_framework.common.yaml#/components/schemas/FlowFrameworkDeleteResponse'
447+
$ref: '../schemas/_common.yaml#/components/schemas/WriteResponseBase'
448448
flow_framework.get@200:
449449
content:
450450
application/json:

‎spec/namespaces/knn.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -390,7 +390,7 @@ components:
390390
in: query
391391
description: The default operator for query string query (AND or OR).
392392
schema:
393-
$ref: '../schemas/knn._common.yaml#/components/schemas/DefaultOperator'
393+
$ref: '../schemas/_common.query_dsl.yaml#/components/schemas/Operator'
394394
knn.search_models::query.df:
395395
name: df
396396
in: query
@@ -518,7 +518,7 @@ components:
518518
in: query
519519
description: Search operation type.
520520
schema:
521-
$ref: '../schemas/knn._common.yaml#/components/schemas/SearchType'
521+
$ref: '../schemas/_common.yaml#/components/schemas/SearchType'
522522
knn.search_models::query.seq_no_primary_term:
523523
name: seq_no_primary_term
524524
in: query
@@ -580,7 +580,7 @@ components:
580580
in: query
581581
description: Specify suggest mode.
582582
schema:
583-
$ref: '../schemas/knn._common.yaml#/components/schemas/SuggestMode'
583+
$ref: '../schemas/_common.yaml#/components/schemas/SuggestMode'
584584
knn.search_models::query.suggest_size:
585585
name: suggest_size
586586
in: query

‎spec/schemas/_common.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1516,6 +1516,11 @@ components:
15161516
type: ['null', string]
15171517
opensearch_version:
15181518
$ref: '#/components/schemas/VersionString'
1519+
optional_extended_plugins:
1520+
type: array
1521+
items:
1522+
type: string
1523+
x-version-added: '2.19'
15191524
required:
15201525
- classname
15211526
- description

‎spec/schemas/flow_framework.common.yaml

Lines changed: 18 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -76,30 +76,6 @@ components:
7676
version:
7777
$ref: '#/components/schemas/version'
7878
minProperties: 1
79-
FlowFrameworkDeleteResponse:
80-
type: object
81-
properties:
82-
_index:
83-
type: string
84-
description: The name of the index where the document was stored.
85-
_id:
86-
type: string
87-
description: The unique identifier of the deleted document.
88-
_version:
89-
type: integer
90-
description: The version of the document after the deletion.
91-
result:
92-
type: string
93-
description: The result of the deletion operation.
94-
enum: [deleted, not_found]
95-
_shards:
96-
$ref: '#/components/schemas/shards'
97-
_seq_no:
98-
type: integer
99-
description: The sequence number assigned to the document after the deletion.
100-
_primary_term:
101-
type: integer
102-
description: The primary term assigned to the document after the deletion.
10379
FlowFrameworkGetResponse:
10480
type: object
10581
properties:
@@ -148,14 +124,6 @@ components:
148124
skipped:
149125
type: integer
150126
description: The number of shards where the deletion was skipped.
151-
query:
152-
type: object
153-
description: The search query to match workflows. Use `match_all` to retrieve all workflows, or `match` to search by specific fields like `use_case`.
154-
properties:
155-
match:
156-
type: object
157-
match_all:
158-
type: object
159127
total:
160128
type: object
161129
properties:
@@ -224,6 +192,23 @@ components:
224192
type: string
225193
user:
226194
$ref: '#/components/schemas/user'
195+
provision_start_time:
196+
$ref: '_common.yaml#/components/schemas/DateTime'
197+
provision_end_time:
198+
$ref: '_common.yaml#/components/schemas/DateTime'
199+
resources_created:
200+
$ref: '#/components/schemas/ResourcesCreated'
201+
ResourcesCreated:
202+
type: object
203+
properties:
204+
workflow_step_name:
205+
type: string
206+
workflow_step_id:
207+
type: string
208+
resource_type:
209+
type: string
210+
resource_id:
211+
type: string
227212
user:
228213
type: object
229214
properties:
@@ -367,7 +352,7 @@ components:
367352
type: object
368353
properties:
369354
query:
370-
$ref: '#/components/schemas/query'
355+
$ref: '_common.query_dsl.yaml#/components/schemas/QueryContainer'
371356
WorkflowSearchResponse:
372357
type: object
373358
properties:

‎spec/schemas/insights._common.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,10 @@ components:
1919
TopQuery:
2020
type: object
2121
properties:
22+
id:
23+
type: string
24+
group_by:
25+
$ref: '#/components/schemas/GroupingType'
2226
timestamp:
2327
type: integer
2428
description: The timestamp of the query execution.
@@ -56,6 +60,13 @@ components:
5660
measurements:
5761
type: object
5862
$ref: '#/components/schemas/Measurements'
63+
GroupingType:
64+
type: string
65+
enum:
66+
- NONE
67+
- SIMILARITY
68+
- none
69+
- similarity
5970
TaskResourceUsages:
6071
type: object
6172
properties:

‎spec/schemas/knn._common.yaml

Lines changed: 15 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -6,25 +6,6 @@ info:
66
paths: {}
77
components:
88
schemas:
9-
DefaultOperator:
10-
type: string
11-
description: The default operator for query string query (AND or OR).
12-
enum:
13-
- AND
14-
- OR
15-
SearchType:
16-
type: string
17-
description: Search operation type.
18-
enum:
19-
- dfs_query_then_fetch
20-
- query_then_fetch
21-
SuggestMode:
22-
type: string
23-
description: Specify suggest mode.
24-
enum:
25-
- always
26-
- missing
27-
- popular
289
Stats:
2910
type: object
3011
properties:
@@ -143,6 +124,20 @@ components:
143124
$ref: '_common.yaml#/components/schemas/ByteCount'
144125
current_size_in_bytes:
145126
$ref: '_common.yaml#/components/schemas/ByteCount'
127+
KnnMethod:
128+
type: object
129+
properties:
130+
name:
131+
type: string
132+
space_type:
133+
type: string
134+
engine:
135+
type: string
136+
parameters:
137+
type: object
138+
additionalProperties: true
139+
required:
140+
- name
146141
TrainedModel:
147142
type: object
148143
properties:
@@ -166,7 +161,7 @@ components:
166161
compression_level:
167162
type: string
168163
method:
169-
type: string
164+
$ref: '#/components/schemas/KnnMethod'
170165
spaceType:
171166
type: string
172167
required:

‎tests/default/flow_framework/workflow/state/search.yaml

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,19 +18,20 @@ prologues:
1818
output:
1919
workflow_id: payload.workflow_id
2020
chapters:
21-
- synopsis: Search for a workflow.
21+
- synopsis: Search for a workflow state.
2222
path: /_plugins/_flow_framework/workflow/state/_search
2323
method: GET
2424
request:
2525
payload:
2626
query:
27-
match_all: {}
27+
term:
28+
workflow_id:
29+
value: ${workflow.workflow_id}
2830
response:
2931
status: 200
3032
payload:
3133
hits:
3234
hits:
3335
- _index: .plugins-flow-framework-state
3436
_source:
35-
state: NOT_STARTED
36-
37+
workflow_id: ${workflow.workflow_id}

‎tests/plugins/knn/docker-compose.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
version: '3'
2+
3+
services:
4+
opensearch-cluster:
5+
image: ${OPENSEARCH_DOCKER_HUB_PROJECT:-opensearchproject}/opensearch:${OPENSEARCH_VERSION:-latest}${OPENSEARCH_DOCKER_REF}
6+
ports:
7+
- 9200:9200
8+
- 9600:9600
9+
environment:
10+
- OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_PASSWORD:-myStrongPassword123!}
11+
- OPENSEARCH_JAVA_OPTS=${OPENSEARCH_JAVA_OPTS}
12+
- discovery.type=single-node

‎tests/default/knn/models.yaml renamed to ‎tests/plugins/knn/knn/models.yaml

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
$schema: ../../../json_schemas/test_story.schema.yaml
1+
$schema: ../../../../json_schemas/test_story.schema.yaml
22

33
description: Delete a k-NN model.
44
version: '>= 2.17'
@@ -41,6 +41,15 @@ prologues:
4141
mode: on_disk
4242
compression_level: 32x
4343
spaceType: l2
44+
method:
45+
name: ivf
46+
engine: faiss
47+
parameters:
48+
nlist: 2
49+
encoder:
50+
name: pq
51+
parameters:
52+
code_size: 1
4453
status: [200]
4554
epilogues:
4655
- path: /movies

‎tests/default/knn/models/search.yaml renamed to ‎tests/plugins/knn/knn/models/search.yaml

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
$schema: ../../../../json_schemas/test_story.schema.yaml
1+
$schema: ../../../../../json_schemas/test_story.schema.yaml
22

33
description: Search for a k-NN model.
44
version: '>= 2.17'
@@ -41,6 +41,25 @@ prologues:
4141
mode: on_disk
4242
compression_level: 32x
4343
spaceType: l2
44+
method:
45+
name: ivf
46+
engine: faiss
47+
parameters:
48+
nlist: 2
49+
encoder:
50+
name: pq
51+
parameters:
52+
code_size: 1
53+
status: [200]
54+
- method: GET
55+
path: /_plugins/_knn/models/model-1
56+
retry:
57+
count: 3
58+
response:
59+
status: 200
60+
payload:
61+
model_id: model-1
62+
state: created
4463
status: [200]
4564
epilogues:
4665
- path: /movies

0 commit comments

Comments
 (0)