You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/async/index.md
+32-30
Original file line number
Diff line number
Diff line change
@@ -7,9 +7,9 @@ has_children: true
7
7
8
8
# Asynchronous Search
9
9
10
-
Searching large volumes of data can take a long time, especially if it’s across warm nodes or multiple remote clusters.
10
+
Searching large volumes of data can take a long time, especially if you're searching across warm nodes or multiple remote clusters.
11
11
12
-
Asynchronous search lets you run search requests that execute in the background. You can monitor the progress of these searches and get back partial results as they become available. After the search completes, you can save the results to examine at a later time.
12
+
Asynchronous search lets you run search requests that run in the background. You can monitor the progress of these searches and get back partial results as they become available. After the search finishes, you can save the results to examine at a later time.
13
13
14
14
Asynchronous search requires Elasticsearch 7.10.2 or higher.
15
15
@@ -25,11 +25,11 @@ You can specify the following options.
25
25
26
26
Options | Description | Default value | Required
27
27
:--- | :--- |:--- |:--- |
28
-
`wait_for_completion_timeout` | Specify the amount of time you plan to wait for the results. Whatever results you get within this time, you can see just like a normal search. The remaining results you can poll based on an ID. The maximum value is 300 seconds. | 1 second | No
29
-
`keep_on_completion` | Whether you want to save the results in the cluster after the search is complete. You can examine the stored results at a later time. | `false` | No
30
-
`keep_alive` | Specify the amount of time the result is saved in the cluster. For example, `2d` means that the results are stored in the cluster for 48 hours. The saved search results are deleted after this period or if the search is cancelled. Note that this includes the query execution time. If the query overruns this time, the process cancels this query automatically. | 12 hours | No
28
+
`wait_for_completion_timeout` | Specifies the amount of time that you plan to wait for the results. You can see whatever results you get within this timejust like in a normal search. You can poll the remaining results based on an ID. The maximum value is 300 seconds. | 1 second | No
29
+
`keep_on_completion` | Specifies whether you want to save the results in the cluster after the search is complete. You can examine the stored results at a later time. | `false` | No
30
+
`keep_alive` | Specifies the amount of time that the result is saved in the cluster. For example, `2d` means that the results are stored in the cluster for 48 hours. The saved search results are deleted after this period or if the search is cancelled. Note that this includes the query execution time. If the query overruns this time, the process cancels this query automatically. | 12 hours | No
31
31
32
-
#### Sample Request
32
+
#### Sample request
33
33
34
34
```json
35
35
POST _opendistro/_asynchronous_search/?pretty&size=10&wait_for_completion_timeout=1ms&keep_on_completion=true&request_cache=false
@@ -99,30 +99,16 @@ POST _opendistro/_asynchronous_search/?pretty&size=10&wait_for_completion_timeou
99
99
100
100
Options | Description
101
101
:--- | :---
102
-
`id` | Use the ID of an asynchronous searchto monitor its progress, get its partial results, and/or delete the results. If the asynchronous search completes within the timeout period, the response won’t include the ID as the results are not stored in the cluster.
103
-
`state` | Whether the search is still running or it has completed and if the results persist in the cluster. The possible states are `RUNNING`, `COMPLETED`, or`PERSISTED`.
102
+
`id` | The ID of an asynchronous search. Use this ID to monitor the progress of the search, get its partial results, and/or delete the results. If the asynchronous search finishes within the timeout period, the response doesn't include the ID because the results aren't stored in the cluster.
103
+
`state` | Specifies whether the search is still running or if it has finished, and if the results persist in the cluster. The possible states are `RUNNING`, `COMPLETED`, and`PERSISTED`.
104
104
`start_time_in_millis` | The start time in milliseconds.
105
105
`expiration_time_in_millis` | The expiration time in milliseconds.
106
106
`took` | The total time that the search is running.
107
107
`response` | The actual search response.
108
-
`num_reduce_phases` | The number of times the coordinating node aggregates results from batches of shard responses (5 by default). If this number increases compared to the last retrieved results, you can expect additional results included in the search response
109
-
`total` | The total number of shards that executes the search.
110
-
`successful` | The number of shard responses the coordinating node received successfully.
111
-
`aggregations` | The partial aggregation results completed by the shards so far.
112
-
113
-
This table lists the possible states.
114
-
115
-
Options | Description
116
-
:--- | :---
117
-
`submitted` | The number of asynchronous search requests submitted.
118
-
`initialized` | The number of asynchronous search contexts successfully created in memory.
119
-
`rejected` | The number of asynchronous search requests rejected.
120
-
`search_completed` | The number of asynchronous search requests that completed with a successful response.
121
-
`search_failed` | The number of asynchronous search requests that completed with a failed response.
122
-
`persisted` | The number of asynchronous search requests whose final result successfully persisted in the cluster.
123
-
`persist_failed` | The number of asynchronous search requests whose final result failed to persist in the cluster.
124
-
`running_current` | The number of asynchronous search requests running on a given coordinator node.
125
-
`cancelled` | The number of asynchronous search requests cancelled when running.
108
+
`num_reduce_phases` | The number of times that the coordinating node aggregates results from batches of shard responses (5 by default). If this number increases compared to the last retrieved results, you can expect additional results to be included in the search response.
109
+
`total` | The total number of shards that run the search.
110
+
`successful` | The number of shard responses that the coordinating node received successfully.
111
+
`aggregations` | The partial aggregation results that have been completed by the shards so far.
126
112
127
113
## Get partial results
128
114
@@ -194,13 +180,15 @@ GET _opendistro/_asynchronous_search/<ID>?pretty
194
180
}
195
181
```
196
182
197
-
You can poll the ID with the `wait_for_completion_timeout` parameter to wait for the results received for the time you specify.
183
+
After the response is successfully persisted, you get back the `STORE_RESIDENT` state in the response.
184
+
185
+
You can poll the ID with the `wait_for_completion_timeout` parameter to wait for the results received for the time that you specify.
198
186
199
-
For asynchronous searches with `keep_on_completion` as `true` and a sufficiently long `keep_alive` time, you can keep polling the IDs until the search completes. If you don’t want to periodically poll each ID, you can retain the results in your cluster with the `keep_alive` parameter and come back to it at a later time.
187
+
For asynchronous searches with `keep_on_completion` as `true` and a sufficiently long `keep_alive` time, you can keep polling the IDs until the search finishes. If you don’t want to periodically poll each ID, you can retain the results in your cluster with the `keep_alive` parameter and come back to it at a later time.
200
188
201
189
## Delete searches and results
202
190
203
-
You can use the DELETE API to delete any ongoing asynchronous search by its ID. If the search is still running, it’s cancelled. If the search is complete, the saved search results are deleted.
191
+
You can use the DELETE API operation to delete any ongoing asynchronous search by its ID. If the search is still running, it’s canceled. If the search is complete, the saved search results are deleted.
Copy file name to clipboardExpand all lines: docs/async/security.md
+8-7
Original file line number
Diff line number
Diff line change
@@ -8,13 +8,13 @@ has_children: false
8
8
9
9
# Asynchronous search security
10
10
11
-
You can use the security plugin with asynchronous searches to limit non-admin users to specific actions. For example, you might want some users to only be able to submit or delete asynchronous searches, while others to only view its results.
11
+
You can use the security plugin with asynchronous searches to limit non-admin users to specific actions. For example, you might want some users to only be able to submit or delete asynchronous searches, while you might want others to only view the results.
12
12
13
-
All asynchronous search indices are protected as system indices. Only a super admin user or an admin user with a TLS certificate can access system indices. For more information, see [System indices](../../security/configuration/system-indices/).
13
+
All asynchronous search indices are protected as system indices. Only a super admin user or an admin user with a Transport Layer Security (TLS) certificate can access system indices. For more information, see [System indices](../../security/configuration/system-indices/).
14
14
15
15
## Basic permissions
16
16
17
-
As an admin user, you can use the security plugin to assign specific permissions to users based on which APIs they need access to. For a list of supported APIs, see [Asynchronous Search](../).
17
+
As an admin user, you can use the security plugin to assign specific permissions to users based on which API operations they need access to. For a list of supported APIs operations, see [Asynchronous search](../).
18
18
19
19
The security plugin has two built-in roles that cover most asynchronous search use cases: `asynchronous_search_full_access` and `asynchronous_search_read_access`. For descriptions of each, see [Predefined roles](../../security/access-control/users-roles/#predefined-roles).
20
20
@@ -24,9 +24,9 @@ If these roles don’t meet your needs, mix and match individual asynchronous se
24
24
25
25
Use backend roles to configure fine-grained access to asynchronous searches based on roles. For example, users of different departments in an organization can view asynchronous searches owned by their own department.
26
26
27
-
First, make sure that your users have the appropriate [backend roles](../../security/access-control/). Backend roles usually come from an [LDAP server](../../security/configuration/ldap/) or [SAML provider](../../security/configuration/saml/), but if you use the internal user database, you can use the REST API to [add them manually](../../security/access-control/api/#create-user).
27
+
First, make sure that your users have the appropriate [backend roles](../../security/access-control/). Backend roles usually come from an [LDAP server](../../security/configuration/ldap/) or [SAML provider](../../security/configuration/saml/). However, if you use the internal user database, you can use the REST API to [add them manually](../../security/access-control/api/#create-user).
28
28
29
-
Now when users view asynchronous search resources in Kibana (or make REST API calls), they only see asynchronous searches submitted by users who have a subset of the backend role.
29
+
Now when users view asynchronous search resources in Kibana (or make REST API calls), they only see asynchronous searches that are submitted by users who have a subset of the backend role.
30
30
For example, consider two users: `judy` and `elon`.
31
31
32
32
`judy` has an IT backend role:
@@ -71,5 +71,6 @@ PUT _opendistro/_security/api/rolesmapping/async_full_access
71
71
72
72
Because they have different backend roles, an asynchronous search submitted by `judy` will not be visible to `elon` and vice versa.
73
73
74
-
`judy` needs to have at least the superset of all roles that `elon` has to see the asynchronous searches that `elon` started.
75
-
For example, if `judy` has 5 backend roles and `elon` one has 1 of these roles, then `judy` can see asynchronous searches submitted by `elon` but `elon` can’t see the asynchronous searches submitted by `judy`. This means that `judy` can perform GET and DELETE operations on asynchronous searches submitted by `elon` but not vice versa.
74
+
`judy` needs to have at least the superset of all roles that `elon` has to see `elon`'s asynchronous searches.
75
+
76
+
For example, if `judy` has five backend roles and `elon` one has one of these roles, then `judy` can see asynchronous searches submitted by `elon`, but `elon` can’t see the asynchronous searches submitted by `judy`. This means that `judy` can perform GET and DELETE operations on asynchronous searches that are submitted by `elon`, but not the reverse.
Copy file name to clipboardExpand all lines: docs/async/settings.md
+4-5
Original file line number
Diff line number
Diff line change
@@ -7,8 +7,7 @@ nav_order: 4
7
7
8
8
# Settings
9
9
10
-
The asynchronous search plugin adds several settings to the standard Elasticsearch cluster settings. They are dynamic, so you can change the default behavior of the plugin without restarting your cluster.
11
-
You can mark them `persistent` or `transient`.
10
+
The asynchronous search plugin adds several settings to the standard Elasticsearch cluster settings. They are dynamic, so you can change the default behavior of the plugin without restarting your cluster. You can mark the settings as `persistent` or `transient`.
12
11
13
12
For example, to update the retention period of the result index:
14
13
@@ -23,7 +22,7 @@ PUT _cluster/settings
23
22
24
23
Setting | Default | Description
25
24
:--- | :--- | :---
26
-
`opendistro_asynchronous_search.max_search_running_time` | 12 hours | Maximum running time for the search beyond which the search is terminated.
27
-
`opendistro_asynchronous_search.max_running_searches` | 20 | Concurrent searches running per coordinator node.
28
-
`opendistro_asynchronous_search.max_keep_alive` | 5 days | The maximum amount of time search results can be stored in the cluster.
25
+
`opendistro_asynchronous_search.max_search_running_time` | 12 hours | The maximum running time for the search beyond which the search is terminated.
26
+
`opendistro_asynchronous_search.max_running_searches` | 20 | The concurrent searches running per coordinator node.
27
+
`opendistro_asynchronous_search.max_keep_alive` | 5 days | The maximum amount of time that search results can be stored in the cluster.
29
28
`opendistro_asynchronous_search.max_wait_for_completion_timeout` | 1 minute | The maximum value for the `wait_for_completion_timeout` parameter.
0 commit comments