Skip to content

Commit b405750

Browse files
nhtruongnhtruong
authored
Updated/Corrected Docs (#270)
* Updated/Corrected Docs - README.md - CLIENT_GENERATOR_GUIDE.md - DEVELOPER_GUIDE.md - ./tools/README.md Signed-off-by: Theo Truong <[email protected]> * # minor corrections Signed-off-by: Theo Truong <[email protected]> * # minor corrections Signed-off-by: Theo Truong <[email protected]> --------- Signed-off-by: Theo Truong <[email protected]>
1 parent 21cdfe9 commit b405750

File tree

4 files changed

+74
-42
lines changed

4 files changed

+74
-42
lines changed

‎CLIENT_GENERATOR_GUIDE.md

+7-8
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,13 @@ In a client, the `search` operations are grouped in to a single API method, `cli
1919

2020
In the [published OpenAPI spec](https://github.com/opensearch-project/opensearch-api-specification/releases), this grouping is denoted by `x-operation-group` vendor extension in every operation definition. The value of this extension is the name of the API action (like `search` or `indices.get_field_mapping`). Operations with the same `x-operation-group` value are guaranteed to have the same query string parameters, response body, and request body (for PUT/POST/DELETE operations). Common path parameters are also guaranteed to be the same. The only differences between operations are the HTTP method and the path. With that in mind, below are rules on how to combine operations of different HTTP methods and path compositions.
2121

22-
- If an operation is marked with `x-ignorable: "true"`, then ignore the operation. Such an operation has been deprecated and has been replaced by a newer one. As far as the clients are concerned, ignorable operations do not exist.
22+
- If an operation is marked with `x-ignorable: true`, then ignore the operation. Such an operation has been deprecated and has been superseded by a newer one. As far as the clients are concerned, ignorable operations do not exist.
2323
- If two operations have identical HTTP methods, but different paths: use the path that best matches the path parameters provided.
2424
- If two operations have identical path, but different HTTP methods:
2525
- GET/POST: if the request body is provided then use POST, otherwise use GET
2626
- PUT/POST: Either works, but PUT is preferred when an optional path parameter is provided.
2727

28-
The psuedo-code that combines the `search` operations into a single API method is as follows:
28+
The pseudocode that combines the `search` operations into a single API method is as follows:
2929
```python
3030
def search(self, index=None, body=None):
3131
if index is None:
@@ -41,20 +41,19 @@ def search(self, index=None, body=None):
4141
return self.perform_request(method, path, body=body)
4242
```
4343

44-
4544
## Overloaded Name
46-
You will also encounter `x-overloaded-param: "metric"` for the `node_id` path parameter of `GET /_nodes/{node_id}` operation in `nodes.info` action. This is a special case where the path parameter is overloaded to accept either a node ID or a metric name. The `client.nodes.info` method when called with either `metric` or `node_id` (but not both), will use `GET /_nodes/{node_id}` operation (even though the path parameter name is `node_id`). When called with both `metric` and `node_id`, it will use `GET /_nodes/{node_id}/{metric}` operation.
45+
You will also encounter `x-overloaded-param: metric` for the `node_id` path parameter of the `GET /_nodes/{node_id}` operation in `nodes.info` action. This is a special case where the path parameter is overloaded to accept either a node ID or a metric name. When the user evokes the `client.nodes.info` method with either `metric` or `node_id` (but not both), the method will use the `GET /_nodes/{node_id}` operation. When evoked with both `metric` and `node_id`, it will use the `GET /_nodes/{node_id}/{metric}` operation.
4746

4847
## Handling Bulk Operations
49-
Some operations accept a bulk of data in the request body. For example, the `bulk` action accepts a bulk of index, update, and delete operations on multiple documents. Unlike other operations where the request body is a JSON object, the request body for bulk operations is a newline-seperated JSON string. The client will automatically convert the request body into a newline-seperated JSON objects. The request body of such operations will be denoted with `x-serialize: "bulk"` vendor extension.
48+
Some operations accept a bulk of data in the request body. For example, the `bulk` action accepts a bulk of index, update, and delete operations on multiple documents. Unlike other operations where the request body is a **JSON object**, the request body for bulk operations is an **NDJSON** (i.e a [Newline-delimited JSON](https://github.com/ndjson/ndjson-spec)). When encountering this type of operation, the client must serialize the request body accordingly, and set the `Content-Type` header to `application/x-ndjson`.
5049

5150
## Parameter Validation
5251
As of right now, most clients only validate whether required parameters are present. The clients do not validate the values of parameters against the enum values or regex patterns. This is to reduce performance overhead for the clients as the validation is already done on the server. However, the list of enum values and regex patterns are often written into the parameter description.
5352

54-
Some clients also check for the validity of query string parameter names to guard the users from typos. If you decide to implement this feature, make sure that it's performant. Scripting languages like Python and Ruby require the code to be loaded into memory at runtime, and constructs used for this feature can be expensive to load, as far as micro-services are concerned.
53+
Some clients also check for the validity of query string parameter names to guard the users from typos. If you decide to implement this feature, make sure that it's performant. Scripting languages like Python and Ruby require the code to be loaded into memory at runtime, and constructs used for this feature can be expensive to load, as far as microservices are concerned.
5554

5655
## Global Parameters
57-
All operations in the spec contain a set of parameters that are common across all operations. These parameters are denoted with `x-global: true` vendor extension. The generated clients should find a way to DRY these parameters.
56+
All operations in the spec contain a set of parameters that are common across all operations. These parameters are denoted with `x-global: true` vendor extension. The generated clients should find a way to DRY these parameters in type definitions and method documentation.
5857

5958
## Default Parameter Values
60-
Parameters can have default values either through schema or the `x-default` vendor extension. When both are present, `x-default` will takes precedence.
59+
Parameters can have default values either through schema or the `x-default` vendor extension. When both are present, `x-default` will take precedence.

‎DEVELOPER_GUIDE.md

+41-27
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,10 @@
33
- [File Structure](#file-structure)
44
- [Grouping Operations](#grouping-operations)
55
- [Grouping Schemas](#grouping-schemas)
6+
- [Superseded Operations](#superseded-operations)
7+
- [Global Parameters](#global-parameters)
68
- [OpenAPI Extensions](#openapi-extensions)
7-
- [Linting](#linting)
9+
- [Tools](#tools)
810

911
# Developer Guide
1012

@@ -20,10 +22,6 @@ The Specification is written in OpenAPI 3, so understanding the OpenAPI 3 specif
2022

2123
To make editing the specification easier, we split the OpenAPI spec into multiple files that can be found in the [spec](spec) directory. The file structure is as follows:
2224

23-
- The API Operations are grouped by namespaces in [spec/namespaces](spec/namespaces/) directory. Each `.yaml` file in this directory represents a namespace and holds all paths and operations of the namespace.
24-
- The data schemas are grouped by categories in [spec/schemas](spec/schemas/) directory. Each `.yaml` file in this directory represents a category.
25-
- The [spec/opensearch-openapi.yaml](spec/opensearch-openapi.yaml) file is the OpenAPI root file that ties everything together.
26-
2725
```
2826
spec
2927
│
@@ -46,7 +44,11 @@ spec
4644
└── opensearch-openapi.yaml
4745
```
4846

49-
Every `.yaml` file is a valid OpenAPI 3 document. This means that you can use any OpenAPI 3 compatible tool to view and edit the files, and IDEs with OpenAPI support will provide you with autocompletion and validation in real-time.
47+
- The API Operations are grouped by namespaces in [spec/namespaces/](spec/namespaces) directory. Each file in this directory represents a namespace and holds all paths and operations of the namespace.
48+
- The data schemas are grouped by categories in [spec/schemas/](spec/schemas) directory. Each file in this directory represents a category.
49+
- The [spec/opensearch-openapi.yaml](spec/opensearch-openapi.yaml) file is the OpenAPI root file that ties everything together.
50+
51+
Every `.yaml` file is a OpenAPI 3 document. This means that you can use any OpenAPI 3 compatible tool to view and edit the files, and IDEs with OpenAPI support will also offer autocomplete and validation in realtime.
5052

5153
## Grouping Operations
5254

@@ -65,16 +67,44 @@ For this reason, every operation *must* be accompanied by the `x-operation-group
6567

6668
## Grouping Schemas
6769

68-
Schemas are grouped by categories to keep their names short and aid in client generation:
70+
Schemas are grouped by categories to keep their names short, and aid in client generation (where the schemas are translated into data types/classes, and divided into packages/modules). The schema file names can be in one of the following formats:
6971

7072
- `_common` category holds the common schemas that are used across multiple namespaces and features.
7173
- `_common.<sub_category>` category holds the common schemas of a specific sub_category. (e.g. `_common.mapping`)
7274
- `<namespace>._common` category holds the common schemas of a specific namespace. (e.g. `cat._common`, `_core._common`)
7375
- `<namespace>.<action>` category holds the schemas of a specific sub_category of a namespace. (e.g. `cat.aliases`, `_core.search`)
7476

77+
## Superseded Operations
78+
79+
When an operation is superseded by another operation with **identical functionality**, that is a rename or a change in the URL, it should be listed in [_superseded_operations.yaml](./spec/_superseded_operations.yaml) file. The merger tool will automatically generate the superseded operation in the OpenAPI spec. The superseded operation will have `deprecated: true` and `x-ignorable: true` properties to indicate that it should be ignored by the client generator.
80+
81+
For example, if the `_superseded_operations.yaml` file contains the following entry:
82+
```yaml
83+
/_opendistro/_anomaly_detection/{nodeId}/stats/{stat}:
84+
superseded_by: /_plugins/_anomaly_detection/{nodeId}/stats/{stat}
85+
operations:
86+
- GET
87+
- POST
88+
```
89+
Then, the merger tool will generate 2 superseded operations:
90+
- `GET /_opendistro/_anomaly_detection/{nodeId}/stats/{stat}`
91+
- `POST /_opendistro/_anomaly_detection/{nodeId}/stats/{stat}`
92+
93+
from their respective superseding operations:
94+
95+
- `GET /_plugins/_anomaly_detection/{nodeId}/stats/{stat}`
96+
- `POST /_plugins/_anomaly_detection/{nodeId}/stats/{stat}`
97+
98+
if and only if the superseding operations exist in the spec. A warning will be printed on the console if they do not.
99+
100+
Note that the path parameter names do not need to match. So, if the actual superseding operations have path of `/_plugins/_anomaly_detection/{node_id}/stats/{stat_id}`, the merger tool will recognize that it is the same as `/_plugins/_anomaly_detection/{nodeId}/stats/{stat}` and generate the superseded operations accordingly with the correct path parameter names.
101+
102+
## Global Parameters
103+
Certain query parameters are global, and they are accepted by every operation. These parameters are listed in the [root file](spec/opensearch-openapi.yaml) under the `parameters` section with `x-global` set to true. The merger tool will automatically add these parameters to all operations.
104+
75105
## OpenAPI Extensions
76106

77-
This repository includes several penAPI Specification Extensions to fill in any metadata not directly supported OpenAPI:
107+
This repository includes several OpenAPI Specification Extensions to fill in any metadata not natively supported by OpenAPI:
78108

79109
- `x-operation-group`: Used to group operations into API actions.
80110
- `x-version-added`: OpenSearch version when the operation/parameter was added.
@@ -87,28 +117,12 @@ This repository includes several penAPI Specification Extensions to fill in any
87117

88118
## Tools
89119

90-
We authored a number of tools to merge and lint specs that live in [tools](tools/). All tools have tests (run with `npm run test`) and a linter (run with `npm run lint`).
120+
We authored a number of tools to merge and lint specs that live in [tools](tools). All tools have tests (run with `npm run test`) and a linter (run with `npm run lint`).
91121

92122
### Merger
93123

94-
The spec merger "builds", aka combines various `.yaml` files into a complete OpenAPI spec. A [workflow](./.github/workflows/build.yml) publishes the output into [releases](https://github.com/opensearch-project/opensearch-api-specification/releases).
95-
96-
#### Auto-generating Superseded Operations
97-
98-
When an operation is superseded by another operation with **IDENTICAL FUNCTIONALITY**, that is a rename or a change in the URL, it should be listed in [_superseded_operations.yaml](./spec/_superseded_operations.yaml) file. The merger tool will automatically generate the superseded operation in the OpenAPI spec. The superseded operation will have `deprecated` and `x-ignorable` properties set to `true` to indicate that it should be ignored by the client generator.
99-
For example, if the `_superseded_operations.yaml` file contains the following entry:
100-
```yaml
101-
/_opendistro/_anomaly_detection/{nodeId}/stats/{stat}:
102-
superseded_by: /_plugins/_anomaly_detection/{nodeId}/stats/{stat}
103-
operations:
104-
- GET
105-
- POST
106-
```
107-
Then, the merger tool will generate 2 operations: `GET /_opendistro/_anomaly_detection/{nodeId}/stats/{stat}` and `POST /_opendistro/_anomaly_detection/{nodeId}/stats/{stat}` from `GET /_plugins/_anomaly_detection/{nodeId}/stats/{stat}` and `POST /_plugins/_anomaly_detection/{nodeId}/stats/{stat}` respectively, if they exist (A warning will be printed on the console if they do not). Note that the path parameter names do not need to match. So, if the actual superseding operations have path of `/_plugins/_anomaly_detection/{node_id}/stats/{stat_id}`, the merger tool will recognize that it is the same as `/_plugins/_anomaly_detection/{nodeId}/stats/{stat}` and generate the superseded operations accordingly with the correct path parameter names.
108-
109-
#### Auto-generating global parameters
110-
Certain query parameters are global, and they are accepted by every operation. These parameters are listed in the [root file](spec/opensearch-openapi.yaml) under the `parameters` section with `x-global` set to true. The merger tool will automatically add these parameters to all operations.
124+
The spec merger "builds", aka combines all `.yaml` files in a spec folder into a complete OpenAPI spec. A [workflow](./.github/workflows/build.yml) performs this task on the [spec folder](spec) of this repo then publishes the output into [releases](https://github.com/opensearch-project/opensearch-api-specification/releases).
111125

112126
### Linter
113127

114-
The spec linter that validates every `.yaml` file in the `./spec` folder to assure that they follow the guidelines we have set. Check out the [Linter README](tools/README.md#linter) for more information on how to run it locally. Make sure to run the linter before submitting a PR.
128+
The spec linter that validates every `.yaml` file in the `./spec` folder to assure that they follow the guidelines we have set. Check out the [Linter README](tools/README.md#spec-linter) for more information on how to run it locally. Make sure to run the linter before submitting a PR.

‎README.md

+10-2
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,8 @@
77
- [Project Resources](#project-resources)
88
- [Code of Conduct](#code-of-conduct)
99
- [Developer Guide](#developer-guide)
10-
- [OpenSearch API Specs](#opensearch-api-specs)
10+
- [Client Generator Guide](#client-generator-guide)
11+
- [Published Spec](#published-spec)
1112
- [Security](#security)
1213
- [License](#license)
1314
- [Copyright](#copyright)
@@ -38,10 +39,17 @@ This project has adopted the [Amazon Open Source Code of Conduct](CODE_OF_CONDUC
3839

3940
See [DEVELOPER_GUIDE](DEVELOPER_GUIDE.md).
4041

41-
## OpenSearch API Specs
42+
## Client Generator Guide
43+
44+
See [CLIENT_GENERATOR_GUIDE](CLIENT_GENERATOR_GUIDE.md).
45+
46+
## Published Spec
4247

4348
OpenSearch API Specs are hosted at https://opensearch-project.github.io/opensearch-api-specification/. See [PUBLISHING_SPECS](PUBLISHING_SPECS.md) for more information.
4449

50+
Click [here](https://github.com/opensearch-project/opensearch-api-specification/releases/download/main/opensearch-openapi.yaml) to download the latest OpenSearch OpenAPI yaml file.
51+
52+
4553
## Security
4654

4755
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/) or directly via email to [email protected]. Please do **not** create a public GitHub issue.

‎tools/README.md

+16-5
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22

33
This folder contains tools for the repo:
44

5-
- [Merger](./merger/): merges multiple OpenAPI files into one
6-
- [Linter](./linter/): validates files in the spec folder
5+
- [Merger](./merger): merges multiple OpenAPI files into one
6+
- [Linter](./linter): validates files in the spec folder
77

88
## Setup
99

@@ -20,15 +20,26 @@ The merger tool merges the multi-file OpenSearch spec into a single file for pro
2020
Example:
2121

2222
```bash
23-
npm run merge -- ../spec ../build/opensearch-openapi.latest.yaml
23+
mkdir -p ../build
24+
export ROOT_PATH=../spec/opensearch-openapi.yaml
25+
export OUTPUT_PATH=../build/opensearch-openapi.yaml
26+
npm run merge -- $ROOT_PATH $OUTPUT_PATH
2427
```
2528

26-
## Linter
29+
As a shortcut, if those parameters are not provided, the tool will use the default values:
30+
- `../spec/opensearch-openapi.yaml` as the root path (i.e. the root file of the repo's [spec folder](../spec))
31+
- `../opensearch-openapi.yaml` as the output path
32+
33+
```bash
34+
npm run merge
35+
```
36+
37+
## Spec Linter
2738

2839
The linter tool validates the OpenSearch spec files in the `spec` folder:
2940

3041
```bash
3142
npm run lint:spec
3243
```
3344

34-
It will print out all the errors and warnings in the spec files. This tool in still in development, and it will be integrated into the CI/CD pipeline and run automatically with every PR.
45+
It will print out all the errors and warnings in the spec files.

0 commit comments

Comments
 (0)