From ac23b17103ebc3fb4caadbb27823a217ea17b6e4 Mon Sep 17 00:00:00 2001 From: Eric Pugh Date: Thu, 30 May 2024 12:14:03 -0400 Subject: [PATCH 01/14] Docker based development for Jekyll Signed-off-by: Eric Pugh --- CONTRIBUTING.md | 13 +++++++++++++ Dockerfile | 15 +++++++++++++++ 2 files changed, 28 insertions(+) create mode 100644 Dockerfile diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f9f1a23f519..5fc8b0b4cb6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -70,6 +70,19 @@ If you want to make minor changes to an existing file, you can use this approach If you're adding a new page or making major changes to the documentation, such as adding new images, sections, or styling, we recommend that you work in a local copy of the repository and test the rendered HTML before submitting a PR. +#### Setting up your Docker based copy of the repository +This assumes you have Docker installed. + +1. [Fork this repository](https://docs.github.com/en/get-started/quickstart/fork-a-repo) and clone your fork. + +1. Navigate to your cloned repository. + +1. Build the documentation website Jekyll image via: `docker build -t documentation-website .` + +1. Start Jekyll via: `docker run -p 4000:4000 -v $(pwd):/app documentation-website` + +Be aware that it takes a while to build the site the first time, eventually you will see ``. + #### Setting up your local copy of the repository Follow these steps to set up your local copy of the repository: diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 00000000000..2caf448e127 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,15 @@ +FROM ruby:3.2 + +WORKDIR /app + +# Odd we don't version Gemfile.lock and therefore have it here to copy in! +COPY Gemfile ./ + +RUN bundle install + +# Expose the default Jekyll port +EXPOSE 4000 + +ENV JEKYLL_LINK_CHECKER="internal" + +CMD ["bundle", "exec", "jekyll", "serve", "--host", "0.0.0.0","--port", "4000", "--incremental", "--livereload", "--trace"] From f0c57cae7d5727c76ef0f9d0e63e61a88ce6a0d8 Mon Sep 17 00:00:00 2001 From: leanneeliatra <131779422+leanneeliatra@users.noreply.github.com> Date: Thu, 30 May 2024 17:23:56 +0100 Subject: [PATCH 02/14] Security best practices - 10 points to consider - #5782 (#7113) * adding top ten security best practices Signed-off-by: leanne.laceybyrne@eliatra.com * changing nav order Signed-off-by: leanne.laceybyrne@eliatra.com * adding to best practices Signed-off-by: AntonEliatra * adding to best practices Signed-off-by: AntonEliatra * adding to best practices Signed-off-by: AntonEliatra * adding bonus tip Signed-off-by: leanne.laceybyrne@eliatra.com * updates to best practices Signed-off-by: leanne.laceybyrne@eliatra.com * integrating Darshits suggestions for improvement and reviewdog fixes Signed-off-by: leanne.laceybyrne@eliatra.com * review suggestions to grammer Signed-off-by: leanne.laceybyrne@eliatra.com * review suggestions to grammer Signed-off-by: leanne.laceybyrne@eliatra.com * review suggestions to grammer Signed-off-by: leanne.laceybyrne@eliatra.com * review suggestions to grammer Signed-off-by: leanne.laceybyrne@eliatra.com * review suggestions to grammer Signed-off-by: leanne.laceybyrne@eliatra.com * reviewdog update Signed-off-by: leanne.laceybyrne@eliatra.com * Apply suggestions from code review Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanneeliatra <131779422+leanneeliatra@users.noreply.github.com> * reviewdog updates Signed-off-by: leanne.laceybyrne@eliatra.com * Update _security/configuration/best-practices.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: leanneeliatra <131779422+leanneeliatra@users.noreply.github.com> * Update best-practices.md Signed-off-by: AntonEliatra * Update best-practices.md Signed-off-by: AntonEliatra * Add editorial comment Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update best-practices.md Signed-off-by: AntonEliatra * Update _security/configuration/best-practices.md Co-authored-by: Nathan Bower Signed-off-by: AntonEliatra * Update best-practices.md Signed-off-by: AntonEliatra * Update best-practices.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: leanne.laceybyrne@eliatra.com Signed-off-by: AntonEliatra Signed-off-by: leanneeliatra <131779422+leanneeliatra@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: AntonEliatra Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower --- _security/configuration/best-practices.md | 133 ++++++++++++++++++++++ 1 file changed, 133 insertions(+) create mode 100644 _security/configuration/best-practices.md diff --git a/_security/configuration/best-practices.md b/_security/configuration/best-practices.md new file mode 100644 index 00000000000..97457cdb4bd --- /dev/null +++ b/_security/configuration/best-practices.md @@ -0,0 +1,133 @@ +--- +layout: default +title: Best practices +parent: Configuration +nav_order: 11 +--- + +# Best practices for OpenSearch security + +Setting up security in OpenSearch is crucial for protecting your data. Here are 10 best practices that offer clear steps for keeping your system safe. + +## 1. Use your own PKI to set up SSL/TLS + +Although using your own public key infrastructure (PKI), such as [AWS Certificate Manager](https://docs.aws.amazon.com/crypto/latest/userguide/awspki-service-acm.html), requires more initial effort, a custom PKI provides you with the flexibility needed to set up SSL/TLS in the most secure and performant way. + +### Enable SSL/TLS for node- and REST-layer traffic + +SSL/TLS is enabled by default on the transport layer, which is used for node-to-node communication. SSL/TLS is disabled by default on the REST layer. + +The following setting is required in order to enable encryption on the REST layer: + +``` +plugins.security.ssl.http.enabled: true +``` +{% include copy.html %} + + +For additional configuration options, such as specifying certificate paths, keys, and certificate authority files, refer to [Configuring TLS certificates]({{site.url}}{{site.baseurl}}/security/configuration/tls/). + +### Replace all demo certificates with your own PKI + +The certificates generated when initializing an OpenSearch cluster with `install_demo_configuration.sh` are not suitable for production. These should be replaced with your own certificates. + +You can generate custom certificates in a few different ways. One approach is to use OpenSSL, described in detail at [Generating self-signed certificates]({{site.url}}{{site.baseurl}}/security/configuration/generate-certificates/). Alternatively, there are online tools available that can simplify the certificate creation process, such as the following: + +- [SearchGuard TLS Tool](https://docs.search-guard.com/latest/offline-tls-tool) +- [TLSTool by dylandreimerink](https://github.com/dylandreimerink/tlstool) + +## 2. Prefer client certificate authentication for API authentication + +Client certificate authentication offers a secure alternative to password authentication and is more suitable for machine-to-machine interactions. It also ensures low performance overhead because the authentication occurs on the TLS level. Nearly all client software, such as curl and client libraries, support this authentication method. + +For detailed configuration instructions and additional information about client certificate authentication, see [Enabling client certificate authentication]({{site.url}}{{site.baseurl}}/security/authentication-backends/client-auth/#enabling-client-certificate-authentication). + + +## 3. Prefer SSO using SAML or OpenID for OpenSearch Dashboards authentication + +Implementing single sign-on (SSO) with protocols like SAML or OpenID for OpenSearch Dashboards authentication enhances security by delegating credential management to a dedicated system. + +This approach minimizes direct interaction with passwords in OpenSearch, streamlines authentication processes, and prevents clutter in the internal user database. For more information, go to the [SAML section of the OpenSearch documentation]({{site.url}}{{site.baseurl}}/security/authentication-backends/saml/). + +## 4. Limit the number of roles assigned to a user + +Prioritizing fewer, more intricate user roles over numerous simplistic roles enhances security and simplifies administration. + +Additional best practices for role management include: + +1. Role granularity: Define roles based on specific job functions or access requirements to minimize unnecessary privileges. +2. Regular role review: Regularly review and audit assigned roles to ensure alignment with organizational policies and access needs. + +For more information about roles, go to the documentation on [defining users and roles in OpenSearch]({{site.url}}{{site.baseurl}}/security/access-control/users-roles/). + +## 5. Verify DLS, FLS, and field masking + +If you have configured Document Level Security (DLS), Field Level Security (FLS), or field masking, make sure you double-check your role definitions, especially if a user is mapped to multiple roles. It is highly recommended that you test this by making a GET request to `_plugins/_security/authinfo`. + +The following resources provide detailed examples and additional configurations: + + - [Document-level security]({{site.url}}{{site.baseurl}}/security/access-control/document-level-security/). + - [Field-level security]({{site.url}}{{site.baseurl}}/security/access-control/field-level-security/). + - [Field masking]({{site.url}}{{site.baseurl}}/security/access-control/field-masking/). + +## 6. Use only the essentials for the audit logging configuration + +Extensive audit logging can degrade system performance due to the following: + +- Each logged event adds to the processing load. +- Audit logs can quickly grow in size, consuming significant disk space. + +To ensure optimal performance, disable unnecessary logging and be selective about which logs are used. If not strictly required by compliance regulations, consider turning off audit logging. If audit logging is essential for your cluster, configure it according to your compliance requirements. + +Whenever possible, adhere to these recommendations: + +- Set `audit.log_request_body` to `false`. +- Set `audit.resolve_bulk_requests` to `false`. +- Enable `compliance.write_log_diffs`. +- Minimize entries for `compliance.read_watched_fields`. +- Minimize entries for `compliance.write_watched_indices`. + +## 7. Consider disabling the private tenant + +In many cases, the use of private tenants is unnecessary, although this feature is enabled by default. As a result, every OpenSearch Dashboards user is provided with their own private tenant and a corresponding new index in which to save objects. This can lead to a large number of unnecessary indexes. Evaluate whether private tenants are needed in your cluster. If private tenants are not needed, disable the feature by adding the following configuration to the `config.yml` file: + +```yaml +config: + dynamic: + kibana: + multitenancy_enabled: true + private_tenant_enabled: false +``` +{% include copy.html %} + +## 8. Manage the configuration using `securityadmin.sh` + +Use `securityadmin.sh` to manage the configuration of your clusters. `securityadmin.sh` is a command-line tool provided by OpenSearch for managing security configurations. It allows administrators to efficiently manage security settings, including roles, role mappings, and other security-related configurations within an OpenSearch cluster. + +Using `securityadmin.sh` provides the following benefits: + +1. Consistency: By using `securityadmin.sh`, administrators can ensure consistency across security configurations within a cluster. This helps to maintain a standardized and secure environment. +2. Automation: `securityadmin.sh` enables automation of security configuration tasks, making it easier to deploy and manage security settings across multiple nodes or clusters. +3. Version control: Security configurations managed through `securityadmin.sh` can be version controlled using standard version control systems like Git. This facilitates tracking changes, auditing, and reverting to previous configurations. + +You can prevent configuration overrides by first creating a backup of the current configuration created using the OpenSearch Dashboards UI or the OpenSearch API by running the `securityadmin.sh` tool with the `-backup` option. This ensures that all configurations are captured before uploading the modified configuration with `securityadmin.sh`. + +For more detailed information about using `securityadmin.sh` and managing OpenSearch security configurations, refer to the following resources: +- [Applying changes to configuration files]({{site.url}}{{site.baseurl}}/security/configuration/security-admin/) +- [Modifying YAML files]({{site.url}}{{site.baseurl}}/security/configuration/yaml/) + +## 9. Replace all default passwords + +When initializing OpenSearch with the demo configuration, many default passwords are provided for internal users in `internal_users.yml`, such as `admin`, `kibanaserver`, and `logstash`. + +You should change the passwords for these users to strong, complex passwords either at startup or as soon as possible once the cluster is running. Creating password configurations is a straightforward procedure, especially when using the scripts bundled with OpenSearch, like `hash.sh` or `hash.bat`, located in the `plugin/OpenSearch security/tools` directory. + +The `kibanaserver` user is a crucial component that allows OpenSearch Dashboards to communicate with the OpenSearch cluster. By default, this user is preconfigured with a default password in the demo configuration. This should be replaced with a strong, unique password in the OpenSearch configuration, and the `opensearch_dashboards.yml` file should be updated to reflect this change. + + +## 10. Getting help + +If you need additional help, you can do the following: + +- Create an issue on GitHub at [OpenSearch-project/security](https://github.com/opensearch-project/security/security) or [OpenSearch-project/OpenSearch](https://github.com/opensearch-project/OpenSearch/security). +- Ask a question on the [OpenSearch forum](https://forum.opensearch.org/tag/cve). From d3f7cbbb8c33136d7ddca442c8d44c7924e4480e Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Thu, 30 May 2024 13:55:29 -0400 Subject: [PATCH 03/14] Explain k in approximate k-NN (#7194) * Explain k in approximate k-NN Signed-off-by: Fanit Kolchina * Additional info Signed-off-by: Fanit Kolchina * Delete engine row in table Signed-off-by: Fanit Kolchina * Add a clarification to the table Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _search-plugins/knn/approximate-knn.md Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower --- _search-plugins/knn/approximate-knn.md | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/_search-plugins/knn/approximate-knn.md b/_search-plugins/knn/approximate-knn.md index 7d3e119349e..39e9da75253 100644 --- a/_search-plugins/knn/approximate-knn.md +++ b/_search-plugins/knn/approximate-knn.md @@ -127,10 +127,25 @@ GET my-knn-index-1/_search } ``` -`k` is the number of neighbors the search of each graph will return. You must also include the `size` option, which -indicates how many results the query actually returns. The plugin returns `k` amount of results for each shard -(and each segment) and `size` amount of results for the entire query. The plugin supports a maximum `k` value of 10,000. -Starting in OpenSearch 2.14, in addition to using the `k` variable, both the `min_score` and `max_distance` variables can be used for [radial search]({{site.url}}{{site.baseurl}}/search-plugins/knn/radial-search-knn/). +### The number of returned results + +In the preceding query, `k` represents the number of neighbors returned by the search of each graph. You must also include the `size` option, indicating the final number of results that you want the query to return. + +For the NMSLIB and Faiss engines, `k` represents the maximum number of documents returned for all segments of a shard. For the Lucene engine, `k` represents the number of documents returned for a shard. The maximum value of `k` is 10,000. + +For any engine, each shard returns `size` results to the coordinator node. Thus, the total number of results that the coordinator node receives is `size * number of shards`. After the coordinator node consolidates the results received from all nodes, the query returns the top `size` results. + +The following table provides examples of the number of results returned by various engines in several scenarios. For these examples, assume that the number of documents contained in the segments and shards is sufficient to return the number of results specified in the table. + +`size` | `k` | Number of primary shards | Number of segments per shard | Number of returned results, Faiss/NMSLIB | Number of returned results, Lucene +10 | 1 | 1 | 4 | 4 | 1 +10 | 10 | 1 | 4 | 10 | 10 +10 | 1 | 2 | 4 | 8 | 2 + +The number of results returned by Faiss/NMSLIB differs from the number of results returned by Lucene only when `k` is smaller than `size`. If `k` and `size` are equal, all engines return the same number of results. + +Starting in OpenSearch 2.14, you can use `k`, `min_score`, or `max_distance` for [radial search]({{site.url}}{{site.baseurl}}/search-plugins/knn/radial-search-knn/). + ### Building a k-NN index from a model For some of the algorithms that we support, the native library index needs to be trained before it can be used. It would be expensive to training every newly created segment, so, instead, we introduce the concept of a *model* that is used to initialize the native library index during segment creation. A *model* is created by calling the [Train API]({{site.url}}{{site.baseurl}}/search-plugins/knn/api#train-a-model), passing in the source of training data as well as the method definition of the model. Once training is complete, the model will be serialized to a k-NN model system index. Then, during indexing, the model is pulled from this index to initialize the segments. From 150dfa7fd1c9ee72b24e7adeca9064fe6eb6205e Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Thu, 30 May 2024 13:56:07 -0400 Subject: [PATCH 04/14] Add version to PR template (#7276) Signed-off-by: Fanit Kolchina --- .github/PULL_REQUEST_TEMPLATE.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 7eccae70523..bbf3b8d0350 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -4,6 +4,8 @@ _Describe what this change achieves._ ### Issues Resolved _List any issues this PR will resolve, e.g. Closes [...]._ +### Version +_List the OpenSearch version to which this PR applies, e.g. 2.14, 2.12--2.14, or all._ ### Checklist - [ ] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the [Developers Certificate of Origin](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin). From d39929d2422b70b82e66a3f9d8d023af092d8102 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Thu, 30 May 2024 14:01:12 -0400 Subject: [PATCH 05/14] Remove formatting from front matter in foreach processor (#7277) Signed-off-by: Fanit Kolchina --- _ingest-pipelines/processors/foreach.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/_ingest-pipelines/processors/foreach.md b/_ingest-pipelines/processors/foreach.md index 72a0ed1420a..d0f962e6185 100644 --- a/_ingest-pipelines/processors/foreach.md +++ b/_ingest-pipelines/processors/foreach.md @@ -1,11 +1,13 @@ --- layout: default -title: `foreach` +title: Foreach parent: Ingest processors nav_order: 110 --- -# `foreach` processor + +# Foreach processor + The `foreach` processor is used to iterate over a list of values in an input document and apply a transformation to each value. This can be useful for tasks like processing all the elements in an array consistently, such as converting all elements in a string to lowercase or uppercase. From acb79c076aa45ae2e3b3fd12a87e61864fb99010 Mon Sep 17 00:00:00 2001 From: Eric Pugh Date: Thu, 30 May 2024 18:05:02 -0400 Subject: [PATCH 06/14] Use a specific Ruby and store the lock file --- .gitignore | 1 - .ruby-version | 1 + Dockerfile | 9 ++- Gemfile | 2 + Gemfile.lock | 197 ++++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 205 insertions(+), 5 deletions(-) create mode 100644 .ruby-version create mode 100644 Gemfile.lock diff --git a/.gitignore b/.gitignore index ae2249e73f7..bc90884f045 100644 --- a/.gitignore +++ b/.gitignore @@ -2,6 +2,5 @@ _site .sass-cache .jekyll-metadata .DS_Store -Gemfile.lock .idea .jekyll-cache diff --git a/.ruby-version b/.ruby-version new file mode 100644 index 00000000000..351227fca34 --- /dev/null +++ b/.ruby-version @@ -0,0 +1 @@ +3.2.4 diff --git a/Dockerfile b/Dockerfile index 2caf448e127..b5877653bb0 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,15 +1,16 @@ -FROM ruby:3.2 +FROM ruby:3.2.4 WORKDIR /app # Odd we don't version Gemfile.lock and therefore have it here to copy in! -COPY Gemfile ./ +COPY Gemfile Gemfile.lock ./ RUN bundle install # Expose the default Jekyll port EXPOSE 4000 -ENV JEKYLL_LINK_CHECKER="internal" +# Enable the link checker +#ENV JEKYLL_LINK_CHECKER="internal" -CMD ["bundle", "exec", "jekyll", "serve", "--host", "0.0.0.0","--port", "4000", "--incremental", "--livereload", "--trace"] +CMD ["bundle", "exec", "jekyll", "serve", "--host", "0.0.0.0", "--port", "4000", "--incremental", "--livereload", "--trace"] diff --git a/Gemfile b/Gemfile index 7825dcd02bf..0612cf6380d 100644 --- a/Gemfile +++ b/Gemfile @@ -1,5 +1,7 @@ source "http://rubygems.org" +ruby "3.2.4" + # Hello! This is where you manage which Jekyll version is used to run. # When you want to use a different version, change it below, save the # file and run `bundle install`. Run Jekyll with `bundle exec`, like so: diff --git a/Gemfile.lock b/Gemfile.lock new file mode 100644 index 00000000000..4bc0b7d15df --- /dev/null +++ b/Gemfile.lock @@ -0,0 +1,197 @@ +GEM + remote: http://rubygems.org/ + specs: + addressable (2.8.6) + public_suffix (>= 2.0.2, < 6.0) + bigdecimal (3.1.8) + colorator (1.1.0) + concurrent-ruby (1.3.1) + em-websocket (0.5.3) + eventmachine (>= 0.12.9) + http_parser.rb (~> 0) + ethon (0.16.0) + ffi (>= 1.15.0) + eventmachine (1.2.7) + ffi (1.16.3) + forwardable-extended (2.6.0) + google-protobuf (4.27.0) + bigdecimal + rake (>= 13) + google-protobuf (4.27.0-aarch64-linux) + bigdecimal + rake (>= 13) + google-protobuf (4.27.0-arm64-darwin) + bigdecimal + rake (>= 13) + google-protobuf (4.27.0-x86-linux) + bigdecimal + rake (>= 13) + google-protobuf (4.27.0-x86_64-darwin) + bigdecimal + rake (>= 13) + google-protobuf (4.27.0-x86_64-linux) + bigdecimal + rake (>= 13) + http_parser.rb (0.8.0) + i18n (1.14.5) + concurrent-ruby (~> 1.0) + jekyll (4.3.3) + addressable (~> 2.4) + colorator (~> 1.0) + em-websocket (~> 0.5) + i18n (~> 1.0) + jekyll-sass-converter (>= 2.0, < 4.0) + jekyll-watch (~> 2.0) + kramdown (~> 2.3, >= 2.3.1) + kramdown-parser-gfm (~> 1.0) + liquid (~> 4.0) + mercenary (>= 0.3.6, < 0.5) + pathutil (~> 0.9) + rouge (>= 3.0, < 5.0) + safe_yaml (~> 1.0) + terminal-table (>= 1.8, < 4.0) + webrick (~> 1.7) + jekyll-last-modified-at (1.3.0) + jekyll (>= 3.7, < 5.0) + posix-spawn (~> 0.3.9) + jekyll-redirect-from (0.16.0) + jekyll (>= 3.3, < 5.0) + jekyll-remote-theme (0.4.3) + addressable (~> 2.0) + jekyll (>= 3.5, < 5.0) + jekyll-sass-converter (>= 1.0, <= 3.0.0, != 2.0.0) + rubyzip (>= 1.3.0, < 3.0) + jekyll-sass-converter (3.0.0) + sass-embedded (~> 1.54) + jekyll-seo-tag (2.8.0) + jekyll (>= 3.8, < 5.0) + jekyll-sitemap (1.4.0) + jekyll (>= 3.7, < 5.0) + jekyll-watch (2.2.1) + listen (~> 3.0) + just-the-docs (0.3.3) + jekyll (>= 3.8.5) + jekyll-seo-tag (~> 2.0) + rake (>= 12.3.1, < 13.1.0) + kramdown (2.4.0) + rexml + kramdown-parser-gfm (1.1.0) + kramdown (~> 2.0) + liquid (4.0.4) + listen (3.9.0) + rb-fsevent (~> 0.10, >= 0.10.3) + rb-inotify (~> 0.9, >= 0.9.10) + mercenary (0.4.0) + pathutil (0.16.2) + forwardable-extended (~> 2.6) + posix-spawn (0.3.15) + public_suffix (5.0.5) + rake (13.0.6) + rb-fsevent (0.11.2) + rb-inotify (0.11.1) + ffi (~> 1.0) + rexml (3.2.8) + strscan (>= 3.0.9) + rouge (4.2.1) + ruby-enum (1.0.0) + ruby-link-checker (0.2.0) + rubyzip (2.3.2) + safe_yaml (1.0.5) + sass-embedded (1.77.3) + google-protobuf (>= 3.25, < 5.0) + rake (>= 13) + sass-embedded (1.77.3-aarch64-linux-android) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-aarch64-linux-gnu) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-aarch64-linux-musl) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-aarch64-mingw-ucrt) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-arm-linux-androideabi) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-arm-linux-gnueabihf) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-arm-linux-musleabihf) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-arm64-darwin) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-riscv64-linux-android) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-riscv64-linux-gnu) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-riscv64-linux-musl) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-x86-cygwin) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-x86-linux-android) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-x86-linux-gnu) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-x86-linux-musl) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-x86-mingw-ucrt) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-x86_64-cygwin) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-x86_64-darwin) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-x86_64-linux-android) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-x86_64-linux-gnu) + google-protobuf (>= 3.25, < 5.0) + sass-embedded (1.77.3-x86_64-linux-musl) + google-protobuf (>= 3.25, < 5.0) + strscan (3.1.0) + terminal-table (3.0.2) + unicode-display_width (>= 1.1.1, < 3) + typhoeus (1.4.1) + ethon (>= 0.9.0) + unicode-display_width (2.5.0) + webrick (1.8.1) + +PLATFORMS + aarch64-linux + aarch64-linux-android + aarch64-linux-gnu + aarch64-linux-musl + aarch64-mingw-ucrt + arm-linux-androideabi + arm-linux-gnueabihf + arm-linux-musleabihf + arm64-darwin + riscv64-linux-android + riscv64-linux-gnu + riscv64-linux-musl + ruby + x86-cygwin + x86-linux + x86-linux-android + x86-linux-gnu + x86-linux-musl + x86-mingw-ucrt + x86_64-cygwin + x86_64-darwin + x86_64-linux + x86_64-linux-android + x86_64-linux-gnu + x86_64-linux-musl + +DEPENDENCIES + jekyll (~> 4.3.2) + jekyll-last-modified-at + jekyll-redirect-from (~> 0.16) + jekyll-remote-theme (~> 0.4) + jekyll-sitemap + just-the-docs (~> 0.3.3) + ruby-enum + ruby-link-checker + typhoeus + tzinfo-data + webrick (~> 1.7) + +RUBY VERSION + ruby 3.2.4p170 + +BUNDLED WITH + 2.5.11 From 7c5a31e5196c36d03a5be199f55135c58600bb61 Mon Sep 17 00:00:00 2001 From: Eric Pugh Date: Thu, 30 May 2024 18:06:10 -0400 Subject: [PATCH 07/14] be specific on which Ruby we support --- CONTRIBUTING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5fc8b0b4cb6..9a5a85d2e1a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -91,11 +91,11 @@ Follow these steps to set up your local copy of the repository: 1. Navigate to your cloned repository. -1. Install [Ruby](https://www.ruby-lang.org/en/) if you don't already have it. We recommend [RVM](https://rvm.io/), but you can use any method you prefer: +1. Install [Ruby](https://www.ruby-lang.org/en/) version 3.2.4 if you don't already have it. We recommend [RVM](https://rvm.io/), but you can use any method you prefer: ``` curl -sSL https://get.rvm.io | bash -s stable - rvm install 3.2 + rvm install 3.2.4 ruby -v ``` From 63b0ad6437dd2c66eedad42ec3d492fe9d2741c2 Mon Sep 17 00:00:00 2001 From: Eric Pugh Date: Thu, 30 May 2024 18:09:33 -0400 Subject: [PATCH 08/14] Restore the running of the linkchecker --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index b5877653bb0..a000bcc6601 100644 --- a/Dockerfile +++ b/Dockerfile @@ -11,6 +11,6 @@ RUN bundle install EXPOSE 4000 # Enable the link checker -#ENV JEKYLL_LINK_CHECKER="internal" +ENV JEKYLL_LINK_CHECKER="internal" CMD ["bundle", "exec", "jekyll", "serve", "--host", "0.0.0.0", "--port", "4000", "--incremental", "--livereload", "--trace"] From f5ddfdc90aed5034d4664ea128879149f209700f Mon Sep 17 00:00:00 2001 From: Eric Pugh Date: Thu, 30 May 2024 18:49:52 -0400 Subject: [PATCH 09/14] since I decided to keep the Gemfile.lock, remove comment Signed-off-by: Eric Pugh --- Dockerfile | 1 - 1 file changed, 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index a000bcc6601..fabd786ced0 100644 --- a/Dockerfile +++ b/Dockerfile @@ -2,7 +2,6 @@ FROM ruby:3.2.4 WORKDIR /app -# Odd we don't version Gemfile.lock and therefore have it here to copy in! COPY Gemfile Gemfile.lock ./ RUN bundle install From add6da45175bde51ca3f49ce741e3b0b5a5406a9 Mon Sep 17 00:00:00 2001 From: Eric Pugh Date: Thu, 30 May 2024 18:50:43 -0400 Subject: [PATCH 10/14] not convinced that we need the .ruby-version checked in --- .gitignore | 1 + .ruby-version | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) delete mode 100644 .ruby-version diff --git a/.gitignore b/.gitignore index bc90884f045..21a53230a1b 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,4 @@ _site .DS_Store .idea .jekyll-cache +.ruby-version diff --git a/.ruby-version b/.ruby-version deleted file mode 100644 index 351227fca34..00000000000 --- a/.ruby-version +++ /dev/null @@ -1 +0,0 @@ -3.2.4 From 69cae5b6271d784ef76460130653258b89e22b50 Mon Sep 17 00:00:00 2001 From: Eric Pugh Date: Thu, 30 May 2024 19:00:10 -0400 Subject: [PATCH 11/14] document how you know the server has started --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9a5a85d2e1a..8a601504d1a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -81,7 +81,7 @@ This assumes you have Docker installed. 1. Start Jekyll via: `docker run -p 4000:4000 -v $(pwd):/app documentation-website` -Be aware that it takes a while to build the site the first time, eventually you will see ``. +Be aware that it takes a while to build the site the first time, eventually you will see `Server running... press ctrl-c to stop`. #### Setting up your local copy of the repository From fa9fdb70af5078e07be1b6d2406968b5e632e4e2 Mon Sep 17 00:00:00 2001 From: David Tippett Date: Fri, 31 May 2024 15:32:09 -0400 Subject: [PATCH 12/14] Added a few handy little features: 1. Environment now is defaulted to run on localhost 2. Added env var's to run in a hosted manner 3. Added documentation to run a demo server hosted Signed-off-by: David Tippett --- CONTRIBUTING.md | 4 ++++ Dockerfile | 4 +++- _config_docker.yml | 1 + 3 files changed, 8 insertions(+), 1 deletion(-) create mode 100644 _config_docker.yml diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8a601504d1a..0418b6ffdf5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -83,6 +83,10 @@ This assumes you have Docker installed. Be aware that it takes a while to build the site the first time, eventually you will see `Server running... press ctrl-c to stop`. +> __Note__: If you would like to run a hosted version of the documentation site you will need to do the following: +> 1. Change the URL in `_config_docker.yml` to either your hostname or ip: `url: "http://10.0.0.5:4000"` +> 1. Change the `HOST` and `JEKYLL_ENV` variables: `docker run -p 4000:4000 -e JEKYLL_ENV=production -e HOST=0.0.0.0 -v $(pwd):/app documentation-website + #### Setting up your local copy of the repository Follow these steps to set up your local copy of the repository: diff --git a/Dockerfile b/Dockerfile index fabd786ced0..6a5adf3bed2 100644 --- a/Dockerfile +++ b/Dockerfile @@ -11,5 +11,7 @@ EXPOSE 4000 # Enable the link checker ENV JEKYLL_LINK_CHECKER="internal" +ENV JEKYLL_ENV=development +ENV HOST=127.0.0.1 -CMD ["bundle", "exec", "jekyll", "serve", "--host", "0.0.0.0", "--port", "4000", "--incremental", "--livereload", "--trace"] +CMD ["bundle", "exec", "jekyll", "serve", "--host", "$HOST", "--port", "4000", "--incremental", "--livereload", "--trace", "--config", "_config.yml,_config_docker.yml"] diff --git a/_config_docker.yml b/_config_docker.yml new file mode 100644 index 00000000000..3c105144382 --- /dev/null +++ b/_config_docker.yml @@ -0,0 +1 @@ +url: "http://127.0.0.1:4000" \ No newline at end of file From d571df13bc2212bebab7f60a72d2a62c339681a9 Mon Sep 17 00:00:00 2001 From: Eric Pugh Date: Fri, 31 May 2024 15:47:14 -0400 Subject: [PATCH 13/14] I exist to make vale happy --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0418b6ffdf5..99fbf65b012 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -84,7 +84,7 @@ This assumes you have Docker installed. Be aware that it takes a while to build the site the first time, eventually you will see `Server running... press ctrl-c to stop`. > __Note__: If you would like to run a hosted version of the documentation site you will need to do the following: -> 1. Change the URL in `_config_docker.yml` to either your hostname or ip: `url: "http://10.0.0.5:4000"` +> 1. Change the `url` setting in `_config_docker.yml` to your URL: `url: "http://10.0.0.5:4000"` > 1. Change the `HOST` and `JEKYLL_ENV` variables: `docker run -p 4000:4000 -e JEKYLL_ENV=production -e HOST=0.0.0.0 -v $(pwd):/app documentation-website #### Setting up your local copy of the repository From efc35be49754fe5f2008ceb1ce4b6d6520d05456 Mon Sep 17 00:00:00 2001 From: Heather Halter Date: Fri, 12 Jul 2024 15:43:18 -0700 Subject: [PATCH 14/14] Apply suggestions from code review Eric is on vacation for a few weeks, so I accepted your changes, Nate. Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Heather Halter --- CONTRIBUTING.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 99fbf65b012..5f4920ba6df 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -70,18 +70,23 @@ If you want to make minor changes to an existing file, you can use this approach If you're adding a new page or making major changes to the documentation, such as adding new images, sections, or styling, we recommend that you work in a local copy of the repository and test the rendered HTML before submitting a PR. -#### Setting up your Docker based copy of the repository -This assumes you have Docker installed. +#### Setting up a Docker-based copy of the repository -1. [Fork this repository](https://docs.github.com/en/get-started/quickstart/fork-a-repo) and clone your fork. +If you have Docker installed, you can use Docker to test any major changes to the documentation website locally by using the following steps: + +1. [Fork this repository](https://docs.github.com/en/get-started/quickstart/fork-a-repo) and clone your fork in the directory of your choice. 1. Navigate to your cloned repository. -1. Build the documentation website Jekyll image via: `docker build -t documentation-website .` +1. Build the documentation website Jekyll image by using the following command: + + ``` + docker build -t documentation-website . + ``` -1. Start Jekyll via: `docker run -p 4000:4000 -v $(pwd):/app documentation-website` +1. Start Jekyll from inside the image using the following command: -Be aware that it takes a while to build the site the first time, eventually you will see `Server running... press ctrl-c to stop`. +When building the site for the first time, it'll take some time for the site to build. When your Jekyll instance is ready, you will see `Server running... press ctrl-c to stop`. > __Note__: If you would like to run a hosted version of the documentation site you will need to do the following: > 1. Change the `url` setting in `_config_docker.yml` to your URL: `url: "http://10.0.0.5:4000"`