User settings YAML and JSON are shown on logs

[kuisathaverat]

When certain errors happend the user settings YAML/JSON are exposed in the logs, these files can contain sensitive data. also some URLs and other sentitive date is show in logs and probably should not.

Readiness Checklist

• I am running the latest version
• I checked the documentation and found no answer
• I checked to make sure that this issue has not already been filed
• I am reporting the issue to the correct repository (for multi-repository projects)

Expected Behavior

Sessitive variables must not be shown in logs like, user settings, URLs, Cloud ID, ...

Current Behavior

A bunch of SENSITIVE_DATA is shown in logs

Error: deployment deletion error
  
  found deployment plan errors: 1 error occurred:
          * deployment [12345668eed8381992499f1858a65482] -
  [elasticsearch][123456b0ceffe46a1af7e5c17509150f8]: caught error: "Plan change
  failed: Some instances were not stopped"
  Origin: /home/runner/.ansible/collections/ansible_collections/oblt/framework/roles/stack_ess/tasks/destroy_cluster.yml:14:11
  
  12         - ess_deployment_id != ""
  13       block:
  14         - name: Undeploy Elastic Cloud deployment
               ^ column 11
  
  fatal: [localhost]: FAILED! => 
      changed: false
      changed_when_result: false
      cmd: /home/runner/work/my-module/my-module/.ci/bin/terraform destroy -no-color
          -auto-approve -lock=true
      msg: |4-
  
          Error: deployment deletion error
  
          found deployment plan errors: 1 error occurred:
                  * deployment [12345668eed8381992499f1858a65482] -
          [elasticsearch][123456b0ceffe46a1af7e5c17509150f8]: caught error: "Plan change
          failed: Some instances were not stopped"
      rc: 1
      stderr: |4-
  
          Error: deployment deletion error
  
          found deployment plan errors: 1 error occurred:
                  * deployment [12345668eed8381992499f1858a65482] -
          [elasticsearch][123456b0ceffe46a1af7e5c17509150f8]: caught error: "Plan change
          failed: Some instances were not stopped"
      stderr_lines: <omitted>
      stdout: |-
          ec_deployment.main: Refreshing state... [id=12345668eed8381992499f1858a65482]
          ec_deployment_elasticsearch_keystore.gcs_credentials: Refreshing state... [id=3810055890]
  
          Terraform used the selected providers to generate the following execution
          plan. Resource actions are indicated with the following symbols:
            - destroy
  
          Terraform will perform the following actions:
  
            # ec_deployment.main will be destroyed
            - resource "ec_deployment" "main" {
                - alias                  = "oblt-test-biwy" -> null
                - apm_secret_token       = (sensitive value) -> null
                - deployment_template_id = "gcp-storage-optimized" -> null
                - elasticsearch          = {
                    - autoscale      = true -> null
                    - cloud_id       = "[SENSITIVE_DATA]" -> null
                    - config         = {
                        - plugins            = [] -> null
                        - user_settings_yaml = <<-EOT
                            [SENSITIVE_DATA]
                          EOT -> null
                      } -> null
                    - hot            = {
                        - autoscaling                           = {
                            - max_size          = "8g" -> null
                            - max_size_resource = "memory" -> null
                          } -> null
                        - instance_configuration_id             = "gcp.es.datahot.n2.68x10x45" -> null
                        - instance_configuration_version        = 1 -> null
                        - latest_instance_configuration_id      = "gcp.es.datahot.n2.68x10x45" -> null
                        - latest_instance_configuration_version = 1 -> null
                        - node_roles                            = [
                            - "data_content",
                            - "data_hot",
                            - "ingest",
                            - "master",
                            - "remote_cluster_client",
                            - "transform",
                          ] -> null
                        - size                                  = "8g" -> null
                        - size_resource                         = "memory" -> null
                        - zone_count                            = 1 -> null
                      } -> null
                    - http_endpoint  = "[SENSITIVE_DATA]" -> null
                    - https_endpoint = "[SENSITIVE_DATA]" -> null
                    - ml             = {
                        - autoscaling                           = {
                            - max_size          = "64g" -> null
                            - max_size_resource = "memory" -> null
                            - min_size          = "0g" -> null
                            - min_size_resource = "memory" -> null
                          } -> null
                        - instance_configuration_id             = "gcp.es.ml.n2.68x32x45" -> null
                        - instance_configuration_version        = 1 -> null
                        - latest_instance_configuration_id      = "gcp.es.ml.n2.68x32x45" -> null
                        - latest_instance_configuration_version = 1 -> null
                        - node_roles                            = [
                            - "ml",
                            - "remote_cluster_client",
                          ] -> null
                        - size                                  = "0g" -> null
                        - size_resource                         = "memory" -> null
                        - zone_count                            = 1 -> null
                      } -> null
                    - ref_id         = "main-elasticsearch" -> null
                    - region         = "gcp-us-west2" -> null
                    - resource_id    = "123456b0ceffe46a1af7e5c17509150f8" -> null
                  } -> null
                - elasticsearch_password = (sensitive value) -> null
                - elasticsearch_username = "elastic" -> null
                - id                     = "12345668eed8381992499f1858a65482" -> null
                - integrations_server    = {
                    - config                                = {
                        - debug_enabled      = false -> null
                        - user_settings_yaml = <<-EOT
                              SENSITIVE_DATA
                          EOT -> null
                      } -> null
                    - elasticsearch_cluster_ref_id          = "main-elasticsearch" -> null
                    - endpoints                             = {
                        - apm   = "SENSITIVE_DATA"
                        - fleet = "SENSITIVE_DATA"
                      } -> null
                    - http_endpoint                         = "SENSITIVE_DATA" -> null
                    - https_endpoint                        = "SENSITIVE_DATA" -> null
                    - instance_configuration_id             = "gcp.integrationsserver.n2.68x32x45" -> null
                    - instance_configuration_version        = 1 -> null
                    - latest_instance_configuration_id      = "gcp.integrationsserver.n2.68x32x45" -> null
                    - latest_instance_configuration_version = 1 -> null
                    - ref_id                                = "main-integrations_server" -> null
                    - region                                = "gcp-us-west2" -> null
                    - resource_id                           = "bc07e363f54f422093c7d00e3af11750" -> null
                    - size                                  = "2g" -> null
                    - size_resource                         = "memory" -> null
                    - zone_count                            = 1 -> null
                  } -> null
                - kibana                 = {
                    - config                                = {
                        - user_settings_yaml = <<-EOT
                              SENSITIVE_DATA

## Terraform definition

# Retrieve the latest stack pack version
data "ec_stack" "latest" {
  version_regex = "latest"
  region        = "us-east-1"
}

# Create an Elastic Cloud deployment
resource "ec_deployment" "example_minimal" {
  # Optional name.
  name = "my_example_deployment"

  region                 = "us-east-1"
  version                = data.ec_stack.latest.version
  deployment_template_id = "aws-io-optimized-v2"

  elasticsearch = {
    hot = {
      autoscaling = {}
    }
  }

  kibana = {
    config = {
        user_settings_yaml = file("./kbn.yaml")
    }
  }

  enterprise_search = {}

  integrations_server = {}
}

Steps to Reproduce

• Create a deployment witn user settings yaml with sensitive information
• Try to destroy the deployment while it is still creating
• Check the logs

Context

Possible Solution

set as sensible any variable that could contains secrets or other private information like URL od cloud ID

Your Environment

• Version used: ECH
• Running against Elastic Cloud SaaS or Elastic Cloud Enterprise and version:
• Environment name and version (e.g. Go 1.9):
• Server type and version:
• Operating System and version:
• Link to your project: