Valkey instance forces replacement

[zdunning13]

TL;DR

After creating a Memorystore for Valkey instance with terraform-google-modules/memorystore/google//modules/valkey?version=15.0.0, any subsequent plan/apply tries to destroy and recreate the Valkey instance no matter what changes were made in terraform, if any.

This is due to desired_auto_created_endpoints changing to null.

Expected behavior

Running a plan with no changes, or making a change to something like labels does not force a recreation of the Memorystore for Valkey instance.

Observed behavior

After any changes, or after no changes, and running a plan/apply result Terraform trying to destroy and recreate the instance due to the following:

desired_auto_created_endpoints { # forces replacement
          - network    = "projects/<project_id>/global/networks/<network_id>" -> null
          - project_id = "<project_id>" -> null
}

desired_auto_created_endpoints does not appear to be something we can set in our Terraform config to ensure the values persist.

Terraform Configuration

inputs = {
  authorization_mode                    = "AUTH_DISABLED"
  automated_backup_config               = null
  deletion_protection_enabled           = false
  enable_apis                           = false
  engine_configs = {}
  engine_version                        = "VALKEY_8_0"
  gcs_source	                        = null
  instance_id                           = "instance_id"
  labels = {}
  location                              = "region_id"
  managed_backup_source                 = null
  mode                                  = "CLUSTER_DISABLED"
  network                               = "network_id"
  node_type                             = "STANDARD_SMALL"
  persistence_config                    = {}
  project_id                            = project_id
  replica_count                         = 1
  service_connection_policies           = {}
  shard_count                           = 1
  transit_encryption_mode               = "TRANSIT_ENCRYPTION_DISABLED"
  zone_distribution_config_mode         = "MULTI_ZONE"
  zone_distribution_config_zone         = null
}

Terraform Version

terragrunt version v0.73.14

Terraform Provider Versions

Providers required by configuration:
.
├── provider[registry.opentofu.org/hashicorp/google] >= 6.30.0, < 7.0.0
└── module.enable_apis
    ├── provider[registry.opentofu.org/hashicorp/google] >= 3.43.0, < 7.0.0
    └── provider[registry.opentofu.org/hashicorp/google-beta] >= 3.43.0, < 7.0.0

Additional information

No response

[antoinedeschenes]

Seeing the same issue, it seems related to this provider change GoogleCloudPlatform/magic-modules#13819

[imrannayer]

@antoinedeschenes @zdunning13
I created valkey instance using this module. Re-ran the terraform apply with and without changes. I dont see valkey instance getting recreated. This is the reason why I kept desired_auto_created_endpoints out of release 15 cos it will recreate the instances that are already created.
I used code from examples dir.

[antoinedeschenes]

On our side, module v15.0.0 currently tries to replace the valkey instance created with desired_auto_created_endpoints on every plan (using provider hashicorp/google v6.42.0):

      - desired_auto_created_endpoints { # forces replacement
          - network    = "projects/<project>/global/networks/default" -> null
          - project_id = "<project>" -> null
        }

To work around it, we added an override.tf file with the desired_auto_created_endpoints block, resolving the issue:

resource "google_memorystore_instance" "valkey_cluster" {
  desired_auto_created_endpoints {
    network    = "projects/${coalesce(var.network_project, var.project_id)}/global/networks/${var.network}"
    project_id = var.project_id
  }
}

[imrannayer]

@antoinedeschenes but the module does not have capability of creating cluster withdesired_auto_created_endpoints. How were you able to create a cluster using this module with desired_auto_created_endpoints?

[antoinedeschenes]

I assume the Terraform provider eventually imports the desired_auto_created_endpoints block during state refresh on subsequent plans

[imrannayer]

@antoinedeschenes I dont think so. I refreshed my state and nothing changes.