fix: Merge Managed and Unmanaged Users (joining the existing_sso_users club)

[alexandertgtalbot]

• The input variable existing_sso_users, while present, was previsously unused. This includes it into a new merged collection that then updates managed as well as unmanaged user memberships.

[alexandertgtalbot]

This simpler attempt should hopefully clear all checks.

[novekm]

/do-e2e-tests

[aws-ia-automator-prod]

End to end test has been scheduled

[aws-ia-automator-prod]

E2E tests in progress

[aws-ia-automator-prod]

E2E tests in progress

[aws-ia-automator-prod]

E2E test has completed with errors. If you are an external contributor, please contact the project maintainers for more information.

[novekm]

Hi @alexandertgtalbot, thanks for the PR! Were you able to do a successful run of terraform apply after making the changes? Unfortunately, the functional tests are failing with the same error mentioned in #53 and #56, an attempt to iterate over a null value:

tests/02_existing_users_and_groups.tftest.hcl... �[37min progress�[0m
--
606 | �[0m  run "unit_test"... �[31mfail�[0m�[0m
607 | �[31m╷�[0m�[0m
Error: �Iteration over null value on locals.tf line 9, in locals:
for this_user in keys(local.sso_users) : [for group in �[4mlocal.sso_users[this_user].group_membership�[0m : {
613 | �[31m│�[0m �[0m  10:         user_name  = local.sso_users[this_user].user_name
614 | �[31m│�[0m �[0m  11:         group_name = group
615 | �[31m│�[0m �[0m  12:       }
616 | �[31m│�[0m �[0m  13:     ]�[0m
617 | �[31m│�[0m �[0m    �[90m├────────────────�[0m
618 | �[31m│�[0m �[0m�[0m    �[90m│�[0m �[1mlocal.sso_users�[0m is object with 1 attribute "testuser"
619 | �[31m│�[0m �[0m�[0m
620 | �[31m│�[0m �[0mA null value cannot be used as the collection in a 'for' expression.
621 | �[31m╵�[0m�[0m
622 | run "e2e_test"... �[37mskip�[0m�[0m
623 | tests/02_existing_users_and_groups.tftest.hcl... �[37mtearing down�[0m
624 | �[0mtests/02_existing_users_and_groups.tftest.hcl... �[31mfail�[0m�[0m

This is the line causing the error:

for group in local.sso_users[this_user].group_membership

The error stems from the example configuration in /examples/existing-users-and-groups. Within this example, an existing user testuser and existing group testgroup are present. However, the testuser is not actually a member of testgroup. This is meant to test for functionality one may have multiple groups, as well as multiple users, but each user not necessarily assigned to a group, yet you want to assign permission sets and account assignments to these entities. See the below code snippet for an example:

module "aws-iam-identity-center" {
  source = "../.." // local example
  # source = "aws-ia/iam-identity-center/aws" // remote example

  # Ensure these User/Groups already exist in your AWS account
  existing_sso_groups = {
    testgroup : {
      group_name = "testgroup" # this must be the name of a group that already exists in your AWS account
    },
  }
  existing_sso_users = {
    testuser : {
      user_name = "testuser" # this must be the name of a user that already exists in your AWS account
    },
  }


  # Create permissions sets backed by AWS managed policies
  permission_sets = {
    AdministratorAccess = {
      description          = "Provides AWS full access permissions.",
      session_duration     = "PT4H", // how long until session expires - this means 4 hours. max is 12 hours
      aws_managed_policies = ["arn:aws:iam::aws:policy/AdministratorAccess"]
      tags                 = { ManagedBy = "Terraform" }
    },
    ViewOnlyAccess = {
      description          = "Provides AWS view only permissions.",
      session_duration     = "PT3H", // how long until session expires - this means 3 hours. max is 12 hours
      aws_managed_policies = ["arn:aws:iam::aws:policy/job-function/ViewOnlyAccess"]
      tags                 = { ManagedBy = "Terraform" }
    },
  }


  # Assign users/groups access to accounts with the specified permissions
  # Ensure these User/Groups already exist in your AWS account
  account_assignments = {
    testgroup : {
      principal_name  = "testgroup"
      principal_type  = "GROUP"
      principal_idp   = "EXTERNAL"
      permission_sets = ["AdministratorAccess", "ViewOnlyAccess", ]
      account_ids = [              // account(s) the user will have access to. Permissions they will have in account are above line
        local.account1_account_id, // locals are used to allow for global changes to multiple account assignments
        # local.account2_account_id, // if hard coding the account ids, you would need to change them in every place you want to change
        # local.account3_account_id, // these are defined in a locals.tf file, example is in this directory
        # local.account4_account_id,
      ]
    },
    testuser : {
      principal_name  = "testuser"
      principal_type  = "USER"
      principal_idp   = "EXTERNAL"
      permission_sets = ["ViewOnlyAccess"]
      account_ids = [              // account(s) the user will have access to. Permissions they will have in account are above line
        local.account1_account_id, // locals are used to allow for global changes to multiple account assignments
        # local.account2_account_id, // if hard coding the account ids, you would need to change them in every place you want to change
        # local.account3_account_id, // these are defined in a locals.tf file, example is in this directory
        # local.account4_account_id,
      ]
    },
  }

}

In this scenario, group_membership is not set (intentionally) which will lead this value to be null. For more context, there are three possible types of users:

• sso_users - users that you wish to create in AWS IAM IdC with Identity Store as the Identity Provider
• existing_sso_users - users that were synced to AWS IAM IdC with SCIM (e.g. from Okta, Entra ID, etc)
• google_sso_users - users that were synced to AWS IAM IdC from Google Workspace. This is separate from existing_sso_users because of the way SCIM works with Google Workspace. More info on this is in Should existing_google_sso_users even be called that?? #46

With that being said, I don't believe we should merge the managed (sso_users) and federated (existing_sso_users or google_sso_users) users - these are intentionally separated due to the different ways they are handled in the module. I believe this PR and the others mentioned above were meant to address this use case outlined in #47 - adding an existing user or group to new permission sets and account assignments.

The only use case I see for this is for existing users/groups that were manually (or otherwise) created outside of the module, but using Identity Store as the IdP. For these resource it is now desired to use these with the module. If this is the case, the way to achieve this would be the use the sso_users variable (since this aligns with IdentityStore as the IdP), by import the those users users into the module with an import block, or recreating the resources and having some sort of migration process for your users.

I haven't tested this, but if using the import block, you'd likely do something like this:

module "iam_identity_center" {
  source  = "aws-ia/iam-identity-center/aws"
  version = "1.0.2"

  # Define the users
  sso_users = {
    "john.doe" = {
      user_name = "jdoe"
      given_name = "John"
      family_name = "Doe"
      email    = "[email protected]"
      group_membership = ["developers", "operations"]
    }
    "jane.smith" = {
      user_name = "jsmith
      given_name = "Jane"
      family_name = "Smith"
      email    = "[email protected]"
      group_membership = ["developers", "operations"]
    }
  }

  # Define the groups
  sso_groups = {
    "developers" = {
      display_name = "Developers"
      description  = "Development team"
    }
    "operations" = {
      display_name = "Operations"
      description  = "Operations team"
    }
  }

  # ... rest of your module configuration (permission sets, assignments, etc.)
}

# Import existing users
import {
  to = module.iam_identity_center.aws_identitystore_user.this["john.doe"]
  id = "d-1234567890/user-id-for-john"  # Format: "identity-store-id/user-id"
}

import {
  to = module.iam_identity_center.aws_identitystore_user.this["jane.smith"]
  id = "d-1234567890/user-id-for-jane"
}

# Import existing groups
import {
  to = module.iam_identity_center.aws_identitystore_group.this["developers"]
  id = "d-1234567890/group-id-for-developers"  # Format: "identity-store-id/group-id"
}

import {
  to = module.iam_identity_center.aws_identitystore_group.this["operations"]
  id = "d-1234567890/group-id-for-operations"
}

Does this seem to achieve what you're looking for?

[alexandertgtalbot]

That's some great feedback @novekm, appreciate all the detail/context provided. Let me rework my attempt.

[alexandertgtalbot]

Another attempt please @novekm . I've added coalesce defaults to protect against the previous null-based errors.

[novekm]

Hi @alexandertgtalbot, so to confirm is this meant to merge sso_users with existing_sso_users? As mentioned above, these were intentionally separate. sso_users to create new users with IdentityStore as the IdP, and existing_sso_users to reference the users that already exist. What is the use case to merge these?

[alexandertgtalbot]

Hi @novekm,

Sorry for the War-and-Peace-level response....

The intention here is to merge both collections supporting both classes of users whilst managing RBAC through IAM IDC group assignments.

Scenario 1

This is my real-world experience in implementing federated auth:

• Users are initially managed using IAM IDC as the primary IDP and assigned roles through group membership, e.g:

sso_users = {
  "user.one" : {
    group_membership = [
      "group1",
      "group2",
      "group3",
    ],
    user_name   = "user.one"
    given_name  = "User"
    family_name = "One"
    email       = "[email protected]"
  },
  "user.two" : {
    group_membership = [
      "group2",
      "group3",
      "group4",
    ],
    user_name   = "user.two"
    given_name  = "User"
    family_name = "Two"
    email       = "[email protected]"
  },
  "user.three" : {
    group_membership = [
      "group3",
      "group4",
      "group5",
    ],
    user_name   = "user.three"
    given_name  = "User"
    family_name = "Three"
    email       = "[email protected]"
  },
}

• I update the inputs to the module (in preparation for federation) from:

inputs = {
  account_assignments = local.account_assignments
  permission_sets     = local.permission_sets
  sso_groups          = local.sso_groups
}

to

inputs = {
  account_assignments = local.account_assignments
  permission_sets     = local.permission_sets
  sso_groups          = local.sso_groups
  existing_sso_users  = local.sso_users
}

• IAM IDC is updated to federate auth to an Entra Ent. App. that has a single empty group. The intention here is to have a single group to make available Entra IDs to IAM IDC in a controlled fashion. IAM IDC Account and Application access is managed through IAM IDC group assignments, not Entra groups (key point).

• SCIM is configured and a select number of users are assigned to the single Ent. App. group.

• After sync'ing there are now a mix of "Manual" and "SCIM" users.

• Given the following user config:

sso_users = {
  "user.one" : {
    group_membership = [
      "group1",
      "group2",
      "group3",
    ],
    user_name   = "user.one"
    given_name  = "User"
    family_name = "One"
    email       = "[email protected]"
  },
  "user.two" : {
    group_membership = [
      "group2",
      "group3",
      "group4",
    ],
    user_name   = "user.two"
    given_name  = "User"
    family_name = "Two"
    email       = "[email protected]"
  },
  "user.three" : {
    group_membership = [
      "group3",
      "group4",
      "group5",
    ],
    user_name   = "user.three"
    given_name  = "User"
    family_name = "Three"
    email       = "[email protected]"
  },
}

• I delete user.three which is presently reported as "Created by" "Manual" (SCIM does not replace users).
• Up to 40 minutes later user.three is present again reported as "Created by" "SCIM".
• Running a TG/TF plan:

...

  # aws_identitystore_group_membership.sso_group_membership["user.three_some_role"] will be destroyed
  # (because key ["user.three_some_role"] is not in for_each map)
  - resource "aws_identitystore_group_membership" "sso_group_membership" {
      - group_id          = "SOMETHING" -> null
      - id                = "d-SOMETHING/SOMETHING" -> null
      - identity_store_id = "d-SOMETHING" -> null
      - member_id         = "SOMETHING" -> null
      - membership_id     = "SOMETHING" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

─────────────────────────────────────────────────────────────────────────────

• With auth federated we cannot create new users in IAM IDC, that is, all users "Manual" and "SCIM" have to be managed as existing_sso_users. Under this condition I now cannot assign users to IAM IDC groups.

Scenario 2

• I want to grant a SCIM user access to an Account through group membership; I find four references to existing_sso_users in main.tf:

# - Identity Store Group Membership -
# New Users with New Groups
resource "aws_identitystore_group_membership" "sso_group_membership" {
  for_each          = local.users_and_their_groups
  identity_store_id = local.sso_instance_id

  group_id  = (contains(local.this_groups, each.value.group_name) ? aws_identitystore_group.sso_groups[each.value.group_name].group_id : data.aws_identitystore_group.existing_sso_groups[each.value.group_name].group_id)
  member_id = (contains(local.this_users, each.value.user_name) ? aws_identitystore_user.sso_users[each.value.user_name].user_id : data.aws_identitystore_user.existing_sso_users[each.value.user_name].user_id)

}

# Existing Google Users with New Groups
resource "aws_identitystore_group_membership" "sso_group_membership_existing_google_sso_users" {
  for_each          = local.users_and_their_groups_existing_google_sso_users
  identity_store_id = local.sso_instance_id

  group_id  = (contains(local.this_groups, each.value.group_name) ? aws_identitystore_group.sso_groups[each.value.group_name].group_id : data.aws_identitystore_group.existing_sso_groups[each.value.group_name].group_id)
  member_id = data.aws_identitystore_user.existing_google_sso_users[each.value.user_name].user_id
  # member_id = (contains(local.this_existing_google_users, each.value.user_name) ? aws_identitystore_user.sso_users[each.value.user_name].user_id : data.aws_identitystore_user.existing_sso_users[each.value.user_name].user_id)

}

as I'm not concerned with Google we'll consider this resource:

resource "aws_identitystore_group_membership" "sso_group_membership" {
  for_each          = local.users_and_their_groups
  identity_store_id = local.sso_instance_id

  group_id  = (contains(local.this_groups, each.value.group_name) ? aws_identitystore_group.sso_groups[each.value.group_name].group_id : data.aws_identitystore_group.existing_sso_groups[each.value.group_name].group_id)
  member_id = (contains(local.this_users, each.value.user_name) ? aws_identitystore_user.sso_users[each.value.user_name].user_id : data.aws_identitystore_user.existing_sso_users[each.value.user_name].user_id)

}

iterates over users_and_their_groups which itself is defined as (locals.tf):

...
  users_and_their_groups = {
    for s in local.flatten_user_data : format("%s_%s", s.user_name, s.group_name) => s
  }
...

which references:

...
  # Create a new local variable by flattening the complex type given in the variable "sso_users"
  flatten_user_data = flatten([
    for this_user in keys(var.sso_users) : [
      for group in var.sso_users[this_user].group_membership : {
        user_name  = var.sso_users[this_user].user_name
        group_name = group
      }
    ]
  ])
...

leaving existing_sso_users un-assignable to IAM IDC groups.

• The second reference:

resource "aws_ssoadmin_account_assignment" "account_assignment" {
  for_each = local.principals_and_their_account_assignments // for_each arguement must be a map, or set of strings. Tuples won't work

  instance_arn       = local.ssoadmin_instance_arn
  permission_set_arn = contains(local.this_permission_sets, each.value.permission_set) ? aws_ssoadmin_permission_set.pset[each.value.permission_set].arn : data.aws_ssoadmin_permission_set.existing_permission_sets[each.value.permission_set].arn

  principal_type = each.value.principal_type

  # Conditional use of resource or data source to reference the principal_id depending on if the principal_type is "GROUP" or "USER" and if the principal_idp is "INTERNAL" or "EXTERNAL". "INTERNAL" aligns with users or groups that were created with this module and use the default IAM Identity Store as the IdP. "EXTERNAL" aligns with users or groups that were created outside of this module (e.g. via external IdP such as EntraID, Okta, Google, etc.) and were synced via SCIM to IAM Identity Center.

  principal_id = each.value.principal_type == "GROUP" && each.value.principal_idp == "INTERNAL" ? aws_identitystore_group.sso_groups[each.value.principal_name].group_id : (each.value.principal_type == "USER" && each.value.principal_idp == "INTERNAL" ? aws_identitystore_user.sso_users[each.value.principal_name].user_id : (each.value.principal_type == "GROUP" && each.value.principal_idp == "EXTERNAL" ? data.aws_identitystore_group.existing_sso_groups[each.value.principal_name].group_id : (each.value.principal_type == "USER" && each.value.principal_idp == "EXTERNAL" ? data.aws_identitystore_user.existing_sso_users[each.value.principal_name].user_id : (each.value.principal_type == "USER" && each.value.principal_idp == "GOOGLE") ? data.aws_identitystore_user.existing_google_sso_users[each.value.principal_name].user_id : null)))

  target_id   = each.value.account_id
  target_type = "AWS_ACCOUNT"
}

which on the face of it reads like it should work but only supports direct assignment to an account, not indirect through group membership.

• The fourth reference:

# Users assignments
resource "aws_ssoadmin_application_assignment" "sso_apps_users_assignments" {
  for_each = {
    for idx, assignment in local.apps_users_assignments :
    "${assignment.app_name}-${assignment.user_name}" => assignment
  }
  application_arn = aws_ssoadmin_application.sso_apps[each.value.app_name].application_arn 
  principal_id    = (contains(local.this_users, each.value.user_name) ? aws_identitystore_user.sso_users[each.value.user_name].user_id : data.aws_identitystore_user.existing_sso_users[each.value.user_name].user_id)
  principal_type  = each.value.principal_type
}

like the Account example above should work fine but is direct assignment instead of group assigned.

Wrap-up

• The only path to group membership is through here:

# - Identity Store Group Membership -
# New Users with New Groups
resource "aws_identitystore_group_membership" "sso_group_membership" {
  for_each          = local.users_and_their_groups
  identity_store_id = local.sso_instance_id

  group_id  = (contains(local.this_groups, each.value.group_name) ? aws_identitystore_group.sso_groups[each.value.group_name].group_id : data.aws_identitystore_group.existing_sso_groups[each.value.group_name].group_id)
  member_id = (contains(local.this_users, each.value.user_name) ? aws_identitystore_user.sso_users[each.value.user_name].user_id : data.aws_identitystore_user.existing_sso_users[each.value.user_name].user_id)

}

as users_and_their_groups never contains existing_sso_users, e.g.:

flatten_user_data = flatten([
    for this_user in keys(var.sso_users) : [
      for group in var.sso_users[this_user].group_membership : {
        user_name  = var.sso_users[this_user].user_name
        group_name = group
      }
    ]
  ])

  users_and_their_groups = {
    for s in local.flatten_user_data : format("%s_%s", s.user_name, s.group_name) => s
  }

the contains(...) lookup on local.this_users will never match users in var.existing_sso_users, e.g.:

member_id = (contains(local.this_users, each.value.user_name) ? aws_identitystore_user.sso_users[each.value.user_name].user_id : data.aws_identitystore_user.existing_sso_users[each.value.user_name].user_id)

making data.aws_identitystore_user.existing_sso_users[each.value.user_name].user_id) redundant/unreachable in the conditional.

This update thus aims to extend a warm hug to our existing_sso_users through inclusivity :)

[alexandertgtalbot]

Just following up on this @novekm, any thoughts?

[alexandertgtalbot]

Okay, I accept defeat :)

For any future travellers finding their way here, here's a quick unblock/fix: https://registry.terraform.io/modules/refractive-space/iam-identity-center/aws/latest