Create gcp infra for Headlamp

[joaquimrocha]

So we start publishing images the Kubernetes way.

[k8s-ci-robot]

Welcome @joaquimrocha!

It looks like this is your first PR to kubernetes/k8s.io 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/k8s.io has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @joaquimrocha. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[joaquimrocha]

Hey everyone. Please let me know if there are other bits related to getting the Headlamp image published by Kubernetes that we should add to this PR already, I understood this was the first step but couldn't grasp yet the entire process.

[mrbobbytables]

/ok-to-test

[ameukam]

We changed the way we bootstrap repositories. Sorry for the lack of docs.

See #8049 as an example

[joaquimrocha]

Thank you @ameukam . I have based these changes on the ones you referenced.

[xmudrii]

Looking at #8049, it also includes a change to groups/restrictions.yaml. I think we probably need it here as well.

[xmudrii]

@joaquimrocha We're still missing an entry to groups/restrictions.yaml, which I think is mandatory, but @ameukam can confirm for sure.

[joaquimrocha]

@joaquimrocha We're still missing an entry to groups/restrictions.yaml, which I think is mandatory, but @ameukam can confirm for sure.

Seems like I missed that too. Thanks for bringing it up. PR updated.

[xmudrii]

/lgtm
/approve
/hold
for @ameukam to take a look

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: joaquimrocha, xmudrii

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• groups/OWNERS [xmudrii]
• infra/gcp/terraform/OWNERS [xmudrii]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[xmudrii]

atlantis plan

[k8s-infra-ci-robot]

Ran Plan for dir: infra/gcp/terraform/k8s-staging-images workspace: default

Show Output

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:

  # module.artifact_registry["charts"].google_artifact_registry_repository.repo has changed
~ resource "google_artifact_registry_repository" "repo" {
        id                     = "projects/k8s-staging-images/locations/us-central1/repositories/charts"
        name                   = "charts"
      ~ update_time            = "2025-05-14T02:24:28.265589Z" -> "2025-05-27T07:59:51.206978Z"
        # (12 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.artifact_registry["cloud-provider-kind"].google_artifact_registry_repository.repo has changed
~ resource "google_artifact_registry_repository" "repo" {
        id                     = "projects/k8s-staging-images/locations/us-central1/repositories/cloud-provider-kind"
        name                   = "cloud-provider-kind"
      ~ update_time            = "2025-05-14T20:12:20.471676Z" -> "2025-05-26T07:53:30.689543Z"
        # (12 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.artifact_registry["csi-vsphere"].google_artifact_registry_repository.repo has changed
~ resource "google_artifact_registry_repository" "repo" {
        id                     = "projects/k8s-staging-images/locations/us-central1/repositories/csi-vsphere"
        name                   = "csi-vsphere"
      ~ update_time            = "2025-05-14T14:57:42.749560Z" -> "2025-05-27T05:20:53.695334Z"
        # (12 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.artifact_registry["dra-example-driver"].google_artifact_registry_repository.repo has changed
~ resource "google_artifact_registry_repository" "repo" {
        id                     = "projects/k8s-staging-images/locations/us-central1/repositories/dra-example-driver"
        name                   = "dra-example-driver"
      ~ update_time            = "2025-05-13T13:45:33.108228Z" -> "2025-05-22T12:20:06.960460Z"
        # (12 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.artifact_registry["gateway-api-inference-extension"].google_artifact_registry_repository.repo has changed
~ resource "google_artifact_registry_repository" "repo" {
        id                     = "projects/k8s-staging-images/locations/us-central1/repositories/gateway-api-inference-extension"
        name                   = "gateway-api-inference-extension"
      ~ update_time            = "2025-05-14T20:43:39.592890Z" -> "2025-05-26T19:46:13.541920Z"
        # (12 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.artifact_registry["ingress-nginx"].google_artifact_registry_repository.repo has changed
~ resource "google_artifact_registry_repository" "repo" {
        id                     = "projects/k8s-staging-images/locations/us-central1/repositories/ingress-nginx"
        name                   = "ingress-nginx"
      ~ update_time            = "2025-05-08T06:26:08.521706Z" -> "2025-05-27T06:50:16.868874Z"
        # (12 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.artifact_registry["jobset"].google_artifact_registry_repository.repo has changed
~ resource "google_artifact_registry_repository" "repo" {
        id                     = "projects/k8s-staging-images/locations/us-central1/repositories/jobset"
        name                   = "jobset"
      ~ update_time            = "2025-05-14T00:31:17.923670Z" -> "2025-05-27T08:00:33.344138Z"
        # (12 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.artifact_registry["kueue"].google_artifact_registry_repository.repo has changed
~ resource "google_artifact_registry_repository" "repo" {
        id                     = "projects/k8s-staging-images/locations/us-central1/repositories/kueue"
        name                   = "kueue"
      ~ update_time            = "2025-05-14T21:00:48.436806Z" -> "2025-05-27T08:00:25.180101Z"
        # (12 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.artifact_registry["lws"].google_artifact_registry_repository.repo has changed
~ resource "google_artifact_registry_repository" "repo" {
        id                     = "projects/k8s-staging-images/locations/us-central1/repositories/lws"
        name                   = "lws"
      ~ update_time            = "2025-05-13T07:55:21.689338Z" -> "2025-05-26T19:45:50.540495Z"
        # (12 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
~ update in-place

Terraform will perform the following actions:

  # module.artifact_registry["headlamp"].google_artifact_registry_repository.repo will be created
+ resource "google_artifact_registry_repository" "repo" {
      + cleanup_policy_dry_run = false
      + create_time            = (known after apply)
      + effective_labels       = {
          + "goog-terraform-provisioned" = "true"
        }
      + format                 = "DOCKER"
      + id                     = (known after apply)
      + location               = "us-central1"
      + mode                   = "STANDARD_REPOSITORY"
      + name                   = (known after apply)
      + project                = "k8s-staging-images"
      + repository_id          = "headlamp"
      + terraform_labels       = {
          + "goog-terraform-provisioned" = "true"
        }
      + update_time            = (known after apply)

      + cleanup_policies {
          + action = "DELETE"
          + id     = "delete-images-older-than-90-days"

          + condition {
              + older_than            = "7776000s"
              + package_name_prefixes = []
              + tag_prefixes          = []
              + tag_state             = "ANY"
              + version_name_prefixes = []
                # (1 unchanged attribute hidden)
            }
        }
    }

  # module.artifact_registry["headlamp"].google_artifact_registry_repository_iam_member.readers["allUsers"] will be created
+ resource "google_artifact_registry_repository_iam_member" "readers" {
      + etag       = (known after apply)
      + id         = (known after apply)
      + location   = "us-central1"
      + member     = "allUsers"
      + project    = "k8s-staging-images"
      + repository = (known after apply)
      + role       = "roles/artifactregistry.reader"
    }

  # module.artifact_registry["headlamp"].google_artifact_registry_repository_iam_member.writers["group:[email protected]"] will be created
+ resource "google_artifact_registry_repository_iam_member" "writers" {
      + etag       = (known after apply)
      + id         = (known after apply)
      + location   = "us-central1"
      + member     = "group:[email protected]"
      + project    = "k8s-staging-images"
      + repository = (known after apply)
      + role       = "roles/artifactregistry.writer"
    }

  # module.iam.google_project_iam_binding.project_iam_authoritative["default--roles/viewer"] will be updated in-place
~ resource "google_project_iam_binding" "project_iam_authoritative" {
        id      = "k8s-staging-images/roles/viewer"
      ~ members = [
          + "group:[email protected]",
            # (18 unchanged elements hidden)
        ]
        # (3 unchanged attributes hidden)
    }

Plan: 3 to add, 1 to change, 0 to destroy.

• ▶️ To apply this plan, comment:

  atlantis apply -d infra/gcp/terraform/k8s-staging-images

• 🚮 To delete this plan and lock, click here
• 🔁 To plan this project again, comment:

  atlantis plan -d infra/gcp/terraform/k8s-staging-images

Note: Objects have changed outside of Terraform
Plan: 3 to add, 1 to change, 0 to destroy.

---

• ⏩ To apply all unapplied plans from this Pull Request, comment:

  atlantis apply

• 🚮 To delete all plans and locks from this Pull Request, comment:

  atlantis unlock

[xmudrii]

atlantis apply

[k8s-infra-ci-robot]

Ran Apply for dir: infra/gcp/terraform/k8s-staging-images workspace: default

Apply Error

Show Output

running 'sh -c' '/usr/local/bin/terraform1.10.5 apply -input=false "/atlantis/repos/kubernetes/k8s.io/8028/default/infra/gcp/terraform/k8s-staging-images/default.tfplan"' in '/atlantis/repos/kubernetes/k8s.io/8028/default/infra/gcp/terraform/k8s-staging-images': exit status 1
module.iam.google_project_iam_binding.project_iam_authoritative["default--roles/viewer"]: Modifying... [id=k8s-staging-images/roles/viewer]
module.artifact_registry["headlamp"].google_artifact_registry_repository.repo: Creating...
module.artifact_registry["headlamp"].google_artifact_registry_repository.repo: Still creating... [10s elapsed]
module.artifact_registry["headlamp"].google_artifact_registry_repository.repo: Creation complete after 11s [id=projects/k8s-staging-images/locations/us-central1/repositories/headlamp]
module.artifact_registry["headlamp"].google_artifact_registry_repository_iam_member.readers["allUsers"]: Creating...
module.artifact_registry["headlamp"].google_artifact_registry_repository_iam_member.writers["group:[email protected]"]: Creating...
module.artifact_registry["headlamp"].google_artifact_registry_repository_iam_member.readers["allUsers"]: Creation complete after 4s [id=projects/k8s-staging-images/locations/us-central1/repositories/headlamp/roles/artifactregistry.reader/allUsers]
╷
│ Error: Error applying IAM policy for artifactregistry repository "projects/k8s-staging-images/locations/us-central1/repositories/headlamp": Error setting IAM policy for artifactregistry repository "projects/k8s-staging-images/locations/us-central1/repositories/headlamp": googleapi: Error 400: Group [email protected] does not exist.
│ 
│   with module.artifact_registry["headlamp"].google_artifact_registry_repository_iam_member.writers["group:[email protected]"],
│   on .terraform/modules/artifact_registry/main.tf line 210, in resource "google_artifact_registry_repository_iam_member" "writers":
│  210: resource "google_artifact_registry_repository_iam_member" "writers" {
│ 
╵
╷
│ Error: Request `Set IAM Binding for role "roles/viewer" on "project \"k8s-staging-images\""` returned error: Error applying IAM policy for project "k8s-staging-images": Error setting IAM policy for project "k8s-staging-images": googleapi: Error 400: Group [email protected] does not exist., badRequest
│ 
│   with module.iam.google_project_iam_binding.project_iam_authoritative["default--roles/viewer"],
│   on .terraform/modules/iam/modules/projects_iam/main.tf line 32, in resource "google_project_iam_binding" "project_iam_authoritative":
│   32: resource "google_project_iam_binding" "project_iam_authoritative" {
│ 
╵

[xmudrii]

/hold cancel

[xmudrii]

/skip

[upodroid]

/override 'atlantis/apply'

[k8s-ci-robot]

@upodroid: /override requires failed status contexts, check run or a prowjob name to operate on.
The following unknown contexts/checkruns were given:

• 'atlantis/apply'

Only the following failed contexts/checkruns were expected:

• EasyCLA
• atlantis/apply
• atlantis/apply: infra/gcp/terraform/k8s-staging-images/default
• atlantis/plan
• atlantis/plan: infra/gcp/terraform/k8s-staging-images/default
• pull-k8sio-groups-test
• pull-k8sio-verify
• tide

If you are trying to override a checkrun that has a space in it, you must put a double quote on the context.

In response to this:

/override 'atlantis/apply'

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[upodroid]

/override atlantis/apply

[k8s-ci-robot]

@upodroid: Overrode contexts on behalf of upodroid: atlantis/apply

In response to this:

/override atlantis/apply

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[upodroid]

/override "atlantis/apply: infra/gcp/terraform/k8s-staging-images/default"

[k8s-ci-robot]

@upodroid: Overrode contexts on behalf of upodroid: atlantis/apply: infra/gcp/terraform/k8s-staging-images/default

In response to this:

/override "atlantis/apply: infra/gcp/terraform/k8s-staging-images/default"

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-infra-ci-robot]

Locks and plans deleted for the projects and workspaces modified in this pull request:

• dir: infra/gcp/terraform/k8s-staging-images workspace: default

[xmudrii]

@joaquimrocha The registry is created and reconciled, you should be able to proceed with the implementation.

[joaquimrocha]

@joaquimrocha The registry is created and reconciled, you should be able to proceed with the implementation.

Thank you for your patience and support!