Fix NGFW add-on instructions

fast/addons/2-networking-ngfw/.fast-stage.env

@@ -1,5 +1,5 @@
 FAST_STAGE_DESCRIPTION="NGFW Enterprise networking add-on"
 FAST_STAGE_LEVEL=2
 FAST_STAGE_NAME=networking-ngfw
-FAST_STAGE_DEPS="0-globals 0-org-setup 1-resman 2-networking"
+FAST_STAGE_DEPS="0-globals 0-org-setup 2-networking"
 FAST_STAGE_OPTIONAL="2-security"

fast/addons/2-networking-ngfw/README.md

@@ -30,22 +30,78 @@ The controlling project is usually one of those already created and managed by t
 
 ## How to run this stage
 
-Once the main networking stage has been configured and applied, the following configuration is added the the resource management `fast_addon` variable to create the add-on provider files, and its optional CI/CD resources if those are also required. The add-on name (`networking-ngfw`) is customizable, in case the add-on needs to be run multiple times for example to create different sets of endpoints and NGFW configurations per environment.
+Once the main networking stage has been configured and applied, the following configuration is added to the org setup stage.
 
-```hcl
-fast_addon = {
-  networking-ngfw = {
-    parent_stage = "2-networking"
-    # cicd_config = {
-    #   identity_provider = "github-test"
-    #   repository = {
-    #     name   = "test/ngfw"
-    #     type   = "github"
-    #     branch = "main"
-    #   }
-    # }
-  }
-}
+First, the new provider file is declared in the `defaults.yaml` file.
+
+```yaml
+# defaults.yaml (snippet)
+
+output_files:
+  # ...
+  providers:
+    # ...
+    2-networking-ngfw:
+      bucket: $storage_buckets:iac-0/iac-stage-state
+      prefix: 2-networking-ngfw
+      service_account: $iam_principals:service_accounts/iac-0/iac-networking-rw
+
+```
+
+Then, the GCS folder (shown here) or bucket for the Terraform state is defined in the IaC project.
+
+```yaml
+# projects/iac-0.yaml
+buckets:
+  # ...
+  iac-stage-state:
+    description: Terraform state for stage automation.
+    managed_folders:
+      # ...
+      2-networking-ngfw:
+        iam:
+          roles/storage.admin:
+            - $iam_principals:service_accounts/iac-0/iac-networking-rw
+          $custom_roles:storage_viewer:
+            - $iam_principals:service_accounts/iac-0/iac-networking-ro
+
+```
+
+And finally, grant extra roles at the organization level to the networking service accounts.
+
+```yaml
+# organization/.config.yaml
+iam_by_principals:
+  # ...
+  $iam_principals:service_accounts/iac-0/iac-networking-rw:
+    - roles/compute.orgFirewallPolicyAdmin
+    - roles/compute.xpnAdmin
+    # add the custom role
+    - $custom_roles:ngfw_enterprise_admin
+  $iam_principals:service_accounts/iac-0/iac-networking-ro:
+    - roles/compute.orgFirewallPolicyUser
+    - roles/compute.viewer
+    # add the custom role
+    - $custom_roles:ngfw_enterprise_viewer
+```
+
+If VPC-SC is used, an additional ingress policy needs to be added to the perimeter to allow the NGFW service agent to reach the Certificate Authority Service. Edit and enable the following policy.
+
+```yaml
+from:
+  access_levels:
+    - "*"
+  identities:
+    # TODO: change to actual NGFW service identity
+    - serviceAccount:service-1234567890@gcp-sa-networksecurity.iam.gserviceaccount.com
+to:
+  operations:
+    - method_selectors:
+        - "*"
+      service_name: privateca.googleapis.com
+  resources:
+    # TODO: change to project number where CAS lives
+    - projects/1234567890
 ```
 
 ### Provider and Terraform variables
@@ -64,7 +120,6 @@ ln -s ~/fast-config/providers/2-networking-ngfw-providers.tf ./
 # input files from other stages
 ln -s ~/fast-config/tfvars/0-globals.auto.tfvars.json ./
 ln -s ~/fast-config/tfvars/0-org-setup.auto.tfvars.json ./
-ln -s ~/fast-config/tfvars/1-resman.auto.tfvars.json ./
 ln -s ~/fast-config/tfvars/2-networking.auto.tfvars.json ./
 
 # conventional place for stage tfvars (manually created)

fast/addons/README.md

@@ -4,39 +4,42 @@ Each of the folders contained here is a separate "add-on" that can be used to ad
 
 Add-ons can be thought of as additional thin layers on top of a stage, that reuse its IaC resources and leverage the same IAM configuration: the same service accounts are used to run the add-on, and state configuration is stored in the same bucket as their "parent stage" under a different prefix.
 
-The only dedicated resources that can be optionally defined for add-ons are to enable CI/CD functionality, so that a dedicated repository can be used to host the add-on code and pipeline.
-
-Add-ons are currently only implemented for stage 1 (resource management and VPC-SC), and stage 2 (networking, project factory, security).
-
 ## Add-on configuration
 
-To configure an add-on:
+To configure an add-on, once you have identified its "parent stage" (networking, security, etc.) you have to configure the org setup stage so that its providers file, and associated GCS resources to host the state file, are created. The following example configures resources for the NGFW Networking add-on.
 
-- its "parent stage" (the stage the add-on augments) needs to be enabled and applied, so that the IaC and IAM configurations the add-on uses are present
-- the `fast_addon` variable in the stage controlling the "parent stage" (boostrap for a stage 1 add-on, resource management for a stage 2 add-on) is configured and the stage applied, so that the add-on provider and optional CI/CD resources are created
-- the provider and relevant FAST output variable files are linked or copied in the add-on folder (e.g. via the `fast-links.sh` script)
+First, the new provider file is declared in the `defaults.yaml` file.
 
-At this point the add-on can be run, and operate on the same folders, projects and resources controlled by its "parent stage".
+```yaml
+# defaults.yaml (snippet)
 
-Add-ons typically generate their own FAST output variable files, which can be optionally consumed by downstream stages.
+output_files:
+  # ...
+  providers:
+    # ...
+    2-networking-ngfw:
+      bucket: $storage_buckets:iac-0/iac-stage-state
+      prefix: 2-networking-ngfw
+      service_account: $iam_principals:service_accounts/iac-0/iac-networking-rw
 
-This is an example configuration of the `fast_addon` variable in the resource management stage, to enable running the NGFW networking add-on. The CI/CD configuration block is optional, and commented out here.
+```
 
-```hcl
-fast_addon = {
-  networking-ngfw = {
-    parent_stage = "2-networking"
-    # cicd_config = {
-    #   identity_provider = "github-test"
-    #   repository = {
-    #     name   = "test/ngfw"
-    #     type   = "github"
-    #     branch = "main"
-    #   }
-    # }
-  }
-}
+Then, the GCS folder (shown here) or bucket for the Terraform state is defined in the IaC project.
+
+```yaml
+# projects/iac-0.yaml
+buckets:
+  # ...
+  iac-stage-state:
+    description: Terraform state for stage automation.
+    managed_folders:
+      # ...
+      2-networking-ngfw:
+        iam:
+          roles/storage.admin:
+            - $iam_principals:service_accounts/iac-0/iac-networking-rw
+          $custom_roles:storage_viewer:
+            - $iam_principals:service_accounts/iac-0/iac-networking-ro
 ```
 
-This configuration will create `tfvars/2-networking-ngfw-providers.tf` and
-`tfvars/2-networking-ngfw-r-providers.tf` provider files in the GCS output bucket and local folder (if configured).
+Once the org setup stage is ready and applied, the add-on can be run using the generated provider files. The `fast-links.sh` can be used to link or copy the relevant files to the add-on, in the same way it's used for regular stages.

fast/stages/0-org-setup/README.md

@@ -654,7 +654,7 @@ Define values for the `var.environments` variable in a tfvars file.
 | name | description | type | required | default |
 |---|---|:---:|:---:|:---:|
 | [context](variables.tf#L17) | Context-specific interpolations. | <code title="object&#40;&#123;&#10;  custom_roles          &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  folder_ids            &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  iam_principals        &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  locations             &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  kms_keys              &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  notification_channels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  project_ids           &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  service_account_ids   &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  tag_keys              &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  tag_values            &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  vpc_host_projects     &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;  vpc_sc_perimeters     &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [factories_config](variables.tf#L37) | Configuration for the resource factories or external data. | <code title="object&#40;&#123;&#10;  billing_accounts  &#61; optional&#40;string, &#34;data&#47;billing-accounts&#34;&#41;&#10;  cicd              &#61; optional&#40;string&#41;&#10;  defaults          &#61; optional&#40;string, &#34;data&#47;defaults.yaml&#34;&#41;&#10;  folders           &#61; optional&#40;string, &#34;data&#47;folders&#34;&#41;&#10;  organization      &#61; optional&#40;string, &#34;data&#47;organization&#34;&#41;&#10;  project_templates &#61; optional&#40;string, &#34;data&#47;templates&#34;&#41;&#10;  projects          &#61; optional&#40;string, &#34;data&#47;projects&#34;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [factories_config](variables.tf#L37) | Configuration for the resource factories or external data. | <code title="object&#40;&#123;&#10;  billing_accounts  &#61; optional&#40;string, &#34;datasets&#47;classic&#47;billing-accounts&#34;&#41;&#10;  cicd              &#61; optional&#40;string&#41;&#10;  defaults          &#61; optional&#40;string, &#34;datasets&#47;classic&#47;defaults.yaml&#34;&#41;&#10;  folders           &#61; optional&#40;string, &#34;datasets&#47;classic&#47;folders&#34;&#41;&#10;  organization      &#61; optional&#40;string, &#34;datasets&#47;classic&#47;organization&#34;&#41;&#10;  project_templates &#61; optional&#40;string, &#34;datasets&#47;classic&#47;templates&#34;&#41;&#10;  projects          &#61; optional&#40;string, &#34;datasets&#47;classic&#47;projects&#34;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
 | [org_policies_imports](variables.tf#L52) | List of org policies to import. These need to also be defined in data files. | <code>list&#40;string&#41;</code> |  | <code>&#91;&#93;</code> |
 
 ## Outputs

fast/stages/0-org-setup/cicd.tf

@@ -26,7 +26,8 @@ locals {
   }
   cicd_project_ids = {
     for k, v in merge(
-      var.context.project_ids, module.factory.project_ids
+      var.context.project_ids,
+      module.factory.project_ids
     ) : "$project_ids:${k}" => v
   }
   cicd_workflows = {

fast/stages/0-org-setup/datasets/classic/organization/.config.yaml

@@ -14,7 +14,7 @@
 
 # TODO: data access logs
 
-# yaml-language-server: $schema=../../../schemas/organization.schema.json
+# yaml-language-server: $schema=../../schemas/organization.schema.json
 
 id: $defaults:organization/id
 contacts:
@@ -83,7 +83,7 @@ iam_by_principals:
     - roles/compute.xpnAdmin
   $iam_principals:service_accounts/iac-0/iac-networking-ro:
     - roles/compute.orgFirewallPolicyUser
-    - roles/compute.xpnAdmin
+    - roles/compute.viewer
   $iam_principals:service_accounts/iac-0/iac-security-rw:
     - roles/cloudasset.viewer
   $iam_principals:service_accounts/iac-0/iac-vpcsc-rw:

fast/stages/0-org-setup/variables.tf

@@ -37,13 +37,13 @@ variable "context" {
 variable "factories_config" {
   description = "Configuration for the resource factories or external data."
   type = object({
-    billing_accounts  = optional(string, "data/billing-accounts")
+    billing_accounts  = optional(string, "datasets/classic/billing-accounts")
     cicd              = optional(string)
-    defaults          = optional(string, "data/defaults.yaml")
-    folders           = optional(string, "data/folders")
-    organization      = optional(string, "data/organization")
-    project_templates = optional(string, "data/templates")
-    projects          = optional(string, "data/projects")
+    defaults          = optional(string, "datasets/classic/defaults.yaml")
+    folders           = optional(string, "datasets/classic/folders")
+    organization      = optional(string, "datasets/classic/organization")
+    project_templates = optional(string, "datasets/classic/templates")
+    projects          = optional(string, "datasets/classic/projects")
   })
   nullable = false
   default  = {}

tests/fast/stages/s0_org_setup/not-simple.yaml

@@ -2399,10 +2399,15 @@ values:
     condition: []
     members:
     - group:[email protected]
-    - serviceAccount:[email protected]
     - serviceAccount:[email protected]
     org_id: '1234567890'
     role: roles/compute.xpnAdmin
+  module.organization-iam[0].google_organization_iam_binding.authoritative["roles/compute.viewer"]:
+    condition: []
+    members:
+    - serviceAccount:[email protected]
+    org_id: '1234567890'
+    role: roles/compute.viewer
   module.organization-iam[0].google_organization_iam_binding.authoritative["roles/essentialcontacts.admin"]:
     condition: []
     members:
@@ -2772,7 +2777,7 @@ counts:
   google_logging_project_bucket_config: 3
   google_org_policy_custom_constraint: 1
   google_org_policy_policy: 37
-  google_organization_iam_binding: 35
+  google_organization_iam_binding: 36
   google_organization_iam_custom_role: 9
   google_project: 3
   google_project_iam_binding: 16
@@ -2793,5 +2798,5 @@ counts:
   google_tags_tag_value_iam_binding: 4
   local_file: 9
   modules: 46
-  resources: 310
+  resources: 311
   terraform_data: 2