feat: Add Dataproc Networking Diagnostic Runbook #120
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR introduces a new diagnostic runbook for Dataproc within `gcpdiag`. The `NetworkingRunbook` aims to help users troubleshoot common networking and permission issues that can affect Dataproc cluster creation and operation.
**Key Changes:**
* **New Runbook:** `dataproc/networking_runbook.py` orchestrates a series of checks.
* **Individual Check Modules:** Added new rule modules under `gcpdiag/runbook/dataproc/` for specific checks:
* `cluster_existence.py`: Verifies the cluster exists.
* `private_google_access.py`: Checks if Private Google Access is enabled on the cluster's subnet.
* `cloud_nat.py`: Checks if Cloud NAT is configured for the subnet if needed.
* `firewall.py`: Looks for overly restrictive egress firewall rules (e.g., deny all to 0.0.0.0/0).
* `security_and_iam.py`: Placeholder for IAM and Service Account permission checks.
* `cloud_vpn_and_interconnect.py`: Placeholder for Cloud VPN / Interconnect checks.
* `dns.py`: Placeholder for DNS configuration checks.
* `private_service_connect.py`: Placeholder for Private Service Connect checks.
* `secure_web_proxy.py`: Placeholder for Secure Web Proxy checks.
* `service_accounts.py`: Placeholder for detailed Service Account checks.
* `vpc_peering.py`: Placeholder for VPC Peering checks.
* `vpc_service_controls.py`: Placeholder for VPC Service Controls checks.
* **Unit Tests:** Each new rule module includes a corresponding test file (e.g., `cloud_nat_test.py`).
* **Reporting Template:** Added `templates/networking.jinja` to format the output of the runbook.
**Details:**
The runbook executes checks in a logical order. Currently, `Private Google Access`, `Cloud NAT`, and `Firewall` checks have initial implementations. Other modules (IAM, DNS, VPC features, etc.) are placeholders and will be fully implemented in subsequent PRs.
This runbook structure allows for modular and extensible network diagnostics for Dataproc.
cjac
force-pushed
the
dataproc-network-runbook-20250805
branch
from
e6e5dbe to
545b492
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a new diagnostic runbook for Dataproc within
gcpdiag. TheNetworkingRunbookaims to help users troubleshoot common networking and permission issues that can affect Dataproc cluster creation and operation.Key Changes:
dataproc/networking_runbook.pyorchestrates a series of checks.gcpdiag/runbook/dataproc/for specific checks:cluster_existence.py: Verifies the cluster exists.private_google_access.py: Checks if Private Google Access is enabled on the cluster's subnet.cloud_nat.py: Checks if Cloud NAT is configured for the subnet if needed.firewall.py: Looks for overly restrictive egress firewall rules (e.g., deny all to 0.0.0.0/0).security_and_iam.py: Placeholder for IAM and Service Account permission checks.cloud_vpn_and_interconnect.py: Placeholder for Cloud VPN / Interconnect checks.dns.py: Placeholder for DNS configuration checks.private_service_connect.py: Placeholder for Private Service Connect checks.secure_web_proxy.py: Placeholder for Secure Web Proxy checks.service_accounts.py: Placeholder for detailed Service Account checks.vpc_peering.py: Placeholder for VPC Peering checks.vpc_service_controls.py: Placeholder for VPC Service Controls checks.cloud_nat_test.py).templates/networking.jinjato format the output of the runbook.Details:
The runbook executes checks in a logical order. Currently,
Private Google Access,Cloud NAT, andFirewallchecks have initial implementations. Other modules (IAM, DNS, VPC features, etc.) are placeholders and will be fully implemented in subsequent PRs.This runbook structure allows for modular and extensible network diagnostics for Dataproc.