Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
315 lines (295 loc) · 18.3 KB

CONTRIBUTING.md

File metadata and controls

315 lines (295 loc) · 18.3 KB
Raw

Contributing

Development Workflow

  • Create a feature branch for each change.
  • Keep pull requests focused (core-only, single-companion-module-only, or docs-only when possible).
  • Keep repository-level Bazel config explicit in scripts/workflows: this repo intentionally has no root .bazelrc (example workspaces own local .bazelrc).
  • For new core-rule fail(...) diagnostics, prefer consistent prefixed wording (for example test_optimization_sync:) or shared helpers in tools/core/common_utils.bzl (fail_with_prefix).
  • Preserve public label contracts from sync outputs:
    • :test_optimization_files
    • :test_optimization_context
    • :module_<sanitized>
  • Keep the three sync contracts separate:
    • static single-service in test_optimization_sync.bzl;
    • static multi-service in test_optimization_multi_sync.bzl;
    • invocation-scoped Go/Python aggregation in test_optimization_manifest_sync.bzl. Manifest sync must not acquire target discovery, consumer naming policy, or a checked-in target/service registry.

Validation Commands

  • Canonical full-repo command:

    • ./bazelw test //...
    • On macOS with Bazel 8.5.1, append --noexperimental_split_xml_generation. The split XML helper can terminate with SIGSEGV independently of the test target.

  • Core module tests (repo root):

    • ./bazelw test //tools/...

  • Go companion tests:

    • cd modules/go && ../../bazelw test //... --override_module=datadog-rules-test-optimization=../..

  • Python companion tests:

    • cd modules/python && ../../bazelw test //... --override_module=datadog-rules-test-optimization=../..

  • Java companion tests:

    • cd modules/java && ../../bazelw test //... --override_module=datadog-rules-test-optimization=../..

  • NodeJS companion tests:

    • cd modules/nodejs && ../../bazelw test //... --override_module=datadog-rules-test-optimization=../..

  • .NET companion tests:

    • cd modules/dotnet && ../../bazelw test //... --override_module=datadog-rules-test-optimization=../..

  • Ruby companion tests:

    • cd modules/ruby && ../../bazelw test //... --override_module=datadog-rules-test-optimization=../..

  • Build examples from root:

    • ./bazelw build //examples/...

  • Verify core/companion module version alignment:

    • python3 tools/dev/check_module_versions.py

  • Python tooling tests:

    • bash tools/tests/python/run_python_tools_test.sh
    • ./bazelw test //tools/tests/python:python_tools_test

  • rules_go variant verification:

    • python3 tools/dev/generate_rules_go_fork_maps.py --check
    • python3 tools/dev/materialize_rules_go_fork.py check --all
    • python3 tools/dev/verify_rules_go_profiles.py --public-denylist tools/dev/private_leak_public_denylist.txt
    • python3 tools/dev/check_release_archive_contents.py
    • python3 tools/dev/diff_rules_go_fork.py --all --write-report

  • Optional Python tooling dependencies (for local script execution):

    • python3 -m pip install --require-hashes -r tools/requirements.txt

  • Local lint prerequisites (match CI tooling):

    • shellcheck (shell lint lane)
    • buildifier (Starlark formatting lane)
    • jq (integration harness payload/CODEOWNERS checks)

  • Optional pre-commit setup:

    • python3 -m pip install pre-commit && pre-commit install

  • Optional Python syntax smoke check when editing tooling:

    • python3 -m py_compile tools/core/validate_payload_schema.py tools/core/schemas/sync_agentless_schema.py tools/tests/integration/mock_dd_server.py

  • Integration harness:

    • Prerequisites: jq (Linux/macOS). Windows harness is PowerShell-only.
    • Linux/macOS: tools/tests/integration/run_mock_server_tests.sh
    • Windows primary entrypoint: pwsh -File tools/tests/integration/run_mock_server_tests.ps1
    • Windows convenience wrapper: tools/tests/integration/run_mock_server_tests.cmd
    • Mixed-runtime uploader changes are not done until both harnesses still pass: they cover single-context, explicit override, multi-context repo selection, and no-match fallback behavior.
    • Manifest-driven Go/Python changes: python3 tools/tests/integration/run_manifest_sync_tests.py --mode full. This proves strict manifest validation, disabled no-op behavior, deterministic output, target-specific cache invalidation, exact doctor targets, aggregate enrichment, and no host Go.
    • Windows manifest-disabled parsing: python tools/tests/integration/run_manifest_sync_tests.py --mode disabled.

  • Test Optimization bootstrap config:

    • For config-gated Go and Python onboarding, keep --config=test-optimization as the only user-facing switch. The shared config entry is common:test-optimization --repo_env=DD_TEST_OPTIMIZATION_ENABLED=1.
    • Go additionally sets the existing rules_go Orchestrion enabled=true build setting. Python-only consumers must not declare that Go-only label.
    • Omitting the config is the documented complete opt-out for Go and Python: metadata repositories use disabled stubs when enabled_by_env = True, Go aliases select local empty targets, and Python keeps the consumer runner without Test Optimization wiring.
    • Other companions retain their existing enablement contract. Do not add enabled_by_env = True to their sync repositories until their runtime wrapper implements and tests the disabled export contract.
    • Do not add a consumer-local Test Optimization bool flag or a second Go Orchestrion repository chooser.

  • Go consumer integration harnesses:

    • Bzlmod default smoke: tools/tests/integration/run_bzlmod_go_integration.sh
    • WORKSPACE default smoke: tools/tests/integration/run_workspace_go_integration.sh
    • WORKSPACE base, rules_go v0_60_0: USE_BAZEL_VERSION=8.4.1 RULES_GO_UPSTREAM=v0_60_0 RULES_GO_VARIANT=base tools/tests/integration/run_workspace_go_integration.sh
    • Bzlmod base, rules_go v0_60_0: USE_BAZEL_VERSION=8.4.1 RULES_GO_UPSTREAM=v0_60_0 RULES_GO_VARIANT=base tools/tests/integration/run_bzlmod_go_integration.sh
    • WORKSPACE base, rules_go v0_61_1: USE_BAZEL_VERSION=8.4.1 RULES_GO_UPSTREAM=v0_61_1 RULES_GO_VARIANT=base tools/tests/integration/run_workspace_go_integration.sh
    • Bzlmod base, rules_go v0_61_1: USE_BAZEL_VERSION=8.4.1 RULES_GO_UPSTREAM=v0_61_1 RULES_GO_VARIANT=base tools/tests/integration/run_bzlmod_go_integration.sh
    • WORKSPACE base, rules_go v0_62_0: USE_BAZEL_VERSION=8.4.1 RULES_GO_UPSTREAM=v0_62_0 RULES_GO_VARIANT=base tools/tests/integration/run_workspace_go_integration.sh
    • Bzlmod base, rules_go v0_62_0: USE_BAZEL_VERSION=8.4.1 RULES_GO_UPSTREAM=v0_62_0 RULES_GO_VARIANT=base tools/tests/integration/run_bzlmod_go_integration.sh
    • Disabled alias gate for a fresh output root: WINDOWS_DISABLED_SMOKE_ONLY=1 RULES_GO_UPSTREAM=v0_60_0 RULES_GO_VARIANT=base tools/tests/integration/run_workspace_go_integration.sh
    • Enabled alias and payload gate with valid Orchestrion pins: WINDOWS_ENABLED_SMOKE_ONLY=1 RULES_GO_UPSTREAM=v0_60_0 RULES_GO_VARIANT=base tools/tests/integration/run_workspace_go_integration.sh
    • Same-output-root disabled then enabled transition for the public central wrapper: CONFIG_TRANSITION_ONLY=1 RULES_GO_UPSTREAM=v0_60_0 RULES_GO_VARIANT=base tools/tests/integration/run_workspace_go_integration.sh
    • Cold bootstrap without host Go, using the Bazel-managed SDK and an isolated bootstrap cache: cache_root="$(mktemp -d)"; CONFIG_TRANSITION_ONLY=1 FORBID_HOST_GO=1 EXPECTED_ORCHESTRION_CACHE_PHASE=extensions.bootstrap_cache_miss XDG_CACHE_HOME="$cache_root" RULES_GO_UPSTREAM=v0_60_0 RULES_GO_VARIANT=base tools/tests/integration/run_workspace_go_integration.sh
    • Repeat that command with the same XDG_CACHE_HOME and EXPECTED_ORCHESTRION_CACHE_PHASE=extensions.bootstrap_cache_hit to prove warm restoration also avoids host Go. Run both commands for WORKSPACE and Bzlmod; CI covers both supported rules_go upstreams.
    • Each script now validates:
      • the same consumer-owned central wrapper call expands to a raw go_test without the named config and to the existing Orchestrion-backed shape with it
      • disabled then enabled resolution on one fixture workspace and output root, with no intervening clean or shutdown
      • no hidden Test Optimization targets, empty Orchestrion aliases, exact disabled stubs, zero metadata HTTP requests, and no payload while disabled
      • normal mode
      • hermetic mode with the inline CI sandbox/network-blocking flags
      • strict BEP fresh/cached uploader behavior
      • structural aquery assertions for the Orchestrion offline module proxy wiring
    • For Go Orchestrion changes, run each relevant script with both ORCHESTRION_MODE=general and ORCHESTRION_MODE=test_optimization; the optimized mode is the standard Go testing Test Optimization path and should still produce payloads.

  • Vendored rules_go variant smoke:

    • RULES_GO_UPSTREAM=v0_60_0 RULES_GO_VARIANT=base tools/dev/run_rules_go_variant_smoke.sh
    • when adding a new upstream, run the same commands with RULES_GO_UPSTREAM=<new_upstream>
    • copies the selected variant into a temp tree, applies the maintainer-only proof overlay, and runs the fast vendored-fork regression set

  • Vendored rules_go extended variant coverage:

    • RULES_GO_UPSTREAM=v0_60_0 RULES_GO_VARIANT=base tools/dev/run_rules_go_variant_extended.sh
    • runs the slower extended regression set against the selected variant

  • Hermetic smoke (mirror CI flags):

    • run the same test commands with sandbox/network-blocking flags from .github/workflows/ci.yml

  • Cross-repo fixture validation for mixed-runtime changes:

    • In ../rules_test_optimization_tests/MODULE.bazel, temporarily enable the documented local_path_override(...) entries for core and each affected companion module so the fixture repo resolves this checkout.
    • If the change touches a rules_go fork tree or related Go bootstrap/orchestrion wiring, also add a temporary local_path_override(module_name = "rules_go", path = "../rules_test_optimization/<registry-resolved-tree-path>") there so the sibling repo resolves the selected local fork.
    • For BEP artifact-resolution changes, run the Go fixture scripts with RTO_LOCAL_ARCHIVE=1 RTO_BEP_ARTIFACT_STAGING=1 so the fixture validates staged BEP artifacts instead of only the pinned release/local-output path:
      • cd ../rules_test_optimization_tests/fixtures/bzlmod-go && RTO_LOCAL_ARCHIVE=1 RTO_BEP_ARTIFACT_STAGING=1 ./runtests
      • cd ../rules_test_optimization_tests/fixtures/workspace-go && RTO_LOCAL_ARCHIVE=1 RTO_BEP_ARTIFACT_STAGING=1 ./runtests
      • cd ../rules_test_optimization_tests/fixtures/bzlmod-go && RTO_LOCAL_ARCHIVE=1 RTO_BEP_ARTIFACT_STAGING=1 ./runtests-hermetic
      • cd ../rules_test_optimization_tests/fixtures/workspace-go && RTO_LOCAL_ARCHIVE=1 RTO_BEP_ARTIFACT_STAGING=1 ./runtests-hermetic
    • Run the relevant fixture entrypoints there before calling the work done.
    • Restore the fixture repo to git_override(...) pins before pushing its PR.

Schema Source Of Truth

  • Canonical schema source is tools/core/schemas/agentless-schema.yaml.
  • Generated artifact is tools/core/schemas/agentless-schema.json.
  • When editing schema structure:
    • update YAML first,
    • run python3 tools/core/schemas/sync_agentless_schema.py,
    • run parity/sync checks:
      • python3 tools/core/schemas/check_schema_parser_parity.py
      • python3 tools/core/schemas/sync_agentless_schema.py --check

CI Lanes

  • bazel-tests:
    • core tests (//tools/...) on Linux/macOS/Windows
    • companion module tests on Linux
    • mock-server integration harness on Linux (.sh) and Windows (.ps1)
    • examples build on Linux/macOS/Windows
    • the Linux result aggregates independent main-suite and mock-server shards
  • bazel-tests-hermetic:
    • core tests with hermetic flags
    • companion module tests with hermetic flags
    • scope policy: Linux-only by design today; non-Linux hermetic expansion is tracked separately to keep CI runtime bounded
  • workspace-compat:
    • one shard per supported rules_go upstream and WORKSPACE/Bzlmod pair
    • general and Test Optimization modes run sequentially inside each shard
    • the Go integration scripts themselves cover normal mode, hermetic mode, and structural aquery checks
  • go-bootstrap-no-host:
    • one shard per supported rules_go upstream and WORKSPACE/Bzlmod pair
    • installs Bazelisk without Go and shadows any hosted-runner go binary with a failing sentinel
    • proves both a cold Bazel-managed-SDK build and warm bootstrap-cache restore without invoking host Go
  • rules-go-variant-smoke:
    • vendored rules_go variant verification and fast fork regression coverage
    • one independent shard per supported upstream
    • Linux-only by design so the PR gate stays fast and stable
  • rules-go-variant-extended:
    • nightly/manual vendored rules_go variant coverage for slower XML, proto, cross, and cgo regression suites
  • Utility/lint lanes:
    • module version alignment check (tools/dev/check_module_versions.py)
    • .bazelversion parity check (tools/dev/check_bazelversion_sync.py)
    • global fork drift checks plus one consumer patch-profile shard per supported rules_go upstream
    • shell scripts, PowerShell, Buildifier, gofmt, schema sync checks, fixture JSON checks, and Python tooling tests
  • Workflow dependency pinning:
    • Keep GitHub Actions pinned by commit SHA and preserve the # vX.Y.Z comment.
    • Use Dependabot (or equivalent) to refresh both SHA and version comment together.

Snapshot Updates

  • Integration snapshots are updated intentionally (never by default).
  • Linux/macOS:
    • UPDATE_SNAPSHOTS=1 tools/tests/integration/run_mock_server_tests.sh
  • Windows PowerShell:
    • $env:UPDATE_SNAPSHOTS = "1"; ./tools/tests/integration/run_mock_server_tests.ps1
  • Windows CMD wrapper:
    • set UPDATE_SNAPSHOTS=1 && tools\\tests\\integration\\run_mock_server_tests.cmd
  • Always review snapshot diffs before committing to ensure they reflect expected behavioral changes.

Maintainer Invariants

  • Root module (MODULE.bazel) must stay runtime-agnostic and avoid non-dev language-rule dependencies. (rules_go is allowed only as dev_dependency for in-repo example builds.)
  • Root module must not declare direct bazel_dep(...) edges to companion modules (datadog-rules-test-optimization-go, ...-python, ...-java, ...-nodejs, ...-dotnet, ...-ruby).
  • Language-specific orchestration stays isolated in modules/<language>.
  • Dev bootstrap wiring in tools/dev/*_bootstrap.bzl is dev-only and cycle-safe.
  • Manifest normalization and repository rendering are deterministic: equivalent input order must produce byte-identical exports, BUILD content, and expected-target JSON.
  • One managed invocation must reuse one manifest path and metadata snapshot across test, doctor, dry-run, and upload. A second invocation refetches once; unchanged stable metadata must retain Bazel test-result cache hits, while telemetry timing facts remain outside test action inputs.
  • The manifest-driven API is additive. Do not change static single-service or static multi-service semantics while modifying it.
  • Validate cross-repository rollout in this order: rules_test_optimization, then rules_test_optimization_tests, then the consumer repository. A green Rule unit test does not replace consumer E2E.

PR Checklist

  • Updated tests for changed behavior.
  • For parser/tooling edits, added malformed-input coverage and verified error diagnostics remain actionable.
  • Ran split-aware validation commands relevant to changed files.
  • Updated docs/snippets for any load-path, module, or API changes.
  • Audited all first-party Markdown and agent skills affected by a public onboarding change; vendored and local historical plan documents are excluded.
  • For manifest-driven changes, proved no committed target/service mapping, exact target-set validation, disabled no-fetch behavior, and deterministic rendering.
  • For Go Orchestrion changes, documented any orchestrion_mode behavior, pin-file or explicit-version behavior, unsupported testify/suite scope, opt-out flags, and payload metadata changes.
  • Updated LICENSE-3rdparty.csv for dependency or vendored-code changes.
  • Confirmed no stale references to removed legacy paths (for example //tools/go:*, replaced by modules/go/... targets).
  • Reviewed timeout metadata when adding new slow tests (--test_verbose_timeout_warnings).
  • Included rationale and risk notes in PR description.

Release Runbook (Core + Companion Modules)

  • CI release gate:

    • Trigger .github/workflows/release.yml via workflow_dispatch (or push a version tag) before publication.
    • The workflow now always executes the full validation suite for release safety.

  • Version alignment:

    • Keep root MODULE.bazel and all companion modules/*/MODULE.bazel versions aligned.
    • Keep each companion dependency on core aligned with the same version.
    • Keep tools/core/common_utils.bzl RULES_VERSION aligned with root MODULE.bazel version.
    • Keep UPLOADER_VERSION in semantic-version format (X.Y.Z) and update it intentionally when uploader runtime behavior changes.
    • Run python3 tools/dev/check_module_versions.py before every release cut.

  • Publication order:

    1. Publish core module metadata/artifacts.
    2. Publish companion metadata/artifacts (depends on core).

  • Companion source mapping:

    • Ensure release metadata maps each companion source to modules/<language> (strip-prefix/subdirectory strategy).

  • BCR metadata checklist (performed in the Bazel Central Registry repo):

    • Core module entry includes MODULE.bazel, metadata.json, and source.json.
    • Companion entries include MODULE.bazel, metadata.json, and source.json.
    • Each companion source.json uses a strip_prefix that resolves to modules/<language> at archive root.
    • Generate scaffolding with BCR helper tooling (for example bazel run //tools:add_module) and then verify each generated file.

  • Pre-announce validation:

    • core tests, companion tests, examples build, integration harness, hermetic lane all green.

  • Rollback notes:

    • If companion release fails, avoid publishing docs that require new load paths until both artifacts are available.