Skip to content

[DOCS] Add cloud store backend sync recipe (#11986) - #12046

Open
TemidayoA wants to merge 1 commit into
fivetran:developfrom
TemidayoA:docs/11986-cloud-store-backend-recipe
Open

[DOCS] Add cloud store backend sync recipe (#11986)#12046
TemidayoA wants to merge 1 commit into
fivetran:developfrom
TemidayoA:docs/11986-cloud-store-backend-recipe

Conversation

@TemidayoA

Copy link
Copy Markdown
Contributor

Closes #11986

  • Description of PR changes above includes a link to an existing GitHub issue
  • PR title is prefixed with one of: [BUGFIX], [FEATURE], [DOCS], [MAINTENANCE], [CONTRIB], [MINORBUMP]
  • Code is linted - run invoke lint (uses ruff format + ruff check)
  • Appropriate tests and docs have been updated
  • For any behavioral change to a data source, validation mechanic, or Expectation, at least one integration test exists in tests/integration/data_sources_and_expectations (see AGENTS.md#integration-test-requirement)
  • CI is green, including linting, mypy type-checking, and tests - this is required for merge
  • If this PR proposes adopting or recommending a particular third-party library or service, any affiliation with it (employment, financial interest, maintainership) is disclosed above for the reviewer's context

Summary

Adds a new docs recipe covering the supported sync-based workflow for storing GX artifacts (Expectations, Validation Results, Checkpoints, Data Docs) in cloud object storage, replacing the removed TupleGCSStoreBackend, TupleS3StoreBackend, and TupleAzureBlobStoreBackend store backends from GX 1.0.

Follows the configure_data_docs/ pattern as requested in the issue:

  • cloud_store_backend_sync/cloud_store_backend_sync.md — main guide (pull → run → push workflow, config-as-read-only, safe results syncing, single-writer Data Docs)
  • cloud_store_backend_sync/_backends/_gcs.md, _aws.md, _azure.md — per-provider partials
  • cloud_store_backend_sync/_examples/cloud_store_backend_gcs.py, _aws.py, _azure.py — tested example scripts, wired via <snippet> tags
  • Registered each example script as an IntegrationTestFixture in docs/docusaurus/docs/components/examples_under_test.py with the corresponding BackendDependencies flag (GCS / AWS / AZURE)

For more information about contributing, visit our community resources.

After you submit your PR, keep the page open and monitor the statuses of the various checks made by our continuous integration process at the bottom of the page. Please fix any issues that come up and reach out on Slack if you need help. Thanks for contributing!

@netlify

netlify Bot commented Aug 7, 2026

Copy link
Copy Markdown

👷 Deploy request for niobium-lead-7998 pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 7943dd9

@TemidayoA

Copy link
Copy Markdown
Contributor Author

PR is up for #11986@joshua-stauffer, could you approve the workflow runs when you get a chance?

@TemidayoA

TemidayoA commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Note on folder structure: the issue suggested placing this content under configure_metadata_stores/ or configure_data_docs/, but since the guide covers both the artifact-sync workflow and the Data Docs single-writer pattern together (as one end-to-end recipe), I put it in its own cloud_store_backend_sync/ folder instead of splitting across the two existing guides.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cloud Store Backend docs recipe

1 participant